The Awful Dilemma

I like fixing bugs… It makes people happy who have their bugs fixed, it makes Krita better, and it can be done in relatively small time laps. And it gives one a sense of having been usefully productive to go to the weekly bug summary, and see oneself in the top-five of bug resolvers. Not that I’m there right now, though I was last week, because sometimes one has to dig deeper.

These weeks I’m working on refactoring Krita’s resource systems. Resource in graphics app parlance are things like brushes, gradients, patterns — mostly small files that are stored somewhere on disk and that are loaded on start up. This code dates back to 2000 or so and was originally designed for a world where people would have a few dozen of each resource installed, and where brushes and patterns wouldn’t be bigger than 64 x 64 pixels.

These days, people want to have libraries containing hundreds of resources, and many are huge, like 5000×5000 pixel images. Krita cannot simply load all of that in memory like we’re doing now. It takes too much memory. It takes too much start-up time. It makes organizing resources too hard for the user. Because it uses the ancient KDE system for finding resources in the installation, local installation and local user folder in a tiered system, some resources cannot be edited, like with kxmlgui customization files, any application update will spell disaster.

The whole system will have to be scrapped. We’ll have to have a buffer between the actual resources on disk and the application — a caching database. I kinda feel like I’m jumping down an akonadi-type rabbit hole!

And then there’s tagging and organizing and all the bugs that 18 years of accretion have both fixed, added and papered over. The codebase is the most amazing mix of simple-minded, fiendishly over-complicated and sometimes downright mis-guided patterns and anti-patterns.

So, I’m coding, for the first time since the export filter warning project a couple of years ago, lots and lots and lots of new code. It’s fun! It’ll take at least two months of solid work, probably more, especially since most of it is actual research…

Still, going so deep and losing oneself in the high of concentrated coding means that bug fixing falls by the wayside — even though the result should end with scores of bugs closed — that I feel pangs of guilt. I know that this or that thing is broken, and my fingers itch! But I find it impossible to really carry all that’s needed for this refactoring in my head, and dig into problems in other systems.

Appimages, Snaps, XDG-Apps^WFlatpaks

Lots of excitement… When Canonical announced that their snaps work on a number of other Linux distributions, the reactions were predictable, sort of amusing and missing the point.

In the end, all this going back and forth, these are just turf wars. There are Redhat/Fedora people scared and horrified that Canonical/Ubuntu might actually set a standard for once, there are probably Canonical/Ubuntu people scared that their might not set a standard (though after several days of this netstorm, I haven’t seen anything negative from their side, there are traditional packagers worried that the world may change and that they lose their “curating” position.

And there’s me scared that I’ll have to maintain debs, rpms, flatpaks, snaps, appimages, OSX bundles, MSI installers, NSIS installers and portable zips. My perspective is a bit that of an outsider, I don’t care about the politics, though I do wish that it isn’t a dead certainty that we’ll end up having both flatpaks (horrible name, by the way) and snaps in the Linux world.

Both the Canonical and the Fedora side claim to be working with the community, and, certainly, I was approached about snap and helped make a Krita snap. Which is a big win, both for me and for snap. But both projects ignore the appimage project, which is a real community effort, without corporate involvement. Probably because there is no way for companies to use appimage to create a lock-in effort or chance monetization, it’ll always be a community project, ignored by the big Linux companies.

Here’s my take, speaking a someone who is actually releasing software to end users using some of these new-fangled systems.

The old rpm/deb way of packaging is excellent for creating the base system. For software where having the latest version doesn’t matter that much for productivity. It’s a system that’s been used for about twenty years and served us reasonably well. But if you are developing software for end users that is regularly updated, where the latest version is important because it always has improvements that let the users do more work, it’s a problem. It’s a ghastly drag having to actually make the packages if you’re not part of a distribution, and having to make packages for several distributions is not feasible for a small team. And if we don’t, then when there are distributions that do not backport new versions to old releases because they only backport bugfixes, not releases, users lose out.

Snap turns out to be pretty easy to make, and pretty easy to upload to Ubuntu’s app store, and pretty easy to find once it’s there, seeing that there were already more than a thousand downloads after a few days. I don’t care about the security technology, that’s just not relevant for Krita. If you use Krita, you want it to access your files. It takes about five minutes to make a new snap and upload it — pretty good going. I was amazed and pleased that the snap now runs on a number of other distributions, and if Canonical/Ubuntu follows up on that, plugs the holes and fixes the bugs, it’ll be a big plus. Snap also offers all kinds of flexibility, like adding a patched Qt, that I haven’t even tried yet. I also haven’t checked how to add translations yet, but that’s also because the system we use to release translations for Krita needs changing, and I want to do that first.

I haven’t got any experience with flatpak. I know there was a start on making a Krita flatpak, but I haven’t seen any results. I think that the whole idea of a runtime, which is a dependency thing, is dumb, though. Sure, it’ll save some disk space, but at the cost of added complexity. I don’t want that. For flatpak, I’ll strike a wait-and-see attitude: I don’t see the need for it, but if it materializes, and takes as little of my time as snap, I might make them. Unless I need to install Fedora for it, because that’s one Linux distribution that just doesn’t agree with me.

Appimages, finally, are totally amazing, because they run everywhere. They don’t need any kind of runtime or installation. Creating the initial AppImage recipe took a lot of time and testing, mainly because of the run-everywhere requirement. That means fiddly work trying to figure out which low-level libraries need to be included to make OpenGL work, and which don’t. There might be bumps ahead, for instance if we want to start using OpenCL — or so I was told in a comment on LWN. I don’t know yet. Integration with the desktop environment is something Simon is working on, by installing a .desktop file in the user’s home directory. Sandboxing is also being worked on, using some of the same technology as flatpak, apparently. Automatic updates is also something that is becoming possible. I haven’t had time to investigate those things yet, because of release pressures, kickstarter pressures and all that sort of thing. One possible negative about appimages is that users have a hard time understanding them — they just cannot believe that download, make executable, go is all there’s to it. So much so that I’ve considered making a tar.xz with an executable appimage inside so users are in a more familiar territory. Maybe even change the extension from .appimage to .exe?

Anyway, when it comes to actually releasing software to end users in a way that doesn’t drive me crazy, I love AppImages, I like snap, I hate debs, rpms, repositories, ppa’s and their ilk and flatpak has managed to remain a big unknown. If we could get a third format to replace all the existing formats, say flatsnapimage, wouldn’t that be lovely?

Wouldn’t it?

Running Krita Snaps on Other Distributions

This is pretty cool: in the week before the Krita release, Michael Hall submitted a snapcraft definition for making a Krita snap. A few iterations later, we have something that works (unless you’re using an NVidia GPU with the proprietary drivers). Adding Krita to the Ubuntu app store was also really easy.

And now, if you go to snapcraft.io and click on a Linux distribution’s logo, you’ll get instructions on how to get snap running on your system — and that means the snap package for Krita can work on Arch, Debian, Fedora, Gentoo — and Ubuntu of course. Pretty unbelievable! OpenSUSE is still missing though…

Of course, running a snap still means you need to install something before you can run Krita while an AppImage doesn’t need anything making it executable. Over the past month, I’ve encountered a lot of Linux users who just couldn’t believe it’s so easy, and were asking for install instructions 🙂

Twenty years of Qt!

I first encountered Qt in Linux Journal in 1996. Back then, I wasn’t much of a programmer: I had written a GPL’ed mail and Usenet client in Visual Basic and was playing around with Linux. I wanted to write a word processor, because that was really missing, back then.

I tried to use xforms, but that didn’t have open source (it was binary only, can you believe it?), and besides, horrible. Since I didn’t particularly care about having a GUI, I tried to use curses, which was worse. I had taken a look at Motif years before, so I didn’t look again. Sun’s OPEN LOOK had a toolkit that looked nice, but wasn’t. For work, I had used Delphi and MFC, and had had to make sense of Oracle 2000. None of those were useful for writing a word processor for Linux.

And then, suddenly, out of the blue (though I remembered some of the names involved with Qt as being involved with my favorite ZX Spectrum emulator), appeared Qt. Qt made sense: the classes were helpfully named, the api’s were clear and sensible, the documentation was good, the look and feel worked fine. It had everything you’d need to write a real application. Awesome!

So I got started and discovered that, in the first place, I didn’t know what makes a word processor tick, and in the second place, I didn’t know C++… So my project foundered, the way projects tend to do, if you’re trying to do stuff all by your lonesome.

Then I changed jobs, stopped working on a broken-by-design Visual Basic + Oracle laboratory automation system for Touw in Deventer, started working with Tryllian, building Java-based virtual agent systems. Fun! I learned a lot at that job, it’s basically where I Was taught programming properly. I discovered Python, too, and loved it! Pity about that weird tkInter toolkit! And I started using KDE as soon as I had a computer with more than 4 megabytes of ram, and KDE used Qt.

Qt still made sense to me, but I still didn’t know C++, though it looked to me that Qt made C++ almost as easy as Java, maybe easier, because there were seriously dumb bits to Java.

Then PyQt arrived. I cannot figure out anymore when that was: Wikipedia doesn’t even tell me when it was first released! But I threw myself into it! Started writing my first tutorials in 1999 and followed up writing a whole book on PyQt. My main project back then was Kura, an alternative to SIL’s Shoebox, a linguistic database application that later got handed over to the Ludwig-Maximilians University in Munich.

I never could make sense out of Java’s GUI toolkit: Swing didn’t swing it for me. But that was work stuff, and from 2003, I worked on Krita. I started doing my own painting application in PyQt, about the time Martin Renold started MyPaint. I quickly decided that I couldn’t do a painting application on my own, just like I couldn’t do a word processor on my own. By that time I had taken a good hard look at GTK as well, and concluded that anyone who’d propose to a customer to base a project on GTK should be sued for malpractice. Qt just made so much more sense to me…

So I found Krita, learned bits of C++, and since then there haven’t been many days that I haven’t written Qt-based code. And without Qt, I probably would have started a second-hand bookshop or something. Qt not just lets me code with pleasure and profit, it is what keeps me sane, coding!

Krita 3.0

Krita 3.0 is going to be the first Qt 5.x-based Krita. April 13th, 2006, we  ported Krita to Qt 4. Seventeen days after porting started, I could publish Krita 2.0 runs!”:

Back then, I was young and naive and deluded enough to think that porting Krita to a new version of Qt would automatically make Krita better. Porting itself was an exciting adventure, and using new technology was fun all by itself.

But porting to Qt 4 was a complete and utter disaster and it took us many years to finally get to a Qt 4-based version of Krita that was ready for users: Krita 2.4, released April 11th, 2012. There were reasons for that beyond the mere porting, of course, including the fact that, like fools, we couldn’t resist doing a complete refactoring of the basic KOffice libraries.

This time, we’re not doing that. But that’s not to say that I’m at all confident that we’ll have a Krita 3.0 that is as good for end users as 2.9. We started porting March 6th. Right now, Krita starts, but you cannot load or save an image and you cannot use any tool. Our input handling is broken because of changes in the event filter handling in Qt. Also, ugly big fonts and no icons. Really simple things, like the list of brush engines, are broken.

I know that we have to port Krita to Qt 5, because Qt 4 is not going to be maintained for much longer, because Linux distributions want to move on, because Qt 5 is being actively developed (except for the parts, like QtWebkit that are being actively deprecated). But it’s taking a lot of effort away from what really counts: making Krita better for end users.

It’s like this…

One can develop software for any number of reasons: because it’s fun to write code, because you get paid for it or because your software makes users happy. I’m going for the last reason. I want to make users happy to use Krita. I want users to have fun using Krita, I want it to be an efficient tool, a pleasure to use.

In order to do that, code-wise, I have to do three things: implement cool new features (including workflow improvements), fix bugs and improve Krita’s performance.

Similarly, I expect people working on libraries or build tools to have as their goal making it possible for me to reach my goals: after all, I’d hope they are writing software for me to use, otherwise I’d better use something else that does help me reach my goals.

Updating our build system and struggling with that because the new way of specifying the list of directories where the compiler looks for header files isn’t compatible with third party software that uses cmake, well, that does not contribute to my goal. This problem has taken four or five people each over four hours of time looking into and it hasn’t been solved yet! Now realize that Calligra has 609 CMakeLists.txt files and about 300 plugins. There are over seventy libraries in Calligra.

Likewise, having to rewrite existing code because someone found a new, better way of handling boolean values in a C library for reading and writing a particular file format is not helping. Your library might be much better quality, now, code-wise, but I, as your user don’t give a damn. Just like my users don’t give a damn what my code looks like; it only needs to work. I only care about whether your software helps me to deliver features to my users. Don’t tell me the new way is cleaner — that’s an illusion anyway. Don’t insult me by telling me the new way will make my maintenance-burden smaller, because you’ve just added a load to it right away.

In general, any change in a library or in a build system that makes work for me without materially improving Krita for Krita’s users is a waste of my time. Doubly so if it’s badly documented. I resent that waste. I don’t have enough time already.

Let’s take this tiny example… QModelIndex::internalId() no longer returns a qint64, but a kind of a pointer abstraction, a signed integer. Well, we have some code that compared that internalId() to -1. This code was written in 2011 by someone who is no longer around. The Qt documentation used to say

“Returns a qint64 used by the model to associate the index with the internal data structure.”

Now it says

“Returns a quintptr used by the model to associate the index with the internal data structure.”

It might just be me… But this change isn’t mentioned in C++ API changes — so, what do we do now? And once we’ve done it, is the style selector for the text tool really better?

Likewise, what do I care that “QCoreApplication::setEventFilter() and QApplication::x11EventFilter/macEventFilter/qwsEventFilter/winEventFilter are replaced with QCoreApplication::installNativeEventFilter() and QCoreApplication::removeNativeEventFilter() for an API much closer to QEvent filtering.” Nothing. Nada. Zilch. I just don’t want to port our event filter to a new api; it worked fine, let it go on working!

I could enumerate examples until dawn, and we’re only a month into porting. We’ve disabled all deprecation warnings, even, because they were so numerous they obscured the real errors.

So, to conclude, I suspect that it’ll take at least six months before the Qt 5 port of Krita is usable by end users, and that we’ll be adding new features and fixing bugs in Krita 2.9 for at least a year to come. Because if there’s one thing that I desperately want to avoid, it’s losing our userbase just when it’s nicely growing because we spend months doing stuff none of our users gives a damn about.

The World is Wrong!!!

“No shit, the world is wrong! It ain’t got a clue! But here, in this one minute video, I will explain what’s wrong and how I have discovered the right way.”

As soon as you encounter something that can be reduced to the above, it’s a pretty fair sign that the author doesn’t know what he’s talking about. Anyone who thinks they’re so unique that they can come up with something nobody else has thought before is, with likelihood bordering on certainty deluded. The world is full of smart people who have encountered the problem before. Any problem.

That means that a video that’s been doing the rounds on how “Computer Color is Broken” is a case in point. The author brings us his amazing discovery that linear rgb is better than non-linear, except that everyone who’s been working on computer graphics has known all of that, for ages. It’s textbook stuff. It’s not amazing, it’s just the way maths work. The same with the guy who some years ago proved that all graphics applications scale the WRONG way! “And how much did you pay for your expensive graphics software?”, he asked. “Eh? You sucker, you got suckered”, he effectively said, “but fortunately, here’s me to put you right!” It’s even the same thing, actually.

Whether it’s about color, graphics, or finding the Final Synthesis between Aristotle and Plato, this is my rule of thumb: people who think everone else in the world is wrong, are certainly wrong. (Also, Basque really is not the mother of all languages.)

And when it comes to color blending or image scaling: with Krita you got the choice. Use 16 bit rgb with a linear color profile, and you won’t see the artefacts, or don’t use it, and get the artefacts you probably were already used to, and were counting on for the effect you’re trying to achieve. We’ve had support for that for a decade now.

Note: I won’t link to any of these kookisms. They get enough attention already.

The Gorilla and the Gibbon

As a Krita developer, I’m not too happy comparing Krita to Photoshop. In fact, I have been known to scream loudly, start rolling my eyes and in general gibber like a lunatic whenever someone reports a bug with as its sole rationale “Photoshop does it like this, Krita must, too!”.

But when we published the news that a group at University Paris 8 had replaced Photoshop with Krita, that comparison becomes inevitable.

Even though it is just one group that used Photoshop for a specific purpose, one that Krita can fill as well. The news got picked rather widely and even brought the krita.org webserver to its knees for a moment. The discussion on Hacker News got interesting when people started claiming that Krita, like Gimp was missing so much stuff, like 16 bit/channel, adjustment layers and so on — even things that we’ve had since 2004 or 2005.

So, where are we, what’s our position? In the first place, with Adobe having about a hundred developers on Photoshop, they can spend thousands of hours a week on developing Photoshop. We’re lucky to get a hundred hours a week on Krita. Of course, we’re awesome, but that’s a huge disparity. We simply cannot duplicate all the features in Photoshop: even if we’d want to, there are not enough developer hours available. And even if there were, there’s the pesky problem of incomplete file format specifications. That means choices have to be made.

We develop Krita for a specific purpose: for people to create artwork. Comics, illustrations, matte paintings, concept-art, textures. Anything that’s not relevant for those purposes isn’t relevant for Krita. No website design, no wedding album editing, no 3D printing, no embedded email client.

But anything that artists need for their work is relevant. And I think we’re doing pretty well in that regard. Krita is an efficient tool that people find fun to use.

If that’s the purpose you use Photoshop for, give Krita a try. If you use Photoshop for something else, don’t bother. Or, well, you can give Krita a try, but don’t be surprised if Krita is missing stuff.

Sometimes, we will do a direct clone of a Photoshop feature: the layer styles dialog Dmitry and I are working on is an example. People want that feature, even if we’ve got filter layers and transformation masks already. It’s going to take about two to three hundred hours just to do all the typing, let alone the actual thinking about how to fit the algorithms into Krita’s image composition code.

But mostly, cloning another application is a bad idea. You are always be running behind, because you cannot clone what hasn’t been released yet, and unless you have more hours per week available than the clonee, you won’t ever have time for introducing unique features that make your project more interesting than the clonee — like Krita’s wrap-around mode. Or the opengl canvas, which we had in 2005, and which Photoshop now also has. (This Nvidia page on how Opengl makes Photoshop more responsive could have been written for Krita. The only thing we miss is embedding a 3D model in your painting, and we’ve already made two Summer of Code students attempt just that.)

So, what’s our roadmap for 2015, if it isn’t “be Photoshop unto all people”?

These are the big issues that we need to spend serious time on:

  • Port to Qt and KDE Frameworks 5. In many respects, a waste of time, since it won’t bring us anything of actual use to our users, but it has to be done.
  • Implement a performance optimization called Levels of Detail. This will make Krita work much faster with bigger images, at the expense of actually doing the pixel mangling several times.
  • Animation. Somsubhra’s animation timeline is a great start, but it’s not ready for end users. We had hoped for a big donation kickstarting this development, but that doesn’t seem likely to materialize.
  • OSX. We’ve got an experimental OSX port, but it’s buggy and broken and missing features. (No HDR painting, OpenGL is broken, the popup palette doesn’t show in OpenGL mode, memory handling is broken — and a thousand smaller issues.
  • Python. Krita has been scriptable in the past. First through KJS, in the 1.x days, then through Kross (which meant javascript, ruby, python). Neither scripting interface exposed all of Krita, or even the right parts. You could create filters in Python, but automating workflow was much harder. There’s a new prototype Python scripting plugin, modelled after Kate’s Python plugin that would make a good start.

To make this possible, we simply have to add hours per week to Krita. Which means starting on the next fund raiser, and publising Krita in more app stores, sell more DVD’s, get more people to join the Development Fund!

Brilliant!

Emanuele Tamponi (Thrain) has managed to implement Kubelka-Monk color mixing for Krita’s mixer palette. This means, simply put, that mixing  works like in real life. It uses the reflectancy of pigments to reliably mix color on the computer and get the results you’ve been taught in primary school.

This is really unique. There is no other software available that does this. Corel Painter doesn’t do this. Photoshop doesn’t do this. There are some academic experiments that do this — but it’s impossible to get the code or the binaries for that.

Hip hip for Emanuele!

(Pics will follow when I’ve either compiled Krita (I broke my tree) or have finished moving house, whichever comes first.)

Krita on Windows

Sven posted his thoughs on the difficulties supporting Krita on Windows. I’d like to dig in a bit deeper here. The first question is, is it worth it?

Well, the highly experimental Calligra installer has been downloaded about 15.000 times. That’s a lot of downloads! Judging from the ratio of referrals from krita.org and calligra.org, Krita is probably the reason for two-thirds of those downloads.

So, apparently there is a demand for Krita on Windows that makes people download it even though we’re extremely explict about its extremely experimental status!

We also know that there are bugs specific to Windows, like the slow move tool, a bug with duplicated layers being broken and so on. We know that the top toolbar looks ugly and that there are other issues where being non-native really is apparent.

Another thing I know is that developing on Windows really is no fun. As the Tomahawk Windows dude says:

We do need developers on Windows too, but no-one in a sane state of mind wants to develop there.. 😉

So what Krita needs is to find a volunteer who completely and utterly disagrees with that quote… Someone who loves Windows, cares about the look and feel of an application on Windows, and who simply loves developing on Windows. If you’re that person, contact me!

The other option would, of course, be to find the money to pay someone to work on Windows.

But we’re not talking 1000 euro/month here — we’re out of the student summer job territory! At a conservative estimate, we’d need between five and seven times as much: 5000-7000 euros of regular income to hire a full-time developer. So we’re out of donations territory, out of Summer of Code territory. It’s a big job, which needs to be done well.

If there are 15.000 downloads of an experimental version of Krita in about three months — maybe there is a market. Maybe we could package Krita, put it in an app store like Intel’s app-up, on Amazon, sell it directly, go with a pay-what-you-like model like Ardour. Whatever — and use the proceeds pay a full-time developer to work on the Windows version of Krita. It’ll, indirectly and directly, also benefit Krita on its main platform.

I’m seriously investigating all the options here, including finding some kick-start funding.

(Note: Krita is GPL, but that doesn’t stop us from charging money for a binary, even though we can’t stop anyone from copying the binary around or reselling it — and I don’t even want to do that. Free software is Free software. Being GPL, proprietary extras and improvements are also out of the question. And that’s the way I like it.)

Oh, and as a side-node: next to the highly experimental Calligra 2.4 installer that includes Krita next to the other apps, there’s also an extremely experimental Krita-only installer that packages Krita fresh from git master. Just to help me figure out how to do it. Get yours while it’s fresh at KO GmbH’s download page.

Call for Photoshop documents

Call for Photoshop Documents

One of the new things in Krita 2.4 (soon to be released) is pretty decent Photoshop file import. Not everything is supported, but you should be able to load RGB, Lab and CMYK files with multiple layers, thanks to the work by Siddharth Sharma.

Next is filling in the missing parts, and export, of course!

But for that we need to have a good, representative corpus of PSD files, with known version numbers. A bit like the corpus we have for applications like Words, Stage and Sheets — http://websvn.kde.org/ trunk/tests/calligratests/.

These office documents are tested regularly using special tools Thorsten Zachmann built for Calligra. We need that for Krita as well.

So I want to call upon all readers here to contribute PSD files to our collection. I need to know the version of Photoshop they were created with and the files will be committed to this public subversion repository, so they must be appropriately licensed or be public domain.

Please help us out! You can send the files to boud@valdyas.org.