Hot tub rocks!

After the horrible accident this afternoon, the KDE hackers made up with the crocodile (after all, it's family of our beloved mascotte), and enjoyed some time in the hottub.



Yes, it was HOT in that thing...


It proved to be very difficult to get them out of hacker mode, though, as they continued to discuss improvements to plasma. It finally took the magic sentence "I've got beer in my fridge" to get them out of the hot tub and in to what could be described as closer to normal life. I must confess that the beer brought them back into discussing plasma... Probably not an incredible good thing considering the effects of alcohol on some people.



(they're back at work...)

KDE release event & google!

So we're in the US now. Never been here before, so I must say, kind' a weird. Interesting, too. Surprisingly good food, the beer imho isn't as bad as pictured before. The people are nice as well, but roads are worse than in Belgium. And Helio bought a PS3 (which I haven't seen in action, but I suppose it's cool too).

Yeah, the event. Really great, really enjoyed yesterday. Did you know google ppl are very cool? Unfortunately, the google marketing ppl don't want us to give them proper credit... They read through my daily article, to ensure I didn't mention them too much. Feels kind'a weird - after all, in a FOSS community like ours, giving proper credit is like a second nature, you know?

So that's why I wanted to thank them here - they can't forbid me to blog... They provided us with a hotel, food, they organized like almost everything - put some very dedicated ppl here, who are helping us any way they can, even helped us finish the beer last night (they were good at that). They're even gonna work saturday just to get us a cool third day... Oh, and they provided a bus to travel from the hotel and back (with a google-minded chauffeur). Did I mention the excellent food? T-shirts printed by them for every KDE hacker? Meanwhile, I'm not even allowed to take a picture of the great google people helping us - yeah, gotta be kiddin', right? Nope, I may not talk about them, use names, nor pictures. 'It should be about KDE'.



Here's a picture of three of the great google ppl helping us to get rid of some alcoholic beverages. Big thanks to them!

Well, this won't help create more cool conspiracy theories, you know... Tuesday I stumbled upon a great one, which was about KDE-on-Windows. Apparently, Google initiated the KDE-on-Windows thing. So I talked about this to the other 4 hackers around me, one of which was actually Holger, the KDE guy who started KDE-on-Windows 5 years ago. He still thought it was because his employer didn't allow him to put linux on his windows-laptop, but when we talked about it with the google people they acknowledged Google indeed was using some search-engine-mojo to brainwash people into doing stuff like that...

Now the talk by Aaron is coming up so I'm gonna go back to writing on the daily article - it gotta be a nice one for today ;-)

Sweet Follows Sour

I think it's really necessary to respond to some criticism seen on the reactions to the latest OSnews article.

I won't go into the article itself, imho it's rather negative, but hey. From an user's perspective, it makes sense to only review 3 or 4 parts of KDE 4 and complain about them, and ignore all the other brilliant pieces of work in there, right?

On to the responses, I found this reaction by dagw to be the most typical.

Let me quote the most important part:

The KDE team obviously shot themselves in the foot with calling it 4.0. I'm sure they had a reason for not calling it Beta or Developer release, but whatever the reason it was a bad one. Especially since every complaint is met with a response of "well what did you expect, it's a Beta software". No matter which way I look at it, the KDE team screwed up this release, and it would probably be in their best interest to admit it and just flat out say, we jumped the gun.

Well. That's painful. So, is he right? Did we make the wrong decision? Let's look at it from a broader perspective for a while. Let's see it in the Grand Scheme of Things to Come.

The big question that should come up is: couldn't we have released what will now be KDE 4.1 as KDE 4.0?

No. Seriously, no. If you think that, I see why you would agree with what dagw said. But it's wrong, for many reasons.

One of those reasons can be summarized as 'community dynamics'. You need to get people into release mode, and we wouldn't have been at KDE 4.0.0 stage right now if we wouldn't have committed to releasing it. Many users will start using KDE 4.0.0 and start reporting bugs, so many corner issues the developers themselves would've NEVER found will be fixed in 4.1 - those would have been there if 4.1 would be our first release. Sure, the current 4.0 won't be picked up by as many ppl as the '4.1-4.0' would have been - but by more than if we would have released another alpha or beta.

A second issue is packaging. KDE 4.0 is relatively hard to package, not due to it being that difficult - packaging it is far easier and faster than KDE 3.x. But it is new, and new always needs adjusting to. CMake, SVN, many new dependencies, many new architectural pieces, changes in the internal structure of the major KDE packages like KDElibs and KDEbase. It'll take a while to get used to those. We probably can't expect distro's to put out KDE 3.5.x quality packages for at least a few months. By the time 4.1 is released, though, they will have some experience, and get it done rather quickly. (if you don't believe me - just check out a few different KDE 4.0 distributions... They differ wildly in terms of stability, features, everything...)

Third, we didn't want to hurt KDE-edu, KDE-graphics, KDE-games and the other parts of KDE that were ready for a release up to a year ago - for an explanation, read my previous blog - Why KDE 4.0 now.

Fourth - underlying issues. Many of the problems in KDE 4.0 can and will be fixed by the KDE hackers (many of them hopefully in KDE 4.0.1 already). But many can't. By pushing the boundaries of technology, you'll be pushed back. We've exposed issues in drivers, architectural issues in X, windowmanagement, Qt, all over the place (if you want to read up about it, aaron seigo has some excellent blogs about it). These simply would've appeared in '4.1-4.0', and would've bit users just as hard as they're biting now.



What I'm trying to say is of course the typical stuff: it is easy to say a decision is wrong if you're standing on the sideline. But the issue is often much more complicated than you think - and indeed, it is. Please, take that into account when you criticize the decision we (as in the KDE community) made. My bold statement is: No good would have come of delaying the release any longer. We would just have delayed progression. Would you want that?

(have a nice day and see you on the other side of the ocean)

Viral marketing rocks!

Hi everyone!

Within a few days the release announcement of KDE 4.0 will be finished and put on the web. You can imagine we, the promo ppl, are working very hard to create a good, quality announcement to do justice to the enormous amount of hard work put into KDE in the last 2 years. After all, we envision KDE 4.0 will be the beginning of something great. I've just been trying KDE 4.0.0, and I can only say I feel proud to be part of this community. I know I haven't written a single line of code - as I simply can't. But I still feel KDE 4.0.0 is a little bit mine ;-)

To guide reviewers and new users through KDE 4.0.0, we have tried to write not just a press announcement and a what's new, like we did for the beta's and RC's, but a much more 'do this and that' like guide. We hope to receive quality reviews that way - after all, if you're thrown in the KDE 4.0.0 desktop, you might feel a bit alienated. It is very different from KDE 3.5.x, after all. I might add that, as far as I know, this was Sebas' brilliant vision - don't underestimate our Master Teddy Bear!

Now we, the promo team, have a request for you all. We need to get the word out. We want everybody to know about KDE 4.0 - and unfortunately, we only have commit rights to kde.org ;-)

So we would like to ask every single one of you to help out. Please submit the release announcement (and the visual guide) to all kinds of news sites. Digg it, Reddit, put it on del.icio.us, shoutwire, but also local media sites like menéame, Fuzz, yigg or Tweakers. Post it on formums, talk about it on IRC, blog about it... Spread the word!

Even though it is the first release in the KDE 4 series introducing many new technologies, I am confident KDE 4.0 presents a stable and usable desktop to its users. I hope this release will encourage people all over the world to explore the exciting possibilities brought by it. In time, we will build and improve on this framework, and bring over all the functionality users have come to expect from the KDE 3 series - and more, of course.

We need to get this message out there - and YOU are the ones who need to do that.

KDE 4.0 - why now?

After a few posts about why we release KDE 4.0 while it's in a not-yet-perfect state, Aaron wrote an excellent post about it. Even though is post was pretty much perfect, I want to add a little thing to it - maybe obvious to most, but probably insightful to others.

As many of you might know, KDE is a large project. Under the KDE umbrella are projects like Kalzium, KTouch and KStars from KDE-EDU (each saw a lot of work). Of course, there are the games, receiving new graphics and features. KDEGraphics, which contains Gwenview (video), Okular, and kolourpaint. And let's not forget the many KDE Base applications like Kate or Juk from the KDE multimedia area. And of course I'm skipping over many other useful KDE applications.

These applications are, for most part, ready for release. Actually, the games and educational applications could have had a release a year ago. The latest OpenSuse, 10.3, already ships some KDE games and KDE Edu applications from KDE 4!

We could let these applications wait for another 6 months - sure. But that would hurt them. They would lose users (no release = no new features & fixes so users will start to look somewhere else). They would not gain new developers (which are, after all, often attracted to cool projects - and you're not that cool if you don't release). They might even lose developers (developers often develop because they want their code to be used. Not sit bit-rotting in a big repository somewhere).

So, a choice had to be made. Wait more, let the parts of KDE 4.0 which aren't up to 3.5 standards mature, while other parts of KDE would slowly start to deteriorate? Or do a release in true FOSS spirit, bring it out while it's fresh, and hope that it will infuse a stream of new developers to help us make it more mature?

Well, you all know what happened. Yesterday tagging. Release in a little over a week. I think the right decision was made.

Last Performance Blog

According to a reply on my previous blog, I could just as well test if having my hamster run in his wheel would increase drawingspeed as long as I don't use valgrind and cachegrind.

Well, I've never used those tools before, but hey. Let's give it a try.

Valgrind and Dolphin (resizing all the time).
Well, valgrind talked about memleaks. I get that. The details of that, however, don't tell me much (of course, that's an understatement).

==21180== LEAK SUMMARY:
==21180== definitely lost: 8,383 bytes in 342 blocks.


The output for individual parts where something is lost is like this:
==21180== 216 bytes in 1 blocks are definitely lost in loss record 175 of 257
==21180== at 0x4021765: malloc (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
==21180== by 0x56BB885: _XimOpenIM (in /usr/lib/libX11.so.6.2.0)
==21180== by 0x56B88CF: _XimRegisterIMInstantiateCallback (in /usr/lib/libX11.so.6.2.0)
==21180== by 0x5699517: XRegisterIMInstantiateCallback (in /usr/lib/libX11.so.6.2.0)
==21180== by 0x503F73D: QXIMInputContext::QXIMInputContext() (qximinputcontext_x11.cpp:361)
==21180== by 0x503E4A5: QInputContextFactory::create(QString const&, QObject*) (qinputcontextfactory.cpp:120)
==21180== by 0x4B928F1: QApplication::inputContext() const (qapplication.cpp:4541)
==21180== by 0x4BD5046: QWidget::inputContext() (qwidget.cpp:245)
==21180== by 0x4C051CC: QWidget::destroy(bool, bool) (qwidget_x11.cpp:853)
==21180== by 0x4BD7D8F: QWidget::~QWidget() (qwidget.cpp:1264)
==21180== by 0x4EADB04: QLineEdit::~QLineEdit() (qlineedit.cpp:357)
==21180== by 0x4E76C8A: QComboBox::setLineEdit(QLineEdit*) (qcombobox.cpp:1599)


Above is with QtCurve as style, btw.

Well, I think that would explain it all if I just had more brains. Unfortunately, I don't, so let's go on to kcachegrind. Again with QtCurve, which, as I mentioned before, draws a bit faster than Oxygen.

KCachegrind
Well, what can I say. Lovely graphs, that's for sure. As far as I can tell, this tool shows where the app is spending it's time. I guess the 100% ld-2.7.so doesn't mean that's what the app does all the time.
The < cycle 13 > I see as LibQtCore.so.4.4.0 doesn't explain much either. Digging down (this kcallgrind tool is pretty cool, even though I don't really understand it) I seem to see a lot of calls to QWidgetBackingStore?!? And there are over 2.4 million calls to isPaintOrScrollDoneEvent, but the % spend there (if that's what the % mean) is just 1.5%... Hmmm, again - big ?????

Or is that second line, self, the one which shows how much a function is called? Sounds like it could be true. In that case, at least 12% of the time of Dolphin (when resizing all the time, of course) is spend in varous libc-2.7.so calls.
The funny thing is that the orders for the libc-2.7.so functions seem to come all from the same Qt function: QListView::paintEvent < cycle 13 >. And then some KFileitemDelegate::paint things.

This could mean QListView is the problem (did I say something about QScrollBar? Does that have something to do with QListView?)? Or does this mean my hamster simply didn't run fast enough?

I'm sorry. This does not make anything clear to me. As I said before, I'm going to wait for someone actually knowledgeable in this area to say something.

A commenter on my previous blog did use callgrind, btw, but his findings were not very clear (to him, I mean, they were obviously not clear to me anyway).

Luckily I received also a few private mails, ranging from "Xephyr does indeed weird stuff, Qt4 apps draw much faster in it than in plain X" to "go on blogging about this". Which I won't, btw, as I never intended to go this far. Sure, many ppl seem to find it interesting, but the complaints about "don't whine, do something", while not making me happy, have a point. I can't do anything, so someone who actually CAN should take over. Or not, in that case - well. Shit happens. Maybe just use GTK apps, then. I had to install a bunch of GTK libs for kcachegrind anyway, for some weird reason:
libgnomecanvas, libgnome, libbonoboui, gnome-keyring, libgnomeui, graphviz, gail, gconf, gnome-vfs and libbonobo (including a bunch of errors about missing gtk things and other stuff, even though kcachegrind works fine)
Why KCachegrind needed that stuf: please let me know. I'd love to get rid of it again.

Edit: aah, probably optional (but apparently compiled in) dependencies of Graphviz... Weird, the graphviz page doesn't mention them, so I still think it's faulty.... So far for Arch linux is mean & lean.

Oh, and Happy new year for those who already are in 2008 when they read this ;-)

Edit2: And the Gwenview author, Aurélien Gâteau, just emailed me to ask me to test some improvements to the panning in Gwenview 4 he made in response to my blogs. So I did test it, and I am happy to tell you all the slower panning in Gwenview in KDE 4 is totally gone - the KDE 4 version is now as fast as the KDE 3 version! Thank you, Aurélien!

Edit3: make the blog sound less like I'll be crying all night because someone said bad things - after all, that's not the point, and I don't want to sound like I won't blog anymore if ppl are mean...

performance and Qt 4.4 again

My previous blog resulted in an large amount of reactions, many of which warrant a reply from me.

I blogged about bad drawing performance, and the big question I had was: what was causing it.

Firstly, a few video's were posted which showed the issue:
http://bw.uwcs.co.uk/kde3_resize.ogg
http://bw.uwcs.co.uk/kde4_resize.ogg

I hope the person posting them won't mind the bandwidth sucking....

Now I ended my previous blog with mentioning this issue seemingly was a bug in Qt, and more precisely, QScrollBar, based on the profiling one reader did in the blog before that. But in the comments section another issue came up: first mentioned by Benji, who discovered to his surprise his 3 year old laptop with integrated (Intel) graphics performed better than his NVidia GeForce 8800GTS... Quite a clue as to where the issue might be, I'd say. An article talking about the upcoming (now out) NVidia driver mentioned quite a huge improvement in XRENDER performance, so that might make a difference. I've compiled and installed the driver, and unfortunately, no visible difference. I can show the graph, but just take my word: Konqi still takes between 10% and 20% CPU when scrolling (that's with dualcores, so it's actually between 20 and 40% of a 2.4 ghz AMD core...). Resizing still barely goes over 5 frames a second (mostly a lot less).

So it probably isn't the NVidia driver - somebody with an old ATI Radeon (pretty good FOSS drivers available) also has the issue. But I do wonder about the apparently good Intel performance. More convincing against the NVidia-does-it case is an comment from Rui, who mentions sub-standard Qt 4 performance on Mac OS X:
Hi, FWIW, on Mac OS X, the last.fm client, which bundles some version of Qt4, is also a lot slower redrawing itslef when resizing than, say, a Finder window, even if said Finder window is displaying the new coverflow view of files.

I'll try getting some numbers later. Maybe checking Qt4's performance on windows would be constructive too. By checking on other systems we are eliminating a whole bunch of probable places where the slowdown could be occurring. This kind of performance problem is quite difficult to measure especially because there are so many variables going under (system frameworks) and above (the app itself) Qt.

So, were are we at. A short overview:

- It is not the Oxygen style.
- It is not Qt doublebuffering.
- It is not 3D/compositing stuff.
- It is not the (lack of) alien widgets.
- It is most likely not the XRENDER performance of the NVidia drivers (but maybe Intel does something right?)

So, we're back at Qt again. Why does Qt 4 drawing seem to perform so much worse on some hardware? It could be that Qt 4 is much more dependent on hardware acceleration - and if that's not done properly by the drivers, drawing suffers. I wouldn't know. But if it IS drivers, the new NVidia driver should probably fix it - and it doesn't. I know there is more to drivers than just XRENDER, so I should probably let others talk about that. Actually, that would be a good thing - maybe a graphics guru (our own Graphics Ninja perhaps?) could chime in, say something?


----DISCLAIMER
Now I also have a disclaimer, as my previous blog also resulted in a comment blessing me by the name of 'whistleblower', and someone else wasn't to happy with the "this is Qt's fault".

I think I was clear enough on this, but I want to repeat it: I don't think this is a huge issue. Sure, it looks bad, but that's about it. The faster application startup we have in KDE 4 far outweighs the slower drawing. Performance is much more than drawing, and imho having to wait for an application to start is much more annoying.

Secondly, my measurements are of course entirely unscientific. After all, the difference on can see with the bare eye must be relatively large to really count. And it might be that there isn't one cause but there are several causes. Maybe it is a combination of the double-buffering, bad drivers, Oxygen style and lack of alien widgets. But I have tried resizing dolphin with the QtCurve style and Qt 4.4 (aka alien widgets) with the new NVidia driver - and I still clearly could see it draw. Oh, and turning the double-buffering of leads to 100% cpu usage when resizing even a slight bit so that won't help either ;-)

Third, maybe it isn't Qt's fault at all - it might be that the way Qt4 works (more hardware acceleration, more use of animations, transparency, etc) just exposes bad performance in X.org and/or graphics drivers. Seriously, I find that very likely. Seeing how many Qt developers seem to care about performance, it seems unlikely they would release something which is so much slower than Qt 3. So it is very well possible it works fine on THEIR hardware. Apparently, the FOSS Intel drivers fare pretty well, so maybe they all have these nice MacBooks and such ;-)

Last, I didn't file a bug. First because I'm still trying to figure out what exactly causes this, and secondly I'm simply not knowledgeable enough to start profiling apps. And also because this process seems to work rather well - with the help from the many readers of this blog, I feel we're getting closer to finding the reason of the issue.

So, while I look forward to a comment from someone who can tell me he/she found the issue, this is in no way a blocker for KDE 4.0 or a horrible thing for Qt 4.x. What I would do is recommend to the KWin developers to set KWin to NOT display content in resizing windows by default. That wouldn't look incredibly cool, but it would look far better than horribly slow resizing windows. Lubos? Rivo? Are you guys reading this?