Rant of the day: well, at least Microsoft is making loads of money...

Sadly, many if not most of our schools today are suddenly pumping lots of extra money into Microsoft, Zoom and other proprietary software companies, because they need online collaboration. We all know there are many alternatives to giving their students' data away to foreign companies but most don't bother. It is annoying, there is always budget for Microsoft, but not for proper, local, privacy-protecting open source solutions, even if those are better. Why is that?

Reputation, I'm convinced, is the main reason for that.

We teach them the wrong thing

Unfortunately, a lot of people try to convince schools, governments, charitable organizations and even companies to not pay anything at all. They are promoting open source solutions as an alternative that is cheaper or free, which just makes it look inferior to management. They are not telling organizations to pay local and open source product companies instead of Microsoft.

Open source/Free Software advocates hammer on "but it is free"! And when they do, THEY probably think of Freedom. But the person they talk to just thinks "cheap and bad", no matter how you try to explain freedom. Nobody gets that, really, even if they nod friendly while thinking what a silly, idealistic nerd you are. Been there, done that.

I love the enthusiasm, yes, but in the end it is not helpful: it presents open source as a crappy but cheaper alternative without any real support. Well, there are a few overloaded volunteer enthusiasts who might do a great job for a volunteer but can't compete with a bunch of full time paid people at Microsoft. So the schools and governments and companies will simply use those 'free' (as in cheap and crappy) services as a stop-gap and then beg their bosses for budget to be able to pay a "proper" Microsoft service. There goes more public money in NOT public code.

We need to stop teaching companies that open source is a crappy, cheaper alternative to proper, paid alternatives from big American companies and instead tell them that they can pay for an open source solution that has real good support, no vendor lock-in, doesn't leak your data, protects your privacy and is actually better in many other ways. That way open source companies can actually hire people to make products better instead of just doing consulting one customer at a time.

And yes, some companies and some business areas have figured this out - Red Hat and SUSE are obvious examples, and projects like OpenStack have lots of paid people involved. But lots of other companies, from Bareos (backup) to Kolab (groupware) have struggled for years if not decades to build a product, instead getting sucked into consulting.

It doesn't work that way

I have seen loads of open source product companies go bankrupt or just give up and become consulting firms because their customers simply expected everything for free and to only pay a bit for consulting. Lots of open source people work at or set up their own consulting firms, occasionally even contributing a patch to upstream - but not building a product. Not that they don't want to, but they quickly find out that working your ass off for a maybe decent hourly rate does not leave you time to actually work on the thing you wanted to improve in the first place.

Indeed, you can't build a good end user product that way. Frank and myself put together a talk about this recently:



I have also recently written an article about this entire thing, explaining why of all the business models around open source, only subscriptions can lead to a sustainable business that actually builds a great product. Will hopefully soon be on opensource.com.

Yeah but volunteers...

Are fundamental to open source, yes, no doubt. At Nextcloud we could not have build what we did without lots of volunteers, heck, nearly everybody at Nextcloud was a volunteer at some point. And yes, all code we write is AGPL, and that, too is important. I am NOT arguing against that, not in the least.

What I say is:

• You can't build a great product without paid developers*
• You can't build a great product on consulting and only getting paid for setting it up/hosting

But let me then also add:

• You can build a better product collaboratively
• And the (A)GPL are the best licenses to do that

I'm sure there are exceptions to those rules, yes. But compare a great product like Krita, see how its developers struggle every day to be able to pay the bills of just a few full-time volunteers. Do you know how they are currently paying most of them? Last time I spoke to Boudewijn, the reality was sad: the Microsoft App store. Yup. How many does Adobe manage to pay to work on its products? Why should our ambition not be to have as many people working on Krita? Of course it should be. And yes, keep it open source. Is that doable?

Of course it is. Well, maybe not Adobe levels, but we can absolutely do better.

Missed opportunities

I said this was a rant, so I do have to complain a bit. My biggest regret is that KDE failed to catch up during the netbook period (around 2005). I believe that it is in no small part because we failed to work with businesses. Idealism can be super helpful and can also totally keep you irrelevant.

KDE is, lately, working more with companies, trying to build up more business around its product. GNOME has been far better at that for a far longer time, by the way. It is hard, and companies like Kolab, struggling for the last ~20 years to make things work, have shown that. Just being a for-profit obviously doesn't solve all problems. Idealism and hard work are not enough to make a business work. But we can do better, and Nextcloud is an example that shows we can. Now not all things are freaking awesome at Nextcloud, really - we work our a**** off and it is hard. We put on our best face in public but sometimes I just want to bang my head on and in the wall...

Still, see the video, read the blog hopefully soon on opensource.com - there are ways.

Thoughts welcome.

Edit:
* let me qualify that statement. You can do it without paid developers in a small project, I dunno, grep or ls or the awesome simplescreenrecorder and tools like that. With those there is a risk of the apps going unmaintained and new ones popping up all the time - look at music players in the KDE community. I'd rather see one well maintained than new ones pop up with all their different flaws, but I totally get that for a volunteer it is often easier and more fun to start fresh. In either case, once you start building something huge, it gets pretty hard without long term dedicated resources. Note that it can be donations-run (like Krita and many others), with a charitable organization. I do think it is about more than 'just' the resources. If somebody 'just' sponsored 25 people to work full-time on Nextcloud, the end result would be different than the situation today. The need to deliver something that makes customers happy (which means focus on details, scalability etc!) and pressure to do things you wouldn't want to do in your free time (developer documentation...) make a big difference.

In any case, I really don't think projects like LibreOffice, Firefox, Nextcloud, KDE or GNOME and the Linux kernel itself would be where they are today without people paid to work on them.

Principles

We recently did a post about the Nextcloud Mission and Principles we discussed at the previous Contributor Week. I guess it is mostly the easy-to-agree on stuff, so let me ruin the conversation a bit with the harder stuff. Warning: black and white don't exist beyond this point.

Open Source

In an internal conversation about some community pushback on something we did, I linked to islinuxaboutchoice.com - people often think that 'just' because a product is open source, it can't advertise to them, it has to be chock full of options, it has to be made by volunteers, it can't cost money and so on...

But if you want to build a successful product and change the world, you have to be different. You have to keep an eye on usability. You have to promote what you do - nobody sees the great work that isn't talked about. You have to try and build a business so you can pay people for their work and speed up development. Or at least make sure that people can build businesses around your project to push it forward.

I personally think this is a major difference between KDE and GNOME, with the former being far less friendly to 'business' and thus most entrepreneurial folks and the resources they bring go into GNOME. And I've had beers with people discussing SUSE's business and its relationship with openSUSE - just like Fedora folks must think about how they work with Red Hat, all the time. I think the openSUSE foundation is a good idea (I've pushed for it when I was community manager), but going forward I think the board should have a keen eye on how they can enable and support commercial efforts around openSUSE. In my humble opinion the KDE board has been far to little focused on that (I've ran for the board on this platform) and you also see the LibreOffice's Document Foundation having trouble in this area. To help the projects be successful, the boards on these organizations need to have people on them who understand business and its needs, just like they need to have community members who understand the needs of open source contributors.

But companies bring lots of complications to open source. When they compete (as in the LibreOffice ecosystem), when they advertise, when they push for changes in release cycles... Remember Mark Shuttleworth arguing KDE should adopt a 6-month release cycle? In hindsight, I think we should have!

Principles

So, going back to the list of Nextcloud's Mission and Principles, I say they are the easy stuff, because they are. They show we want to do the right thing, they show what our core motivation was behind starting this company: building a project that helps people regain control over their privacy. But, in day to day, I see myself focus almost exclusively on the needs of business. And you know what, businesses don't need privacy... That isn't why we do this.

Oh, I'm very proud we put in significant effort in home users when we can - our Simple Signup program has cost us a lot of effort and won't ever make us a dime. The Nextcloud Box was, similarly, purely associated with our goals, not a commercial project. Though you can argue both had marketing benefits - in the end, a bigger Nextcloud ecosystem helps us find customers.

I guess that's what keeps me motivated - customers help us improve Nextcloud, more Nextcloud users help us find more customers and so both benefit.

Pragmatism and the real hard questions

Personally, I'd add an item about 'pragmatism' to the list, though you can say it is inferred from our rather large ambitions. We want to make a difference, a real difference. That means you have to keep focused on the goal, put in the work and be pragmatic.

An example is the conversation about github. Would we prefer a more decentralized solution? Absolutely. Are we going to compromise our goals by moving away from the largest open source collaboration network to a platform which will result in less contributions? No.... As long as github isn't making our work actively harder, does not act unethically and its network provides the biggest benefits to our community by helping us reach our goals, we will stay...

More questions and the rabbit hole

Would you buy a list of email addresses to send them information about Nextcloud? No, because it harms those users' privacy and probably isn't even really legal. Would you work with a large network to reach its members, even if you don't like that network and its practices? Yes - that is why we're on Facebook and Twitter, even though we're not fans of either.

Let's make it even harder. How about the choice of who you sell to. Should we not sell to Company X even if that deal would allow us to hire 10 great developers on making Nextcloud better for the whole world and further our goals? Would you work with a company that builds rockets and bombs to earn money for Nextcloud development? We've decided 'nope' a few times already, we don't want that money. But what about their suppliers? And suppliers of suppliers? A company that makes screws might occasionally sell to Boeing which also makes money from army fighters... Hard choices, right?

And do you work with countries that are less than entirely awesome? Some would argue that would include Russia and China, others would say the USA should be on a black list, too... What about Brazil under its current president? The UK? You can't stop anyone from using an open source product anyway, of course... It gets political quick, we've decided to stick to EU export regulations but it's a tough set of questions. Mother Teresa took money from dictators. Should she have? No?

It might seem easy to say, in a very principled way, no to all the above questions, but then your project won't be successful. And your project wants to make the world better, does it not?

Conclusion?

We discuss these things internally and try to be both principled and pragmatic. That is difficult and I would absolutely appreciate thoughts, feedback, maybe links to how other organizations make these choices. Please, post them here, or in the comments section of the original blog. I can totally imagine you'd rather not comment here as this blog is hosted by blogger.com - yes, a Google company. For pragmatic reasons... I haven't had time to set up something else!

There's lots of grey areas in this, it isn't always easy, and sometimes you do something that makes a few people upset. As the Dutch say - **Waar gehakt wordt vallen spaanders**.



PS and if you, despite all the hard questions, still would want to work at a company that tries to make the world better, we're hiring! Personally, I need somebody in marketing to help me organize events like the Nextcloud Conference, design flyers and slide decks for sales and so on... Want to work with me? Shoot me an email!

The Washington Post again demanded that tech companies create special 'golden keys' for authorities to keep and use for access to private communication. Protected by a warrant, of course. For the benefit of this discussion (which is really getting old), I just put together the reasons why it is a dumb idea.

First of all. It is a pure fantasy, an entirely unrealistic wish of the Pink Unicorn variety that it is possible to create a key which only the US goverment (and other sanctioned agencies) would have access to. It is technically not possible. Ever. I explained that before so let me now just quote Bruce Schneier:

"We have one infrastructure. We can't choose a world where the US gets to spy and the Chinese don't. We get to choose a world where everyone can spy, or a world where no one can spy. We can be secure from everyone, or vulnerable to anyone. And I'm tired of us choosing surveillance over security."

And let's be clear - we've been over this, the Clinton government wanted a similar thing with the Clipper chip and as security researcher Matthew Green pointed out:

Clipper is only one of several examples of 'government access' mechanisms that failed and blew back on us catastrophically.

A second issue with the proposal is that it doesn't do anything. Just like all the spying programs that came before in this and previous decade. Here's Bruce talking about that, here the Guardian, the Newyorker, Wired and Washingtonsblog. Whatever these spy programs do - from spying on German Chancellor Merkel to US congress (that's the Washington Post itself!) to the United Nations and Unicef - the government spying programs certainly don't target or are helpful against terrorism or pedophilia or any of the other stuff they are claimed to be for. And neither will these 'golden keys' be used to catch terrorists.

Last, and this should already be blindingly clear if you see the list above of some of the targets of surveillance, you should doubt if the government agencies will abide by the rules - they haven't in the past.

I also want to point out that the very reason we're having this conversation in the first place is because we're idiots.

Teacups and storms!

warning: wall-of-text blog. Don't read!

I see quite a bit of misconceptions and unhappy-ness around the new search infrastructure in KDE. While a series of small patches has probably fixed most of that by now, if you want to know what happened, read on. This is based on a mail I send to the openSUSE-KDE list a few days ago. And based on what I figured and understood from the developers - I'm not exactly a coder myself, so I'm sure some minor errors have seeped in. I'd be happy to fix them!

Why?

For starters, I would strongly suggest you read this dot article to get a bit more of the background behind it. Some people wondered if the new search got enough testing. We did quite a big testing push for this release, see this call to action and the liveCD I kept updated. There was social media stuff, too, of course.

If you've read the article on the dot, you will understand why the new search (baloo is the technical term, which I'll avoid using) came to be: it is orders of magnitude faster and more reliable and otherwise simply re-uses most of the previous infrastructure so it should not really introduce new issues. For almost all users, it should give a big improvement over the previous version.

New issues?

This turned out to be true and wrong at the same time. The new search was so much more efficient at indexing, it could totally clog up the disk on certain systems. Depending on the kernel version, settings and hardware, it was possible that reading and writing would overwhelm Linux' I/O system and slow down the entire computer to a crawl until indexing was done.

Usually, this would not take long, as the new search indexes so fast. A few minutes is enough for many gigabytes of data. But certain files can not be handled properly by the indexer - among them, text files over 20 megabyte. They take a long time. When it detects this, Search will put these files on a 'bad list' and not index them again.

Note that the old search had these same issues, but they were less noticeable among the general performance problem it had... It would eat lots of CPU frequently anyway, mostly because its database (virtuoso) was simply not very suitable for desktop usage!

As the new indexer indexes in batches of 40 (for performance reason), it needs some time to detect the problematic file. In 4.13.0, it has a time-out of 5 minutes on such a batch, then do the two halves again to see which half contains the bad file, then cut those remaining 10 in half again and so on until it knows what file exactly is faulty. You can imagine this can take a while, and the 5 minute timeout has now been shortened to about 2 so this should go faster. Also, the indexer has been improved to deal better with these files. Of course, very few people have such large text files lying around, but those who do did get bitten painfully, and are probably happy that this has been mostly (but not completely...) resolved.

Lastly, indexing will stop immediately on laptops when the power plug is taken out, to not shorten your battery life.

I hope the users who did get in trouble (often relatively early adopters) understand why we released Search: for all testers and developers, it worked and provided huge benefits. Why make users suffer by not releasing if you have something so much better?

About testing

For users of repositories like openSUSE's KDE:Current or distributions which were quick to ship this release, it can happen you bumped into these and other issues. Certainly the developer, Vishesh, and everybody who helped testing (including myself) wanted to try out everything to make sure it was stable. I installed this on my work laptop, despite the risk, so I could test in a realistic scenario. And I've send various screenshots of issues, including performance problems to Vishesh, who promptly fixed them. Few people are as responsive, responsible and hard working as he is.

Which makes it all the more frustrating that a lot of people are loudly yelling at him and others for their work. I can get that some people have no time to test. I accept that. But then THEY should accept that that might mean that their scenario, be it hardware, usage, or configuration, is NOT TESTED. Unfortunately, we can't delay releasing forever until every user has tested all our software - that would never happen. If it works for all testers, we release it. There is no sane alternative.

If you have problem with that - it is the Universe you have a beef with, not us. This is the reality with volunteer work you're not paying for.

Configuration

Some users were unhappy that most of the configuration of Search had disappeared. As most efforts went into making the new search as fast and unobtrusive as possible, the UI received little love. The most important thing, the ability to exclude some or all user data from indexing, is there. Indexing can be disabled by simply adding your home folder to the 'exclude' list - there is nothing to be indexed at that point, so Search won't index anything.

Few users needed more: the ability to configure specifically what files are indexed or not, for example. This is available in the configuration files, and a GUI has been made available by Lindsay, one of the people unhappy about the lack of configurability: here. She had something nice to say about the code ;-).

You can see the GUI in the gif on the right. I expect that the group of users in need of these advanced features but not capable of installing this KCM by hand is extremely small, and in a newer release perhaps a 'advanced' button or something similar can be added. If you want to read more about this, perhaps the discussion in this google Plus thread is interesting, especially the one from Thomas Pfeiffer at the end.

On or off by default?

Search has been enabled in Plasma Desktop for quite a while now, and the new Search had proven itself for all testers to be far superior to the previous release. There seemed to be no reason to not enable it. Moreover, Search is something you'd expect in 2014. Every competitor ships it by default, and most users, not even aware it exists, use it. Those more technical users who are aware of the existence of the various search technologies are often smart enough to be able to turn it off if they want to.

Seriously, Vishesh is convinced a Raspberry Pi running Search would be able to index an terrabyte hard drive over USB in less than 15 minutes. Honestly, he said 5 minutes to me, but I just can't believe that so I tell people 15.... It'd be fun if somebody could try this out and tell us how long it really takes.

Why can't I choose?

Some users on the openSUSE KDE list complained in various forms and shapes about a 'lack of choice'. Now there is no 2nd search infrastructure for Qt/KDE applications ATM, so until somebody writes one, it seems rather pointless to ask developers to put in extra work and create extra complexity for the ability to swap out Search with something else.

What about KNotes

Quite a few users had trouble with KNotes: it didn't properly import their notes and misses some functionality compared to the old version. Why did this get shipped?

This is, like many things, a matter of resources. The old version could no longer be maintained. At that point, we can drop it. Or rewrite it such that it can be easily maintained. A developer was willing to do the latter, so that is what happened. The rewrite looses some functionality, and it clearly has a bug: not always importing all notes properly. This WAS tested and it worked for the developers and at least the people who tested, so I guess some users were just unlucky. Or did not help test...

Again, the same: if you would have helped test, the importing could have been more robust. In this case, too, I know the developer who did this work personally, and he is extremely responsive to bug reports. If you had no time, that is fair, but then accept that we don't get paid to test or can't pay people to test, so this is all we could do. We would love to get 100 extra, paid developers and another 100 full-time testers. And pink unicorns.

Of course, if you think we made the wrong choice and should simply have dropped KNotes, you can simply remove it yourself and tell yourself that KDE did not have enough volunteers to maintain the old version and KNotes is no more. That, too, sucks and you can blame any random person (just pick a colleague next to you in the office for example) for not stepping up and maintaining KNotes. An alternative is to be happy that somebody at least did the work they did to keep it around and hope that they will find time to add the functionality missing now. And not be discouraged by all the negativity and drop the app completely...

Again, I'm sorry for the issues. So are many KDE developers. But we can't change the world as it is and yelling at us doesn't help. On the contrary, it does the opposite: there is little reason to put in your free time when people just yell at you for doing something for them for free.

Documentation

With the new Search being, well, new, not much documentation has been updated yet. I've put in a few hours to update the online documentation here. If you are missing anything, please add it!


If you have questions, ask in the comment section. But if you comment, at least be reasonable and realistic. We can't bend reality so don't expect us to. And be fair and put in some effort to understand what we do and why before you complain. I think that that is not too much to ask.

As Bruce Lee said: A wise man can learn more from a foolish question than a fool can learn from a wise answer.

I hope the above helps answer some questions!