On Open Source, forking and collaboration: Nextcloud 9 is here!

The nature of Open Source is, in a sense, dualistic. It encourages collaboration through the threat of not collaborating--a fork. When I was approached by Struktur AG to join them to work on ownCloud and Spreed, I loved the idea. I always wanted an ecosystem around ownCloud, which is why I pushed things forward like our collaboration with Western Digital Labs and Collabora, matters of no business interest to the company I worked for. I believe a stronger ecosystem benefits everybody.

Ecosystems and confidence

A major point which makes open source so beneficial for businesses is that it puts pressure on suppliers to offer great service and support. If they don't, another can enter the market and out-service them. Tight control over the community tough things like CLA and trademark makes it hard to grow such an ecosystem and negates some of the benefits of open source for customers.

Luckily, in the end, the AGPL license protects the future of a project, even if its steward clings to power. From conversations with Niels early on, it was clear to me that he has a very different and very confident view on his ability to run a real open source company. His history at Red Hat results in frequent comparisons. And indeed, Red Hat runs things the right way, even supporting a project like CentOS which many other companies would consider an existential threat to their business model. Just as their investment in opensource.com shows: they aim to grow the pie, not grab a bigger slice.

former 'enterprise feature' done right (and open)

I'm super proud and happy that we could announce today, with our first release, that Nextcloud will not be doing proprietary code. No closed apps means no inherent conflict between sales and community management/developers within the company, but a full alignment in one simple direction: servicing the customer.

And if you wonder about the collaboration with Collabora/LibreOffice Online and with Western Digital: yes, of course, we'll go full steam ahead and will facilitate where we can! No, we're not afraid that either would 'compete' with us: both will complement and strengthen the ecosystem. So we will work together.

Why? Because the core contributors and founder shared an ambitious goal for Nextcloud: be THE solution for privacy and security.

You are Nextcloud, too - what we will do for contributors

Cool stuff we want to do more with!

Based on feedback collected from many contributor members we've defined some plans and already made changes to how Nextcloud will be developed. Improved transparency and governance, focus on stability and architectural improvements and other improvements are covered in this blog. Much more is coming, you can join the conversation right now on our forums!

Community Input

January 2015, I ran a contributor survey to see what the ownCloud community thought about the processes, development focus and our work at the company. I shared the results by the end of April and pushed internally for the feedback to be taken serious. Some of the changes were implemented but many others were left for a future project to push forward. And Nextcloud will.

feedback and changes

Nextcloud aims to build a sustainable business, not limited by short-term, next-quarter thinking. The relationship with our community of contributors and users is central to our plans.

To quote Frank on this:

The company shouldn't be involving the community more in decision making; that's the wrong way of looking at it. There shouldn't be a fundamental separation to begin with!

And that's what we want. Saying "we're more open" just means being a more friendly ruler - Nextcloud aims to be a participant, not a king, benevolent or not! That is not to say that there should not be any direction but it shouldn't be dictated by a company anymore. Of course, people can decide what they work on, and the company gets to decide what it pays its employees to do. Now there are changes in how we manage our employees too, with far less micromanagement and more freedom. But that's for another blog.

Let's go over the specific pieces of feedback mentioned in the email and received from contributors in other ways and note how Nextcloud intends to address them.

Development

ownCloud is fun and relatively easy to contribute to, with a mostly well running review process and release cycle. There were some practical requests and suggestions as well as concerns about the strain the growth of our project has put on the core developers.

Dealing with Pull Requests

A major issue as detailed in many comments was that it often takes too long for pull requests to be merged. That is, contributions are not handled fast or at all. The result is that, with Core moving fast, contributions get out of sync, no longer apply and are effectively lost. As the graphs below show, the number of pull requests taking longer than 6 months to be merged is rising rapidly while the company is contributing less to development relative to volunteers. Don't get me wrong, it's great to have a growing community! But the support for development from the company needs to keep up with the pace.

Respecting contributions by being responsive and getting them merged will be our number one development priority at Nextcloud. As research by Mozilla has shown, reacting swiftly to contributions is crucial for growing community and we intend to grow and nurture our contributor base, recognizing outside input as a key driver of growth and success.

More stability

A general point made was that it'd be good to focus more on stability and performance. Some of that has been implemented with the 8.x series and automated testing improvements done over the last year. An especially sore point in terms of stability is the upgrade process, as was very visible with the 9.0 release that is still not available for users of the built in updater app. We will soon blog about the Nextcloud plans with regard to the updater.

Architectural improvements

It was mentioned that some parts of ownCloud are in need of serious architectural love and refactoring. ownCloud has been traditionally rather restrained in this regard and people worried that this "impairs competent developers". While being conservative is important with regards to building a platform (stability and compatibility!) many improvements made their way into the 8 and 9 releases. To preserve a healthy balance, we want to introduce an Architecture team to make decisions that have a big impact on the code base. More details will follow.

Another area of improvement would be to communicate more about architectural changes. Frank has already done a series of blogs about Federation in the past and more will follow.

Apps: support for Calendar, Contacts and Spreed

Many pointed out that apps are extremely important for ownCloud and we should work more on that. Frank has always emphasized the importance of building a platform and for Nextcloud this will be a central goal.

Nextcloud will officially support the Calendar and Contacts apps and supercharge their development. The Spreed.ME app will bring fully supported audio and video chat to ownCloud. We'll also invest in growing and improving our API for these and other applications.

Process

Some smallish process improvements were requested. More logical labels and tags, for example, which have been pretty much cleaned up since then. Another thing was that big pull requests are often a pain in the ass to get merged and we should tell contributors to cut their work in smaller pieces. This was added to the documentation.

Decision making

Most people were positive about the technical direction of ownCloud - test-driven, stability, architectural work, those were great improvements. Decision making processes in the technical area were not considered very transparent. Comments were even more harsh about the project-wide decision making process.

People feel decisions are often done behind closeddoors. Nextcloud will address this, in part by a new architecture team and in another part by getting rid of most 'hidden' communication channels like internal IRC and mailing lists. We also plan on talking more about our goals and plans in blogs and such.

Longer term planning is a major sticking point: there is little of it public. We need to discuss, together, how to do longer term planning. This doesn't fit too well with github. Thoughts welcome!

Communication channels

Several people have noted that we've got too many, confusing and overlapping communication channels. We've already eliminated one: mailing lists. We still have a newsletter for those who want to follow us and the blog roll on nextcloud.com/news. For technical discussions we keep using github (which now links commits to pull requests so you can find the discussion behind code) and we'll discuss more general subjects on the forum. Speaking of which, it's now on discourse - a massive improvement I'd say. And email fans can use email to communicate with people on the forum!

Governance

It was already mentioned here and there but there are two other big changes. First, we want the Nextcloud trademarks to be owned by the community, like the ownCloud one should have been. So we will set up a foundation soon which will control the trademark (not have it sub-licensed!) and more in the future.

Second, we will get rid of the Contributor License Agreement. You don't need to sign anything to contribute to Nextcloud.

Third, without CLA there are no proprietary apps part of Nextcloud. We won't be artificially crippling Nextcloud just to get some checkmark on a feature list on the enterprise side. At the same time, of course much functionality is needed for companies, stuff that they need (and home users don't). We will provide that for sure, including migration path, but this time as stand-alone tools. No more exclusivity for a single company, allowing it to do things others can't for legal reason. Our power is in employing the people who write the code, so we can give the best support and develop the best features. If another comes and tops us, well, we should've done better.

Users

There will be improvements for users, too. Already mentioned were our plans to support the Calendar and Contacts apps, Mail too, perhaps more. And of course with Spreed.ME we will integrate open source, WebRTC based video conferencing. There is more coming - for a future blog!

That's all? Nope.

Now I know this is a long blog with lots of details. No surprise, it is based on things we've wanted to improve for many years but could not. Now we can and we will. This is not the end of it, other suggestions and thoughts are more than welcome. Get involved!

Use ownCloud provided Packages, then VM, then Zip, no distro packages.

Last week I wrote a blog about why running an old ownCloud release isn't more stable and you should upgrade. There are many ways to install, run and upgrade ownCloud. What is best depends on your situation but some general rules of thumb can be given.

Use ownCloud Provided Packages If You Can

The best solution from a security and stability point of view are the official ownCloud packages, provided you have the basic know-how needed to run your own Linux server.

Packages give you the advantage of a relatively clean and easy upgrade process, with the ownCloud team taking care of any special steps which have to be taken. The upgrade itself will still have to be kicked off by the system administrator (see our latest update) but you won't risk forgetting to remove old files, correcting file permissions and so on.

We do not recommend using distribution packages, for several reasons:

•  First, there will inevitably be a lag between what distributions ship and what we make available.
•  Second, it happens that distributions don't grab the right code (like relying on a git tag rather than final zip files) or miss changes in dependencies which can break installations.
•  Third, many distributions are reluctant to upgrade to newer ownCloud releases, which means you can get stuck on older versions. While security fixes are often still back ported, being on, say, ownCloud 7.0.4+dfsg-4~deb8u3 means you have missed out on many ownCloud fixes and improvements. And of course, they continue to ship major server releases and ownCloud client versions until long after we've abandoned them.
•  Four. Not offering the latest ownCloud bugfix version also means upgrading can become a tricky problem: we recommend to upgrade to the latest bugfix version in a series before upgrading to the next for a reason. ownCloud 7.0.12 will contain bugfixes related to the upgrade to 8.0.x, fixes for problems you might run in to when you try to upgrade from your distribution's older bugfix release to a new major release. On top of that, our upgrade code looks at the ownCloud version to know how to upgrade - good luck if you run a Frankenstein version. Also, we don't support skipping ownCloud releases on upgrade. So once you're ready to upgrade your distribution with ownCloud 7.0.ancient to a new distro with ownCloud 8.2.shiny, well, it won't work.
• Last but not least. The distribution packagers try to do weird shit, shoehorning ownCloud (and other web apps) into their rules made for C/C++ apps. You'll find packagers trying to move the ownCloud configuration php file out into the /etc folder or split up ownCloud core in separate packages because we maintain some external components as part of our setup. Of course, this breaks in beautifully surprising ways and provides few if any of the benefits they are hunting for. It is also a performance issue. And problems will only get bigger with the upcoming code integrity checker.

I blogged on owncloud.org about the issues with distribution packages (and the distribution model in general) earlier.

"the misguided policy of not updating to even bug fix releases which some distributions have is nothing but harmful for users"

The last of the five points, distributions struggling with projects which don't fit their rules, leads to things like Iceweasel. If you're new to the world of Linux distributions, this is a tale you'll love: on Debian and perhaps soon, Fedora, Firefox is named Iceweasel. Debian does not want to update software because "that would break stuff", instead, they forked Firefox to backport security issues. I'll leave it up to you to decide how secure it is to have packagers 'maintain' software long after the project itself has moved on...

Of course, it also leads to changes coming in big chunks - we've learned that users adapt to changes far better and easier if they come small and regular. That is why mobile apps and many websites introduce changes in an Agile way. ownCloud and linux desktop KDE also increased their release speed for this reason.

I understand the reasons behind the rules and sometimes it has a pay-off which is worth the extra effort. But, as always when things are taken to an extreme, it is now blocking distributions from adapting to 2016 and onwards. The result is that technologies like Docker and the sandboxed apps the GNOME and systemd teams are working on will make much of the distributions irrelevant. A future distro will be a small base on which you run some kind of application image which comes with its dependencies built in. Is that the most secure solution, perfectly disk-space or memory efficient? Perhaps not. But by chasing perfection, distributions failed to attain good-enough.

Anyhow, enough rambling, let's talk about what to do if packages don't work for you.

Use a VM if a Server is Hard

Not everybody is comfortable on the Linux command line and while there is excellent documentation on running and securing a Linux server, it might make perfect sense to go for the Virtual Machine option. Since ownCloud 8.1 we've provided official ownCloud VM images and there are also several third party images available. This way, you get a server installation which has been set up by our experts to be secure out of the box. Upgrading and responding to immediate security threats is still in your hands but we've done what we can to make it easy.

Use Zip Files or Installer if all else fails

On some systems, like cheap hosting or a NAS, your control over the system simply does not extend to the installation of packages or running a Virtual Machine. Then, the zip files (or, for convenience, the installer) are your last resort. While the zip file is easy to use ("just extract and go"), it requires special upgrade attention which might discourage you from staying up to date. And that introduces a security risk! Also, doing the upgrade process from the web UI has limitations and risks on large installations.

Now you've learned why you should upgrade your ownCloud and what installation method to use. Time to get started!

Western Digital Labs and ownCloud

We published a blog post about a collaboration between WDLabs and ownCloud. WDLabs is, in their words, "a division of Western Digital focused on accelerating new solutions". I'd take that as "they look for more ways to sell hard drives" and they have been playing with some interesting stuff. One such thing is their PiDrive Kit. This kit includes a connector cable which powers both the harddrive and the Pi from one power supply, simplifying the setup. We're now working with WD Labs to build something from this starting point.

WD Labs, meet ownCloud

When they approached us a few weeks ago, they told us they had been toying with a more complete kit with bundled operating system based on berry boot. This lets you choose an OS on first boot and they had put together an image with ownCloud as one of the options. They thought it was really cool and asked if we felt the same and had any ideas of what we could do with it. Well, sure, I wrote not too long ago about What's holding ownCloud back and being able to offer potential users a plug-and-play device with ownCloud on it is something we'd love.

So did they. We quickly received a prototype of a kit with Berryboot and ownCloud on it as well as a case and a Pi included. After some more discussions, we developed a plan with which we hope to get some community help to make this happen.

We need your help

You have to know - this is not something ownCloud, Inc. is very much involved with. It simply is not a commercial endeavor - Inc. works for enterprise customers and we haven't found any of them yet running ownCloud on a Pi. So, while WD Labs will take care of the hardware and has committed themselves to doing a serious number of ownCloud branded devices complete with pre installed OS in the first quarter of next year, it will depend on our free time and community help to develop the OS.

And there's serious work to be done. Berryboot can be configured to boot a headless ownCloud but right now, to even boot, you have to connect a keyboard and monitor. And booting isn't even the issue - finding and configuring an IP and then connecting to the Pi is. And once you've done that, the system has to be configured to be accessible from outside so you can use the ownCloud clients on your mobile phone and laptop to always find and connect to your ownCloud and give you the ability to share with friends and family.

There are folks in our community who have experience with this, building our community VM for example (hi Daniel!) and we'd like to solicit their help. We have 10 prototypes to send out to people who want to help build this ready-to-go ownCloud, the blog has more details on how to get one. And if there is a lot of interest, we'll get more prototypes put together, though I think it'll be tight to get them out before Christmas.

Note that it is totally cool, even advantageous, to send in a proposal ad a team. Together you can get more done!!!

Of course - without this kit there is still a LOT you can do to help out and there is no reason to wait for us sending out anything! Help get a discussion started about what this should look like, how to get there and who does what. Subscribe to the developer mailing list and get started!

Future

Now we know the Raspberry Pi 2 isn't the very best device to run ownCloud on, in terms of performance. It won't run with 100 users, indeed, though it isn't too bad for a few users at home. But, aside from the fact that there is a lot of room for tuning (PHP 7, for one, should help a lot, see some stats here), we're not married to the Raspberry with a single hard drive. It is a popular device and thus a great place to start but WD Labs has already shown us prototypes using a Rasberry Pi Compute device and 2 hard drives and we could work with devices like the Banana Pi or more powerful boards, too. That won't happen immediately, it simply depends on how well the first version turns out and the interest from users.

That means it matters a lot now: can we make this work? I'll sure put a portion of my free time in this as I'm quite excited about it. You too?

ownCloud workshops - two down, two to go.

some had already given up at this time...

The success rate is going up - where, at the first ownCloud workshop at the openSUSE conference, we had no successful installation, yesterday in Helsinki we reached a two-out-of-five. Both workshops had around 20 participants but usually people collaborated in groups of 3-5, following my guidelines on how to get ownCloud up and running on a Banana Pi development board.

Whoah, two out of five?

I admit it isn't easy and partially, that is intentional. The instructions in the document are sparse, especially for newbies - but even an experienced Arch'er threw his towel in the ring after almost three hours. Then again, what is a workshop for if not for enjoying the struggle of learning something new? And certainly everybody did that - struggle and learn new things.

The winning team still hard at work

The second team to get ownCloud running

Those who have any experience installing ownCloud know the difficulty can not be in ownCloud - and indeed, once you are set up with a running Banana Pi with SSH access, installing ownCloud is a matter of minutes. But getting there is no picnic! Why?

The hardest part by far is with the networking part of the workshop - the moment you've SSH'ed into the Banana Pi, 85% of the work is done. The challenge is significant - requiring not just Linux on the host laptop (yesterday, Ubuntu got 4 new users as neither Mac nor Windows were up to the task) but also handling stuff like tcpdump, Wireshark and a bunch of low-level command line tools like dd, mount, dpkg, ssh and so on. For most participants, the hardest challenges were:

• Windows and Mac. I'm sure they are awesome operating systems, but something which is a few mouse clicks on a Linux system (sharing the wifi internet connection over a Ethernet cable with the Banana Pi) seems virtually impossible. On Linux, NetworkManager makes it as easy as creating a new wired connection, choosing "shared network" under the IPv4 tab and ticking the "this connection requires IPv4". Now, just enable this network after connecting the Pi and done. I have no experience with Windows or Mac whatsoever, but if nearly 15 IT students with internet access can't figure out how to make these operating systems do the same thing - I can only assume it is hard.
• Command line familiarity. If you're new to Linux, an instruction like "mount the USB stick and copy over the data to the Pi" takes more than a few minutes and requires you to learn at least two new tools and looking through system logs. 
• New tools. You'll be looking for alternatives to Linux commands like dd on Windows and Mac first, and once you've given up on your familiar platform you get to learn tools even most Linux users rarely need. 
• Geeks. "Can't you do this easier with TCPDump?" "You can do this with the ip command too, you know" - I could only reply "Try, and if it works, show me how." Rest assured, I learned a few things - but haven't changed my instructions. Yet.

Seriously, I love it. Three hours of a workshop full of people trying various ways of skinning the cat. To me, the fact that nobody got completely through my instructions merely means they had a good time on the way. At least, that's what they said - "the best workshop I've been at" is just awesome to hear.

And... working! A Banana Pi was the reward.

For the next workshop (coming Thursday at the Open Tech Summit in Berlin), I'll further streamline the instructions (but not too much!) and I mean to put in some 'advanced' challenges for those who simply know too much about Linux to stay busy for 3 hours. I want to know how to do this without Wireshark and NetworkManager, to name two things...

of course - food and beer as reward for hard work

I'll also have to make Linux mandatory. The USB sticks with instructions and software I provide will be openSUSE live sticks for the next workshop, as I can't expect everybody to have Linux by default on their laptops. And of course you can use Windows or Mac if you really want and are up for the challenge, I don't mind including instructions for these platforms. But nobody got them working yet so expect some struggle.

You will also need a laptop with a working ethernet port, no amount of creativity has been enough to work around the physical limitation of not being able to plug in the other end of the networking cable...

Besides the satisfaction of getting ownCloud running, the first to succeed earns the Banana Pi they worked with!

If you can't make it to Berlin, I will give workshop later this month also in Dubrovnik (see my earlier blog) and if you'd like to have this workshop close by - let me know, perhaps we can arrange something.

Why open source works

Trying to explain why open source works, you can of course point to the Cathedral and the Bazaar by Erik. But the kernel development process shows it happening 'in real time', every day, and that's a major reason why I so enjoy reading the weekly LWN.

Competition FTW

A recent article explained how two patch sets are developed, impacting the Huge TLB feature. One is about improving reference counting in the handling of huge pages to allow a huge page to be mapped in both 'huge page mode' and in 'normal page mode' by different processes. This is a first step towards being able to use transparent huge pages with the page cache, giving it access to the performance benefits of huge pages. This has been in the works for a while as it is a very complex change. But an alternative has recently surfaced, adding transparent huge page support to the tmpfs filesystem. This also has pieces needed to support huge pages in the page cache, but in a very different way.

Why on earth would you want developers to spend so much time on two competing solutions for a problem? Isn't there One to Rule Them All?

Finding that perfect solution

I am sure that there is a Perfect Solution. I would simply suggest that nobody knows it! Like in economics, where an equilibrium exists but no single person can define it, software development has become complicated enough that competition of ideas often leads to the best (or at least better) solutions.

Companies, building their own cathedral in-house, would put product managers and developers in a room. Let them come up with the Perfect Solution. But they won't, because of Joy's law:

"No matter who you are, most of the smartest people work for someone else"

Impressive results

This is the key to why open source works and works so well. No single development project in the world even approaches the size of the Linux kernel project - no less than 1400 people contributed in under six weeks. With millions of lines of code developed by tens of thousands of people over 20 years time and with an extremist attitude towards backwards compatibility, you wouldn't expect the trend to be "go faster". It is.

Bringing over almost 12,000 different people from 1,200 different companies together in the last decade, the kernel competes with Wikipedia for the title of humanity's largest collaborative effort. Oh, and I'm sure the masses building the pyramids were big, too, they just didn't integrate the experience of all their collaborators into its architecture as well as the kernel and Wikipedia do ;-)

Cookie licking

Many other projects struggle to learn from the kernel's success. Success which has many aspects, one of which exemplified here, a rule Frank sometimes talks about: no cookie licking!

In short, cookie liking is preventing others from working on something by claiming you are (but not really making progress). Extending it a bit, I would say that, while collaboration is good, if you feel you can do better -- or just want to sample a different approach:

"never be deterred to build an alternative solution to a problem"

Because it's space for competition within projects that makes open source work better.

First Berlin meetup of the year!

Next week will be the first Berlin ownCloud meetup of 2015. As the ownCloud Desktop client team is having a code sprint hackathon meeting get together at the office, they'll be at the meetup. So if you wanted to hack on the client, fixing or adding something and live in Berlin - join at the meetup!

The meetup in February I unfortunately can't join but we've already decided that the meetup in March will be All About Devices™ - we will have some Banana Pi, Raspberry Pi etc devices around to play with, do some performance testing and optimization and see where we can take 'em! If you're interested, you can RSVP already.

See you in Berlin?!?

If you're not in Berlin and are sad that you don't meet ownCloud people - check out our events pages to see what can be done about that!

How else to help out

Yesterday I blogged about how to help testing. Today, let me share how you can facilitate development in other ways. First of all - you can enable testers!

Help testers

As I mentioned, openSUSE moved to a rolling release of Factory to facilitate testing. KDE software has development snapshots for a few distributions. ownCloud is actually looking for some help with packaging - if you're interested, ping dragotin or danimo on the owncloud-client-dev IRC channel on freenode (web interface for IRC here). Thanks to everybody helping developers with this!

KDE developers hacking in the mountains of Switzerland

Coding

Of course, there is code. Almost all projects I know have developer documentation. ownCloud has the developer manual and the KDE community is writing nothing less than a book about writing software for KDE!

Of course - if you want to get into coding ownCloud, you can join us at the ownCloud Contributor Conference in in two weeks in Berlin and KDE has Akademy coming just two weeks later!

And more

Not everybody has the skills to integrate zsync in ownCloud to make it only upload changes to files or to juggle complicated API's in search for better performance in Plasma but there is plenty more you can do. Here is a KDE call for promo help as well as KDE's generic get involved page. ownCloud also features a list of what you can do to help and so does openSUSE.

Or donate...

If you don't have the time to help, there is still something: donate to support development. KDE has a page asking for donations and spends the donations mostly on organizing developer events. For example, right now, planet KDE is full of posts about Randa. Your donation makes a difference!

You can support ownCloud feature development on bountysource, where you can even put money on a specific feature you want. This provides no guarantees - a feature can easily cost tens to hundreds of hours to implement, so multiple people will have to support a feature. But your support can help a developer spend time on this feature instead of working for a client and still be able to put food on the table at home.

So, there are plenty of ways in which you can help to get the features and improvements you want. Open Source software might be available for free, but its development still costs resources - and without your help, it won't happen.

ownCloud numbers

Last week, we went over some numbers related to ownCloud. Things like the number of people who contributed in the last 12 months or the speed of code flowing in on average. The numbers are impressive and you can read about them in this press release.

Analysis

Numbers can tell you a lot. One thing is of course particularly cool: the numbers are big. Really big. ownCloud has had almost 300 people contribute code to it in the last 12 months. That is a lot. Some perspective: wordpress has had 52 contributors over its lifetime! Drupal: 149. phpbb: 190. Mediawiki: 534. Joomla: 483. VLC media player: 662. ownCloud has had 566 contributors over its lifetime. This is just one metric out of many, and the comparisons are between often wildly different projects so take it with some salt.

One thing I think you can safely conclude: ownCloud is certainly in the big leagues. Looking at our competition, the ownCloud Client team alone (59 contributors over its life time) is bigger than any other open source file sync and share technology.

Why numbers

We primarily want to keep an eye on numbers to see if we are doing well or not. Anecdotal evidence is important (I really like to read all the positive feedback on the #ownCloud7 release) but hard numbers are very important too. For example, if we see fewer new people join ownCloud, we can see if we can improve developer documentation or have to offer better help for new developers on IRC.

We have good reasons to keep an eye on that. Open Source projects typically have a huge turnover (60%/year is normal), requiring us to keep attracting new contributors. Not only that, ownCloud Inc. has hired many community members and, through its marketing and sales machine, is increasing the number of ownCloud users enormously. We do numbers on our user base internally, and the number we make public (about 1.7 million at the moment) is a rather conservative estimate. And growing quickly: Germany's upcoming largest-ever cloud deployment will bring ownCloud to half a million users!

What effect does that have? For one, paid developers can create a 'freight train' effect, accelerating development to a point where it is hard for volunteers to catch up. This is a reason why it is good to split up the apps from the core and to improve the API offered by ownCloud. This makes it easier to keep changes more localized and easier to follow. Another effect is that the growing popularity of ownCloud brings more people to our mailing lists and forums, asking questions. That is a tough issue. Improvements in documentation can help here, but we can also think about other tools and ways to answer questions.

Conclusions

We can't stare ourselves blind on numbers, and we won't. Real life matters more: that is why we are working hard on preparing the ownCloud Contributor Conference later this month! But it is cool to see confirmed what we already thought: ownCloud is a very significant Free Software community. Not just its size, but also in what we are doing and how we do it!

There still is plenty of work to be done so come help out and liberate more data!

ownCloud 7 awesomeness

One of the things I love from the Free and Open Source software world is that doing things in the open simply leads to better solutions. Resources are often constrained, polish might be lacking, but frequently from the seemingly chaotic processes emerges brilliance.

The upcoming ownCloud 7 has one of those things: server to server sharing. You see, for a long time, I and others have been asking the ownCloud desktop client developers for a feature: support for syncing multiple ownCloud installations. That way, files from our corporate ownCloud installation (of course we dogfood here at ownCloud Inc.!) and my private ownCloud could both be synced to my desktop and laptop. Unfortunately, while it has been on the todo for a while, it just kept pushed down by more urgent feature work.

While you could get it to work already by running multiple clients and playing with the config file locations, it seemed a bit brittle to me and I just accepted this feature wasn't there yet.

Only today, during my bike ride to c-base, did I realize that ownCloud 7 actually introduces this feature. And I even wrote a sneak preview about it! It is the server to server sharing that solves this issue.

What is Server to Server sharing?

Let's step back. What is the major thing that 'private' clouds don't have which public clouds do? Other than the NSA snooping, that is...

Well, you're all alone, of course. If you want to share a file with another student from uni, you have to create an account or use a shared link. He/she will then have to visit your ownCloud to be able to work with you. It gets quickly messy with a lot of files.

This is where Server to Server sharing comes in. You can simply share a link and your collaboration partner can add this, either a folder or a single file, to his or her own ownCloud. They can put the file wherever they want in their folder structure (we've gotten rid of that rigid shared folder concept in ownCloud 7!) and work with you like the file was on their own instance.

That means they can also locally sync the file with their sync client by just putting it in a folder that is synced!

Boom. You don't have to create and manage multiple server accounts in your sync client, creating a folder on your system for each server you work with. No, you just add the share to your ownCloud and put the files wherever you want - in one folder, or grouped by subject - whatever works best.

Now tell me that isn't awesome ;-)

There's more

Of course, this is just a first step to bringing ownCloud servers closer to each other. Our goal is full 'federation' of data: transparent sharing between servers so they can act as one cloud, protecting privacy while giving you the convenience of sharing, collaborating and communicating with friends, family, collegues and more. Once you connected to another ownCloud, you should be able to share files with the users on that server seemlessly - and other data, like contacts or music! You should be notified when things are shared (a news feed!) and be able to comment and chat. This however still needs work. If you're interested in helping to make this happen, consider joining us at the ownCloud Contributor Conference from August 26-31 in Berlin, Germany!


* Note that you can have kind-of server-to-server sharing with ownCloud 6 by manually mounting folders from other servers via webdav. This of course comes with various limitations (you do have to give out webdav credentials, to name one) so the new server to server sharing is a massive improvement...

ownCloud development in April

Github in action

Frank and myself thought it would be nice to have a weekly development update around the ownCloud community. It makes it easier for people to follow what is going on, we can't all follow all mailing lists and git logs and forums and so on. To start this up I went through information sources like the development mailing list, blogs, news and more and compiled an article about the whole month of April. If you all like it, I can start doing this weekly.

Now I can't follow everything either, of course, so it would be helpful if you, dear readers, could send what YOU know is going on to me! That would ensure I don't miss any cool things. Mail me or ping me over G+, twitter or somewhere else! Any suggestion is welcome.

UI mockups of new features

Development

So, let's start with what happened in ownCloud development in April, separated by core, cloud and clients. Note that what is below is not guaranteed to end up in a release! We might re-design things, defer to a later release to give it more testing or change the feature otherwise...

ownCloud core

• Work is happening to refine the oC 7 public API.
• Lots of work on LDAP improvements (much of that landed in 6.0.3, see this blog)
• Some work is done for improving support for high dpi screens! See PR 1 and PR 2 for details.
• Progress on a tool for conversion of data between various databases
• Some neat 'ajaxification' in the core, making way for:
• infinite scrolling for files app got merged!
• file sorting (by clicking column headers) is under development, needs review and help...
• The new user management from this pull request is almost ready to be merged!
• There is a project going on to make the normal ownCloud UI more usable on mobile devices. Help is of course very welcome as this is a lot of work!

Good old fashioned paper mockup

ownCloud apps

• work is going on to improve the view on what happened on your ownCloud in the activity app
• Infinite scrolling & neater layout for pictures app are coming.
• A much improved music app is coming, and a variety of features is under development, like new playlist functionality.

ownCloud clients

• The iOS saw work on multiple-downloads support. Not finished yet but a future version should include this. There was also work on the 'favorite files' feature.
• The Android client got the InstantUpload video branch finally merged. This means that once released, the client can upload video instantly, either always or only when on wifi. Thanks to zerginator for doing the hard work!
• The first beta for the 1.6 release of the desktop client came out and a bunch of bugs got fixed based on the feedback that came in. If you have not tried the new release, help us test it so we can make it even better. Read more about this upcoming release in this blog by Danimo.

Devel mailing list discussions

The entire month saw lots of bugfixes fly by on the devel mailing list. The news app, for example, now deals better with a host of websites, but also got reworked to be finally independent from the app framework. Meanwhile, there's a firefox plugin being developed for the News app, courtesy of our Outreach Program for Women participation.

Visual bugreports with screenshots.

Listing apps

Jan-Christoph announced that he had put together a page listing the central ownCloud apps as well as mobile and desktop clients. There is also a section 'external apps to integrate with'. With this overview he hopes to encourage collaboration over fragmentation. See the page here.

Vincent Petry brought a python client library to the ownCloud list, spawning a discussion about the need for perhaps a JavaScript library as well.

In other news

The bugfix release ownCloud 6.0.3 brought new bugfixes and improved LDAP performance.

The wider FOSS world saw Canonical retire Ubuntu One, to which we replied offering Ubuntu users to come and try out ownCloud. As Frank wrote when Box announced they'd open source some of the tools they use in-house for system administration and engineering:

"Depending on a vendor is always risky: the vendor can go bankrupt, be bought or raise prices. In the world of web services it also often happens that vendors decide to discontinue functionality they deem not crucial enough. It could just be something you depend on!"

ownCloud can be run on your own servers, not requiring you to move your data to an untrusted, third-party server farm." See his blog post.

Conclusion

I hope you liked the ownCloud development update of April! Obviously, the weekly ones should be shorter than this one. Let me know in the comments what you think about doing this every week. If you all think it is a good idea, I'll move this to the ownCloud blog. 

Get-involved workshops at the KDE/ownCloud/openSUSE booth at LinuxTag

Tomorrow, 9:45, LinuxTag and DroidCon open the doors in Berlin. The even presents a staggering number of sessions (see LinuxTag on Thursday alone featuring 57 talks). This is in part due to the addition of evening sessions, which are open for the public (no ticket needed). My own talk about the relevance of the Linux Desktop is one of these evening sessions. This is in parallel with LinuxNacht so we'll have to see how many people will choose talks over beer...

Workshops

At the combined openSUSE/ownCloud/KDE booth I've organized short workshops, given by contributors to these projects, designed to help you get involved with these (and other!) Free Software projects. There is only room for about 10-12 people per workshop so you will get some real attention from the developer doing the workshop. It also means you should make sure to be there on time to secure a spot!

We try to give the workshops tree times so if one is full, you can come back the next day (due to availability of people this didn't work for all workshops).

The program

Thursday

10:00	Testing Linux with openQA (Bernhard Wiedemann)
11:00	Introduction to hacking ownCloud file synchronization (Daniel Molkentin)
12:00	AppArmor Crash Course (Christian Boltz)
13:00
14:00	Build your first ownCloud App (Arthur Schiwon)
15:00	Writing your first KDE application (Sebastian Gottfried)
16:00	Packaging with the Open Build Service (Marcel Kühlhorn)
17:00	Internal security mechanisms used by the German eID card (Joerg Schilling)

Friday

10:00	Testing Linux with openQA (Bernhard Wiedemann)
11:00	Introduction to hacking ownCloud file synchronization (Daniel Molkentin)
12:00	Hacking PostfixAdmin (Christian Boltz)
13:00	Bareos Backup - Rear Disaster Recovery workshop (Maik Außendorf and Gratien D'haese)
14:00	Build your first ownCloud App (Georg Erhke)
15:00	Writing your first KDE application (Sebastian Gottfried)
16:00	Packaging with the Open Build Service (Marcel Kühlhorn)
17:00	Internal security mechanisms used by the German eID card (Joerg Schilling)

Saturday

10:00	Testing Linux with openQA (Bernhard Wiedemann)
11:00	Introduction to hacking ownCloud file synchronization (Daniel Molkentin)
12:00	AppArmor Crash Course (Christian Boltz)
13:00	Bareos Backup - Rear Disaster Recovery workshop (Maik Außendorf)
14:00	Build your first ownCloud App (Georg Erhke)
15:00	Writing your first KDE application (Sebastian Gottfried)
16:00	Packaging with the Open Build Service (Marcel Kühlhorn)
17:00	Internal security mechanisms used by the German eID card (Joerg Schilling)

Participants to the workshops get a Club Mate to keep them awake and quench their thirst.

Our booth space is in Hall 6, Number D11 - next to the LinuxTag info stand. See you there tomorrow!

Building Converging UIs

I just blogged about my article on linux.com about the just-released KDE Frameworks 5 sneak preview.

Convergence in 2010: Plasma Netbook

Converging Form Factors

On the Frameworks, one can soon expect to see releases of KDE's Plasma Workspaces. A Technology Preview of Plasma 2 has already been released and this ambitious project has not lost any of its goals. Today, I noted that ZDNet's Steven J. Vaughan-Nichols wrote about what he expects from Ubuntu in 2014. There, he quotes Jono Bacon talking about formfactor convergence. And the intarwebs are full of people making jokes that Microsoft is copying Ubuntu with their single UI for multiple devices. But let's not forget where they got their ideas...

What is real ambition?

I would argue that neither Apple, Microsoft, GNOME, nor Ubuntu/Canonical are even half as ambitious as the KDE Community in the area of convergence. They are all merely catching up to the state of KDE technology in 2010. In that year, the KDE community released Plasma Netbook, a plasma-based shell optimized for another form factor: the netbook. With far more advanced convergence than anybody today has yet shown: Plasma Netbook and Plasma Desktop share well over 90% of their code, as opposed to not even sharing toolkit or display shell (Ubuntu) and having a completely separate desktop (Apple). On Plasma, widgets can dynamically adjust to the constraints of their environment, be it on a panel, free-form on the desktop, full-screen, in a window or in a tiled environment. And yes, the different form-factor optimized shells be switched on-the-fly. No separate login or account, no loss of functionality, no separate applications for each shell, nothing like that. It just works. See this blog from 2011 to get an idea how the tablet plans were doing.

Plasma on desktop/netbook/phone in 2011.

I understand what Microsoft is doing - trying to build a single user interface for vastly different devices. And I guess we've all seen how it does not work - Apple is smarter, in that regard. Underlying technology can of course be re-used but you simply can not make a UI which works equally well on a 75 DPI 24" screen with mouse & keyboard, on a 455 dpi touch phone, on a 300 DPI touch tablet and a 64" television with Kinect or something like that...

Instead, the Plasma team has build a technology which separates presentation from logic, allowing you to build UI's which adapt dynamically to the needs of the form factor.

Below is a video of Plasma's awareness of the container size in action, like it has functioned since it was released in 2008.


Now I don't deny that in terms of resources, MS and Apple are so far ahead they can make pigs fly. Our more advanced architecture and ideas can't really compete with what they do and as long as we have 0.5% market share on the desktop, Free Software will probably not get the resources it needs to polish things to the same degree. That's a bit frustrating but it doesn't mean we frequently do things far beyond what the big boys do: anybody who has seen the introduction of the upcoming release of Mac OS X and knew about KDE Connect knows we're way ahead of them when it comes to phone-desktop integration ;-)

Moving Forward

All this technology will be brought to a new level with the release of Plasma 5, where your workspace will be able to smoothly morph into a different form factor without even a hickup. So, when it comes to the convergence of formfactors, KDE is lightyears ahead of what the competition is even aiming for. And outside of that, we're doing awesome stuff, for sure.

Did I say something about the power of innovating in the open? That's what I'm talking about.

Giving constructive feedback

Feedback is often equated to 'yelling'. But it doesn't have to be - feedback is, after all, central to learning. And in Free Software, where learning is a major motivation for many participants, proper feedback is very important.



Many of you have seen how Linus can yell at people and unfortunately, some mistake that for effective communication. It is not. But what DOES constitute effective feedback?

Three Rules

Plenty of books have been written on the subject. To save you the reading effort, I'll summarize the most important lessons here: three rules for how to give constructive feedback.

RULE 1: find the right time and place (or don't do it at all)

Both you and the other person should be in the right place, mentally and physically, for the feedback. For example, don't embarrass a person in front of a crowd. Feedback is best given in private, unless of course it is explicitly asked for in a group. But even then, think about what you say.

And have the right attitude towards whatever has to change: everybody makes mistakes. Constructive feedback should be building up, not breaking down. None of us is born as a super coder, we all had to learn. If you and/or the other person are angry, it will be a shouting match. If there is no way for the other person to improve, why frustrate him or her with feedback?

The time element is crucial as well. Feedback should be timely: giving feedback on an event that took place weeks ago, most likely long forgotten by the subject, offers little value. Be sure you have a certain event or action in mind and can be specific about when, where, who was involved and what the results were.

Rule 2: Describe, don't judge or infer

Most important, constructive feedback has to focus on the facts of behavior in a concrete case. And on its consequences, not on the person and who they are. That sounds easier than it is. It is best to follow this format when giving feedback:

• describe the specific behavior/action/code itself
• describe how it made YOU feel (emotional feedback) or the effect it has on $SUBJECT (technical)

To focus on behavior use adverbs to describe action, rather than adjectives describing qualities. Help the other person understand the impact their actions have.

Let me give some examples:

• unhelpful: "You were really an asshole in the meeting today" -> personal and pretty darn judging
• unhelpful: "You were trying to derail the discussion! -> inferring goals
• helpful: "When you talked so loudly during the discussion you made me feel really anxious"
• unhelpful: "Your patch sucks" -> not concrete
• helpful: "If you write code like $EXAMPLE you will break $DESIRED_BEHAVIOR"
• unhelpful: "You always forget to add comments to the code!" -> generalization
• helpful: "Yesterday when you checked in the code for X, you did not add comments in $FILE, so it took me a while to figure out how it did fit in"

This also works with compliments:

• unhelpful: "You're a great coder!" -> unspecific
• helpful: "When you commented the code with that high level overview you made it easy to understand the flow of the class"

Negative feedback which is too vague just frustrates: there is nothing to improve. But unspecific positive feedback is not that great either: people frequently doubt your motives ("Is she trying to get something from me?") and it even makes some people feel insecure.

Note that when giving feedback, people have a tendency to emphasize or even exaggerate. Feedback tends to land much harder than you think, even with people who seem impervious to criticism. It's better to bring it softly and remind people later then to lay it on heavy and make them feel inadequate or not appreciated.

Rule 3: Sandwich it

It matters how you introduce the feedback. If you weren't asked for it, note clearly that you'd like to give some feedback. Perhaps this is a bad time - allow the other to point that out and if so, just let it go.

Then, seek for balance. Certainly there are things to improve, but there is always something that was done well. We tend to focus on the negative. To prevent that, 'sandwich' the negative part with positive feedback: start with something that went well, follow with the part that needs improvement, finish with another part that went well or a silver lining. Not only does this help digest the feedback better, it tends to force you to think about it more as well.

If you can, try to give some tips or hints on how to improve. Share how you dealt with this issue in the past, perhaps. Here, too, be as concrete as you can.

After giving the feedback, let the other talk. It is not unlikely that the recipient will act defensive. Don't take that as meaning the message did not land - most people have trouble admitting mistakes when confronted with them, even if they are brought the Right Way™. Let them respond, say you understand - most importantly, don't pile up more 'evidence'. You've done what you can, it is now their responsibility to take it - or not. If you get no response you might want to ask a open question like: "What do you think".

Note that you can give too much feedback. Improving oneself is a lot of hard work - and one can work only on so many things at once! Recognize progress, even when it is slow, and leave room for mistakes. If a colleague tends to make certain typical mistakes you don't always have to point them out. Give him or her time and opportunity to self-correct, silently accept some of the mistakes and space out the feedback a little.

Feedback 101

There's something to receiving feedback, too, of course. Note that following the rules above isn't always easy and sometimes people are frustrated and do yell. Try not to get angry - feedback, any kind, can help you improve. Even if it is not brought to you in a nice way.

Giving and taking constructive feedback is a hugely useful skill to have - it helps you as well as others to learn and get better. And while most of our communication takes place online, these rules apply as much if not more so. Thanking somebody for a patch, starting by commenting on the good side of it, before you hack and slash in on the less-than-great parts, adding concrete ideas for improvements, and finishing with a high-note in proper sandwich style: it can make the difference between getting an improved patch or never hearing from the contributor again.

If you have comments, feel free to share them ;-)

Getting involved in Free Software

I frequently get the question, by mail or over social networks:

But how do I get involved in $PROJECT?

Now a common answer is 'just do it' while others often point to resources like the KDE Developers Beginners Guide and Contribute to openSUSE, or write a simple how-to for building a package. But I usually don't actually reply with links to any of those. Mostly, people have found these resources by themselves.

What they want to know now is how to, you know, actually do it! And as that question can actually be answered rather project-independent so I thought it would be useful to write it down here.

Step one - Build the Code and get Familiar

After reading the various guides and how-to's, you set up a development environment. Be sure you can run the unstable application(s) or hack on a package. Getting that up and running is a very good first step.

I would also subscribe to mailing lists, read the blogs and hang out in IRC. Just watch what is going on: it will teach you the culture of the community and that's crucial to get stuff done later.

Step Two - Hack Something

Perhaps you will already find bugs, then: trying to report them is good, trying to fix them is better. It will not be easy to fix them but that is when you can ask for help on IRC, forums or here!

If you don't find any bugs you want to fix, perhaps you can think about what to add, what to change. What do YOU think is important and what needs to be done? It doesn't matter if you pick something yourself or find a todo list or wiki page of the project and pick something there. The hardest part will be: JUST DO IT. Get hacking. You'll get stuck, that is OK: read documentation and when you can't figure it out, just ask for help. Above all: don't give up until you are done! I would suggest not to pick something too big. A one-liner patch will take you a day, easily, and might not seem important, but this first step matters a lot. Don't try to fix the entire user interface or work flow with your first change! For example, fixing code style to comply with the project rules is already a perfectly fine first step.

Step Three - Get it in

Now you got something, so get it back to the project. On Github or on the Open Build Service you do a merge request; in other projects you have different work flows varying from sending the patch by mail to using review board or other tools. It doesn't matter.

This won't be easy, but that is mostly due to you being so anxious about it: it can take quite a while for the developers to review the patch and they're sure to have some comments on it. Don't be discouraged and do know that you're allowed to poke them: two weeks waiting isn't much in many projects but it's enough that you are entitled to poke somebody. Send a follow-up mail or ping somebody you know is active in the project on IRC and ask politely if they know how you can get feedback or if you did something wrong in the way you asked.

Note that your patch will not always go in, even in modified form. Not all projects might be OK receiving code style fixes or consider what you fixed a bug. Again, the hardest part here is to not be discouraged. You can ask if they can at least tell you that what you did made some kind of sense; and/or ask for a suggestion for something easy to fix. If they're half-way decent they will be nice about it! If not, perhaps you picked the wrong community to help out in...

Step Four - That's all!

When you manage to fix or hack ONE thing, you have done the hardest part of getting involved in Free Software: you GOT STARTED. From then on, you can find other things and really Make a Difference.

Really

Seriously, there is no more to it. The hardest part is of course the doing itself - but no amount of preparation can make that easier. The most important tip in this is to not give up, the second most important one is to ask for help. All developers had to go through this. Some locked themselves up in a basement, others spend a week banging their head against a computer screen. All of them created a ugly beast of a patch, article, image or whatever your first attempt is. That is perfectly fine. Accepting that is probably the third most important tip ;-)

Of course, I can recommend to check out Open Advice as it contains the lessons of quite a few smart folk. The short essays make for a nice read before bed time or so.

Have a lot of fun and enjoy hacking ;-)

Consensus decision making

Jono blogged about respect in community discussions. I have zero to say on the storm-in-a-teacup (his words) that started it other than, perhaps, suggest that when there are waves, there is wind. But whatever direction that wind blows, I'd like to focus on something else. Jono made the following statement:

Ubuntu is not a consensus-based community. Consensus communities rarely work, and I am not aware of any Open Source project that bases their work on wider consensus in the community.

I'm not entirely sure what he means with consensus and community here. He himself defines community as "a collection of people (or animals) who interact with one another in the same environment". Consensus decision making, according to Wikipedia, is:

"a group decision making process that seeks the consent of all participants. Consensus may be defined professionally as an acceptable resolution, one that can be supported, even if not the "favourite" of each individual"

Talking consensus

Let me take this as an opportunity to address a common misconception about consensus: that consensus means full agreement. The Wikipedia entry already points out that the outcome has to be 'acceptable', one that 'can be supported'. This matters: Jono probably meant to say that there is no sizeable community where everybody fully agrees on every decision and I can't imagine he is wrong on that. But that is not what consensus means.

(dis)agreement

The reality is that in a large and diverse group of people, it is impossible to really reach full agreement on any sufficiently complicated matter. Making decisions on agreement of all participants thus doesn't work. Consensus, instead, allows a decision to be made even in the face of disagreement. Essentially, it is a form of democracy without voting.

Ever heard the phrase: "Let's agree to disagree"? That is it: at some point in a decision making process, consensus requires some of the participants to be mature enough to step out of the way and let a decision actually get made. And others need to respect them for that.

Voluntairy

What makes consensus different from voting?

Usually, those in a small minority are the ones who have to (wo)man up and accept that the decision and project is more important than them. The main difference between voting however, where minorities (anything below 50%, usually) don't get their way, is that it is not mandatory. In some cases, the minority can get their way and it can be the majority which steps back and lets them. And even if that doesn't happen, the difference between being forcefully over-ruled and gracefully accepting that you can't always win is big.

A second key point is that ruling by consensus requires discussion, much more than voting does. You can't make decisions by consensus without informing people of the choices - you have to know what you (dis)agree with. Certainly, a community where a few take decisions without talking about it does not decide based on consensus.

Last, the two are not incompattible. It makes all the sense in the world to occasionally do an 'opinion poll' (as opposed to doing a decisive vote) to aid the decision making process. This is valuable input for a consensual decision: vocal supporters of either side can create rather distorted views on how strong the support for a certain opinion really is.

Trust and respect

So I think Jono is wrong when he states that there are no communities which decide based on consensus - KDE is an example of one, Gnome does it often and it's pretty much the way of the Geeko, too. Others usually prefer to vote (Debian) or have a more top-down structure like Ubuntu. There are many ways to Rome, as they say. Being aware of that is a good thing - and being dismissive of ways other than yours is not.

I want to add that Valerie Zimmerman made an excellent argument for the importance of trust and respect. No structure of decision making works without these - trust that those who disagree will have the courage to agree-to-disagree, trust that the majority is right or trust that those who decide for you make the right decisions. And respect each other while debating it.