Client-side, server-side and zero-knowledge end-to-end encryption in Nextcloud

Android encryption in action

Yesterday, Nextcloud published plans, designs and code for end-to-end encryption in the clients. A recent report from Forrester named data encryption as one of the top global Cybersecurity trends for 2017, so this is a big deal! I thought it'd be good to go over the differences between client-side, server-side and end-to-end encryption and position what we developed.

End-to-End Encryption

End-to-end encryption is usually associated with communication channels, think chat or video calls. Signal, Telegram and lately WhatsApp employ end-to-end encryption. So what does it mean?

The main property of end-to-end encryption is that the data is encrypted on the one end and decrypted on the other end, so only the sender and receiver can read it.

As an example, take the Nextcloud Video Calls app. When you share a link for a call and the other person joins, the Nextcloud Server essentially gives the other user the address of your computer and helps you connect to each other. It will also send signals when others join the call, when you mute and so on. But the actual call takes place between your system and that of your conversational partner, directly. And before it is sent out, your browser will encrypt the data for the other side to decrypt. This way, nobody in between, like your internet provider, can listen in!

The main downside of the term is that the definition of that 'end' can vary: you can say that normal https encryption is 'end-to-end', defining one end as the browser and the other end as the server. So this brings us to the difference between server-side and client-side encryption.

Server-side encryption

Server-side encryption serves to protect data on or going through a server: as soon as the data arrives, the server encrypts it. When you use a cloud storage like Amazon S3 or a Dropbox account or a FTP at another office with Nextcloud, our Server-side Encryption encrypts the data before it gets sent to the other storage and decrypt it only after it has been retrieved. Server-side encryption thus protects your data from access by a third party storage solution.

Note that encryption in the browser is essentially server-side encryption: the code that does the work comes from the server and thus the server controls what is going on. If a server is compromised, the attacker (or evil system administrator or government agency) can simply make a minor modification to that code so it gives them access to your private key!

When the data is on the server itself, however, the server admin has access to it as the keys are managed by the server. In case of Nextcloud, we encrypt that key with your login, making it impossible for the server to decrypt the files unless you are logged. That is called "protection of data at rest", but, of course, you log in all the time if you have the client running.

So the limitation of server-side encryption lies in what it does not protect you from: the server being compromised.

Client-side encryption

This is why server-side encryption is often contrasted with client-side encryption, which is what is employed by the end-to-end encryption Nextcloud introduced today. Where server-side encryption happens after transmission to the server, we encrypt the data on the Android, iOS or desktop client already. Then, only at the receiving end, it is decrypted again. That receiving end can be another device owned by the same user or a device owned by another user who has been given access to the data. But not anyone in between or otherwise not authorized! This is sometimes also called a zero-knowledge privacy: at no point in time can the server have any knowledge of the data.

The process of creating keys

A little bit about keys

Usually, public and private keys are used for the encryption and decryption. How does that work?

Everybody has a set of closely related keys, a 'public' and a 'private' key. They work a bit like a mail box works: anyone can throw a letter in the box, but only the mailman (or woman) can open and empty it. The public key is, as the name implies, public. Anyone can use it to encrypt something. But to decrypt the result, the private key is needed!

Nextcloud generates a public and private key pair the first time a user enables end-to-end encryption in their client. The public key gets signed by the server with 'certificate' to verify the user identity (our Cryptographic Identity Protection feature) and stored there for other users to encrypt files to that they wish to share. The private key gets encrypted with a locally, app-generated very secure and very long (12 word!) passcode which is displayed to the user and then the key gets stored on the server as well.

Another device owned by the user can download the private key, the user can enter the 12 word passcode to decrypt it, and this device will then also be able to encrypt and decrypt files. That is how you add your phone and desktop after you enabled end-to-end encryption on your laptop.

You can learn much more about exactly how the encryption and decryption work on the end-to-end encryption web page we made as well as the whitepaper you can download there.

Enterprise challenges

While consumers use end-to-end encryption for chat for years, larger organizations like companies and governments have struggled to find solutions that protect their sensitive data without disrupting productivity and legal requirements for audit logs.

Indeed, many solutions create a new layer over existing solutions like Box and Dropbox or have poor key management, making sharing cumbersome and less secure. A real enterprise solution needs to take the burden of complexity away from users and, if it needs to be anywhere, put it on the shoulders of the system administrators. Nextcloud has done exactly that, not even allowing users to create their own, potentially insecure password and making adding devices easy. The 12-word passcode can be recovered from any of the devices an user owns and it is possible to enable a system wide recovery key. If enabled, users will get warned of this and the system administrator gets to see and note down the key once and only once. After that, the code is destroyed and no attacker would be able to steal it, provided the server administrator puts it in a safe location: we'd recommend an actual safe.

Another important feature for enterprises is support for a Hardware Security Module which could be used to generate user certificates. Without it, our design does not allow users to switch identities (as this could be abused by an attacker with control over the server) but this restriction can be relaxed if a secure HSM is in the mix.

You can learn more on our webpage about end-to-end encryption!

Happy Birthday ownCloud

Seven years ago at Camp KDE in San Diego, Frank announced a project to help people protect their privacy, building an alternative to Dropbox: ownCloud.

I was there, sharing a room with Frank at the infamous Banana Bungalow. Epic times, I can tell you that - there was lots of rum, lots of rain and loads of good conversations and making new friends.





Since then, a lot has changed. But the people who started building a self-hosted, privacy protecting alternative in 2010 and 2011 are still on it! In 2011, a first meetup was held, and the 5 participants at that meetup recently got on stage at the Nextcloud conference to recall some good memories:



Of course, today we continue the work at Nextcloud, that just yesterday published its latest bugfix- and security update. It is great to see so many people have stuck with us for all these years - just this month, the KDE sysadmins migrated their ownCloud instance to Nextcloud!

We'll keep up the good work and you're welcome to join, either if you're looking for a job or just want to code. In both cases I can promise you: working with such a motivated, dedicated, professional team is just plain amazing.

I also published a blog on our Nextcloud blog about this milestone.

EDIT: By the way - there's a meetup tonight in C-Base, B'lin, 19:00 - would be fun to drink a beer on ownCloud's birthday and talk about the future! Join! It will be at least until 10 or so, so if you can't be there before then - still come! ;-)

Why use ZIP instead of TAR?

I've been asked recently why ownCloud zipps its files instead of tarring them. .tar preserves file permissions, for one, and with tar.gz or tar.bz2 you have compression too.

Good question. Let me start by noting that we actually have both: zip and tar.bz2. But why zip?

A long time ago and far, far away

In the beginning, we used tar.bz2. As ownCloud gained Windows Server support, we added zip. Once we dropped Windows support, we could have killed the zip files. But we had reasons not to: tar is, sadly, not perfect.

Issues with Tar

You see, tar isn't a single format or a 'real' standard. If you have a platform other than plain, modern Linux, think BSD or Solaris, or the weird things you can find on NAS devices, tar files can get you in trouble. Unlike zip, tar files also can have issues with character format support or deep folders. We've had situations where upgrades went wrong and during debugging we found that moving to zip solved the problem miraculously... And, as ownCloud, we're squarely focused on the practical user experience so we keep zip, alongside tar.bz2.

See also the GNU tar manual if you want to know more about the various tar formats and limitations.

Sadly, sometimes it is impossible to find one thing that works for everyone and in every situation.

Tarred turtle pic from wikimedia, Creative Commons license. Yes, that's a different tar, I know. But - save the turtles!

Patching ownCloud: get your fix NOW

you can also get quickly loading kittens!

Sometimes, there's something broken. Or, there's a feature you want in your ownCloud installation now. Not after the next release, NOW!

Well, if the code has been developed but not yet released, you can have it. If you're willing to put in a little work and take a bit of risk!

Let me show how it works.

What we want

Say, you just added 3000 holiday pictures to ownCloud. When you load the folder with the images in the web UI, it is very slow as all thumbnails have to be generated and provided to your browser! What if ownCloud could simply generate all of them during, say, tonight?

Awesome Oparoz has developed code which gives the occ command the ability to run through your ownCloud, either in its entirety or per user account, and generate all thumbnails. But this code hasn't been merged: Oparoz wants to create code tests first before he merges this, to ensure the quality of the code base. Sadly, he hasn't had time to create tests and nobody else has stepped in yet...

(edit: some years later, this has become the super cool Preview Generator app you can just install from here.)

How we get it

How to get this code? Github has this cool trick where you can add .patch behind a pull request URL, like this: https://github.com/owncloud/gallery/pull/387.patch.

Patch your ownCloud with it! This is how:

1. log in to your server
2. navigate to the Gallery app folder
3. grab the patch: wget https://github.com/owncloud/gallery/pull/387.patch
4. test patching: patch -p1 --dry-run < 387.patch
5. If you get only a checking [filename] you're all good!
6. If you have errors check if you are on the right ownCloud release (the patch might be for the latest master and not work on the release you have) and see if you are in the right sub folder.
7. Now run patch -p1 < 387.patch
8. Go and enjoy your feature...
9. If you want to undo the change run: patch -p1 -R < 387.patch

NOTES:

• of course - everything's Nextcloud now - where most the development on former ownCloud code base is happening.
• be careful running random code from github in your Nextcloud, just like experimental apps - this is untested by definition.
• Also, if you go and add all kinds of patches which end up burning down your house and killing your kittens you have only yourself to blame. I gave you a knife, YOU decided to stick it in your eye.
• Last but not least, if it works (or doesn't) report back, this is called 'testing' and very valuable to us!!! It will help get the code merged sooner.
• Idea to blog about it came from the ownCloud meetup evening before publishing - we still do monthly meetups in Berlin and other cities for Nextcloud ;-)

Get Started With ownCloud App Development in Six Steps - the Quick and Dirty Way!

ownCloud Mail, a great newcomer

What's simpler than downloading a zip file, extracting it and running a command in the resulting folder to get an ownCloud server up on localhost?

Yes, it can be that simple, though it might require a few minor tweaks and you have to make sure to have all ownCloud dependencies installed.

Note that this is useful if you want to develop an ownCloud app. If you want to develop on the ownCloud core, a git checkout is the way to go, get started here. Feedback on this process is highly appreciated, especially if it comes with a pull request for our documentation of course ;-)

Step 1 and Two: Dependencies

• Install PHP and the modules mentioned here
  Your distro should make the installation easy. Try these:
  
  • openSUSE: zypper in php5 php5-ctype php5-curl php5-dom php5-fileinfo php5-gd php5-iconv php5-json php5-ldap php5-mbstring php5-openssl php5-pdo php5-pear php5-posix php5-sqlite php5-tokenizer php5-xmlreader php5-xmlwriter php5-zip php5-zlib
  • Debian: apt-get install php5 php5-json php5-gd php5-sqlite curl libcurl3 libcurl3-dev php5-curl php5-common php-xml-parser php5-ldap bzip2
• Make ownCloud session management work under your own user account.
  Either change the path of php session files or chmod 777 the folder they are in, usually something like /var/lib/php (debian/SUSE) or /var/lib/php/session (Red Hat).

The Final Four Steps

• Download the latest ownCloud zip file.
• Extract it in a folder where you want to get going.
• Enter the folder and execute: php -S localhost:5000
• Visit http://localhost:5000 in your browser

ownCloud should present you with its installation steps! Give your username and password and you're up and running with SQLite.

Alternative with OCDev

An alternative is to use OCDev which you can grab here. After installation, you run
ocdev setup core

See the app development tutorial here.

Start with the app

Now you create a subfolder in the owncloud/apps with the name of your app and put in a skeleton. With OCDev:
ocdev startapp MyApp

By hand, you can copy an existing app and hack that up ;-)

It's probably wise to now get going with the app development tutorial here. Be sure to check out the changelog, we try to make sure the latest changes are noted there so even if we didn't manage to fully update the tutorial, you can find out what will and won't work in the changelog. Also, be sure to update the links to get the latest dev doc - this all links to 9.0, once that is out it is probably better to directly target 9.1 and so on.

Your input is very much welcome! If you run through these steps and get stuck somewhere, let me know and I'll update the documentation. Or, of course better still, do a pull request on the documentation right in github. You don't even have to do a full checkout, smaller fixes can easily be done in the web interface on github.

Thanks, good luck, and have fun building ownCloud apps!

Western Digital Labs and ownCloud

We published a blog post about a collaboration between WDLabs and ownCloud. WDLabs is, in their words, "a division of Western Digital focused on accelerating new solutions". I'd take that as "they look for more ways to sell hard drives" and they have been playing with some interesting stuff. One such thing is their PiDrive Kit. This kit includes a connector cable which powers both the harddrive and the Pi from one power supply, simplifying the setup. We're now working with WD Labs to build something from this starting point.

WD Labs, meet ownCloud

When they approached us a few weeks ago, they told us they had been toying with a more complete kit with bundled operating system based on berry boot. This lets you choose an OS on first boot and they had put together an image with ownCloud as one of the options. They thought it was really cool and asked if we felt the same and had any ideas of what we could do with it. Well, sure, I wrote not too long ago about What's holding ownCloud back and being able to offer potential users a plug-and-play device with ownCloud on it is something we'd love.

So did they. We quickly received a prototype of a kit with Berryboot and ownCloud on it as well as a case and a Pi included. After some more discussions, we developed a plan with which we hope to get some community help to make this happen.

We need your help

You have to know - this is not something ownCloud, Inc. is very much involved with. It simply is not a commercial endeavor - Inc. works for enterprise customers and we haven't found any of them yet running ownCloud on a Pi. So, while WD Labs will take care of the hardware and has committed themselves to doing a serious number of ownCloud branded devices complete with pre installed OS in the first quarter of next year, it will depend on our free time and community help to develop the OS.

And there's serious work to be done. Berryboot can be configured to boot a headless ownCloud but right now, to even boot, you have to connect a keyboard and monitor. And booting isn't even the issue - finding and configuring an IP and then connecting to the Pi is. And once you've done that, the system has to be configured to be accessible from outside so you can use the ownCloud clients on your mobile phone and laptop to always find and connect to your ownCloud and give you the ability to share with friends and family.

There are folks in our community who have experience with this, building our community VM for example (hi Daniel!) and we'd like to solicit their help. We have 10 prototypes to send out to people who want to help build this ready-to-go ownCloud, the blog has more details on how to get one. And if there is a lot of interest, we'll get more prototypes put together, though I think it'll be tight to get them out before Christmas.

Note that it is totally cool, even advantageous, to send in a proposal ad a team. Together you can get more done!!!

Of course - without this kit there is still a LOT you can do to help out and there is no reason to wait for us sending out anything! Help get a discussion started about what this should look like, how to get there and who does what. Subscribe to the developer mailing list and get started!

Future

Now we know the Raspberry Pi 2 isn't the very best device to run ownCloud on, in terms of performance. It won't run with 100 users, indeed, though it isn't too bad for a few users at home. But, aside from the fact that there is a lot of room for tuning (PHP 7, for one, should help a lot, see some stats here), we're not married to the Raspberry with a single hard drive. It is a popular device and thus a great place to start but WD Labs has already shown us prototypes using a Rasberry Pi Compute device and 2 hard drives and we could work with devices like the Banana Pi or more powerful boards, too. That won't happen immediately, it simply depends on how well the first version turns out and the interest from users.

That means it matters a lot now: can we make this work? I'll sure put a portion of my free time in this as I'm quite excited about it. You too?

Home Automation, AI and the Crownstone

Waiting for that future where you're dressed automatically? I'd rather do that myself and don't have a Roomba yet but as tech marches on, we'll get more automation in our homes.

With more and more 1984, Brave New World and Farenheit 451 being brought to this century by our esteemed leadership and our corporate overlords, I'm hoping that this tech will be ours, rather than theirs.

Enter a nice player in the "automate stuff" arena: Dobots. In their own words:

Our goal: Really smart buildings. Really smart robots. We're a startup in AiTech with applications in CleanTech and GreenTech.

What makes them nice is that they are real Open Source folks, putting what they code on github under a free license and working with and building on open tech.

They previously did industrial cleaning robots and other stuff. Currently, they are running a kickstarter for a smart power outlet as a first foray into home automation. EDIT: they've instead decided to, for the first run, only create EU plugs as there was little interest in the US. You can order them on the crownstone.rocks website.

Enter the Crownstone

How is AI relevant for power outlets? More than you'd think. One example is SLAM. SLAM (Simultaneous Localization And Mapping) tries to answer two key questions in Robotics:

• Robot Localization: “Where is the Robot?”
• Robot Mapping: “What does the world around the Robot look like?”

(read more about SLAM here)

Dobots has turned this around in the crownstone, using these algorithms to map your house and localize people in it. Yes, the Crownstones know where THEY are and where YOU are. While not exactly a localization tech, the Crownstone uses Bluetooth LE for this so it works with pretty much any device. And the best part: their localization code is, in Javascript, to be found on github!

This means you need no motion detectors or cameras to know where people are in the house, keeping things simpler and cheaper. But Bluetooth LE is used for managing the Crownstones, too. Rather than turning to WiFi or (expensive) central control hubs, Dobots lets the Crownstones automatically create a mesh network. Your smartphone acts as control hub (apps on github, of course) though you can also take a Raspberry Pi, plug in a bluetooth usb module and have it act as hub. That way, even when you're not at home, you can control devices or get notificationsthat, say, your television, computer or other devices are getting unplugged. It uses XMPP and WebRTC to get through your firewall and guess where you can find the Cordova based Raspberry Pi app...



But there's another innovation in there which helps avoid extra devices and complicated configuration: extremely fine grained power usage monitoring. Yeah, it can tell you how much power your fridge and TV use but more importantly: it can automatically detect which device you connect to it! So when you leave the house, it can kill power to your tv but not your fridge... I've been told they think they eventually will be able to figure out different makes and models of laptops based on their power signature. Now is that cool, or what?

The algorithms to do this are currently Matlab code but once finalized, they'll be on github, too.

Dobots regularly works with students who get to play with their tech and come up with new, innovative use cases. And this works wonders. Just some results from a recent hackathon:

• Team 1 developed ChildLock. Devices are only turned on when adults are in proximity.
• Team 3 developed Start VR. Virtual reality that allows you to picture your own furniture in an Ikea store.
• Team 4 introduced Never Lose. Lights indicate for elderly people where they have lost items with iBeacons.
• Team 5 used Crownstones to indicate the way to store employees.
• Team 6 worked on Tommy. An AI that analyzes patterns of daily life to combat loneliness.
• Team 7 developed Any Morning. Your phone using the Crownstones guides you to your morning routines to make you leave your home on time.
• Team 10 implemented Tipspromenad. Kids have to find objects in a place combined with solving puzzles for fun!
• Team 11 developed SpotOn. In emergency situations lights indicate how to flee a building

All together

The Crownstone has some nice advantages over the competition:

• Functional. Dimmer, iBeacon, a current measuring device, a standby killer and more in one.
• Cheap: you get 2 ready-made or 3 do-it-yourself for Eur 75 but there's no need for a hub, motion sensors or other stuff.
• Open. Using open protocols and code on github and I expect interesting applications to come from the community.

I did an interview with their COO - for more background, read it on LinuxVeda.

So I say - if you are looking to start with some basic home automation, go and get some Crownstones and get started!

ownCloud workshops - two down, two to go.

some had already given up at this time...

The success rate is going up - where, at the first ownCloud workshop at the openSUSE conference, we had no successful installation, yesterday in Helsinki we reached a two-out-of-five. Both workshops had around 20 participants but usually people collaborated in groups of 3-5, following my guidelines on how to get ownCloud up and running on a Banana Pi development board.

Whoah, two out of five?

I admit it isn't easy and partially, that is intentional. The instructions in the document are sparse, especially for newbies - but even an experienced Arch'er threw his towel in the ring after almost three hours. Then again, what is a workshop for if not for enjoying the struggle of learning something new? And certainly everybody did that - struggle and learn new things.

The winning team still hard at work

The second team to get ownCloud running

Those who have any experience installing ownCloud know the difficulty can not be in ownCloud - and indeed, once you are set up with a running Banana Pi with SSH access, installing ownCloud is a matter of minutes. But getting there is no picnic! Why?

The hardest part by far is with the networking part of the workshop - the moment you've SSH'ed into the Banana Pi, 85% of the work is done. The challenge is significant - requiring not just Linux on the host laptop (yesterday, Ubuntu got 4 new users as neither Mac nor Windows were up to the task) but also handling stuff like tcpdump, Wireshark and a bunch of low-level command line tools like dd, mount, dpkg, ssh and so on. For most participants, the hardest challenges were:

• Windows and Mac. I'm sure they are awesome operating systems, but something which is a few mouse clicks on a Linux system (sharing the wifi internet connection over a Ethernet cable with the Banana Pi) seems virtually impossible. On Linux, NetworkManager makes it as easy as creating a new wired connection, choosing "shared network" under the IPv4 tab and ticking the "this connection requires IPv4". Now, just enable this network after connecting the Pi and done. I have no experience with Windows or Mac whatsoever, but if nearly 15 IT students with internet access can't figure out how to make these operating systems do the same thing - I can only assume it is hard.
• Command line familiarity. If you're new to Linux, an instruction like "mount the USB stick and copy over the data to the Pi" takes more than a few minutes and requires you to learn at least two new tools and looking through system logs. 
• New tools. You'll be looking for alternatives to Linux commands like dd on Windows and Mac first, and once you've given up on your familiar platform you get to learn tools even most Linux users rarely need. 
• Geeks. "Can't you do this easier with TCPDump?" "You can do this with the ip command too, you know" - I could only reply "Try, and if it works, show me how." Rest assured, I learned a few things - but haven't changed my instructions. Yet.

Seriously, I love it. Three hours of a workshop full of people trying various ways of skinning the cat. To me, the fact that nobody got completely through my instructions merely means they had a good time on the way. At least, that's what they said - "the best workshop I've been at" is just awesome to hear.

And... working! A Banana Pi was the reward.

For the next workshop (coming Thursday at the Open Tech Summit in Berlin), I'll further streamline the instructions (but not too much!) and I mean to put in some 'advanced' challenges for those who simply know too much about Linux to stay busy for 3 hours. I want to know how to do this without Wireshark and NetworkManager, to name two things...

of course - food and beer as reward for hard work

I'll also have to make Linux mandatory. The USB sticks with instructions and software I provide will be openSUSE live sticks for the next workshop, as I can't expect everybody to have Linux by default on their laptops. And of course you can use Windows or Mac if you really want and are up for the challenge, I don't mind including instructions for these platforms. But nobody got them working yet so expect some struggle.

You will also need a laptop with a working ethernet port, no amount of creativity has been enough to work around the physical limitation of not being able to plug in the other end of the networking cable...

Besides the satisfaction of getting ownCloud running, the first to succeed earns the Banana Pi they worked with!

If you can't make it to Berlin, I will give workshop later this month also in Dubrovnik (see my earlier blog) and if you'd like to have this workshop close by - let me know, perhaps we can arrange something.

Why open source works

Trying to explain why open source works, you can of course point to the Cathedral and the Bazaar by Erik. But the kernel development process shows it happening 'in real time', every day, and that's a major reason why I so enjoy reading the weekly LWN.

Competition FTW

A recent article explained how two patch sets are developed, impacting the Huge TLB feature. One is about improving reference counting in the handling of huge pages to allow a huge page to be mapped in both 'huge page mode' and in 'normal page mode' by different processes. This is a first step towards being able to use transparent huge pages with the page cache, giving it access to the performance benefits of huge pages. This has been in the works for a while as it is a very complex change. But an alternative has recently surfaced, adding transparent huge page support to the tmpfs filesystem. This also has pieces needed to support huge pages in the page cache, but in a very different way.

Why on earth would you want developers to spend so much time on two competing solutions for a problem? Isn't there One to Rule Them All?

Finding that perfect solution

I am sure that there is a Perfect Solution. I would simply suggest that nobody knows it! Like in economics, where an equilibrium exists but no single person can define it, software development has become complicated enough that competition of ideas often leads to the best (or at least better) solutions.

Companies, building their own cathedral in-house, would put product managers and developers in a room. Let them come up with the Perfect Solution. But they won't, because of Joy's law:

"No matter who you are, most of the smartest people work for someone else"

Impressive results

This is the key to why open source works and works so well. No single development project in the world even approaches the size of the Linux kernel project - no less than 1400 people contributed in under six weeks. With millions of lines of code developed by tens of thousands of people over 20 years time and with an extremist attitude towards backwards compatibility, you wouldn't expect the trend to be "go faster". It is.

Bringing over almost 12,000 different people from 1,200 different companies together in the last decade, the kernel competes with Wikipedia for the title of humanity's largest collaborative effort. Oh, and I'm sure the masses building the pyramids were big, too, they just didn't integrate the experience of all their collaborators into its architecture as well as the kernel and Wikipedia do ;-)

Cookie licking

Many other projects struggle to learn from the kernel's success. Success which has many aspects, one of which exemplified here, a rule Frank sometimes talks about: no cookie licking!

In short, cookie liking is preventing others from working on something by claiming you are (but not really making progress). Extending it a bit, I would say that, while collaboration is good, if you feel you can do better -- or just want to sample a different approach:

"never be deterred to build an alternative solution to a problem"

Because it's space for competition within projects that makes open source work better.

LAX, SCALE, KDE, SUSE, GNOME and ownCloud

Lobby of the venue

Back home. Tired and jetlaggy, but satisfied: SCALE rocked!

SCALE loves ownCloud

The 13th South California Linux Expo was awesome! It is the biggest LinuxFest in the USA. While decidedly different in nature from Europe's biggest Linux event that that took place just three weeks prior (FOSDEM), we met similarly enthusiastic existing and future users. Conversations were also similar: about half the visitors already knew ownCloud, often using it or planning on deploying it; and the other half was more than a little delighted to hear about it, often exclaiming they had been looking for 'something like that' for a while. Negativity was extremely rare: I don't recall a single negative comment at SCALE (merely a few people who liked ownCloud but had no use for it personally), FOSDEM had one conversation starting unpleasantly but quickly turning around - even though one feature of ownCloud wasn't up to snuff, the user was happy with the experience as a whole.

Before the action started!

For most users, ownCloud was simply a wonderful product and they used it at home, deployed it for customers or managed it in their company. Some asked what features were coming or just arrived in ownCloud 8, or asked about the state of specific features and in more than one occasion they very enthusiastically told me how excited they were about ownCloud, how they loved it and how they were telling everybody to use it!

ownCloud to-go

Those who didn't know ownCloud were almost invariably surprised and excited. I can't count the times I heard "wow, why did I never hear about this before" and "dude, I've been looking for something like this for ever!". Often, people wondered how long ownCloud had been around (we just turned five), if it was open source (yes, with love), how many people contributed to it (719 and counting) and how many users it has (we guestimate over 2 million, with 500,000 in this single deployment alone). Oh, and, does it scale? The deployment linked above and a mention of users like CERN can put most concerns to rest. Yes, ownCloud scales from Raspberry Pi to Atom Smashing size.

What came up a few times as barriers to their future usage of ownCloud was pretty much what I discussed before. Running a server at home is not easy and I walked by the EFF booth to ask about progress on Let's Encrypt to ask about the progress of solving one aspect of that problem: more easily getting SSL certificates. I was told the project is on track for the 2nd half of this year.

Frank and Bryan Lunduke

It is wonderful to have such energizing, positive, enthusiastic users - and to have such an enthusiastic booth crew to talk to them as well. At the booth we had Frank, Matt, Ron, Camila and myself. Awesome it was and we had great fun! Below a timelapse video of Saturday morning. It was still rather quiet but it is nice to see us jump around!

Stuff and talk

Just like at FOSDEM, we brought ownCloud stickers, hand outs explaining ownCloud to users and developers as well as some posters for the booth and pins to give out. This was all very much appreciated - I estimate we gave out about 400 hand outs and 500 or so stickers as well as about 50-100 pins.

Sunday at 3PM, I gave a talk about Privacy and ownCloud, with Frank finishing off with a section about his talk at MIT where he discussed ownCloud's Federated Cloud sharing feature and where it is going. The talk was well received; I think the angle I took to privacy (inspired by my background in psychology) spoke to the audience and Frank's description of federation and how it's done in ownCloud was very interesting. owncloud.org and owncloud.com will feature blogs with some more information about this soon.

Friends

Big, big booth!

I also walked by the booths of 'old friends' - the openSUSE/GNOME/KDE crew in particular, it was awesome to meet them. Some I hadn't seen in years, others I met for the first time. They did an amazing job and richly deserve the reward they earned for most Stunningly Amazing Booth Crew (don't know the real name of the booth award but that's what it should be). If you think that 'just' GNOME an KDE being incorporated in the openSUSE booth isn't enough - Master Planner of the Booths Drew aims to bring in Enlightenment and XFCE as well next year. Supposedly a Trello board has been set up already. I bet it won't be long before it has grown to the point where the SCALE organization needs to give the 'openSUSE booth & friends' a separate hall at SCALE...

I have to note that it was thanks to our green friends that I could hang up the ownCloud flyers - they lend me some (green!) tape to do that.

The KDE booth had a bunch of terribly cool stickers (I only now realize I forgot to get one for myself!) as well as the "frameworks 5" flyers. I could only bring, like, 5 t-shirts and a dozen old 'join-the-game' flyers so I'm glad Bert Yerke and his wife, who formed the awesome local KDE team, had created the other materials. We already discussed 2016, as they have plenty of ideas on how to improve the booth!

Awesome stickers...

If you, dear reader, want to help out at the KDE or ownCloud booth next year - let me know, either in the comments or by mail. I can promise you: it is awesomely fun and by far not as scary as you might think! Bert and Matt and everybody who has ever been at a KDE, openSUSE, ownCloud or other FOSS booth can attest to that: it is a great way of getting involved and making a big difference!

Bonus points for who finds a suitable meaning for the one item in the title which isn't yet an acrynym ;-)

FOSDEM 2015

FOSDEM was crazy.

ownCloud booth between Diaspora and Tor

Regulars know it always is but it somehow managed to surprise me still.

The ownCloud booth was pretty much continuously swamped with visitors and even with 3-4 people there almost all the time, we could barely talk to everybody who had questions - let alone approach people who might have. So, sore feet, but still a blast. My biggest regret is that I barely left the booth to check out other projects I love. Luckily, ownCloud was flanked by awesome projects!

Diaspora, Tor, KDE, openSUSE...

I had a chat with the Diaspora folks at the booth next to us and greatly look forward to the upcoming release. They had a nice flyer-y paper which also included some development stats with the number of active contributors and such, a very useful thing to have so you can quickly see how a project is doing. Diaspora had a hard time since the crazy start, but things are picking up again and 66 people contributed to this important project over the last year.

I only visited the KDE booth a few times for a chat and to check out the new t-shirts (we brought a bunch of the 'old' style t-shirts with us and Ovidiu brought the new ones), not much different from openSUSE. I did talk to the usual subjects and met Douglas. Great to hear the merger of Factory and Tumbleweed is working out very well.

The Open Build Service continues strong and it was interesting to hear some thoughts on a conversation with Debian developers. They are (finally?) also working on reproducible builds, being bitten by some of their processes. I mean, if you become a Debian developer, you're nearly root on all the build systems and have a lot of fun with all the supported architectures. Compared to the robust, transparent processes and easy dealing with multiple architectures and different distribution versions on OBS - it seems a pointless struggle to me. It's not like OBS is evil, proprietary code, nor obscure and barely maintained! But I guess change is hard in such a bureaucratized organization.

Oh, and I look forward to the openSUSE Conference in den Hague! There is good progress with the organization of the event. Perhaps some Debian core developers should participate and have a look at OBS before Lennert solves the problem in a way that makes packaging irrelevant ;-)

Tor - darn, despite standing next to them, I didn't have time for a good conversation and I'm glad some FSFE friends walked past our booth or I wouldn't have known about the FSFE's upcoming valentine campaign I hope we can participate in!

marketing material package assembly

ownCloud

I mentioned our booth was busy. Wow. We have been creating some marketing materials and this was a bit of a test run. I send out materials in envelopes, and I thought - "let's bring 8 packages and see how they fit. Those left can be taken by the other team members to other events."

Result:

• stickers were out (from all packages!) by 3PM on Saturday, flyers on Sunday morning. I need to re-assess the configuration of the packages.
• The developer and user flyers are not easily (or at all) identifiable as such. And the usage of red on the developer flyer triggers people to grab that one first. Looks like my education as psychologist failed me here, I should have known!
• Everybody who had a look at the user flyer could tell me what ownCloud was about - the contents are good.
• People are baffled by the fact that a PHP app that runs on a Raspberry Pi can scale to +500K users at universities or organizations like CERN. We can explain this better!

So, both in terms of organization and contents of the marketing materials, some good ideas on improvements.

Besides the materials, the conversations were mostly 'the usual': incredibly up-beat. Of course, nothing is perfect but the vast majority of visitors was very positive and enthusiastic about ownCloud. Over half had it running and was curious about the upcoming release, while the other half either already knew about it and was planning to play with it, or was very delighted to hear what it was as they had been looking for a way to get out of the proprietary file storage world.

... rulez

I swear, if the average citizen was anything like the FOSDEM visitor, the ownCloud user base would be twice the size of the proprietary clouds combined already... Of course, the world isn't like that and ownCloud is has to overcome some hurdles still.

Then again, if you see the recent announcements like OpenCloudMesh, universities and public research institutions around the world are diving all-in on ownCloud. At FOSDEM, two staff members from the ULB IT team (where FOSDEM takes place) came by the booth to share they were experimenting with ownCloud for staff and students and I talked to many employees and owners of a variety of companies busy testing or deploying ownCloud.

With the quickly growing number of ownCloud providers, perhaps the future for ownCloud (and data ownership for consumers and businesses alike) lies in such deployments, rather than people running their own servers. This is already a huge decentralization compared to everybody on one of the 4 or 5 clouds from Google and friends, which would help our privacy and security.

If things continue like this, the number of 2 million ownCloud users will have to be revised up to triple that before 2016 is here. I'd welcome that!

First Berlin meetup of the year!

Next week will be the first Berlin ownCloud meetup of 2015. As the ownCloud Desktop client team is having a code sprint hackathon meeting get together at the office, they'll be at the meetup. So if you wanted to hack on the client, fixing or adding something and live in Berlin - join at the meetup!

The meetup in February I unfortunately can't join but we've already decided that the meetup in March will be All About Devices™ - we will have some Banana Pi, Raspberry Pi etc devices around to play with, do some performance testing and optimization and see where we can take 'em! If you're interested, you can RSVP already.

See you in Berlin?!?

If you're not in Berlin and are sad that you don't meet ownCloud people - check out our events pages to see what can be done about that!

What's Holding ownCloud Back?

In the recent article about the ownCloud event program, I pointed out that while ownCloud has 2.5 million users, it is a drop in the ocean looking at the number of Internet users (a little over 4 billion today). The announcement of the "Let's Encrypt" initiative from the Electronic Frontier Foundation, Mozilla and others prompted me to write this: it is one step in the direction of removing the limitations holding back wider ownCloud adoption. What does the future hold?

Easy ownCloud

As Frank pointed out in his blog on the future of PHP, ownCloud has ease of use as an explicit and very important goal. And while the technological choices made aren't always so exciting and bleeding edge, they do result in ownCloud being very easy to deploy on a very wide range of devices. Plenty of tutorials exist showing it running on everything from Rasberry Pi devices to big iron at organizations like CERN, where physicists looking for the origins of the universe are routing hundreds of terrabytes of data through their CernBOX build on ownCloud, sharing and collaborating on the data analysis.

Limitations

Unfortunately, there are limitations outside of what ownCloud can directly control.

In the database area, SQLite is default because it requires no manual setup whatsoever. But performance suffers when an installation has more than a trivial amount of data. When sharing with more than 15 users or indexing your mp3 connection, SQLite usage leads to frequent time-outs and other issues!

Another, more serious issue, is the architecture of the current internet. Most users are set up at home behind a firewall provided by their internet router. While this provides some additional security, it is mainly because the limited number of unique addresses available in the still widely-used 'IPv4' protocol. It simply is impossible to assign a unique address to each device connected to your internet at home. But this means your server will not be reachable when you're not home, unless you adjust some settings on your router. While we can configure some routers automatically, most we can't and as every router is different, an easy 'generic' how-to can't be provided either.

A third issue is that an ideal ownCloud platform would be small and cheap devices like the Raspberry Pi, but these are almost all based on 32bit CPU's. Due to technical limitations in the platform ownCloud builds on, this means you won't be able to have it handle files bigger than about 4 gigabyte! That is a big limitation if you'd like to store your virtual machine or Blue Ray collection on your ownCloud.

The fourth issue I see is security. While not the biggest problem of the three, setting up a server to be secure, including a decent SSL certificate, is not easy. I personally couldn't figure it out and while I'm new to server things, I am not a technology hater by any means. My parents wouldn't ever be able to figure it out and more importantly, they wouldn't want to!

Solutions

These four issues to wider ownCloud adoption aren't the only ones, but as far as I can tell, the biggest. So how do we deal with it?

There are several routes to an even easier ownCloud installation. Having a pre-setup operating system in the form of a container (Docker?) or a virtual machine can take care of much of the trouble around database setup and help a lot with the security issue. However, it can't run on light hardware like a Raspberry Pi and doesn't deal with the file size problem.

When it comes to the address limitations, the internet is slowly transitioning to IPv6 which will provide more unique addresses for each person than IPv4 offered in total (see here how Google explains IPv6). So, essentially, we just have to wait for this problem to be solved.

The hardware problem is also working on solving itself: the upcoming new swath of ARM CPU's (and Intel CPU's targeting the embedded market) are fully 64 capable so while current-gen Raspberry Pi devices (and other embedded devices like routers!) aren't perfect for ownCloud, a year from now many new devices will be perfectly capable of providing a great ownCloud experience.

The Electronic Frontier Foundation's "Let's Encrypt" initiative offers a (partial) solution for the security issue. Without it, a pre-configured ownCloud system will most likely be set up to use a self-signed certificate. While secure in principle, it always warns visitors of the self-signed state and thus isn't ideal. Let's Encrypt provides an automated and more importantly free (in terms of cost) solution for this.

And now

While I'd love for all these changes to be implemented yesterday, in reality we simply have to wait for the transitioning to IPv6 and 64bit CPU's. In the mean time, we can already start working on integrating Let's Encrypt into virtual machine and Docker images with a pre-configured MySQL (or MariaDB) and perhaps recommend people to run them on a 64bit capable system like a modern NAS or a NUC. The ownCloud-in-a-Box image on SUSE Studio is a great start!

Meanwhile, getting ownCloud ready to run on a wider range of devices and perform a wider range of 'cloudy' functions like running as backend of the Chromebook devices (see this page and ping me if you want to get involved) should be on the agenda as well. I personally look forward to more 'social' integration in ownCloud, like the ability to comment on images or other data and share these with the people you share files with. We're on it, tags sharing is integrated for ownCloud 8 and a generic metadata repo was created (empty still). Get involved if you can!

Obviously, telling people about ownCloud is still important - which is what the ownCloud event program is all about - and help is welcome. Go to owncloud.org/promote and share the love!

How else to help out

Yesterday I blogged about how to help testing. Today, let me share how you can facilitate development in other ways. First of all - you can enable testers!

Help testers

As I mentioned, openSUSE moved to a rolling release of Factory to facilitate testing. KDE software has development snapshots for a few distributions. ownCloud is actually looking for some help with packaging - if you're interested, ping dragotin or danimo on the owncloud-client-dev IRC channel on freenode (web interface for IRC here). Thanks to everybody helping developers with this!

KDE developers hacking in the mountains of Switzerland

Coding

Of course, there is code. Almost all projects I know have developer documentation. ownCloud has the developer manual and the KDE community is writing nothing less than a book about writing software for KDE!

Of course - if you want to get into coding ownCloud, you can join us at the ownCloud Contributor Conference in in two weeks in Berlin and KDE has Akademy coming just two weeks later!

And more

Not everybody has the skills to integrate zsync in ownCloud to make it only upload changes to files or to juggle complicated API's in search for better performance in Plasma but there is plenty more you can do. Here is a KDE call for promo help as well as KDE's generic get involved page. ownCloud also features a list of what you can do to help and so does openSUSE.

Or donate...

If you don't have the time to help, there is still something: donate to support development. KDE has a page asking for donations and spends the donations mostly on organizing developer events. For example, right now, planet KDE is full of posts about Randa. Your donation makes a difference!

You can support ownCloud feature development on bountysource, where you can even put money on a specific feature you want. This provides no guarantees - a feature can easily cost tens to hundreds of hours to implement, so multiple people will have to support a feature. But your support can help a developer spend time on this feature instead of working for a client and still be able to put food on the table at home.

So, there are plenty of ways in which you can help to get the features and improvements you want. Open Source software might be available for free, but its development still costs resources - and without your help, it won't happen.

Why developers should not be testing

Short answer: because you should.

When somebody asks about their missing pet feature in KDE or ownCloud software, I always trow in a request for help in the answer. Software development is hard work and these features don't appear out of nowhere. There are only so many hours in a day to work on the a million things we all agree are important. There are many ways to help out and speed things up a little. In this blog I'd like to highlight testing because I see developers spend a lot of time testing their own software - and that is not as good as it sounds.

Developers also do testing!

You see, developers really want their software to be good. So when a Alpha or Release Candidate does not receive much testing from users, the developers take it on themselves to test it.

Developers testing software has two downsides:

• Developers tend to test the things they wrote the software to do. It might sound obvious, but usually the things that break are things the developer didn't think off: "you have 51,000 songs? Oh, I never tested the music app with more than 4,000" is what I heard just yesterday.
• And of course, it should be obvious: early and lots of testing speeds up development so you get those features you want!

Take two lessons from this:

• If you want things to work for you, YOU have to test it.
• If you want those other features, too, helping out is the name of the game.

It isn't hard

In the past I wrote an extensive article on how to test for KDE and ownCloud, too, has real nice testing documentation.

If you want to get on it now, Klaas Freitag just released ownCloud client 1.7 alpha 1 and openSUSE has moved factory to a rolling release process to make it easy to help test. KDE Applications 4.14 is at the third beta and the Release Candidate is around the corner.

Your testing doesn't just save time: it is inspiring and fun. For everybody involved. For added kicks, consider joining us at the ownCloud Contributor Conference in in two weeks in Berlin and KDE has Akademy coming just two weeks later!

Help make sure we can get our features done in time - help test and contribute your creativity and thoughts!


note: I'm not argueing here against testing by developers, rather that users should help out more! Of course, developers should make sure their code works and unit tests and automated testing are great tools for that. But I believe nothing can replace proper end-user testing in real-life environments and that can only really be properly done by end users.

ownCloud 7 awesomeness

One of the things I love from the Free and Open Source software world is that doing things in the open simply leads to better solutions. Resources are often constrained, polish might be lacking, but frequently from the seemingly chaotic processes emerges brilliance.

The upcoming ownCloud 7 has one of those things: server to server sharing. You see, for a long time, I and others have been asking the ownCloud desktop client developers for a feature: support for syncing multiple ownCloud installations. That way, files from our corporate ownCloud installation (of course we dogfood here at ownCloud Inc.!) and my private ownCloud could both be synced to my desktop and laptop. Unfortunately, while it has been on the todo for a while, it just kept pushed down by more urgent feature work.

While you could get it to work already by running multiple clients and playing with the config file locations, it seemed a bit brittle to me and I just accepted this feature wasn't there yet.

Only today, during my bike ride to c-base, did I realize that ownCloud 7 actually introduces this feature. And I even wrote a sneak preview about it! It is the server to server sharing that solves this issue.

What is Server to Server sharing?

Let's step back. What is the major thing that 'private' clouds don't have which public clouds do? Other than the NSA snooping, that is...

Well, you're all alone, of course. If you want to share a file with another student from uni, you have to create an account or use a shared link. He/she will then have to visit your ownCloud to be able to work with you. It gets quickly messy with a lot of files.

This is where Server to Server sharing comes in. You can simply share a link and your collaboration partner can add this, either a folder or a single file, to his or her own ownCloud. They can put the file wherever they want in their folder structure (we've gotten rid of that rigid shared folder concept in ownCloud 7!) and work with you like the file was on their own instance.

That means they can also locally sync the file with their sync client by just putting it in a folder that is synced!

Boom. You don't have to create and manage multiple server accounts in your sync client, creating a folder on your system for each server you work with. No, you just add the share to your ownCloud and put the files wherever you want - in one folder, or grouped by subject - whatever works best.

Now tell me that isn't awesome ;-)

There's more

Of course, this is just a first step to bringing ownCloud servers closer to each other. Our goal is full 'federation' of data: transparent sharing between servers so they can act as one cloud, protecting privacy while giving you the convenience of sharing, collaborating and communicating with friends, family, collegues and more. Once you connected to another ownCloud, you should be able to share files with the users on that server seemlessly - and other data, like contacts or music! You should be notified when things are shared (a news feed!) and be able to comment and chat. This however still needs work. If you're interested in helping to make this happen, consider joining us at the ownCloud Contributor Conference from August 26-31 in Berlin, Germany!


* Note that you can have kind-of server-to-server sharing with ownCloud 6 by manually mounting folders from other servers via webdav. This of course comes with various limitations (you do have to give out webdav credentials, to name one) so the new server to server sharing is a massive improvement...