Five Reasons To Upgrade Your ownCloud

On our user mailing list users occasionally report problems with quite old ownCloud releases. Often community members like Chris suggest to upgrade to a newer major version and use official repositories rather than those provided by distributions like Debian. That is good advice. I'll share 5 reasons why an older version isn't more stable and talk about the issues with distribution packages in a follow-up blog.

Older Releases and Stability

The oldest ownCloud release supported, according to owncloud.org/security, is ownCloud 7.0.x, with x currently being at 12. That is, this release has had 12 updates fixing stability, performance and security issues. One could thus argue that this release is more stable than newer releases like 8.0.10, 8.1.5 and our latest stable, 8.2.2.

There are five reasons why that is not really true.

openCloudMesh brings ownCloud to research

1. ownCloud Grows

The rule-of-thumb that an older release has had more use and thus issues have been shaken out is not really true with ownCloud.

The ownCloud user- and developer community is growing all the time. ownCloud 8.2 will have more users in its lifetime than 7.0 had, which had far more than, say, 5.0 ever did and so on. At some point after their release, these newer versions will already have had more users and thus more potential discovery of obscure problems than their older, still supported counterparts. If you're interested in quantifying this, we try and give an idea of our estimated user base in our time line of ownCloud history. There'll be an update early next year with our user estimate as of today, but count on at least double the number of last year.

2. Testing Continuously Improves

With the growth of ownCloud's user and developer community also come more tools and processes for testing. For example, during the 8.0, 8.1 and 8.2 development cycle we've increasingly introduced automated testing provided by the CERN-developed Smashbox tool, which is now routinely used to determine if there have been any regressions in complicated syncing and sharing scenario's. Besides Smashbox, other tools have been added to the roster and manual testing has been improved significantly as well. Older releases have simply not had the benefit of this testing and thus there is the chance of corner case issues still lingering.

3. Back Porting is Limited

Due to many users running recent ownCloud versions and the continuous improvements to testing, most bug are initially found and fixed in the latest or second-latest ownCloud release. From there they are back (or forward) ported to the others, that is, integrated in older releases. Due to the large changes in each ownCloud release, integrating fixes far back often makes little sense and generally speaking, we backport to the latest stable ownCloud version and the one before. Of course, security fixes and fixes for very severe or very simple issues are often brought all the way back to the oldest supported release.

4. Clients Take Advantage Of Server Features

The various ownCloud clients for desktop and mobile operating systems are developed alongside, though not in lock-step with the ownCloud server. Various features which improve reliability and performance of syncing require both client- and server side changes. Thus, running a newer ownCloud client with an older server means you miss out on features which could help protect your data or at least save you from getting some conflict files. For example, there's work going on to have checksums on files, a precursor to the much requested ability to sync file changes rather than the entire file. This deals with the rare but not impossible cases where files get mangled while in transit or on storage.

5. New Features Improve Reliability

The checksum feature coming with ownCloud 9.0 is not the only 'pro-active data defense' improvement to ownCloud. We will introduce features like detecting apps which break ownCloud and code integrity checking. Earlier we introduced file locking, experimental in ownCloud 8.1 and enabled by default in 8.2. This protects files from concurrent changes which in rare situations could result in errors or even data loss. Especially for large, enterprise installations, these situations might not be that rare due to large numbers of users simultaneously accessing the same data and thus the benefits of upgrading includes getting rid of an entire class of impossible to reproduce issues.

So What Version Should I Run?

Above, I gave five reasons why near outdated ownCloud releases do not tend to be any more reliable than newer ones. Rather, the opposite is true, as newer versions get more scrutiny and thus have more issues found and fixed and have received new features which benefit the reliability of both server, client and their interaction.

Home Users

For home users, I still recommend to run the stable or plain latest release channel. On the whole, the benefit of new features and performance improvements in an up to date release far outweigh the stability advantage of older versions, especially if ownCloud is ran in a typical (LAMP) setup. We release ownCloud versions only after extensive testing and the vast majority of issues found shortly after a release is related to either scalability for large instances or non-standard environments like enterprise databases, caching solutions and so on. Most testers and developers use ownCloud on a recent Linux/Apache/MySQL/PHP setup and thus you can expect PHP 7 on a bleeding edge Apache to be surprisingly reliable, even when compared to a old Debian release. Note that you should test yourself if you really want to be sure that an upcoming release works smooth for you!

Enterprise

If you value stability above all else (that is, over features and performance improvements in newer versions), it is best to track ownCloud releases with a N-1 strategy: upgrade to one release before the latest about 1-2 months after a new version comes out. That is, it would be about time to upgrade to ownCloud 8.1 by now and when 9.0.2 is made available, now to be expected some time in May, 8.2 is your best bet. If you want to be sure the new ownCloud version runs great on your infrastructure and the upgrade goes smooth with your setup, I strongly suggest to get involved in testing ownCloud. It provides the best and only guarantee it works for you™

Of course, IF you value stability to this degree, you are most likely an enterprise user and you should seriously consider getting in contact with ownCloud, Inc. which can help you decide far better than a blog post what version is perfect for you. Besides advice, support and deployment tools, you get a heads-up on security, stability and performance related issues (and work around solutions) and of course have access to our enterprise features.

If you are still running ownCloud 7.0.x and it is running satisfactory, I can imagine you don't want to change. That is fine for about another 2 months (until 7.0.x is deprecated). However, if you experience any trouble, any advice other than 'upgrade' is probably unwise: I don't think it is worth the trouble of trying to fix issues with 7.0 when you will have to upgrade to 8.0 in a few months anyway.

I know, upgrading can be a pain, it is work and all that. But so are problems in old versions of software you're running and even more so is security. We're working on a new upgrade process for 9.0.

And realize you don't do your users a favor by keeping software 'the same'. "Big Bang" releases steepen the learning curve by making users swallow too many new features, and increase the likelihood of compatibility issues with other systems in the environment. Much of the web and apps (especially on mobile) is moving to faster release cycles for this reason.

In all cases, use ownCloud from the official, ownCloud-provided repositories you can find on owncloud.org/release-channels. I blogged about how to install ownCloud (packages, VM, zip files etc) here.

We're coming to FOSDEM and SCALE!

The conference season 2016 is coming. ownCloud will be running booths at the largest events in EU and the USA: SCALE 14x in LA and FOSDEM 2016 in Brussels!

  

SCALE 14x

The 14th SCALE moves to a new location in Pasadena, promising to be bigger and better than ever. I'm hoping the much greater distance from the LA Airport will be worth it, but I'm not taking a chance: I want to be there. Like last year, Matt McGraw has committed to helping out, Frank can unfortunately not make it. We could thus certainly use a third helper at our booth!

As Matt can attest, it is not difficult - and certainly a lot of fun to tell people about the awesome that is ownCloud and help them if they have questions. You don't need to know ownCloud inside out to be able to help, many questions are repeat questions you'll quickly learn to answer and at least half the time you're just there to pitch ownCloud or hear how much people love it.

If you're up for it, let me know!

I'll be giving two talks, one about ownCloud on embedded devices like Pi's and one in which I attest you're not as clever as you think.


Timelapse video from the booth last year ;-)

ownCloud booth between Diaspora and Tor

FOSDEM

Last year, we had a real good time at FOSDEM. We were sandwiched between Tor and Diaspora, a great spot to be in - privacy corner, so to say. We had massive numbers of people come by, ask questions, give comments - it was real great to talk to so many people.

This year, I'm counting on the same busy time. We've got a booth again, don't know where it is yet but we'll have a team to answer any questions you might have. As always, if you're up for helping out, let me know!

Western Digital Labs and ownCloud

We published a blog post about a collaboration between WDLabs and ownCloud. WDLabs is, in their words, "a division of Western Digital focused on accelerating new solutions". I'd take that as "they look for more ways to sell hard drives" and they have been playing with some interesting stuff. One such thing is their PiDrive Kit. This kit includes a connector cable which powers both the harddrive and the Pi from one power supply, simplifying the setup. We're now working with WD Labs to build something from this starting point.

WD Labs, meet ownCloud

When they approached us a few weeks ago, they told us they had been toying with a more complete kit with bundled operating system based on berry boot. This lets you choose an OS on first boot and they had put together an image with ownCloud as one of the options. They thought it was really cool and asked if we felt the same and had any ideas of what we could do with it. Well, sure, I wrote not too long ago about What's holding ownCloud back and being able to offer potential users a plug-and-play device with ownCloud on it is something we'd love.

So did they. We quickly received a prototype of a kit with Berryboot and ownCloud on it as well as a case and a Pi included. After some more discussions, we developed a plan with which we hope to get some community help to make this happen.

We need your help

You have to know - this is not something ownCloud, Inc. is very much involved with. It simply is not a commercial endeavor - Inc. works for enterprise customers and we haven't found any of them yet running ownCloud on a Pi. So, while WD Labs will take care of the hardware and has committed themselves to doing a serious number of ownCloud branded devices complete with pre installed OS in the first quarter of next year, it will depend on our free time and community help to develop the OS.

And there's serious work to be done. Berryboot can be configured to boot a headless ownCloud but right now, to even boot, you have to connect a keyboard and monitor. And booting isn't even the issue - finding and configuring an IP and then connecting to the Pi is. And once you've done that, the system has to be configured to be accessible from outside so you can use the ownCloud clients on your mobile phone and laptop to always find and connect to your ownCloud and give you the ability to share with friends and family.

There are folks in our community who have experience with this, building our community VM for example (hi Daniel!) and we'd like to solicit their help. We have 10 prototypes to send out to people who want to help build this ready-to-go ownCloud, the blog has more details on how to get one. And if there is a lot of interest, we'll get more prototypes put together, though I think it'll be tight to get them out before Christmas.

Note that it is totally cool, even advantageous, to send in a proposal ad a team. Together you can get more done!!!

Of course - without this kit there is still a LOT you can do to help out and there is no reason to wait for us sending out anything! Help get a discussion started about what this should look like, how to get there and who does what. Subscribe to the developer mailing list and get started!

Future

Now we know the Raspberry Pi 2 isn't the very best device to run ownCloud on, in terms of performance. It won't run with 100 users, indeed, though it isn't too bad for a few users at home. But, aside from the fact that there is a lot of room for tuning (PHP 7, for one, should help a lot, see some stats here), we're not married to the Raspberry with a single hard drive. It is a popular device and thus a great place to start but WD Labs has already shown us prototypes using a Rasberry Pi Compute device and 2 hard drives and we could work with devices like the Banana Pi or more powerful boards, too. That won't happen immediately, it simply depends on how well the first version turns out and the interest from users.

That means it matters a lot now: can we make this work? I'll sure put a portion of my free time in this as I'm quite excited about it. You too?

Home Automation, AI and the Crownstone

Waiting for that future where you're dressed automatically? I'd rather do that myself and don't have a Roomba yet but as tech marches on, we'll get more automation in our homes.

With more and more 1984, Brave New World and Farenheit 451 being brought to this century by our esteemed leadership and our corporate overlords, I'm hoping that this tech will be ours, rather than theirs.

Enter a nice player in the "automate stuff" arena: Dobots. In their own words:

Our goal: Really smart buildings. Really smart robots. We're a startup in AiTech with applications in CleanTech and GreenTech.

What makes them nice is that they are real Open Source folks, putting what they code on github under a free license and working with and building on open tech.

They previously did industrial cleaning robots and other stuff. Currently, they are running a kickstarter for a smart power outlet as a first foray into home automation. EDIT: they've instead decided to, for the first run, only create EU plugs as there was little interest in the US. You can order them on the crownstone.rocks website.

Enter the Crownstone

How is AI relevant for power outlets? More than you'd think. One example is SLAM. SLAM (Simultaneous Localization And Mapping) tries to answer two key questions in Robotics:

• Robot Localization: “Where is the Robot?”
• Robot Mapping: “What does the world around the Robot look like?”

(read more about SLAM here)

Dobots has turned this around in the crownstone, using these algorithms to map your house and localize people in it. Yes, the Crownstones know where THEY are and where YOU are. While not exactly a localization tech, the Crownstone uses Bluetooth LE for this so it works with pretty much any device. And the best part: their localization code is, in Javascript, to be found on github!

This means you need no motion detectors or cameras to know where people are in the house, keeping things simpler and cheaper. But Bluetooth LE is used for managing the Crownstones, too. Rather than turning to WiFi or (expensive) central control hubs, Dobots lets the Crownstones automatically create a mesh network. Your smartphone acts as control hub (apps on github, of course) though you can also take a Raspberry Pi, plug in a bluetooth usb module and have it act as hub. That way, even when you're not at home, you can control devices or get notificationsthat, say, your television, computer or other devices are getting unplugged. It uses XMPP and WebRTC to get through your firewall and guess where you can find the Cordova based Raspberry Pi app...



But there's another innovation in there which helps avoid extra devices and complicated configuration: extremely fine grained power usage monitoring. Yeah, it can tell you how much power your fridge and TV use but more importantly: it can automatically detect which device you connect to it! So when you leave the house, it can kill power to your tv but not your fridge... I've been told they think they eventually will be able to figure out different makes and models of laptops based on their power signature. Now is that cool, or what?

The algorithms to do this are currently Matlab code but once finalized, they'll be on github, too.

Dobots regularly works with students who get to play with their tech and come up with new, innovative use cases. And this works wonders. Just some results from a recent hackathon:

• Team 1 developed ChildLock. Devices are only turned on when adults are in proximity.
• Team 3 developed Start VR. Virtual reality that allows you to picture your own furniture in an Ikea store.
• Team 4 introduced Never Lose. Lights indicate for elderly people where they have lost items with iBeacons.
• Team 5 used Crownstones to indicate the way to store employees.
• Team 6 worked on Tommy. An AI that analyzes patterns of daily life to combat loneliness.
• Team 7 developed Any Morning. Your phone using the Crownstones guides you to your morning routines to make you leave your home on time.
• Team 10 implemented Tipspromenad. Kids have to find objects in a place combined with solving puzzles for fun!
• Team 11 developed SpotOn. In emergency situations lights indicate how to flee a building

All together

The Crownstone has some nice advantages over the competition:

• Functional. Dimmer, iBeacon, a current measuring device, a standby killer and more in one.
• Cheap: you get 2 ready-made or 3 do-it-yourself for Eur 75 but there's no need for a hub, motion sensors or other stuff.
• Open. Using open protocols and code on github and I expect interesting applications to come from the community.

I did an interview with their COO - for more background, read it on LinuxVeda.

So I say - if you are looking to start with some basic home automation, go and get some Crownstones and get started!

ownCloud Server 8.2 is here

And on time following the new three month release cycle! The previous release, ownCloud Server 8.1, was all about stability, performance and security. Now, more work has been put in the user experience. That shows in the new sidebar, the new gallery app and refreshed style but for system admins, there's a large number of new control points as well.

Gallery app

For end users, the design improvements are nice but the new Gallery app is truly a transformation. It is essentially the 'Gallery Plus' app, developed by Olivier Paroz. It is essentially a full rewrite based on the new app framework, after which lots of features and performance improvements were made. The app has been around for a while and Olivier has solidly proven himself as maintainer so I'm really glad this app is now a part of the core ownCloud experience. Below is Oliviers' talk from the ownCloud Contributor Conference, introducing the new Gallery to the community!

Notifications

The new notifications are cool, too - only one app supports them, for now, and that one isn't there by default so you won't see notifications until apps get updated. But as soon as they get adapted to the new API, accepting share invitations or getting notified of changes becomes easier and nicer. If you want to play with the notifications right now, go get the announcementcenter app and send some news to your users to see how it works!

New Appliance and Proxy app

Technically not new, as both were announced at the conference and made available since then, but super awesome: we have an official ownCloud appliance. This will make installing ownCloud a lot easier for a lot of people - and bring back some kind of Windows Server support, too.

On top of that, the appliance contains the ownCloud Proxy app, which is really great for lowering the bar even further. Simply put, ownCloud Proxy enables you to take a laptop and an Internet connection, connect to your privately hosted ownCloud from anywhere, without requiring you to make any changes to your local network settings – no router configuration, no DNS entries, no domain name registration. The ownCloud Proxy service relies on a partner who provides the service for a fee. Right now, there is one partner, pageKite. It would rock if somebody else saw business in this, too, of course!

Now, go, go and get it. Note that the VM isn't available as of right now (14:05) because it's still building... An 8.1.3 based one is there, though.

Help ownCloud rock SCALE and FOSDEM in 2016!

FOSDEM and SCALE are respectively Europe and North America's biggest FOSS events and, of course, we'd love to run a booth there again. We had a good time last year, just check out see my overview blog and detailed blogs about FOSDEM and SCALE. It is time to start preparing again to have as much fun and impact as last year!

FOSDEM

For FOSDEM we will request a booth again and like last year I am sure it will be very well visited so we need help talking to the visitors!

My experience from last year was that many use ownCloud (and love it). These users often are interested in hearing and seeing what is coming, so I usually have a demo machine with me.

People new to ownCloud are almost invariably very interested and you can help them get started.

Note that you don't need any particularly deep insight in ownCloud to be able to help. For people new to ownCloud, a general overview of how it is for you as a user is already a huge help and we always have people around who can help with the harder questions.
 
Besides the booth, it'd be great if we get some talks in at various ownCloud-related devrooms, like the decentralization room and such. See some info on this page and stay tuned for the announcements of the devrooms.

  

SCALE

The 14th SCALE moves to a new location, promising to be bigger and better than ever. I sure want to be at that epic first in a new venue and so should you!

For SCALE, too, we'll try to get us a booth again and we do expect it to be very well visited just like last year. SCALE is a very cool event with many friendly folks. It surprised me how many people already knew about ownCloud but there were still hundreds we could delight with the knowledge a real free solution exists for their cloudy needs. We can really use some hands with this!

And like with FOSDEM - you don't need to be an ownCloud expert to be able to help out. Being able to explain the concept from the users' point of view is the most important thing!

I'll shoot in a talk or two but - if you have anything you'd like to talk about, SCALE, too, has a call for papers open.

For both events we have the ability to help you with travel and hotel costs if need be. Just contact me directly about that and we can figure things out!

ownCloud needs you!

Lightning Fast

For the last two years, we had only lightning talks & workshops at the ownCloud Contributor Conference. This is an exceptionally good model for creation-type events like ours and your event might benefit from it, too.

Why

To find the best way of presenting content to visitors you must define goals for an event. If your event focuses on creating, building, developing or making and collaboration between participants who might not know each other yet is important, a track of lightning talks is a great way to kick off.

What

Lightning talks are very short sessions (3-15 minutes, usually gravitating around 5 minutes) which are typically scheduled in a single track. This means that the entire audience of an event is in the room and explains why the talks have to be short. Even if some of the subjects aren't interesting for everybody, the next comes in just a few minutes.

These talks provide an opportunity for people to present what they work on and for the audience to find out what is going happening in the project. Perhaps more importantly, the audience can find out who to talk to - connecting names and faces to subjects.

Indeed, due to their nature, lightning talks do not go very deep. This is as much a strength as it is a weakness, though, as presenters are forced to get rid of most of the content so they focus on what matters most. Overhead like lengthy personal introductions, many examples or the setup of demos falls to the wayside and a single point emerges.

How

As the format is a little more unforgiving than 30 or 60 minute 'tech talks', it is a good idea to practice in advance. Luckily, the short duration means that rehearsing the talk 5 or 6 times isn't a big time drain.

For the organization of an event it is important to get the slides from the participants well in advance. This not only forces the participants to be prepared but, as there is no time to even switch presentation files, you can prepare them in a single go. I demand slides in PDF and concatenate them in a single file.

To keep the audience hooked, schedule the lightning talks with much variation. Vary technical subjects with process-oriented talks and social ones. Look for a light note occasionally and be sure not to push to many down the throat of the visitors without an occasional break. In that time, people can look up the presenters and dig a little deeper, process the inspiration and relate it to their own interests.

It also makes sense to have them in the morning and/or the first day, so they provide a starting point for, especially, new participants. They can set the tone, perhaps not like a keynote does, but more practical.

If you additionally need more in-depth sessions and technical talks, you can schedule them in parallel after the lightning talks. This also gives speakers in the lightning talks the opportunity to invite people to these sessions, giving participants some insight in what they will cover.

My personal todo for the next #ownCloudConf includes practicing all lightning talks with all speakers in a one-on-one video call, both to solidify the deadline and increase quality.

BREAKING: Netneutrality more complex than you thought!

It was simple: netneutrality is good. Companies shouldn't be able to buy their way on a fast lane! That stifles innovation and competition and risks ruining the internet. Just like John Oliver explained it! But now the Brazilians are making things complicated.

Brazil was one of the first countries to introduce strong Net Neutrality laws, points for them. But now, Brazilian banks and local government are paying for the data bundles of users! Heresy! Why?

Well, many Brazilians can't afford a data bundle. Yet they need to bank, or order new passports. And it turns out that handling people in person at the office is more expensive for the banks and local governments than have them use an app on their phone. So, they made a deal with some local providers: users, even without a data bundle, can do their banking online and order their passports without paying. That seems like a win-win.

Zero rating, as this practice is called, exempts some services from from the data bundle - exactly what Brasil is doing. It is used widely in India ("internet.org") and in Chile it offered many people access to a limited set of internet services - until it was outlawed. But in a country where only a quarter of the citizens has access to broadband internet, aren't we doing the population a disservice by taking away their internet access, however limited?

Zero rating is essentially the equivalent of a collect call - the receiver pays. What is wrong with that? Even wikimedia supports zero rating!

It isn't win-win but lose-it-all

The thing is - the provider will be the gate keeper of what services you can. You are allowed only on a piece of the internet, being blocked not by technical boundaries but by a business model. A model which allows providers to extract more money from their business than they otherwise would have - not by offering more services, but by offering less.

The result will inevitably be lower data caps because it forces more companies to pay for zero rating! This is exactly what happens in Canada, where $45 gets you 2GB of data - compare that to the price of 8 dollars for the same amount in Finland. Canada is now changing the rules. Cable providers have figured that out, too, and try imposing limits while excepting certain services. And indeed, when providers introduce zero rating, prices go up!


Interestingly, when zero-rating is squashed, the opposite happens. When the government forbade zero rating in the Netherlands, its largest provider KPN responded by doubling their users' data caps without a price hike.

Thus, my suggestion to the Brazil government would be: work with providers to get indiscriminate data bundles to more users, rather than empowering providers to control their users' Internet usage.


Zero rating exist by virtue of artificial Internet scarcity in the form of usage caps and it is not part of the solution to bringing Internet access to everybody. It is part of the problem.

Special people

After a rant on G+ I thought it'd be nice (for me, at least) to share a thought: we have an urge to put certain people on a pedestal because it helps our own identity and self esteem.

We need to feel superior

Self Esteem is very important for us - Maslov put individuality on top of the piramid for a reason. We need it to function, be happy in life.

So our brain lies to us

But how do you feel special and unique when you're not? Our brain lies to us, causing our illusion of superiority. I say 'our' because this is a near universal issue: 90% of people in pretty much any profession feels they are better than average, despite skills pretty much always following a bell curve (statistics speak for "half the people is worse and half is better than average").

Our brain is in charge of maintaining that positive sense of identity and has a series of tricks to keep that.

For example, identity depends on contrast. So we tend to exaggerate differences with others who are close to us. See for example countries who make fun of each other - it is inevitably between peoples very similar. Some interesting experiments were done with group behavior at a young boys' summer camp in the US in the 50's. Read a bit about this here if you're interested. You'll realize some of the problems we have in society are ingrained in our brains - a point I've made in an earlier blog.

Then there is the Self-Serving Bias which:

is the belief that individuals tend to ascribe success to their own abilities and efforts, but ascribe failure to external factors.

We do that? Yes, we do. Examples are everywhere, and some honest introspection will show you. Why do we do it? Because it makes us feel better about ourselves! If something goes wrong, it's the fault of the world. If things go well, I DID AWESOME! (towards other people we show the related Fundamental Attribution Error). Just to get an idea of the impact of environment, watch this TED talk by James Flynn about the increase in IQ over the last 100 years (the Flynn effect, indeed).

Another important strategy is self justification: it is how we deal with a perceived discrepancy between what we believe about the world and what we see (also called 'cognitive dissonance'). Wikipedia again:

Internal self-justification refers to a change in the way people perceive their actions. It may be an attitude change, trivialization of the negative consequences or denial of the negative consequences. Internal self-justification helps make the negative outcomes more tolerable and is usually elicited by hedonistic dissonance. For example, the smoker may tell himself that smoking is not really that bad for his health.

External self-justification refers to the use of external excuses to justify one's actions. The excuses can be a displacement of personal responsibility, lack of self-control or social pressures. External self-justification aims to diminish one's responsibility for a behavior and is usually elicited by moral dissonance. For example, the smoker might say that he only smokes socially and because other people expect him to.

And there are many more of these biases which maintain our belief in ourselves.

Note that these persistent errors in judgment are part of a normal and healthy personality! But it is good to be aware of them and their effects on relationships, both private and in society at large.

Special or not

So, the rich have their brain lies to them, maintaining their illusory superiority. Now we can understand why somebody on top of the world feels that it is justified that he/she is paid more per hour than much of the world population earns in a year.

But why do we support this illusion by buying auto-biograpies and looking up to the Steve Jobs and Fords and Warren Buffets like they are such special people?

Because they support the narrative that we all need: the self made (wo)man.

There are always people who have a worse life than us - significantly so, often. As Sam Harris points out in a painful description of an iPad user, things are going very wrong in the world. So we have the need to justify ourselves, feel superior over the poor. We have many strategies for that - Sam mentions religion as one. Another one is the idea that we make our own life, supported by the biases I described.


I think that the stories of these great, wonderful people we've partially made up help us justify the thought that people who are worse off than us have only themselves to blame. We have a need to deny the harsh reality that the world isn't fair and we would be in their situation if the marbles would've fallen slightly different. 

The Washington Post again demanded that tech companies create special 'golden keys' for authorities to keep and use for access to private communication. Protected by a warrant, of course. For the benefit of this discussion (which is really getting old), I just put together the reasons why it is a dumb idea.

First of all. It is a pure fantasy, an entirely unrealistic wish of the Pink Unicorn variety that it is possible to create a key which only the US goverment (and other sanctioned agencies) would have access to. It is technically not possible. Ever. I explained that before so let me now just quote Bruce Schneier:

"We have one infrastructure. We can't choose a world where the US gets to spy and the Chinese don't. We get to choose a world where everyone can spy, or a world where no one can spy. We can be secure from everyone, or vulnerable to anyone. And I'm tired of us choosing surveillance over security."

And let's be clear - we've been over this, the Clinton government wanted a similar thing with the Clipper chip and as security researcher Matthew Green pointed out:

Clipper is only one of several examples of 'government access' mechanisms that failed and blew back on us catastrophically.

A second issue with the proposal is that it doesn't do anything. Just like all the spying programs that came before in this and previous decade. Here's Bruce talking about that, here the Guardian, the Newyorker, Wired and Washingtonsblog. Whatever these spy programs do - from spying on German Chancellor Merkel to US congress (that's the Washington Post itself!) to the United Nations and Unicef - the government spying programs certainly don't target or are helpful against terrorism or pedophilia or any of the other stuff they are claimed to be for. And neither will these 'golden keys' be used to catch terrorists.

Last, and this should already be blindingly clear if you see the list above of some of the targets of surveillance, you should doubt if the government agencies will abide by the rules - they haven't in the past.

I also want to point out that the very reason we're having this conversation in the first place is because we're idiots.

Meetup and a release party in B'lin?

Today at 7 it is ownCloud meetup time in C-Base again (and also in Munich, by the way!). As ownCloud Server 8.1 is scheduled for the beginning of next month, I hope to grab a daily ownCloud snapshot, put it on a Banana Pi and run the cool Smashbox testing tool against it to see how it fares.

Even more interesting, I hope to see if I can install older ownCloud releases on other Pi's and compare the performance changes between 7, 8 and 8.1 for example. Performance is one of the big improvements that should be coming in 8.1, with 4x speed up of things like file syncing. I'd like to see if I can measure that!

I have several Pi's and (after having given workshops on how to install ownCloud on them) plenty experience in getting them running, but no experience with Smashbox whatsoever. So - I can use some help. If you're interested in joining the fun, be there tonight!

On a different note, ownCloud comes out in the beginning of next month - that means that the NEXT meetup is essentially a release party at C-Base. Sir, yes, sir - release party time! See you there?

There'll be a meetup on the 6th of July in Nuremberg, I guess that's a pre-release party as the release is set for the 7th. Munich also has a meetup on July 15, like Berlin.

ownCloud workshops - two down, two to go.

some had already given up at this time...

The success rate is going up - where, at the first ownCloud workshop at the openSUSE conference, we had no successful installation, yesterday in Helsinki we reached a two-out-of-five. Both workshops had around 20 participants but usually people collaborated in groups of 3-5, following my guidelines on how to get ownCloud up and running on a Banana Pi development board.

Whoah, two out of five?

I admit it isn't easy and partially, that is intentional. The instructions in the document are sparse, especially for newbies - but even an experienced Arch'er threw his towel in the ring after almost three hours. Then again, what is a workshop for if not for enjoying the struggle of learning something new? And certainly everybody did that - struggle and learn new things.

The winning team still hard at work

The second team to get ownCloud running

Those who have any experience installing ownCloud know the difficulty can not be in ownCloud - and indeed, once you are set up with a running Banana Pi with SSH access, installing ownCloud is a matter of minutes. But getting there is no picnic! Why?

The hardest part by far is with the networking part of the workshop - the moment you've SSH'ed into the Banana Pi, 85% of the work is done. The challenge is significant - requiring not just Linux on the host laptop (yesterday, Ubuntu got 4 new users as neither Mac nor Windows were up to the task) but also handling stuff like tcpdump, Wireshark and a bunch of low-level command line tools like dd, mount, dpkg, ssh and so on. For most participants, the hardest challenges were:

• Windows and Mac. I'm sure they are awesome operating systems, but something which is a few mouse clicks on a Linux system (sharing the wifi internet connection over a Ethernet cable with the Banana Pi) seems virtually impossible. On Linux, NetworkManager makes it as easy as creating a new wired connection, choosing "shared network" under the IPv4 tab and ticking the "this connection requires IPv4". Now, just enable this network after connecting the Pi and done. I have no experience with Windows or Mac whatsoever, but if nearly 15 IT students with internet access can't figure out how to make these operating systems do the same thing - I can only assume it is hard.
• Command line familiarity. If you're new to Linux, an instruction like "mount the USB stick and copy over the data to the Pi" takes more than a few minutes and requires you to learn at least two new tools and looking through system logs. 
• New tools. You'll be looking for alternatives to Linux commands like dd on Windows and Mac first, and once you've given up on your familiar platform you get to learn tools even most Linux users rarely need. 
• Geeks. "Can't you do this easier with TCPDump?" "You can do this with the ip command too, you know" - I could only reply "Try, and if it works, show me how." Rest assured, I learned a few things - but haven't changed my instructions. Yet.

Seriously, I love it. Three hours of a workshop full of people trying various ways of skinning the cat. To me, the fact that nobody got completely through my instructions merely means they had a good time on the way. At least, that's what they said - "the best workshop I've been at" is just awesome to hear.

And... working! A Banana Pi was the reward.

For the next workshop (coming Thursday at the Open Tech Summit in Berlin), I'll further streamline the instructions (but not too much!) and I mean to put in some 'advanced' challenges for those who simply know too much about Linux to stay busy for 3 hours. I want to know how to do this without Wireshark and NetworkManager, to name two things...

of course - food and beer as reward for hard work

I'll also have to make Linux mandatory. The USB sticks with instructions and software I provide will be openSUSE live sticks for the next workshop, as I can't expect everybody to have Linux by default on their laptops. And of course you can use Windows or Mac if you really want and are up for the challenge, I don't mind including instructions for these platforms. But nobody got them working yet so expect some struggle.

You will also need a laptop with a working ethernet port, no amount of creativity has been enough to work around the physical limitation of not being able to plug in the other end of the networking cable...

Besides the satisfaction of getting ownCloud running, the first to succeed earns the Banana Pi they worked with!

If you can't make it to Berlin, I will give workshop later this month also in Dubrovnik (see my earlier blog) and if you'd like to have this workshop close by - let me know, perhaps we can arrange something.

Open Tech Summit Berlin, openSUSE Conference and more

This is a fun month. Not only are we moving forward with the ownCloud Contributor Conference (some cool interviews coming out soon), but there's a sudden avalanche of events this month. The ownCloud.org blog already wrote about it - we have had FOSDEM, SCALE, Chemnitz and may others I didn't attend myself. Find out about the openSUSE conf from last week and the upcoming OTS in Berlin!

Bunch'o'geekos

Last weekend I was at the openSUSE Conference which was a great event. I gave a workshop on how to install ownCloud on embedded devices (like Raspberry Pi and Banana Pi) with about 20 participants and while it wasn't as easy as I had hoped, everybody had fun.



There were two great BBQ evenings as well, with beer and time to talk to many, many old and young geeko friends.

I absolutely had a great time - thanks, everybody, for working on the conference and being there! I look forward to next years' event.

Upcoming

This Friday I'll give a workshop at Happy Hacking Day 2015 in Helsinki and later this month I will keynote at the DORS/CLUC conference in Zagreb, Croatia.

Open Tech Summit Berlin

Next week, May 14, is a home game, as I'll give a workshop at the Open Tech Summit at the Kalkscheune in Berlin.

I will spend a few more words on this as the event has been announced rather late, yet it looks like the place to be for Berliners next week!

The event will feature talks, workshops and panels with topics ranging from open hardware to design, graphics, software, start ups and digital policies. In a barcamp style track there is space for adhoc meetings, lightning talks and breakout sessions. There will also be dedicated workshops for kids and maker enthusiasts, where you can make your own gadgets (TV-B-Gone, upgrade knitting machines, your own traffic lights) and Fashiontec wearables. In the evening it will have an “OpenTech-Himmelfahrt” lounge and the Linux Professional Institute offers Linux certification at a discounted rate.

Speakers will include a bunch of interesting people from Freifunk, Mozilla, VLC, 3D printers and many other cool projects, discussing everything from software to hardware hacking.

Workshop and discounted access to OTS

And of course, Blizz and myself will run a workshop on how to install ownCloud on cool devices. We have some with us, but it's recommended you bring your Raspberry Pi, power, a SD card, a laptop and a network cable. You can bring your server, desktop or laptop for installation of ownCloud too, of course. You can register for the workshop here and if you haven't yet registered or gotten tickets for the event - here's a nice discount.

ownCloud Meetup to test devices in Berlin

Coming Wednesday, it's time for the monthly Berlin ownCloud meetup again. Last month, we wanted to play with some little development boards, install ownCloud on them and see what they could do. But we had over a dozen new participants join, turning the meetup mostly in a 'how to get an ownCloud development environment up and running' session.

That was great fun and I'd love to see everybody again - but the devices are also in the office, looking at us like they want attention. So, if you're interested in playing with Banana Pi, Raspberry Pi, cubietruck and similar stuff, come and join our meetup. Wednesday, starting at 7 at c-base Berlin in the main room, we'll grab some tables and bring devices, power cords and chargers and get going.

We'll see how far we can get, but goals include getting some decent performance statistics of the differences between the various devices and ownCloud releases as well as finding some ways of optimizing the ownCloud performance on these boards.

Note that ANYBODY, seriously, ANYBODY is welcome. We'd be more than happy to help you get started with any type of ownCloud hacking, so if you don't care about Raspberry Pi's but rather hack on 2048 CPU clusters - fine with me. Feel free to bring one and we'd be happy to turn it into a fun toy ;-)

Oh and if you tend to get hungry in the evening, bring something to eat. We'll probably grab some pizza.

You can RSVP here but you're welcome in any case!

Dealing with our flaws in thinking

This is a follow-up on a post about the limited human rationality. In that post I described some facts - just a few - that perhaps left you a little more in doubt about your cognitive abilities. Or at least more aware of the limitations our human condition comes with!

Consequences

Unfortunately, the mentioned and the many other flaws in our thinking have consequences for decision making in our society, especially when there's money to be made. The lobby of weapon manufacturing is rather stronger than that of companies creating anti-slip mats in showers and car manufacturers, well, security is merely a factor increasing the costs of cars so there's little incentive for them to hammer on that issue either. The combination of our innate inability to judge the likelihood of these and other things to harm us and the financial pressure on politicians results in massive over-spending on what is in essence irrelevant or even dangerous and harming our society. The NSA, for one, stupidity around net neutrality is another and the war on drugs is third rather prominent example. And now Ebola, of course - a disease so unlikely to kill you, you're probably more likely to be killed by a falling piano.

I think it is pretty clear, as I mentioned above, that politics and business happily abuse our lack of rationality. But probably more often, 'the system' causes issues by itself, as the insanely huge political divide in the US shows. It pays of for the media to put extreme people in front of their audience - and today, we have a country where you can't discuss politics at the office because people see the world so vastly different that only conflict can come out of a conversation. Think of the biases I discussed earlier: these world views aren't likely to get fixed easily, either.

Never attribute to malice that which is adequately explained by stupidity.

I don't think anybody set out to create this divide - but it is with us now.

Now indeed, the media are part of a system working against us. They get rewarded for creating an impression of problems; and they are often uninformed and biased themselves. As John Oliver pointed out, we don't put a believer in regular human abductions by aliens in front of a camera to debate a scientist, attempting to give both sides of the debate an equal chance! We pity the few who still don't get that this, and many other issues, are settled.


Yet this is what often happens in areas where science has long come to a conclusion. Not just the moon landing but also vaccinations, global warming, evolution and a great many more things. Take the recent "Snowden wants to come home" media frenzy!

I don't think any of that is intentional. It's the system rewarding the wrong things. We are part of that 'system': we prefer news that supports our view point; and we prefer new and exciting things - a balanced point of view is boring.

Quality decision making gets harder and harder.

Dealing

One way of dealing with disagreement has been to essentially declare all facts 'up for discussion'. It all depends on your point of view, proponents of this idea say. But reality isn't as malleable as relativists make it out to be. You can choose to leave your house through the front window on the 3rd floor, but gravity's a bitch. It's nice that some want to value everybody's opinion, but the universe imposes limits to that.

We have to realize that the world is real. People can be right or wrong about it and the choices we make matter!

As a society, we need to find new ways to make decisions in a healthy way. We've done good things - we eliminated polio and smallpox, diseases that were around for many thousands of years and nobody has had them in a long, long time. River blindness is hopefully next, and others will follow. We also drove half the worlds' animals near extinction and are abusing this planet to the point where it just will become a much more hostile place in a century or two unless we change something. You can guess I'm not much into libertarianism - it is clear that we can and do impact the world and going it all alone does not solve the tragedy of the commons style issues we have. There's a problem - and the inherent complexity of the world is certainly part of it, as is our lack of rationality.

How do we deal? I used to be an optimist - when I discovered the Internet, I thought it would democratize knowledge (it has) and news (not so much). Social media, sites like Digg where people vote on what the 'best news' is, it seemed a new and improved reality. No more single points of failure. No journalists who can be bought or oppressed. And then there were open source communities, with their flat structure of decision making, ideals of equality and meritocracy. Democracy would thrive!

Reality was harsh. The Internet has allowed us to hide in our corners with like minded people. It has lots of good stuff (if you're not into economy or net neutrality, this is a good read on both) but the Internet didn't kill conspiracy theories, it fuelled them. And open source works for some, but has its own issues of inequality (and that is just one problem).

Perhaps technology can help - Google has apparently found ways to find out what's true and what isn't. I'm not so sure. I wonder what it would do to sites like this proving that even with mere facts you can create conspiracy nonsense.

Methods to the rescue

I think the solution has to be found in a system or a process in the way the scientific method works. Wikipedia describes it as follows:

The scientific method is a body of techniques for investigating phenomena, acquiring new knowledge, or correcting and integrating previous knowledge.

I prefer to call it a process, rather than a 'body of techniques'. The key is that if left to common sense, humanity decides that life emerges from lifeless matter until more than 2000 years later Louis Pasteur shows it really, really doesn't (except for this). The scientific method thus aims to take human decision making out of the equation, or at least, rigorously deal with the biases that cloud our judgment. Lots of books have been written on the subject of philosophy of science - I got my portion by way of Chalmers, worth reading.

However it works, key is that while science relies on people and thus makes mistakes, it has a process for dealing with these mistakes, correcting them over time. Confidence is gained over long periods and the result is that we have been largely refining our knowledge gained since the scientific method became widely used, rather than rewriting the world as we know it over and over again. Yes, Newton's theories on physics still stand - quantum mechanics and Einsteinian physics merely refines it, providing better results in areas Newton can't reach. Uncertainty exists in science, but only at the 'edges', where new knowledge is created. While many facts of evolution are debated, since evolutionary synthesis, we've settled on a core which is as solid as Newton's ideas about gravity; climate models might be imperfect today but much of what we discovered does not have to be debated over again and again.

Method for decision making

We have methods, systems, processes for decision making, too. Democracy is one, the trias politica part of it. But it has flaws and needs some refinement, ideally in the opposite direction of Citizens United. I don't think we can make a Philosopher King system work, so whatever we come up with has to be a bureaucracy, evolved from today's system. I think decentralization is part of the solution (majors should rule the world?), but we live together on this planet so there have to be over-arching structures, too.

I know there is research being done on the topic. And we've already come up with some strategies like the advocate for the devil.

What exactly the solution should look like - don't ask me. I'm a psychologist, I can merely tell you not to trust people and their gut instincts. If this feels like an anti-climax, well, it should. We will have to come up with solutions together - not one blogger alone!

Soon?

But we should hurry.

I believe, with self-described plutocrat Nick Hanauer, that the pitchforks are coming. Perhaps the militarization of police and governments (the NSA in particular) disrupting online security are attempts of governments to prepare for social unrest.

Maybe.

What I'm certain about is that humanity can't continue the way it is functioning now. If social inequality doesn't put a stop it, the depletion of natural resources or religious fundamentalists will. The Dutch would say: the wall will turn the ship.

Let's see how hard we'll hit it.

Why can the NSA do what it does?

This is part one of two blogs about how we make decisions and how our lack of rationality results in much of the mess we have today. I'll start with addressing the title of the blog - spying.

Much has been said about NSA by very qualified people like Bruce Schneier, comparing the NSA to

the Maginot Line (..) in the years before World War II: ineffective and wasteful.

The costs in terms of civil liberties and money resulted in one confirmed case where somebody was caught thanks to NSA spying (probably unfairly, though).

Like most technical people, I'm not impressed but very worried about the erosion of our civil rights, through the NSA spying and in other ways. And I am sure I share with others the impression that if only politicians and the general public knew more about the problem, we wouldn't make such bad decisions.

At the same time, I know I'm probably wrong about that. Like most people, I also care about global warming; health care; poverty; war; and the countless other things arguably Wrong With The World. And collectively, we know all there is to know about them.

Somebody, or a small cabal, must be causing this, then - an argument you often hear about many things gone wrong.

Is there a cabal? Let me invoke Hanlon's razor:

Never attribute to malice that which is adequately explained by stupidity.

Because I think it's the human condition that got us here, not malice of anybody in particular. We just, collectively AND individually, fail at making the right decisions.

So the question should be: what makes us so unreliable? So easy to lie to, especially in groups? Why do we believe conspiracy theories that often require us to believe far more fantastic things than the reality they try to disprove? How can fans kill people only in South Korea?

I'd like to dig into that a bit in this post, more of an essay than a blog, I suppose. The immediate reason is the mess surrounding the NSA (probably not news to most readers of my blog), where one-liners and the inherent complexity of the issue have ensured most people I talk to don't see the problem.

Reality is that the more you know, the harder it is to have a firm opinion. In reality, often conflicts are like Israel vs Palestine - if you pick a side, you're wrong. The complexity of real life issues makes it easy for governments and companies to play people - I feel an urge to point to Russia, but how do we know they are not right claiming Israel-backed Neo-Nazis used US supplied weapons to shoot down Flight 17? You can point out that historically, (neo)Nazis and Jews haven't gotten along very well. That's a fact. But so is the support of the EU for 'political reform' (overthrowing a legitimate, democratically elected government) in Ukraine. How valuable are facts and reason in a complicated situation?

I don't want to talk about the facts of any specific situation here. I'm not an expert in pretty much every relevant domain and neither are all of you! But we do have decision power - that is how democracy works. Luckily, with the Internet today, it is possible to have the most important facts around. And unfortunately, it does not lead to better decisions. So I want to look at it from my background in psychology and talk about how we deal with knowledge and how we make decisions.

Most of the time, we make decisions using 'common sense'. Our gut feeling, which was useful when we were still in Africa. But what works when you're hunting antelope might just not do the trick when you have to decide for an insurance company.

Why we don't get it

The problem lies in cognitive biases, described by Wikipedia as a "pattern of deviation in judgment". That's a nice understatement if you ask me.

A cognitive bias is a remainder of our Africa times, setting us up for certain errors. You'd be surprised how much of our thinking is in our genes. It is why most people are afraid of snakes and spiders: somehow, our genes have programmed our brain to quickly learn to fear snake and spider like objects, but be totally fine with bunnies and flowers.

To understand how limited our ability for logical thinking is, let's remind ourselves of Optical Illusions. I'm sure you've seen many by now and you should realize how flawed our vision is. Yet we have evolved to 'see stuff' for millions of years. We have a big part of our brain dedicated to it. And we use our eyes all day long, every day of our life. Yet, these images keep confounding us. To top it all off, there is change blindness, showing it isn't just bad - we are terrible.

Logic, on the other hand, is quite new. While some brilliant people were inventing math and logic thousands of years ago, most humans kept busy hunting, later growing food. And most of the time, in day to day life, we work on the automatic pilot. There's a reason we learn to recognize objects with little effort as babies already but have to be taught the mere basics of math during excruciatingly long sessions at school! Humans are naturally absolutely horrible at math and logic, to the point where somebody with some reasonable knowledge of probability could have won half the riches of the ancient world by gambling - probability theory was only developed in the 17th century.

And if you think you did well in those 16 years of school, answer this:

A bat and ball together cost $1.10. 
The bat costs $1.00 more than the ball.
How much does the ball cost?

See the answer two chapters further, just before the end.

It shouldn't be a surprise to learn that the list of cognitive biases is as long as it is. Most of the time, instead of 'proper' logic thinking, we use heuristics, shortcuts to 'good enough' solutions. They work great - if you're hunting antelopes.

To state the obvious: we're not. To go back to the NSA - we're letting them go wild. And: global warming, poverty, war... We're not doing that well, as Douglas Adams pointed out:

“This planet has - or rather had - a problem, which was this: most of the people living on it were unhappy for pretty much of the time. Many solutions were suggested for this problem, but most of these were largely concerned with the movement of small green pieces of paper, which was odd because on the whole it wasn't the small green pieces of paper that were unhappy.”

Some biases explained

You might have clicked some of the links above and seen some examples already, but lets bring this to life with a familiar example: why are so many people afraid of flying?

Let's turn the question around. SHOULD you be afraid of flying? There's a risk, certainly. But it is a well known fact (I won't even link to anything) that the chances of an accident when traveling by plane are larger on the way to and from the airport than while in the air! So why do many people still feel flying is dangerous? It is due to the way our brain does statistics. It's nothing like a calculator...

Our brains have ways of estimating how likely something is. They do that by digging in our memory: the easier it is to recall multiple instances of something, the more likely it is to happen. That makes sense - if you found food three times when walking a certain path, there is a high likelyhood of you finding more there rather than on a path where you never found anything to eat. It's clear where you should go if you are real hungry.

But our memory isn't terribly precise. First of all, it stores information in relation to emotion. This means when you're sad you can remember sad things better and when you're happy, happy events come to mind easier. It also stores things better when they are associated with strong emotions. This makes sense: events which upset, anger or scare you are probably more important than those which don't elicit any particular emotional response. You'll remember less details from your home-work travel from Tuesday a week ago then that crazy ride in a roller coaster, even though the latter didn't even get you to anywhere special.

This effect causes us to make massive errors in estimating things like the likelihood of getting a car accident, slipping in the shower or getting killed by a terrorist. And flying, of course - plane accidents feature prominently in the press and cause a lot of anxiety. But let me remind you:

Never attribute to malice that which is adequately explained by stupidity.

These accidents are worth talking about - don't blame the media for reporting the news. It's just that we fail at judging how likely it is that it happens to us.

And this is just one flaw related to just one aspect of how we think - statistics. We're not too good when it comes to statistics and causality either, as XKCD points out below.



Our failure to handle statistics can be abused quite easily, and not just in gambling or other situations where statistics obviously play a role. In marriage counseling, a therapist might ask you to try and come up with five good things about your partner. This wouldn't be too hard and put you in a more positive mood. If the therapist was evil, however, he/she could ask for ten good things instead. While you might get to ten, a number that should give more confidence than a mere five, paradoxically the difficulty in coming up with such properties is not going to be good for the relationship.

And there's (again) no malice needed. Stereotypes, for example, are known to cause behavioral changes without awareness of the subject ("you do something but don't know why you are doing it"). In an interesting experiment, it was shown that if you let a group of white people and people of color complete a standardized test, they can either do equally well, or not. It depends on what you tell them it is a test of "diagnostic of intellectual ability" or not!


That's right: when they know it is an IQ test, African-Americans score significantly worse than white people, while they actually do a little better when the same test is done without any mention of its diagnostic abilities. They let the stereotype come true, unintentional and unconscious.

An awesome example of how prejudices influence us on a fundamental level can be found in this great video of a talk by Mahzarin Banaji, who shows to an audience of liberal, well educated scientists the gender bias in their brains. Watch the section with the test below and while she does the test, participate! Speak out the 'left' and 'right' so you can experience the effect for yourself.


If you participate, you will notice that you easily do the first three tasks - but the fourth is inexplicably harder. This shows the fundamental bias in your brain: you take twice as long to answer a question that isn't congruent with your unconscious gender prejudice! Professor Mahzarin notes that she, herself, exhibits this same bias - about 75% of men and women do. Racism exists even in people with the best of intentions.

Nobody can be blamed for this - it is how we, humans, function, just another one of our many flaws. Stereotypes are central to how we function, influencing our behavior at every corner in our lives. They ARE the biases, the short cuts of thought, themselves!

What does this all add up to? For one, you can't judge how dangerous something is. That is why we spend lots of time and money to reduce risks that are tiny, see also this and this.

Confirmation bias

Let's discuss one more bias, confirmation bias. This is one particularly nasty bugger:

Confirmation bias, also called myside bias, is the tendency to search for, interpret, or recall information in a way that confirms one's beliefs or hypotheses. It is a type of cognitive bias and a systematic error of inductive reasoning.

Remember, the inductive reasoning Wikipedia mentions here is the one based on probability: after you've seen over 50 expensive cars in a new place you're visiting, you might be inclined to think it is a rich city. Maybe. But remember what I said earlier: how does your brain assess probability? Memory recall.

Wikipedia mentions explicitly that your brain has a tendency to remember things that fit the theory you are trying to verify! That means that it will fail to bring to mind the slum around the airport you saw from the airplane. Yes, your brain will explicitly take away your ability to properly judge a situation.

It sure makes you feel more confident about your decision. But not any less wrong.

And that is just the memory side of this bias. Countless studies have shown we explicitly look for evidence that fits our expectation. A truly impressive example is in the video below:

A little background on this experiment is here and an opinion piece by the LA Times connects it to how white people demonize people of color in the aftermath of Fergusson. Of course, when it comes to minorities there are many biases in play, this is just one of them.

About the Bat and the Ball. The bat was a dollar more expensive than the ball, and no, the ball isn't 10 cents with the bat being 1 dollar. That would make the bat only 90 cent more expensive. 5 ct and 1 dollar and 5 cent. Of course, if you did the math, you wouldn't get it wrong. But if you think quickly - you are. Because you're then using a shortcut. Sadly, you do that most of the time, and not just when math is involved.

The human intellect is particularly good at over-estimating itself. We even consider unrealistic positivism about oneself a healthy attitude - people who are more realistic are clinically depressed. There's a sad list of self deceptions here. And if you think that high intelligence means less cognitive biases - think again. It might even be that the opposite is true!

In a way, this video gives a philosophical view on the subject, inspired by Plato's cave:


I hope that with some of the facts and thoughts above, I've made you a little bit more humble, dear reader. Because if we're to solve the problems we have, we need exactly that...

Read part two for some thoughts on the consequences and dealing with our limitations. Feel free to comment below!