The concurrency crunch

We, programmers, had hit a wall.

For the past half of a century, we were writing programs in basically the same way, and we could trust accelerating hardware performance to keep accommodating our ever-more-complex abstractions (I'm looking at you, object orientation, functional programming, and lazy evaluation) and the comfort features that made our lives so much easier (garbage collection, bytecode VMs, dynamic typing) without giving it much thought.

But now, processor speeds have hit a wall. And with it, the whole guild of programmers.

Processors are becoming harder to make much faster. There are problems with cooling and energy efficiency, and the wiring has become so thin that even solar flares can change bits. (And they actually do – just ask anybody why server RAM is much more expensive now.)

What we can do now is to put more cores into processors, more processors into servers, and connect servers together. Sounds easy, right?

No, it's not that easy. #

Nearly all software is still designed to do one thing at a time. Writing programs that actually take advantage of parallelism require a huge, huge change of thinking from how we were programming before.

Oh yeah, we've been using threads for double digit years now, say, to separate a GUI from background processing. But that still uses only two cores then. And not full time. If you're reading this on a newish notebook then it probably has at least four cores, and your servers have 12-24. We're going to have hundreds of cores in a single server in a few years.

It's not just a waste not to use them – it's that now you can't hope to become faster or serve ever more users without utilizing more cores somehow. No chance. Nada. The parallel shall inherit the earth.

But the problem is that writing parallel programs is hard. It requires a totally different thinking from your programmers than sequential programming (which is hard to begin with – that's why the majority of the population simply can't program), and the complexity can quickly become mind-boggling.

Prepare to lose at least as much percentage of programmers as we've lost at C's pointers-to-pointers, the famous "asterisk asterisk". You could either grok that and then you could actually write programs, or you could limp around for a few years before moving on to tech support, sales, or middle management. (Then we've hid the inherent complexity with increasing CPU performance using the techniques mentioned earlier. But this time that won't work.)

What's the plan then? #

There is no way to automatically convert sequential programs to parallel ones. To fully utilize parallelism, software has to be redesigned and rearchitected.

But we're actually very lucky with the architecture of the World Wide Web. It's based on an inherently stateless request-response based model. This means that the web can go quite far by running many instances of the same sequential program, one for each incoming request.

For this to work, it's best if your language barely even support parallelism – like PHP. PHP is actually doing quite well in the multi-core and multi-server world, since it's stateless like the web itself, so your web server can simply start up lots of instances of the same PHP program, and you can start up lots of web servers, and since every web request "lives" only for a few milliseconds, the model works. It's not perfect, but it works quite well for now. (It only breaks down when you need coordination between them.)

We're using new algorithms. For example, one can use map/reduce, which is parallelisable, instead of algorithms which are not. Databases and other backend software are trying to help with this.

There are a few tools that automatically try to find parallelisable parts in existing code. Processors themselves are partly doing that.

But the best programmers are making the jump themselves, and are ready to tackle the complexity of parallelism – I would say that most of them are probably ready by now. Even if they are still working on sequential programs currently, the good ones are already getting fluent in parallelism. Just watch them hang around in Erlang and Haskell meetups, or poking around with Scala and Akka (and also getting into Elixir lately).

This is a warning from the man who had been there #

But not every programmer will be able to change his/her thinking. Global locking just won't cut it anymore – vector and matrix clocks, Lamport timestamps, leader election algorithms and Paxos protocols are the new "asterisk asterisk".

The bar is being raised as we speak.

(You might be interested in reading more about Erlang, and when I personally recommend to use it.)