Making a Dent in the Universe

Leadership is the art of getting someone else to do something you want done because he wants to do it.
– Dwight Eisenhower

We all strive to make our dent in the universe. And while too much concentrated power is certainly a bad thing, nobody wants to be entirely powerless.

If power is the ability or capacity to influence the state of the world, then being able to influence the physical world (stones, trees, rivers) may be the primary form of power. The archetypal lone cowboy is always certain to have enough power to provide food for himself and defend himself. But influence over the physical part of our world is a rather crude form of power, compared to the ability to influence the social part of our world – people.

Technology is usually primariliy considered in its capacity to influence the physical world (e.g. building a dam) rather than the social world. But since the invention of mass media technologies – starting with the printing press, then radio, television and now the internet – this has changed a lot. While not requiring all the same skills as mastering other kinds of technology, mastering a certain communications medium (e.g. writing books, blogs or filming) has become a very effective skill in influencing the world. Even more so in the highly connected and collaborative world we live in today.

A very simple form of the ability to influence people is “getting them to do what you want them to”. There seem to be mainly two strategies to this end. Either, you convince them that your idea is so great that they voluntarily invest their time in its support, or you simply pay them to do so. Or you do both.

In an industrialized society, where every year a larger part of the simple mechanical work can be done by machines, your are dependent upon motivated people that don’t simply execute what you paid them for, but actually understand and believe in your idea and want to actively be part of making it a reality. But in our current society, people also need money to get by.

So you’d think convincing them of your idea and paying them at the same time would be the smartest thing to do. But alas, whenever you introduce money as a motivator, things get more complicated. The warm and fuzzy feeling people get when doing some good for free might fade, you have to decide whether to give some people more money than others, etc.

So what do you do? It might very well be that either the state will provide a basic income to all its citizens, or that companies will do so for their employees – as long as they seem to be doing something broadly in the interest of the company. In the meantime, you can hope to find or found a company doing something close to that, or roll your own and look for some crowd funding.

Parallel and Concurrent Programming Models

This is a simple (non-exhaustive) categorization of different approaches for parallel and concurrent programming, inspired by Parallel and Concurrent Programming in Haskell (PDF). They state that parallelism and concurrency are two solutions to two different problems (as do others).

• Parallelism (i.e. data parallelism, is deterministic)
  • static and dynamic scheduling
  • forking, yielding (directed graph of data flowing: dataflow parallelism)
• Concurrency (i.e. threads or task parallelism, is nondeterministic thus cannot be expressed in purely functional code)
  • locking shared resources
  • blocking/synchronous channels (e.g. CSP) and asynchronous message passing (e.g. Erlang)
  • cancellation: asynchronous exceptions
  • software transactional memory (STM), i.e. group code into atomic blocks

Instead of concurrency, event loops with callbacks (as in Node.js) can be used to solve the problem of multiple external agents interacting with each other.

The paper also notes:

In the world of concurrency and parallelism, there is good reason to believe that no one size fits all programming model for concurrency and parallelism exists, and so prematurely committing to one particular paradigm is likely to tilt the language towards favouring certain kinds of problem.
[...]
We might wonder whether the compiler could automatically parallelise programs for us. After all, it should be easier to do this in a pure functional language where the only dependencies between computations are data dependencies, and those are mostly perspicuous and thus readily analysed. In contrast, when effects are unrestricted, analysis of dependencies tends to be much harder, leading to greater approximation and a large degree of false
dependencies. However, even in a language with only data dependencies, automatic parallelisation still suffers from an age-old problem: managing parallel tasks requires some bookkeeping relative to sequential execution and thus has an inherent overhead, so the size of the parallel tasks must be large enough to overcome the overhead. Analysing costs at compile time is hard, so one approach is to use runtime profiling to find tasks that are costly enough and can also be run in parallel, and feed this information back into the compiler. Even this, however, has not been terribly successful in practice [Harris and Singh]. Fully automatic parallelisation is still a pipe dream. However, the parallel programming models provided by Haskell do succeed in eliminating some mundane or error-prone aspects traditionally associated with parallel programming.

Gratitude and why you shouldn’t expect things to “just work”

I’m utterly convinced that the key to lifelong success is the regular exercise of a single emotional muscle: gratitude.

– Geoffrey James (hat tip to swissmiss)

Continuing the line of thought from my two previous posts, it feels like many of us have (or at least I have) come to expect many things that probably shouldn’t be expected. What I mean is this conception of “how things are supposed to work”, and the believe that somewhere on this planet there are those “true professionals” that do things that way. That conception, that if you adhere to a couple of smart principles, including the scientific method, sound reasoning and a lot of grit, you eventually end up with a process that is “the right way”. And the thought that if you do things that way and do them thoroughly, you will eventually come up with a “proper” product, solution, or at least a satisfactory conclusion concerning that particular problem. And if you don’t, you are just doing it wrong, have too little experience and probably aren’t (yet) one of those “true professionals”.

But what I’ve come to realize, is that those “true professionals” don’t really exist. That even the most professional experts, most of the time, have not much of a clue of what they are actually doing and are just making things up along the way. Of course, experience and formal training help, but even the best of us, most of the time, have to navigate a vast decision tree with simply too little information.

So what’s that got to do with gratitude? Well, if you expect that things can, should and are done “the right way” (that is assuming you are not working with or using a product of complete idiots, or believe you are yourself one), then you expect things to “just work” in the “expected way”. If they don’t, you are disappointed. But what’s even worse: if they do, you just ignore the fact that it just worked, without ever feeling any gratitude. Simply because that’s what you were expecting all along, that things work the way they are supposed to. So however things turn out, either you get disappointed or don’t feel much at all.

On the other hand, if you were to expect that things usually don’t “just work”, it is much easier to feel gratitude every time they actually do. And you will have a much happier life, giving you the power to tackle harder problems (not expecting to solve them on the first attempt).

On uncertainty, success and failure

The truth is, most things aren’t as predictable as we’ve come to believe them to be. There are just way too many factors at play, too many unknown variables, and too many variable we don’t even know that we would have to take into consideration.

As noted in these two great TED talks by Alain de Botton and Elizabeth Gilbert, our perception of who is responsible for our success has changed a lot. Before the modern era, poor people were called “unfortunates”, now they are simply “losers”, implying that it is mostly their own fault that they’re in this miserable situation they are now. Never before in history have we as humans believed so much in our own power to make informed decisions, to enforce our will on the world around us. Much was always believed to be in the hand of the gods or some other transcendent entity, but now it’s always our own fault when something doesn’t turn out the way it was supposed or planned to. In the tragedy, the ancient greeks had even an entire genre of drama dedicated to letting the audience empathically follow the stories of people that were simply struck by very bad luck.

It is true that the physical world at certain small scales is quite accurately modelled by sciences like physics and chemistry and that this has enabled us to achieve things our ancestor didn’t even dare to dream about. But the bigger and more complex things get, the harder to model them and be precise about them it is. Some might argue that intuition is better suited to the job then, but this is only sometimes true. Usually a combination of both, intuition and the scientific method, yields the best results. But still, usually things just don’t turn out exactly the way we predicted.

So should we stop trying just because the outcome is not as predictable as we’ve been led to believe? No, of course not. I think we should try nonetheless. And maybe we are lucky and it works. And maybe we fail and try to do better next time. And we fail again. And again. And again. And then we succeed. (Or at least we’ve helped others to come closer to the solution.)

Maybe rapid prototyping and short feedback loops help figuring out when we go into the wrong direction. And systems- and design thinking help seeing the bigger picture.

On optimism, being wrong and truth

A very good TED Talk by Tali Sharot on The optimism bias tells us something many of us have known (or shall we say suspected) already for a long time: the brains of healthy (non-depressed) people are fundamentally tilted towards optimism. For example, even when told that 40% of all marriages break, newly married people still don’t think there is a possibility they’ll get divorced. Likewise we underestimate the likelihood that bad things will happen to us personally. Sharot suggests that when doing financial budgets or when in dangerous situations, we should remember that we have this innate bias and calculatingly adjust for it. But most of the time, the human optimism bias is a good thing. Otherwise we would never dare to venture into anything new or risky.

The flipside of all this optimism is obviously that we always think we’re right. There is a great TED Talk by Kathryn Schulz: On being wrong. We will do almost anything to avoid having that nasty feeling creep up on us when we realize we were wrong. When somebody disagrees with us, we rather think that he or she either:

1. doesn’t have the same information as we do,
2. is plain stupid, or if neither of those apply,
3. that he’s plain evil.

Rather seldom the thought crosses our minds that we ourselves might be wrong. Or that different people have simply different models of the world that they think in and in which they integrate new information, and that there might be a multitude of valid worldviews. So we should pause more often and think about all the complexities of our world, and allow ourselves to recognize that we may not know, or that we may be wrong.

To the smart fellow who chips in now and says we should just abide to the scientific method and reason objectively: 1) objectivity does not exist, see intersubjectivity. 2) read the closing words of this unsettling article by Jonah Lehrer in The New Yorker about the ‘decline effect’ when trying to reproduce the results of scientific studies in fields such as medicine or psychology.

Although many scientific ideas generate conflicting results and suffer from falling effect sizes, they continue to get cited in the textbooks and drive standard medical practice. Why? Because these ideas seem true. Because they make sense. Because we can’t bear to let them go. And this is why the decline effect is so troubling. Not because it reveals the human fallibility of science, in which data are tweaked and beliefs shape perceptions. (Such shortcomings aren’t surprising, at least for scientists.) And not because it reveals that many of our most exciting theories are fleeting fads and will soon be rejected. (That idea has been around since Thomas Kuhn.) The decline effect is troubling because it reminds us how difficult it is to prove anything. We like to pretend that our experiments define the truth for us. But that’s often not the case. Just because an idea is true doesn’t mean it can be proved. And just because an idea can be proved doesn’t mean it’s true. When the experiments are done, we still have to choose what to believe.

While I always say that it’s problematic to apply the term “prove” or “proved” to anything other than pure mathematics (in science, theories can only be substantiated or falsified), I agree with the point being made. In anything that involves living beings (as opposed to e.g. physics or chemistry), interactions pretty quickly become so complex that it is hard to tell anything with certainty. That doesn’t mean that the scientific method might not be the best (or shall we say one very good) way to approach these problems as well. But we should keep in mind that just because some theory has been substantiated by a scientific study or even two, that doesn’t mean that it’s the absolute truth.

So the bottom line is that it is good to question things every once in a while, to remember that we actually know pretty little, to remember that we’re wrong way more often than we think. And to really consider other people’s worldviews and try to learn from them. But at some point, if you want to stay sane, you have to start believing in something again. How could you ever start a venture or bring any change to the world without believing in it in the first place? That’s when our innate optimism bias is essential. So that we do something – even when all the estimates say that the odds are low – to see whether it might be possible nonetheless. And sometimes it is. Just sometimes it is.

Haskell, XQuery, JavaScript or Opa for web development

This is a follow up on a previous post of mine, XQuery, Node.js and Opa – which is the “new Java” of web development?. So here comes the table…

	Haskell	XQuery	JavaScript	Opa
Purely functional	as pure as it gets	not so pure but mostly functional	cumbersome to write functional code	not pure but functional
Statically typed	yes	yes	no	yes
integrates well with a database	with RDMSs (via HaskellDB) and with a native Haskell store called acid-state	with XML-DBs	with MongoDB or CouchDB	with MongoDB or CouchDB
Web server	good	good, e.g. Sausalito or eXist	Node.js	e.g. Node.js
Web client (browser)	experimental Haskell to JavaScript compilers	experimental, see XQuery in the Browser	great (native)	good (compiles to JavaScript)
GUI code executed seamlessly on client and server	no	no	yes, using Meteor or Derby	yes, integrated
Templating system	yes, e.g. HStringTemplate	not that I know of	yes, e.g. Handlebars	yes
Parallelism and distribution	wide range of facilities for parallelism, maybe less distribution on multiple servers?	current implementations spawn a new process for each request, inside that no parallelism afaik	requests handled by the Node.js event loop which is non-blocking but unfair and starving. Inside WebSockets can be used	uses Node.js, but automates distribution

… and now some explanations. Keep in mind that these are my research notes, I’m no expert (yet ).

This started when I was thinking how nice a language Haskell was, purely functional, monads, lazy evaluation and a rich static type system. What more can you ask? Going from top to bottom in the table, unfortunately Haskell doesn’t integrate that well with NoSQL databases as XQuery and JavaScript do with XML databases and MongoDB respectively. When you have to use a different language to query your database than what you use for the application tier, you have an impedance mismatch: you have to convert all the data back and forth and have to think in different ways depending on what tier you’re on as well as having to decide what should be handled in what tier. A similar impedance mismatch arises when you have to use a different language on the client-side (i.e. the browser where we’re stuck with JavaScript) and on the server side. Ideally, you write your GUI code once and it is transparently executed either on the server (upon the initial page load) or on the client (upon further navigation by the user). To pull this off, it seems to help a lot to have a templating system which also separates presentation and computation/business logic. Finally, it would be nice if you could easily (and without introducing hard to find errors) parallelize your code to run on multiple cores of a machine as well as distribute it to run on a cloud. Maybe I’ll have to look at Clojure or Erlang for that as well.

Opa seems to do a lot of things right, the question is how it fares in practice and also how open its application stack really is. For Haskell, we’ll have to wait until it compiles properly to JavaScript, and I’ll have to look at whether there is a database that performs and integrates well with Haskell, otherwise it seems to rule (except that it’s hard to learn). XQuery seems well suited for large amounts of XML data and JavaScript is pretty easy to learn, so it might become the new (concurrent) PHP. So, we’ll see…

Infinite economic growth, space and population growth

Growth in energy consumption

If we pause to think for a moment, we all know that infinite growth of energy consumption is not possible on a finite planet. This fact can be mathematically proven, assuming the physical laws of thermo-dynamics that is. If you want to see the math, take a look at this excellent blog post – at the current annual energy use growth rate (which has been quite steady for the last 350 years), within 400 years, we will need more energy than what the surface of the earth (including oceans) receives from the sun.

Economic growth

However, the conclusion that economic growth requires a growth in energy consumption cannot absolutely be proven but requires some common sense thinking. Besides the fact that economic growth has historically never been achieved without a raise in energy consumption (see this animation on Gapminder), it could be argued that by some means (e.g. hefty energy or carbon taxes), the economy could be forced to consume less energy (or at least to stop consume more), and be made to grow in the unphysical sector (e.g. services, information economy). It seems unlikely that politicians could ever push through such a regime because the moment something like that is proposed, everybody starts shouting that higher energy prices are bad for ‘the economy’ and we wouldn’t want to risk losing jobs, would we? But even if such a regime were enforced, we would wind up in a pretty strange world were the unphysical sector would dominate the economy and the prices for physical goods eventually became almost zero relative to the whole economic production und thus relative to a person’s income. So food (and all other physical resources) would become so cheap for all the people working in the unphyiscal sector, that it would basically be free. (See the argument in more detail.) I cannot imagine such a world becoming reality, can you? So what that leaves us with is that infinite economic growth implies infinite growth in energy consumption which is impossible on a finite planet.

Space

While expanding into space would buy us some more time, the mathematics of exponentially increasing energy consumption would not even work out in space (see this already mentioned blog post). Additionally, sending energy or physical resources from space down to earth, would only put additional strain on the environment and climate of our small planet. Mind you, I’m all for going to space. But not in order to retain growth (that’s quite impossible in the long run), but simply in order to learn more about our universe as well as to put humanity’s eggs in one than more basket, should one or another catastrophy make our home planet unlivable (think nuclear/chemical war, climate change, major asteroid/meteroid impact, etc). Unfortunately, the political will in the US and Europe to build a permanent presence in space is dwindling (the US has even lost its capability to ship astronauts to the International Space Station, momentarily relying on Russia for that). So our hopes must rest on private companies, China or Open Source ventures.

Population growth

When you start talking about infinite growth on a finite planet, people often mention population growth. While it is undoubtedly true that infinite population growth on a finite planet is just as impossible as infinite economic growth, I think it is morally quite something different to ask families all over the world to stop having more babies than to ask the drivers of economic growth to stop making more profits. We know that it easier for the economy to grow when population increases and easier to degrow when population would dwindle. But there are plenty of examples of countries having positive economic growth while negative population growth (e.g. Eastern European states in the early nineties) as well as countries being in a recession while still having population growth (e.g. Sub-Saharan African states). Meanwhile, the majority of industrialized countries have a population growth rate of less than one percent while often retaining economic growth rates of more than two percent. (Again, have a look at this animation on Gapminder.)

In a future post I’ll look at the drivers of economic growth…