Better life index – a post mortem
Today, OECD launched the Your better Life Index, a project on which I had been involved for months.
The three main lessons are:
- Just because data is presented in an interactive way doesn’t make it interesting. The project is interesting because the design was well-suited to the project. That design may not have worked in other contexts, and other tools may not have worked in this one.
- The critical skill here was design. There are many able developers, but the added value of this project was the ability of the team to invent a form which is unique, excellent and well-suited for the project.
- Getting a external specialists to develop the visualization was the critical decision. Not only did we save time and money, but the quality of the outcome is incomparable. What we ended up with was far beyond the reach of what we could have done ourselves.
The less short
Before going any further, I remind my kind readers that while OECD pays my bills, views are my own.
Early history of the project
OECD had been working on measuring progress for years and researching alternative ways to measure economy. In 2008-2009 it got involved in the Stiglitz-Fitoussi-Sen commission which came up with concrete recommendations on new indicators to develop. It was then that our communication managers started lobbying for a marketable OECD indicator, like the UN’s Human Development Index or the Corruption Perception Index from Transparency International.
The idea was to come up with some kind of Progress Index, which we could communicate once a year or something. Problem – this was exactly against the recommendations of the commission, which warned against an absolute, top-down ranking of countries.
Eventually, we came up with an idea. A ranking, yes, but not one definitive list established by experts. Rather, it would be a user’s index, where said user would get their own index, tailored to their preferences.
Still, the idea of publishing such an index encountered some resistance, some countries did not like the idea of being ranked… But at some point in 2010 those reserves were overcome and the idea was generally accepted.
Our initial mission
It was then that my bosses asked a colleague and I to start working on what such a tool could look like, and we started working based on the data we had at hand. I’ll skip on the details of that but we first came up with something in Excel which was a big aggregation of many indicators. It was far from perfect (and further still from the final result) but it got the internal conversation going.
Meanwhile, our statistician colleagues were working on new models to represent inequality, and started collecting data for a book on a similar project, which will come out later this year (“How is Life?”). It made sense to join forces, we would use their data and their models, but will develop an interactive tool while they write their book, each project supporting the other one.
From prototypes to outsourcing
It wasn’t clear then how the tool would be designed. Part of our job was to look at many similar attempts online. We also cobbled some interactive prototypes, I made one in processing, my colleague in flash. Those models were quite close to what we had seen, really. My model was very textbook stuff, one single screen, linked bar charts. Quite basic too.
I was convinced that in order to be marketable, our tool needed to be visually innovative. Different, yes, but not against the basic rules of infovis! No 3D glossy pie or that kind of stuff. Unique, but in a good way. There was also some pressure to use the existing infovis tools used by OECD. We have one, for instance, which we had introduced for subnational statistics and which was really good for that, and which we have used since then in other contexts, with mixed success. My opinion was that using that tool as is on this project would bury it.
That’s my 1st lesson here. I’ll take a few steps back here.
In 2005, the world of public statistics was shaken by the introduction of gapminder. The way that tool presented statistics, and the huge success of the original TED talk – which attracted tens of millions of viewers – prompted all statistical offices to consider using data visualization, or rather in our words to produce “dynamic charts”, as if the mere fact that gapminder was interactive was the essence of its success. The bulk of such initiatives was neither interesting nor successful. While interactivity opens new possibilities, it is a means and certainly not than an end in itself. Parenthese closed.
At this stage, the logical conclusion was that we needed to have a new tool developed from scratch, specifically suited for the project. Nothing less would give it the resonance that we intended. My colleague lobbied our bosses, who took it to their bosses, and all the way to the Secretary-General of OECD. This went surprisingly well, and soon enough we were granted with a generous envelope and tasked with finding the right talent to build the tool.
So our job shifted from creating a tool and campaigning for the project to writing specifications that could be understood by external developers. We had to “unwrite” our internal notes describing our prototypes and rewrite them in a more abstract way, trying to describe functionality rather than how we thought we could implement it (i.e. “the user can select a country” rather than “and when they click on a country, the middle pane changes to bla bla bla”.)
Being a governmental organization we also had to go through a formal call for tenders process, where we’d have a minimum number of bidders and an explicit decision process that could justify our choices.
This process was both very difficult and very interesting. Difficult because we had many very qualified applicants and not only could we only choose one, but that choice had to be justified, vetted by our bosses, which would take time. And it was rewarding because all took a different approach to the project and to the selection process. What heavily influenced the decision process was (nod to the 2nd lesson I outlined) whether the developers showed potential to create something visually unique. We found that many people were able to answer functionally to what we had asked. But the outcome probably wouldn’t match the unspoken part of our specifications. We needed people who could take the project beyond technical considerations, and imbue it with the creative spirit that would make it appealing to the widest audience.
Working with a developer
When we officially started to work with the selected developer – a joint effort by Moritz Stefaner and RauReif, some time had passed since we had introduced the project to them. When Moritz started presenting some visual research (which by the way has very little to do with the final site) I was really surprised by how much this was different what we had been working on. And that’s my 3rd lesson here.
We had become unable to start again from a blank sheet of paper and to re-imagine the project from scratch. We were too conditioned by the other projects we had seen and our past prototypes that we lacked that mental agility. Now that’s a predicament that just can’t affect an external team. Besides, even if we had the degree of mastery of our developers in flash or visual design (and we don’t), we still had our normal jobs to do, meetings to attend and all kind of office contingencies, and we just couldn’t be that productive. Even if we had equivalent talent inhouse, it would still had been more effective to outsource it.
What I found most interesting in our developers approach is that it underplayed the accuracy of the data. The scores of each country were not shown, nor the components of that score. That level of detail was initially hidden, which produced a nice, simple initial view. But added complexity could be revealed by selecting information, following links etc. At any time, the information load would remain manageable.
Two things happened in a second phase. On one hand, Moritz had that brilliant idea of a flower. I instantly loved it, so did the colleagues who worked with me since the start of the project. But it was a very hard sale to our management who would have liked something more traditional. Yet that flower form was exactly what we were after: visually unique, a nice match with the theme of the project, aesthetically pleasing, an interesting construction, many possibilities of variation… Looking back, it’s still not clear how we managed to impose that idea that almost every manager hated. The most surprising is that one month after everybody had accepted that as an evidence.
On the other, the written part of the web site, which was initially an afterthought of the project, really gained in momentum and importance, both in terms of contents and design. Eventually the web site would become half of the project. What’s interesting is that the project can cater to all kinds of depths of attention: it takes 10 seconds to create an index, 1 minute to play with various hypotheses and share the results on social networks, but one could spend 10 more minute reading the page of a country or of a topic, and several hours by checking the reference texts linked from these pages…
Fast forward to the launch. I just saw a note from Moritz that says that we got 60k unique visitors and 150k views. That’s about 12 hours after the site was launched (and, to be honest, it has been down during a couple of these 12 hours, but things are fine now)!! those numbers are very promising.
When we started on that project we had an ambition for OECD. But beyond that, I hoped to convince our organization and others of the interest of developing high-quality dataviz projects to support their messages. So I am really looking forward to see similar projects that this one might inspire.