If you think of data visualization as a great way to bring together heaps of interesting data and, well, a visual language, one of the most exciting areas in terms of room for improvement has to be how surveys are visualized.
In conversations with my pollster friend @laguirlande, I often find myself regretting that when reading an opinion survey, we see answer tallies in isolation. That is: we can see what proportion of people gave the first answer to question A, and what proportion of people gave the second answer to question B. But what we we don’t know is, out of the first group, how many find themselves in the second?
Another frustration: survey data is very tabulated. Pollsters know respondents age, gender, and all kind of categories they can fit their respondents in. But again when reading the results, more often than not it is not possible to utilize this structure to filter answers by category – do men react like women to the issue?
Yet opinion surveys are always interesting and topical, else they wouldn’t be paid for. And the interestingness may lie there: little nuggets of data that could have escaped analysts could be found by readers as they try various manipulations.
And then last week, Chanel 4 released the sexperience 1000, a project around the “Great British Sex Survey” by Ipsos MORI, which as its name implies is a large-scale survey on sexual practices of UK citizens.
The project walks us through around 20 questions where little icons represent individual respondents. When the reader changes questions, the icons arrange themselves in bars or circles to form a chart.
Yes, filtering! Yes, cross-tabulating!
It is then possible to click on a column label to “track” that group and find out what they’ve been up to.
Here, for instance, we can select the group of people who had over 101 partners and follow them through another question: the longest period without intercourse, for instance.
The selected individuals will show in green, so we can see that some of them went through long periods of abstinence.
It is also possible to select one interesting individual and see all of their answers.
It is also possible to filter the individuals according to many categories. Here, for instance, we can see that a majority of respondents had intercourse in a car. Including quite a few who don’t drive or don’t have a car!
Great interactivity, yet legibility could be improved
Those functions are truly great and encourage the reader to explore. Yet the choice of representation makes it a bit difficult to understand the answers.
Going back to the first chart I showed about the number of partners for instance, this representation highlights the mode, the most frequent answer to the question. In this case, this is the 11-20 bracket with 121 answers. A close second is just 1 partner (120 answers). The composition of the answer brackets has a lot of influence on that, because if we had made a 5-9 bracket for instance it would have outweighed both (217). Also, they chose not to directly represent the people who had 0 partners, which are over 7% of the respondents.
More fundamentally, I suppose that people want to know where do they fit in the distribution. Are they normal? Less then the norm? Or better? From that chart it’s very difficult.
With that in mind, it’s more relevant to show a cumulative chart.
Less than 5! who would have known?
The site choose to display bar charts when the answer is quantitative, but to present circles for qualitative questions. This is fine when several circles are displayed: it’s ok to figure out if one is bigger than the other.
But it’s much harder when only one is shown, like for the question on places I’ve shown above. It’s difficult to know whether this is a “big circle” or “small circle” without any indication of the total size of the sample. I only know that the majority of respondents had intercourse in a car because of the small number 550 next to the circle, but there is no graphical way to show that.
It’s even harder to compare proportions across groups which have different sizes.
Here, for instance, I’m trying to see whether people with an iPhone are more likely to cheat on their partners. What I see is two grey bubbles – it’s relatively easy to say that one is bigger than the other – then two smaller blue and pink ones, which are harder to compare one to another because they are smaller. What’s even harder is to assess whether the ratio between the larger balls is greater or smaller than the one between the smaller balls. However, the proportion between both balls (that is the share of the respondents who have an iPhone) is relatively easy to figure out, too bad it doesn’t answer the question at all.
To compare across groups of different size, you just can’t escape switching to proportions. I understand the design choice and enjoy its consistency but in this case it goes against the function of the site. I feel that in this case, the designers chose playability and aesthetic appeal over ease of getting questions answered, which is not a bad choice per se condidering the subject and audience, even if there are more academic possibilities. At any rate, the Sexperience 1000 shows the way survey results could be displayed and is a great improvement over the current situation.