DataViz Critique

Earlier this week, I attended the one day “Stats for America” conference at the Bureau of Labor Statistics. It was an interesting conference that focused on the use and communication of data and analyses from several federal agencies. My primary reason for attending was to see the last (well-named, I must say) session of the day: “What would Edward Tufte say about the Charts and Graphs coming out of Washington”.

One of the four presentations in this panel consisted of a series of critiques and remakes of graphs from one of the represented federal agencies (of which there were several representatives in the audience). Over the next 10 minutes or so, the speaker proceeded to excitedly critique a selection of graphs from this agency, including colors, annotations, and basic graph types, even giggling at some of the elements in the graphs. As a neutral audience member, I felt the speaker treated the entire exercise flippantly and casually, laughing at times, tearing the data visualization apart.

I’m watching a speaker flippantly tearing an agency’s dataviz apart. And people from that agency are in the room. I don’t know if they’re offended or not, but as an observer, I find this painful.

— Jon Schwabish (@jschwabish) June 3, 2019

It was only later that I learned two things about the presenter. First, this speaker tends to speak in a fairly casual, half-laughing tone. Second, the presentation was based on a report that was commissioned by the agency to (presumably) critique and remake some of their graphs. It’s possible I missed the part of the presentation at the very beginning where this was described in detail (though I don’t think I did), but here’s the thing: if the presenter had thought carefully about the audience and how the criticism might be perceived, both by those who work in the agency and independent listeners like me, the speaker might have re-thought the approach and demeanor. As a friend recently told me, “intent is not always magical” and, as in many cases, it’s not necessarily what you say, but how you say it.

Critique in the data visualization field has long been a source of discussion. From Fernanda Viegas and Martin Wattenberg’s famous article on critique to Cole Nussbaumer Knaflic’s recent podcast episode to Elijah Meeks’ Medium article late last year, the field continues to evolve in how we criticize other people’s work. As a field, we seem to struggle with where data visualization critique falls in relation to critique in other fields like art (e.g., movies and food) and data analysis (e.g., peer reviewed academic journal articles). How polite do we need to be? When do we need to provide alternative approaches? When do we need to make those alternative approaches? Should we alert people that we are going to critique their work?

Always makes me uncomfortable. Any viz can be torn apart. You don’t know what their constraints were. You don’t know what the client (if there was one) insisted on. Many critics pretend their opinion is gospel.

— Stephen Hampshire (@shampshire) June 3, 2019

What is true about critique in data visualization that may not be true in some of these other fields like movies and food, is that the critic does not always know the different constraints (e.g., workflow, bosses, colleagues), intended audience (e.g., technical audience), and tools. Ideally, critique and discussion should benefit everyone, and in the case of presenting data visualization critique, this not only includes the speaker and recipient, but also other readers or audience members.

We should all learn NOT to do this. I know because I’ve done it myself in the past.

You can critique, of course, but first be acknowledge production circumstances, goals, and limitations. And be constructive, presenting alternatives as open to discussion and criticism, as well.

— Alberto Cairo (@AlbertoCairo) June 3, 2019

I’ll admit that I have certainly been guilty of being a bad critic. I have criticized graphs and websites without full consideration of the different constraints the producer may have been under. Or I didn’t completely offer alternative ideas or approaches. But I try to do better, to keep more perspectives in my mind and to take a more balanced approach in my criticism.

All this being said, I also don’t think we need to be too soft about this—if we are brave enough to put work out into the world, we should also be brave enough to take comments and criticism. Providing and receiving criticism is part of publishing data visualizations and is also what can help move the field forward to ultimately create better visualizations.

It might help to think about these kinds of critiques as public conversations. These conversations can benefit the visualization creator by providing them with feedback and alternative ideas, and they can also help people who are viewing these visualizations. In any conversation, tone and form matter—how you say something can be as important as what you say. We should think of critique not as a simple take-down of someone’s work but instead as a way to build up someone’s work and an ongoing evolution of the field.

I have long said that we should be more comfortable with critique in data visualization but without context the remarks I make might seem arbitrary and mean-spirited. –Elijah Meeks

Look, I’m not trying to call anyone out here (I haven’t used any names in this post). And maybe I was the only uncomfortable person in the room. But whenever we critique people’s work—be it on a blog, podcast, or presentation—we should be cognizant of the different people who are reading or listening to our critique and how they might perceive that critique. We might not be able to make everyone happy, but the more we can give greater thought to this, the better our community can be.

Thanks to Alyssa Fowers and Alberto Cairo for helping me think through some of these issues, and to the various folks in my Twitter feed for sharing their thoughts on critique.

Featured image by William Iven on Unsplash