Vidya Setlur is the director of Tableau Research. She leads an interdisciplinary team of research scientists in areas including data visualization, multimodal interaction, statistics, applied ML, and NLP. She earned her doctorate in Computer Graphics in 2005 at Northwestern University. Prior to joining Tableau, she worked as a principal research scientist at the Nokia Research Center for seven years. Her personal research interests lie at the intersection of natural language processing and computer graphics to better understand data semantics and user intent to inform the meaningful visual depiction of data.
Interpreter turned analyst, Bridget Cogley brings an interdisciplinary approach to data analytics. As Chief Visualization Officer at Versalytix, her role uplifts data visualization within the org and helps shape the vision. Her dynamic, engaging presentation style is paired with thought-provoking content, including ethics and data visualization linguistics. She has a deep interest in the nuances of communication, having been an American Sign Language Interpreter for nine years. She is currently a Tableau Hall of Fame Visionary. Her work incorporates human-centric dashboard design, an anthropological take on design, ethics, and language. She extensively covers speech analytics and open text. Prior to consulting, Bridget managed an analytics department, which included vetting and selecting Tableau, creating views in the database, and building comprehensive reporting. She also has experience in training, HR, managing, and sales support.
Episode Notes
Functional Aesthetics for Data Visualization
Webinar about the book
Vidya | Tableau Research | Twitter
Bridget | Tableaufit | Twitter | The Logic of Dashboards presentation (YouTube)
Paper: Striking a Balance: Reader Takeaways and Preferences when Integrating Text and Charts by Chase Stokes, Vidya Setlur, Bridget Cogley, Arvind Satyanarayan, and Marti Hearst
Versalytix
Stroop Effect
Tableau User Groups
VisComm
Information is Beautiful Awards
Other recent books
- Jen Christiansen, Building Science Graphics: An Illustrated Guide to Communicating Science through Diagrams and Visualizations
- Nigel Holmes, Joyful Infographics: A Friendly, Human Approach to Data
Related Episodes
Episode #211: Jock D. Mackinlay
Episode #209: The Flerlage Twins
Episode #202: Lindsay Betzendahl
Episode #201: Leland Wilkinson
Sponsor
Partnerhero: to waive set up fees, go to http://partnerhero.com/policyviz and mention “PolicyViz” during onboarding!
New Ways to Support the Show!
With more than 200 guests and eight seasons of episodes, the PolicyViz Podcast is one of the longest-running data visualization podcasts around. You can support the show by downloading and listening, following the work of my guests, and sharing the show with your networks. I’m grateful to everyone who listens and supports the show, and now I’m offering new exciting ways for you to support the show financially. You can check out the special paid version of my newsletter, receive text messages with special data visualization tips, or go to the simplified Patreon platform. Whichever you choose, you’ll be sure to get great content to your inbox or phone every week!
Transcript
This episode of the PolicyViz podcast is brought to you by PartnerHero. PartnerHero provides customer service outsourcing that’s built for the needs of small businesses and high growth startups. So if you’ve ever needed help with your data visualization work the way I do, sometimes I need someone to help me scrape some data or make a graph or edit a blog post, I use PartnerHero. They have flexible terms, they can help you scale quickly, there’s quality assurance baked into every product and every program, and they have offices around the world which can also help you with different languages and different nuance in your writing or your visualizations. So if you’re ready to bring an outside customer support to help your startup that feels like it’s part of your existing team, check out PartnerHero. Head on over to partnerhero.com/policyviz to book a free consultation with their solutions team. Mention you heard about PartnerHero from PolicyViz, and they’ll waive the setup fee.
PolicyViz Podcast Episode #230: Vidya Setlur and Bridget Cogley
Welcome back to the PolicyViz podcast. I am your host, Jon Schwabish. Happy New Year, everybody. I hope you had a great New Year, a great holiday. Glad to have you back listening to the show for another great set of episodes, I’m going to take you all the way through June of this year, I’ve got a whole set of great guests coming your way to kick off the new year. I’m really excited to bring you the authors of the new book Functional Aesthetics for Data Visualization, Vidya Setlur from Tableau, and Bridget Cogley, we talk for a while.
I won’t lie to you, when I get folks to come on the show, I say, yeah, we’re going to chat for 25 or 30 minutes, and then we have a little bit of a chat before we actually start recording, make sure we’re all on board of what we’re going to talk about in the topics, and sort of have a list of questions we’re ready to do. And we were chatting for a while before we recorded this, and then we recorded, and I couldn’t stop, it was just the conversation was so great. I think Vidya and Bridget are onto something with this book, sort of, moving the data visualization field into the next stage of its evolution.
Obviously, a lot of my writing is on best practices and step by steps and introductory pieces. But there is a need and there is a market for that next phase of the field – how do we sort of think about data visualization, less as here’s a chart, go read it; here’s a bar chart, see if you can get in five seconds, to thinking about the next form of data visualization as a language, not just as visual icons or that sort of content.
So Vidya, Bridget and I talked for almost an hour in this, but I think it’s a great conversation. I think you’re going to really enjoy it. So again, Happy New Year, I hope you’ll enjoy this first episode of the PolicyViz podcast of 2023. So here’s my conversation with Bridget and Vidya.
Jon Schwabish: Hi Vidya. Hi, Bridget. Good morning early for you, Vidya right?
Vidya Setlur: Hi. Yeah, it’s 7:40, it’s not too bad.
JS: But you’re already at the office and ready to go.
VS: Yeah, looking forward to this.
JS: That’s a pretty good start. Well, thanks so much for coming on the show. I’m going to, for folks who are watching the video, I’m going to hold up your new book, Functional Aesthetics. Great new book. Love it. I’ve already ruined it with all of my writing and my tags and folded pages and everything. I think it’s great. I want to dive into a few sections of it, but I thought we’d start with just introductions, maybe just tell folks who you are and where you’re working, and then, we can talk about how you two got hooked up. You tell the story in the book, but maybe we’ll give a preview for folks, and then we could talk about some content. So Vidya, do you want to start?
VS: Yeah. Well, thanks for having me here. I am the Director of Tableau Research. I have been at Tableau for 10 years. I lead a really awesome team of interdisciplinary research scientists in the areas of data visualization, multimodal interaction, applied ML and NLP. I got a PhD in computer graphics and NLP in 2005 from Northwestern, and I was at Nokia Research seven years before I joined Tableau.
Bridget Cogley: I’ll also just add, she writes a lot of papers and collaborates on a lot of papers, so she’s really good at bringing together these teams of people that wouldn’t necessarily normally collaborate. She’s like, hey, let’s work on this, and I’ve gotten to be a part of it once, which is really nice.
VS: Yeah, and hopefully there will [inaudible 00:04:43].
BC: I’m Bridget Cogley, and I’m the Chief Visualization Officer over at Versalytix. I’m also the cofounder; and then, I am a Tableau Hall of Fame visionary, so I’m old enough in Tableau world to be retired, or at least semiretired, and put out to the pasture as far as like recognition goes. I started kind of using Tableau itself back in 2010, and before that did a lot of analysis in Excel. And then, I started my career path as an American Sign Language interpreter. So when you read the book, I’m the practitioner voice in that book, whereas Vidya is the researcher, and really, together we come up with a lot of these spitball theories where it’s like, well, I think this thing is true.
JS: Right. And you have this model in the book that I want to get to, that tries to piece these together. But maybe we could start with how you two got together, and teamed up to actually write this book.
BC: So it started with a Tableau user group over in Wisconsin, and so, they reached out to me and said, hey, you know, and I was doing this logic of dashboards talk, which was my whole, you know, how do you build dashboards, kind of, with a logical frame in mind. And we were doing kind of a two-TUG tour in Wisconsin. So we started out in Madison, and then, the next day were due to show up in Milwaukee, and they had mentioned Vidya was going to be there, and they were like, well, we’re trying to figure out transport and stuff. I’m like, well, she’d just ride with us, because we drove our teeny tiny Fiat, all the way out there, because this is what we do. It’s the Midwest for us, and so, it’s like, you just get in your car, you drive for however many hours, and what you do.
And so, we get there, and what was really cool is Vidya went first, and she presented, and I was just watching her talk. I mean, I can remember vividly getting chills, because I was like, this is the missing piece to my talk, because I was really going from a semantic lens of, like, you think about describing a room, and how do you do that, and calling out where people miss certain features of it, or they don’t do certain things grammatically, whereas in American Sign Language, there’s actually a grammatical convention to how you describe a room. And Vidya, you definitely need to add to this.
VS: Yeah, it was my first TUG, Tableau user group, so I really didn’t know what to expect, and my talk was not a traditional DataViz talk, so I had stuff on semantics and user intent, and I had a stroop effect exercise, which we talk about in the book, just showing people how our brain works, and how, if information does not align with or semantically align with how we see the world, our brain starts playing games with us.
And so, it was sort of this hands-on, participatory exercise, and so, I did my thing and then Bridget came up, and she had this, like, lovely picture that kind of instigated the need for creating not only beautiful dashboards, but something meaningful, and all of a sudden, for me, I was like, wow, this is such a beautiful segue from what I presented to what she’s doing. And yeah, she was gracious enough to take me in their little Fiat car to Milwaukee…
BC: And I want you to picture that, so my husband’s driving, because I’m lazy, and then, I’m in the passenger seat, and then, Vidya is behind my husband, her suitcase’s in the back, and I swear, she’s like packed in there, a little teeny tiny sardine, because it’s a Fiat, it is [inaudible 00:08:24] and she is happy as a clam back there, just smiling, and grateful. I would just be like, oh, but…
VS: Yeah.
BC: We were happy, and we talked the whole time.
VS: Talked the whole time, yeah, I feel bad for Mike, but he was a good sport. But I will say that, in addition to the similarities of the way we thought about data visualization, we also bonded over food, I mean, I kid you not.
BC: Which is the important part, let’s be real.
JS: Obviously, absolutely.
VS: We’re both vegetarian, we both love Indian food, and we started searching on Google Maps where is the closest Indian restaurant to go to, and that was the whole premise of that book cover. There’s like this map of all the Indian restaurants. That’s our conversation thread above the Fiat car, and [inaudible 00:09:19].
BC: And I just want to point out, like, these are actual Indian restaurants, this is real, this is not…
JS: This is like the Easter egg in the book.
BC: It is. This is the [inaudible 00:09:30].
JS: Yeah.
BC: And if you get the story behind the cover, and that’s why it’s like it’s not a traditional DataViz cover, like, they kept coming back to us with like these real futuristic, like, the dashboard samples you always see, it’s the Pixabay, it’s the Unsplash, you go there and you buy, like, the five dashboard templates that they have. And we really wanted a lot of personality, and we wanted to kind of just showcase the journey. And so, it’s like, we’re just going to do this, so it’s a very metaphorical DataViz without being a DataViz.
JS: So in this car trip journey, did you at the end of that – I don’t know, what is it, an hour and a half between Madison and Milwaukee, something like that – at the end, were you like, we need to team up on something, we don’t know what it is, do we need to, like, where did you end up at the end of that weekend?
BC: So we were going to – we chatted back and forth, and then, it’s like, well, we should do a 2020 talk together. And so, we were really working…
VS: We were also thinking about the Tableau Conference.
BC: Yeah.
VS: Right. And then, the pandemic happened, and the conference really didn’t happen. I’m trying to recall the exact set of events that happened, but yeah, go ahead.
BC: You’d started the talk, and then we kept having more content and more content, so we shared this outline, and it’s like, well, we should just do a book. And I love Vidya for this, because she’s got some hustle, she’s like, okay, we are putting together a book proposal, she sent in publishers left and right. And, I mean, she’s just on it.
JS: Just on it, yeah.
VS: Well, the story there was we formed a social bubble with my son and his friends and their parents, and we all went to Lake Tahoe, and we stayed in this huge house. This was all before we were all vaccinated. And the world was falling apart and it was like, okay, what’s the worst that can happen. So, wrote up a book proposal based on the content that Bridget and I had come up with for our original talk, and then, just sent it to publishers, various publishers all over, and we were like, okay, let’s wait and watch. And then, within a few days, we started getting responses from these publishers, and we’re like, whoa, okay, this is actually…
BC: It’s happening. It’s really…
JS: Yeah, that’s exciting. So let’s talk about content of the book, because in that story, you both use the word semantics. And so, I want to start there. So can you and maybe, I don’t know, I’ll try to pick people so that we’re not over talking, so we’ll start with Bridget maybe. Can you define for folks, what you mean by semantics as it applies to data visualization?
BC: So semantics is the study of how we draw meaning in communication, and the whole premise of my initial talk is dashboards are a form of communication. A lot of times we think we’re making a [inaudible 00:12:25] or we’re making a widget, and they’re really communicating. And so, that’s a part of it, Vidya, I want you to chime in on this as well, because like, we really went kind of back and forth, and what I find is that most people are focused on the perceptual side of this, that this is a perceptual creation. And it’s like, no, no, no, no, there’s so much semantic resonance. When Vidya showed the stroop effect, it’s like, that’s the thing I’d been running up against, and I didn’t have a word for it. I just knew that every time if I use the color over and over and over again, or if I used – if I crossed the wire, if you will, on color, it got really confusing, and I kind of found this little secret sauce, where if I did certain things with color, it worked extraordinarily well to an effect that I couldn’t explain.
VS: Yeah, I mean, so that is the formal meaning, and what we try to explain in the book is, as humans, we are trying to make sense of the world, we’re trying to figure out what things mean, what two people imply by saying something, doing something, seeing something, and semantics is a way of just formalizing that notion of meaning. And we often do this in various forms of communication, and this is also, I think, one common thread that Bridget and I share where she comes from the American Sign Language point of view, and there is a certain type of rhetoric that goes on with respect to communication, but there’s also communication in the form of language that arises from my background in natural language processing.
And so, we really wanted to impress on the reader that visualizations are a form of language used for communication, and there is a set of practices, some of them are taught through design, because when you think about graphics design, we are taught how to emphasize what is most important and deemphasize what is less important. And so, we kind of took that notion which primarily resided in the area of perception and design, and brought in semantics and understanding user intent, which is a way of expressing your goal when you’re looking at information, so we could provide a deeper understanding of how visualizations work, because they are not just a simple graph, right? They contain a message, and they’re trying to communicate something to us, and how do we actually use that form of communication to get people to take action. And so that was kind of where we were going with the book.
BC: And I really liked that Vidya introduced this idea of analytical conversation, and you think about how we converse back and forth, both with ourselves, so we start this initial pass with the data as an analyst, where we have a conversation with the data, and that happens on a very intimate register, where we’re digging into it, we’re exploring it, and we have a bunch of shortcuts, because it’s only us in that conversation; so we can really embed a lot of that shortcut information where we understand what we’re saying, but nobody else does.
The challenge is then when we take that intimate conversation, and we try to present it to the world, and we don’t put in the affordances to make other people able to understand it and navigate it, they struggle. And in the book, we highlight this in part, it’s the paper towel problem, it’s like you have this great experience at a restaurant, you go to the bathroom, and this is like a huge problem for me, you wave your hands in front of that cute little paper towel machine, and you don’t get any bloody paper towels. So then you sit there and you do all these antics to get paper towels, and it finally spits out like maybe an inch or two our paper. You pull at it, you’re wiping your fingertips, and then, you do what I do, which is the toddler thing, you’re wiping on your pants.
And it’s just like, and then, you know, here you are in this nice restaurant with wet pants, and it’s like, yeah, that’s how, again, but we do this to our users all the time in that our intent is we want them to do a certain thing, we want people to use less paper towels, but we still want them to have paper towels. It’s just that the signals get mixed up, and the wrong thing happens. And that’s, you know, we have that intimate conversation, when we’re exploring the data, we don’t put out enough exposition to the users to truly follow the conversation, nor do we do it in a cohesive manner. I mean, it ends up like charts on a page, which makes sense to us, because we have the verbal linkages that they do not.
JS: Right. Do you think that’s just human behavior that we just get so deep into, in this case, our data, that we just forget that our user hasn’t been neck deep in the data for six months the way we have?
BC: I’m actually going to push on that a little bit where it’s we’ve not been trained in DataViz to expose the information. You think about children go to school, they learn an essay writing template, they learn all these ways to expose information, that’s a literate society, and we are entrenched in a literate society. You cannot go anywhere without seeing something in writing. You see signs. You see all sorts of things – even in my car, I have all sorts of stuff that’s written, and it’s really, really hard to navigate the world if you can’t read extraordinarily.
And with data visualization, this is another competency skill, so with data graphic, where we talk about like numeracy as the basis, then literacy, and then, this is the third here. And so, I constantly mention data visualization is the third tier, because it’s not just, okay, we’re getting information from it, but it’s becoming a primary source of information. We are actually learning directly from the chart, i.e., COVID, where we are starting to see case trends, we’re starting to see a lot more visualization incorporated often as the lead in a news story, rather than the supplement. So that is the primary way of getting the information.
VS: I do also want to add that with data visualization, rightfully so, it started by helping people understand how our human visual system works, and what are some core perceptual principles that come into play explaining, for example, when a bar chart should be used, or when a pie chart can be used, or like stack bars may not be good for comparing values. So we have been taught some of the do’s and don’ts from a perception standpoint, but that’s kind of where the message has just been, and we wanted to take that to the next level, because there are some higher order cognitive processes that go into play, including thinking about dashboards and communication as a conversation.
And so, that goes beyond just the sheer perceptual qualities of a chart, because now, barring paper, most visualizations are interactive. There is this back and forth, and so, how does a person, how does the user who is interacting with a dashboard that an author has spent time on, walk away with some mental model of understanding what the data is about, and that happens through that back and forth interaction that tends to not be expressed as clearly when you just stay in the lair of perception.
BC: And that’s also where I kind of channel Marshall McLuhan a little bit, because the medium ends up the message. And so, when you’ve got interactive dashboards, that’s a very different message than something delivered on paper, it transforms how we communicate, it transforms the ways in which I can communicate. So if I’m printing on paper, that one chart really needs to suffice. If I’m building out an interactive dashboard, I can actually split the task amongst several charts. And so, that’s why I don’t often need to try to make this really complicated nested chart. I can split that task up, and let people dig into that, drill into that, and get that in a very different manner.
JS: Right. So you’ve both mentioned different types of modeling and thinking of ways to sort of maybe structure or formalize the way that we create, and then, ultimately, consume data visualizations. So I wanted to ask about this model that you have in the book that is sort of almost the through line, it’s the sort of conceptual and visual model that sort of comes through in this triangle. And I don’t want to describe it, I want to let you all describe it, but I thought maybe Bridget, you could start with this model and how people can think of implementing that into their own process of creating their own visualizations, but also working with their colleagues and their teams; because I think that’s the other piece that I really pull out of this book is that, yeah, you could go off and work on your own, but there is a part of this that requires a team, and if you have these different elements and think about it in that way, you can ultimately be more successful.
BC: So one of the figures we have in the book is kind of it’s a pseudo-triangle is really what it ends up being, and it represents to me the shift. So you start learning charts, and it’s a very elementary, just as Vidya had talked about, these are the types of charts you have, here’s your library of charts, and it’s a very pictorial representation. And to me, this is when you think about learning to read, you’ve got doctors, so you’ve got a lot of these very elementary books that supplement with pictures, and it’s not pictorial learning stage. And so, you start learning kind of these real basic graphics, and these real basic words, and that’s the pictorial stage.
And usually, as a practitioner, when you’re in that stage, it feels overwhelming, because it is. You are often trying to operate at a much higher level than you truly have the skills for, and I know because I was there. It was really intimidating, and then, you get all the books, and there’s a lot of resources, as you’ve mentioned, to learn “how to do it right”, how to refine, how to reduce, how to remove, and that ends up being that perceptual stage. And Vidya, you hit on this as well, and I’ll let you expand further on that.
But then we start shifting into where it’s like, we’ve done this drastic shift, and so, you actually see literally that pendulum shift from pictorial to perceptual. But then, there’s this other shift that happens, and that’s where we propose that instead of trying to move back and forth, what you actually do is you move up to a higher plane, and that’s that semantic phase. And so, then the graphic kind of draws down, and it’s fuzzy. I mean, it’s a really fuzzy kind of graphic intentionally. And then you finally wrap around everything with intent, and this is, particularly, Vidya, you can really talk about this, because that’s where a lot of your work lies.
VS: Yeah, I mean, Bridget, you succinctly described our model. What is kind of interesting with the semantic layer is there is this spectrum of very concrete concepts, which I think someone who has been doing data visualization, even for a few years, can understand or grok. But there’s also these fuzzy notions of language that need to be expressed through visualizations, like, if I am looking at a neighborhood of houses in Seattle, and I want to look for the best house to buy, what does best mean. The author who creates the dashboard might have their own mental model of what best means, but best could mean very different things for even the three of us. Right? Is it a good walk [inaudible 00:24:12]? Is it proximity to restaurants? Is it a good school district?
So that is where intentionality comes into play, like, what is the goal of this visualization, what sort of audience does it need to reach, and what is the audience’s goals in terms of how they want to consume and interact with a visualization or a dashboard to meet their needs, and there needs to be a way to embrace that fuzziness in semantics, where there’s either clear directive, and we might get into this topic of using text with charts, because I think text is a very effective way of either enriching visual communication, or it could exist on its own, kind of, going back to what Bridget alluded to with the medium is the message.
We are exploring other types of media beyond just the traditional dashboard, and so, other forms of communication might come in to bear, and that is why intentionality is sort of the glue that helps the author with certain directors in terms of how the dashboard needs to be crafted to meet a certain goal, and also provides guidelines or scaffolds to the user or the interactor, so that they understand what the goal of that dashboard might be.
BC: And what’s really neat about this whole process is I was able to draw in interpreting models, and you think that you’ve got language transfer, and how does this relate to data visualization, but it really, really does, because to me, I see my work, not as, oh, this is so different from interpreting, but I’m actually interpreting from data through charts. I’m rendering a message that somebody else is designed to take, and that takes into account where it’s happening. So where is this message occurring? When it’s interpreting, it’s occurring at a doctor’s office, or maybe in a court of law.
When I’m doing data visualization, it may be occurring at a business, where I’m giving this to a high level executive, or maybe this is on a flat panel screen where people are walking by it daily. All of that informs how I create that message. So that setting, and that kind of place matters. And then, you’ve got tone, what is the intent or the tone of this, and you really want to set that mood, and we really care a lot about that. And you can do that by color, you can do that by arrangement, there’s all these – and that was my logic of dashboards talk was really how do you create that mood, how do you create that message, and that also affects looking at, am I putting this on a telephone. And so, I’m scrolling this way, and that affects interactivity. So am I using a mouse where I’ve got a lot more refined clicking space, or am I using my finger where it’s actually a really kind of fuzzy, not very specific space. All of that has to be taken into account.
JS: You have both, so far, used terms like evolution, the next tier, kind of, looking ahead, and I wanted to ask that your book, along with some others that are either just out or coming out, Nigel Holmes has a book that I think just came out, Jen Christiansen has a book that’s on its way – these books feel like they are the next evolution, or the next tier of the data visualization book, the visualization library, as it were. There are and always will be the needs for the intro books for the grammar, the language, like, we need punctuation, so where’s the rugged spot, and how do you push the boundaries. And maybe Vidya, we’ll start with you, like, do you view this book as that next evolution? I don’t want to say next level, because that’s not really fair, but next evolution in the DataViz library, the DataViz field?
VS: Yeah, I think there are few levels or layers of data visualization that we’re trying to pull on. First of all, at least, from my standpoint, data visualization is no longer a field just for academia. I come from research, and we often get into, you know, let’s run a perceptual experiment to assess how useful this chart is, and I’m not trying to discount that it’s not important. It’s absolutely important, but it’s not just that. And we have reached a point where there is so much of wealth of knowledge that practitioners have brought into the field with Bridget, and so many more, I’ve seen this particularly with the Tableau community that I have been part of, and there hasn’t really been much effort in trying to bridge the two worlds.
I feel like the academic community and the practitioner community, we do talk and care about similar stuff, we just have different ways of expressing it, and there are kind of different sides to that same coin, so to speak. And so, we wanted this book to sort of bridge those two worlds together, where we come together on these common topics, and we share these different perspectives. So that was, I think, the first step, because I feel like there’s a lot of books that are either skewed more towards the practice side or more towards the research side, and so, we wanted to help bridge that.
And then, the second aspect is, yes, I mean, perception is just one form of that equation, right? There’s obvious questions that people need to understand, you know, how do you actually discern different magnitudes of values, compare different values, when is the bar chart more effective than a pie chart or vice versa. But to us, I think the most interesting set of questions is when we actually think of visualizations as a form of communication, and with our kind of diverse backgrounds that both really consider visualizations as a form of communication, we really wanted to bring to bear that it’s a language.
And yes, the punctuation is important, but let’s not stop there, let’s try to come up with ways in which we can string words and phrases together and come up with actual sentences that help assign meaning to what we see, and how do we use icons and colors, and have – and really paint a very deep understanding of how visualizations work. And we have also moved to a place where we are thinking of other forms of seeing and understanding data, and it’s not just through visual form, we have chatbots, we have Slack and Microsoft Teams, where people are asking questions, and sometimes you may not need a chart, you may need text or a different type of modality to bring insights to people.
So we really want people to kind of understand the breadth and depth of the field, and provide some sneak peek into where the field is heading towards as we share in the latter parts of the book.
BC: And I want to pull on a couple of threads there, because there’s a broader data representation that we’re starting to hit into. And I’ve seen that term kind of mentioned by a few other people as well, so it’s not my term, so don’t – but when you think about data sonification, when you think about physicalization, and just being able to represent data in a myriad of ways, I mean, some of this is really, really old. We’ve done this for countless of millennia. But then, some of this is really, really new. And so, intersecting a little bit of that, to me, is also part of the conversation.
I do want to go back a little bit where Vidya was talking about kind of research and practice, and what I found for me, it’s like, in the practitioner community phase, we’re always looking at research as proof, like, well, I did this thing, and I want proof. And so, we see research as that definitive proof. And what was really cool is we were talking one time, and working on a chapter, and it’s like, for Vidya, it’s like, something from research rolls out into practice, that is proof. And it was just like this lightbulb moment for me, like, we really are two sides of that coin, and that was the beauty in working together is really being able to kind of see these things come together, really see how they play together.
And then, what’s been really fun for me, at least, is seeing the research projects that come from the book, so we’ve written this book, we already did one research into text and charts, and it came initially from something I thought was throw away. We had talked about this, I put a segment in the text and charts kind of chapter just about over-texting. And a lot of times what we do is we divorce the text from the visual, so you have this huge long paragraph, and then, a chart, and they’re separated. And it’s not really useful, and so, you just have this big block of text on a chart, and they’re not really playing together. And so, we broke it up, and we had an example where you could see the text in the chart, and then, when we did the research, we really didn’t find that there was a limit to the text. I mean, we were using line charts, and I do think that that has a potential effect, but we were testing how much annotation could we put on this thing before people said it’s too cluttered, and we never hit that point. I would say we were modest.
VS: Yeah, there was no notion of over-texting when we actually did the research, which was kind of interesting.
JS: Okay, so I want to make two points, and then, move on to another question. So first is I would be remiss if I didn’t mention the VIS Con workshop, which is one of the IEEE workshops, that is trying to do this bridging. So I’ll put that in the show notes for people who want to check it out, but there is, I think, there is this clear movement to try to bridge the gaps. And Bridget, to your point you just made, I mean, this is a point that I talked about with all of my people I work with, and all my clients, I mean, it is funny to me, when I talk to people about let’s make your chart title more active, let’s tell the story, the argument in the chart title, and they’ll say, especially government folks, they’ll say, well, we can’t do that, because we’ll be deemed as being partisan or not being objective. And I’ll say, okay, yeah, I get that, let’s not do that, but let’s see what you wrote about it in the report. And Bridget, just like you just said, like 99 times out of a 100, 999 times out of a 1000, the sentence in the report, in the text is the argument, and then, they move on to the next thing. And there’s still this separation, I think between the visual and the text, which is amazing to me.
BC: It doesn’t surprise me at all, because we actually have this exact same problem with interpreting, it’s like, oh well, but I’m not in the room. It’s like, no, you really are. And there’s this whole model from interpreting called demand control schema. And what it does is it acknowledges your impact on the message, and so, you’re not this neutral party, the whole myth of neutrality is just that, it’s a myth. It’s a story we like to tell ourselves, and console ourselves that, oh, it’s our way of exiting harm, and it’s not.
And so, you’re an active participant in crafting that message, and regardless of who you are, you’re in the room, and typically, neutrality is only afforded to certain types of people, and that’s the other thing that we’re not necessarily discussing. But you are an active participant, and you are shaping that analysis, and in the interpreting world, you’re responsible for crafting a message that people understand. I can remember very early in my career, when I was still an interpreting student, I was sitting with a friend of mine, an interpreter came in to interpret forum, and she was explained to him that they were going to do tests to figure out whether the tumor he had was malignant or benign. And she spelled those two words, she literally spelled malignant or benign, and when she left, he looked at me and said, what did she say. And I had my friend, we don’t know if you have cancer or not, but they’re going to test and find out. And that’s what we do with data visualization all the time, we take no ownership over the message, we simply pass through, and we do a disservice to our users, because we’re not helping distill that message. I mean, that is the goal.
VS: Yeah. Okay, so on text, so I want to read this for listeners, because I think this might be my favorite sentence from the whole book. So this is in kind of towards the beginning, okay, so you both write: charts are not intuitively read, instead, consumers rely on outside narration, expanded supplemental text, and numeracy to navigate what the visualization shows. I mean, I think this is so important, and so great, and I want to give you just – I mean, I don’t even know if I have a question here, other than maybe my question is, do you think that when people say, a chart should be instantly understood, or like the three-second rule, I’m going to guess you both think that that’s not true. I mean, I don’t think it’s true, because of this exact point about text. So I don’t really have a broader question here, other than just give you a chance to talk about the importance of text, you’ve already mentioned a little bit, so maybe we’ll start with Bridget, I see you’re like, you’re raring to go on this question.
BC: Yeah, it’s [inaudible 00:37:25] bouncing back and forth, so I have a few things to say about that. So charts to me are like classifiers, and in English, a classifier word is a word like bundle. So if I talk about a bundle, you have no clue what I’m talking about, you know I’m talking about agglomerate of things, but it could be a bundle of words, it could be a bundle of software, it could be a bundle of books, and you have no context for what it is. [inaudible 00:37:48] tool for expressing data, and we fill them with intent, and we use perception to help guide users through it. But you have to have the words to convey what that thing is, you have to have the numbers to really provide a sense of scale.
And so, that’s where, to me, charts are a form of classifiers, and I can take that one step further, American Sign Language, we have these classifiers, where it’s like, I can take a car, and I’ll make this what I call a three-handshape, my thumb, index, and middle finger all three out while the other two are closed, and I can drive this thing around. And I can either, if it’s a car, I have to tell you, it’s a car and provide some context that if it’s bouncing up and down, maybe I’m on hills. But if I just do this by itself, it’s not meaningful. I have to tell you this is a car or helicopter for it to have meaning.
And so, that’s kind of part one. The three-second rule, I have a lot of probably personal rants about that, because our communication isn’t that efficient, and we have this, you know, and that’s where you end up [inaudible 00:38:53] bar chart hell, where everything is a bar chart, because that’s the fastest thing to understand. But back to the kind of the thinking fast and slow methodology, you want to have people be able to dig in and have that deep dive, you want to be able to unfurl information. And when you think about text expositions, we’re not just writing five-word sentences all the time, you have to vary your sentence, like, you have to give people something interesting to chew on. So that’s kind of my quick version of the rant, and Vidya, please feel free to chime in.
VS: Yeah, I mean, in the research community, I would say that visualizations are often contrasted with alternative forms of representation, like, tabular forms or written descriptions. But in reality, most charts are displayed with some accompanying charts, whether it’s titles, annotations, or captions. And so, there has been an actual push within the research community that text should be considered co-equal to visualizations, and calling on researchers to devote more attention to readability and how do you actually integrate both text and charts in terms of their takeaways, users’ takeaways.
And so, there’s been kind of growing body of work that explores that role of text that plays in visual analysis, and, in fact, there have been studies that have shown that users don’t always prefer charts, especially in chatbots, they actually just prefer text; and with modalities like voice, I mean, there’s no form factor that affords for any sort of visual display. And so, coming up with really pithy ways of sharing insights about the data becomes very pertinent when the modality is not conducive for any sort of elaborate visual representation.
BC: Which also intersects with accessibility, I mean, making sure that when people are using screen readers, they’re getting an equivalent message, and this is another area where we’ve historically fallen flat. So it’s that modality and the more you kind of think multimodal, the more inclusive you make that message.
VS: Yeah, and it’s inclusive for everybody, yeah, so I think accessibility is one big piece. And then, our technology has been moving towards automated or semi-automated data narratives that either accompany these charts, or they’re just shared with readers on a regular basis. So textual description has shown to be pretty influential with respect to these visual components, and that’s why we decided to have a chapter dedicated just for texts and charts. And as Bridget mentioned, we had a paper that was presented at the IEEE visualization conference that really goes into further understanding when is text preferred over charts, and how do the various semantic levels of text influence both the readers takeaway of what they’re getting away from the data, but also their own preferences, you know, is it text just describing statistical features in the chart, all the way to higher level takeaways? So it’s a very interesting field, and I feel like we need to pay more attention to text, because it is a first class citizen.
BC: And what was really interesting about that paper is we found that certain levels of text worked better in certain locations. So it’s like, if you’re making a very general statement, that’s a great place for a headline; if you’re starting to know physical notes about the trend, so it’s like it’s trending up, or this is a peak or – and you’re talking about what transpired, it’s best to do that in place. And to me, this really aligns well with American Sign Language, if I’m talking about certain things, I’m going to tightly reference so that deictic referencing, I’m making a space for it, I’m pointing, I’m using a lot of close and space behaviors of this incident right here. And all of that helps foster that communication.
I want to hammer one point about text a little further, and that is that I had a couple of conversations on Twitter somewhat recently around I don’t have success deploying scatterplots, is what I saw other people saying, and I’m like, I’ve never had a problem deploying a scatterplot. But I always annotate less than more, or I provide additional contextual clues. And then, I’ll also supplement with additional charts, so that way, when people are hovering over this piece, they’re getting additional information about what it is. And all of that’s just, you know, it’s that land marking, it’s the, you know, am I truly going the right way and deciphering this in the way that I should.
JS: Right. We’re coming out of the Information is Beautiful award, and I was fortunate enough to be able to judge a couple of the categories. And the thing that came out for me this year was the writing around some of these longer scrollytelling pieces was just really not that good. And it just, I think reinforces your message here, which is, the text in and around the graphs is just so important, and it’s like another skill set that we as DataViz creators need to have.
VS: Yeah.
BC: Absolutely. And we need to provide voice for it.
JS: Right. I mean, this whole, like, three-second rule thing or whatever, however many seconds people want to put on it, you know, if I showed you a bar chart with five bars and no text on it…
BC: It’s meaningless.
JS: It’s meaningless, right, exactly. So there needs to be some text around it, and how much text and where you put it, depends on all these factors that you’ve been talking about.
VS: Yeah, and it goes back to the conversation metaphor. I mean, text is an effective way to ground the conversation. You need to provide context so that people can be successful, and they are having a conversation with others, and it’s a very similar metaphor.
JS: Yeah, it’s interesting the parallels between the way I think we traditionally think about what language is versus DataViz, which is a visual language, and maybe we just haven’t been thinking about it in kind of the, I don’t want to say the wrong way, but we haven’t really been thinking about it in sort of a, I don’t know, merged way.
BC: We’ve treated it as a pictorial representation, and it’s not. It’s a lot more nuanced than that. It’s got a lot more systematic capabilities. I mean, we’ve seen that as far back as Grammar of Graphics, and being able to formulate it so that you can construct a wide variety of visualizations. And so, to me, moving into the next step of how do you construct these so that multiple charts are working together to have that conversation, integrating intent because that intent piece, we’ve really underestimated. And then, we really underestimate the semantic systems that allow us to express that message.
JS: So we’ve been going for a while, and I feel like we could keep talking for a while. I do want to end on one last thing, because at the last chapter of the book, you provide, well, it’s across several pages, but you provide essentially a big planning critique type of grid. And I was hoping you’d talk just a little bit about what the grid is, and how you thought about it, because it’s different than some other ones I’ve seen out there, this is very binary. It’s like did you do this thing, yes or no. And I also am curious about how you’ve used it, probably, Bridget, I think the question’s sort of different for each of you, Bridget probably in your own work or in work with clients, and then, Vidya, if you’ve used these sorts of things in teaching, and how students have sort of reacted to that. So I don’t know who to start with, maybe – I don’t know, who wants to start about talking about the grid itself?
VS: Bridget would start with how the whole thing came about, and then…
JS: Okay, yeah.
BC: So for me, the book is it’s a big book, I mean, there’s a lot there. And, for me, sometimes there’s a challenge of how do you take something very conceptual, and put it into play, how do you do it, how do you make it work. And I love books like Switch by Chip and Dan Heath, where you can download a workbook, you can literally do this process. And so, to me, that was a part of my model, and then, I used to train and mentor interpreters, I did a lot of training. And so, I actually took some of the materials from certification exercises, and had actual training benchmarks, and I liked the very Boolean yes or no, or it’s not applicable in this case, so we’ve got this 108-point checklist, if you will, of, did you do this thing or not.
And if you give people a fudge factor of, well, maybe I did this thing, you end up with these really fuzzy numbers. And so, I didn’t want it to be a score, because, A, the score wouldn’t be the same, and I feel like it’s a false equivalency. I wanted to focus more on, yes, this thing was done, no, it wasn’t, and these are literally the things I can either go fix, or I, at least, need to have a reason for why I don’t think I should do it.
JS: Yeah.
BC: So it’s more of a conversation piece than a grading piece in my mind.
JS: Yeah.
BC: Now, I have, I will say, I have used it for grading, I have literally made it zero and one, and I have occasionally put in a 0.5 just to evaluate maturities of organizations, or to look at a workbook and say, this is where you are today, and this is where you can go, and particularly, highlighting certain sections. So we’ve gone in, every chapter has parts pulled from it. And then, we’ve also pulled in these triangles, so we have these landmark triangles calling out certain key points, and we’ve actually put it right in the tool as well. So you can go back and find the thing that it references. So it’s not just they pulled it out of a hat, this is real in the book, and truly, every point can be kind of tied back to something.
JS: And to your point, and then Vidya, I want to give you a chance to talk about too, but to your point about how a grid like this can be used, it’s like any other skill, right? You can use it at the very beginning to be like, did I do this, yes or no, to a more nuanced thing as you become more experienced and maybe your data get more complex or something, where there’s [inaudible 00:49:16] but yeah, I think you’re right on, Bridget, that you’ve been talking about this whole hour, it depends on who the audience is, depends on your experience, and all of these things that a grid like this, you can use it to help grow and you might not, you know, you start today as a data visualization person who just learned whatever tool or Tableau or whatever it is, and you use it, yes/no, and five years from now, you’re like, well, I’ve expanded this, I’ve grown it in very different ways, and now I use a scale, and I’m focusing on different things.
BC: And you can see it by section, which to me is what’s key, because you can trace it back to, I’m not using text enough, my cohesion systems aren’t there.
JS: Right. So Vidya, I wanted to give you a second here to talk about whether you’ve been using it in either research or teaching.
VS: Yeah, I think, I mean, actually in both. So in research, we actually took a bunch of these heuristics from the checklist, and we kind of appropriated that to how to evaluate dashboards for facilitating what we call kind of a cooperative conversation between the dashboard by itself and the user, and I hesitate to say, reader, because these dashboards are interactive and they’re non-static.
But most recently, I just came back from a wonderful trip to India, and I used a lot of the book as a textbook in class, where I taught 60 students, who had no background in data visualization, over five weeks. And I use the checklists for two main goals, one of them was, most of these students, even though they did not have any formal training or any class on data visualization, were familiar with charts, because of how prevalent these charts are, you know, as simple as looking at a map, or looking at the news channel where they see political information and trends.
So I really wanted them to understand how to even critique visualizations that they see around them, when do you trust a visualization, how do you understand it. So we used the checklist or a modified form of that, so that I could get them to kind of talk about the visualization and critique it in a way with some guidelines, some guardrails. And as we moved along the course, and as these students learned how to create their own visualizations, either using Tableau or D3, I got them to self-evaluate their own creations through the checklist, and given that they had done this exercise on other people’s dashboards, they were familiar with the language and the expectations of the checklist to try it on themselves, and some of them as part of their final group project, took existing dashboards, identified certain places where they could improve the dashboard, and then, reran the checklist on their improved dashboards to see if it actually swayed the needle.
And we found in general that, and I did this checklist of exercise even with practitioners out in the world beyond just students, and what we found was people were pretty good at understanding kind of basic graphic design when creating charts, because there’s so much prevalent literature out there, but the use of icons, the use of semantics across charts, thinking more deeply about color, thinking about the placement of text, the use of scaffolds to guide the user in terms of where they should look at, being more thoughtful about which charts should be made larger, and where they should be placed, I think were things that sort of came out when they looked at the checklist, which I thought was kind of fascinating.
BC: And what I like as a practitioner is a lot of times, in the practitioner space, we provide feedback on our opinion, what we like, and it kind of becomes I’m going to make you in my image, and from an interpreting standpoint, you don’t like that. I mean, you just, because you have a rendition, you have a reason for the rendition, and I like that this tool puts the control back into the author’s hands. It lets you think about this is where the system is struggling, and these are the things you can think about doing, but I’m not explicitly saying make this green, make that purple, do this thing, this thing needs to be the size. I’m empowering the author to make those decisions so that they can take into account their intent and the semantic systems they want to leverage for this particular rendition. You’ll notice I’m going to keep hammering the word rendition, because it’s not – and if you put seven chart makers together, or seven data visualization people, you’re going to have seven different renditions, and it’s not that one is right and six are wrong.
JS: Yeah, right. Wow, this was fantastic. Not surprised. Great book, love it. Love where you’re headed with this in terms of where you’re providing, I think, a resource and service really to the DataViz field to sort of help move us forward, in the way we should be thinking about DataViz. So Bridget, Vidya, thanks so much for coming on the show. Thanks for staying longer than we’d planned. I know I usually say, like, oh 25-30 minutes, we’ll just chat, but this was so interesting, I couldn’t stop this. So thanks so much for coming on the show, really appreciate it.
BC: Thank you.
VS: Thank you so much, it’s a lot of fun.
And thanks everyone for tuning into this week’s episode of the show, I hope you enjoyed that. I hope you’ll check out their book, Functional Aesthetics, great book. Also check out the website, and, of course, check out all the links in the show notes. There’s a lot of stuff in there, so go explore it. Take a look. And, of course, if you have a chance, go check out PartnerHero, the sponsor of this week’s episode of the show; and if you’d like to support the show, please consider reviewing it on your favorite podcast provider. If you’d like to sign up for the Winno app, you can do a free version, you can do a paid version; or if you’d like to support the show financially check out Patreon, PayPal, or any of the other ways that I am providing content and connecting with you. So until next time, this has been the PolicyViz podcast. Thanks so much for listening.
A whole team helps bring you the PolicyViz podcast. Intro and outro music is provided by the NRIs, a band based here in Northern Virginia. Audio editing is provided by Ken Skaggs. Design and promotion is created with assistance from Sharon Sotsky Remirez. And each episode is transcribed by Jenny Transcription Services. If you’d like to help support the podcast, please share and review it on iTunes, Stitcher, Spotify, YouTube, or wherever you get your podcast. The PolicyViz podcast is ad-free and supported by listeners. But if you would like to help support the show financially, please visit our Winno app, PayPal page or Patreon page, all linked and available at policyviz.com.
