Lilach Manheim Laurio leads the Data Experience Center of Excellence at Visa, where she helps data practitioners across the company to elevate the quality of their data products, and improve their skills in data visualization and data experience design. Lilach’s data visualization work blends together a background in art history, library science, and human-centered information design, along with a passion for visual metaphor and pun.

Lilach has served as a Tableau Zen master (2018-2019), Tableau Public featured author, and co-organizer of her local Tableau user group chapter. She has contributed as guest author to the Tableau blog and the Nightingale journal, writing about design and user experience in data visualization. She has also spoken on topics ranging from visual metaphor to dataviz critique at Tableau conferences and user groups across the U.S.

Lilach holds a Bachelor degree in Art History and a Master of Library and Information Science (MLIS).

Episode Notes

Lilach | Web | Twitter | Tableau Public

Visa Chart Components
Elevating Data Experiences framework
Chris DeMartini (Twitter)
Frank Elavsky (Twitter)
Data Visualization SocietyThe Shape Parameter of a Two-Variable Graph (banking to 45 degrees paper from Cleveland, McGill, and McGill)

Related blog posts:

Books

Related Episodes

Episode #230: Vidya Setlur and Bridget Cogley
Episode #222: Richard Brath
Episode #213: Elevate Your DataViz Team

Sponsor

User Interviews connects researchers with quality participants. Participants earn money for their feedback on real products. There is a high demand for software developers and engineers to provide feedback on products being created for developers.

New Ways to Support the Show!

With more than 200 guests and eight seasons of episodes, the PolicyViz Podcast is one of the longest-running data visualization podcasts around. You can support the show by downloading and listening, following the work of my guests, and sharing the show with your networks. I’m grateful to everyone who listens and supports the show, and now I’m offering new exciting ways for you to support the show financially. You can check out the special paid version of my newsletter, receive text messages with special data visualization tips, or go to the simplified Patreon platform. Whichever you choose, you’ll be sure to get great content to your inbox or phone every week!

Transcript

This episode of the PolicyViz podcast is brought to you by User Interviews. User Interviews connects researchers with quality participants who earn money for their feedback on real products. So there’s high demand right now for software developers and engineers to provide feedback on products that are being created for developers. So if you want to help shape the future of the tools that we use in the data, data visualization, data communication field, this is your opportunity to provide that feedback to product developers. So you can go in and you can sign up for free, you can apply for your first study in other five minutes, and they will send you updates for surveys that are going to be related to the work that you do, so you can actually customize what opportunities you’re going to see from User Interviews. Now, most studies, at least, what I’ve seen here are less than an hour, they pay over $60 for an hour worth of work. Some studies are more the focus group, the one on one conversations, and those pay even more money up to several hundred dollars. So there’s some opportunity here not only to help shape the future of technology, but also to earn some money for your time. So if you’re ready to earn extra income for sharing your expert opinion on software development, engineer hardware/software, head over to userinterviews.com/hello to sign up and participate today.

Welcome back to the PolicyViz podcast. I am your host, Jon Schwabish. Hope you’re having a good start to the year, I hope you enjoyed the last few episodes of the show. We’ve been dealing with some interesting pieces on different data visualization books, and today, it’s actually going to be no different because on this week’s episode of the show, I am fortunate enough to be chatting with Lilach Manheim Laurio who I’ve been chatting with a lot lately, because I’ve been thinking more about how we as a field critique data visualizations, and the different types of ways that we critique data visualizations, and so, Lilach and I have been emailing back and forth. She’s been so kind and generous to take a lot of time reading through this very long post that I’m publishing today, along with this podcast episode about the field of DataViz critique. And Lilach has a book coming out later in the year about this concept of critique. How do we do it? How can we do it better, both as a critic and as someone receiving criticism? And how can we do that not just as individuals, but also how can we do that within teams and within organizations? 

And so, what you’re going to hear in today’s conversation is the start of Lilach’s work on this at Visa where she and her team created this data experiences framework. It’s like 10 pages long, it’s terrific, and it gives you a lot of ways to think about generating critique and to think about all the different aspects of data visualization that you might want to think about and talk about and try to improve upon. And I’ll link to that in the show notes, there’s a lot of books that we talk about here as well, not just in data visualization, but also in the UX/UI fields, so I link to those as well. So I’d really encourage you to listen to today’s conversation, I’d encourage you to check out the blog post that I’m publishing along with today’s conversation about my sort of view on DataViz critique and what we as a field need to do and what we need to do better, and where I think we need to focus our attention, rather than just saying, oh, this graph is garbage, this graph is great, what we need to do sort of to move the field forward. 

So I hope you’ll listen to today’s conversation, hope you’ll check that out, hope you’ll check out all the other resources on policyviz.com, on my YouTube channel, my Twitter feed, my Winno feed, wherever you like to connect with me. And if you have comments or questions, please let me know, you can connect with me at all those different places. So here is today’s conversation of the PolicyViz podcast with Lilach Manheim Laurio. 

Jon Schwabish: Hi, Lilach, good afternoon. Welcome to the show. 

Lilach Manheim: Hi Jon. Thanks so much for having me. 

JS: I mean, this is really exciting, because – I mean, no one really knows for us, but, like, we’ve been emailing at length for like a while about a lot of different topics, primarily what we’re going to talk about today. So this is very nice to actually see your face and chat in person. 

LM: Definitely. And definitely, thank you for your patience and openness to my diatribes on [inaudible 00:04:23]. 

JS: No, it was great. I mean, this is the thing that is, I mean, I don’t know, I think everybody has their own fears of feedback and rejection and criticism and something like you have. For me, it’s always been writing, like, that’s always the thing. It’s like you just have to learn to embrace the feedback and the critique, and that’s the only way to get better. And that, of course, is a great segue to what we’re going to talk about today, because you are working on a book on critique and feedback, which I’m really excited to talk about, but you’ve already done a bunch of that work, and I’m sure, at least, some of the listeners of the show are familiar with the data experiences. Is that really a checklist per se? 

LM: You can call it a critique framework. 

JS: A critique framework, yeah, from you and Frank Elavsky, and a couple of others, I think. So I’d like to talk about that.

LM: [inaudible 00:05:15].

JS: Yeah, and talk about the book and how exciting that is that’s coming out. So maybe we can just start, you can talk just, sort of, give a little background and how you ended up at Visa, and what you do there on the DataViz side? 

LM: Sure. So I think I have – I’m one of those, like, got into DataViz through a bit of a unique pathway. So I started, initially studied art history, and then, a few years later did my Master’s in Library and Information Science with a focus on information seeking behavior, which is really like a fancy, I guess, library term way of saying, it’s how people, when you have something to research, how do you go about looking for information. And then, I really actually have really got into, while I was doing that degree, kind of, how people deal with information overload, and how you kind of interact with a lot of information and decide when you’ve found enough information to answer your question or complete your task. 

And so, during kind of the last year of my program, I kind of, a little bit, accidentally discovered DataViz and Tableau, and it was like the light from the heaven, like, it was just such a crazy mashup of everything that I loved, and kind of learned, or had been doing before that, I never really anticipated using anything I learned in my art history degree, outside of museum. But definitely, I love how I can bring all of those things into DataViz, so I spent a few years kind of building dashboards and doing the BI thing. And I think just as I got further into it, and especially getting more involved in the Tableau community, I really started falling more in love with the teaching piece, and how can I enable others to improve their, especially the design piece of the skill set. 

And so, the opportunity opened up, I’d say, definitely a big part of it was the Tableau community, because otherwise, I would have known Chris DeMartini who’s running that group, but it was a really good opportunity to work in a center of excellence, which is kind of more the enabling piece. And I think that it really, really appealed to me, especially, about the Visa team is that it’s, we like to call it a small but mighty team. But we’re really, I think, partly because of the size of the team, but also, kind of, like the nature of working in a somewhat big company, we’re really focused on just kind of like tool agnostic data visualization. 

So, I mean, I think that’s just a really, a super interesting challenge, a chance to be creative, a little not just in making DataViz, but how do you kind of abstract out a little, but not too much. Like, what makes something a good data experience, a good data [inaudible 00:08:38] and really helping people, because it’s not just tools. It’s even like the difference between BI folks who are making dashboards for other people to do interactive analysis, or someone like a data scientist presenting results [inaudible 00:08:58] these are very different types of things that we’re making, experiences that we are making.

JS: Yeah, and I assume, both internal like data scientists internal to Visa, but also communicating with banks and customers and the external piece too.

LM: Yeah, Visa does a fair amount of data products that are kind of sold to customers.

JS: Right. 

LM: [inaudible 00:09:23]. I think you can kind of get a little sense of that. There’s a program that, I think there’s some, like, information on the developer Visa external website. It’s called VAP – I should know what that stands for, Visa Analytics Product. But also we have the VCC, which is Visa Chart Components, fairly recently put out there open source. So definitely, I think, because we have some, like a fair amount of the data product that are made are beyond just internal units, we have a bit more of the opportunity to do something like open source the [inaudible 00:10:11]. 

JS: Yeah, that’s great. So, I guess, we can start with the data experiences framework. So I want to start with where or why that came about, like, this is like a 10-page framework of doing a better job of providing feedback for all these different types of visualizations that you just mentioned across, I mean, I don’t know how many it is, I want to say, like, 20 different domains. So I’m curious, like, was that sort of demand driven within Visa, or was it supply driven where you and Chris and people on the team were like, we definitely need to do this better, let’s create this experience, this framework. 

LM: Yeah, that’s a good question. So I think it’s somewhat demand driven, and then that we kind of got asked, like, within our org, like, how can we improve the maturity of data products, and, obviously, that’s part of our team’s mission, but how can we measure whether a dashboard is good or not so good, and kind of how can we [inaudible 00:11:25] have some consistency in saying, these are the things that should be addressed to improve the quality of the experience. But the great thing was that we had, we got a lot of flexibility in terms of like how to do that, and I think, especially, because of our focus on being tool agnostic, that was kind of like one of the challenges of like what are things that are universal. But still specific enough that, like, it can actually help a person to improve visualization. 

JS: Right. 

LM: And I think the other thing is that, like, we really, kind of, I think Visa generally has a pretty high level of maturity, at least, compared to most companies for the design maturity, so we have a whole team for the design system, and also accessibility. So there’s already, like, people are already used to doing a pretty thorough accessibility check for any product, not just this product. And so, we wanted to see how we can kind of infuse more of human centered design, so how can we – I think, generally, because we’re data people, we think about the data, so when we design, we’re thinking about how can we best present this data, rather than what does a human need to do with the data first, and then, what information can we help them give them to do that. And I think similar with critique, a lot of times we focus on the things in the data visualization, like, do you have a good title, or did you – I don’t know, did you pick the right chart. These are all very system focused. In our case, the system is also data [inaudible 00:13:19] focused. 

So we really wanted to do something that was, instead of a checklist of going through, like, did you do this and this and this, like, being able to look at a dashboard or a DataViz, and saying, is it accomplishing these things. And so, it was great, because I kind of, I was like, oh, I wonder if we could take some of the concepts from heuristic evaluations, which are very well established in the UX field, but apply them more to a data product, and this is where I really love – this is where our team is awesome, because Chris was like, okay, what would it look like, just run with it. 

And so, I looked at some of the key UX usability heuristics, and then, there’s also been, like, I’ll say, a good amount of work on how to apply some of those heuristics to dashboards. Some of them are like very domain focused, like, the one that I looked at that I cite in the framework is like the, I think it had to do specifically with a domain of health information in a hospital setting. But still, it’s a good foundation to extract from. So we kind of took that as a starting point, and [inaudible 00:14:54] a lot of kind of mapping very information architecture kind of bring storming work. I’d like, okay, if you take something like visibility of system status as one where you want to have filters or really anything that a user interacts with, like, to be free users to not have to work to find that, so obviously for… 

JS: Right. So filter or a dropdown, yeah. 

LM: Yeah. So for DataViz, obviously, it’s a little, like, there’s some unique things that that applies to, so that could apply to, like, do you have legends that are easy to find, or something like that. So it’s a process of kind of like coming up with what would some questions be specific to a DataViz that would test for that heuristic, and then, expanding on that to what’s maybe some additional heuristics that aren’t yet, you know, because we want to go beyond just usability. So one that I know we added was it kind of has a somewhat similar, not necessarily heuristic, but more principal in UX of you want something to be useful. So usable is important, obviously, but useful as well. 

And so, we basically called it valuable, but we really tied it to what you learn, or what you’re able to learn from using a product. So does it answer the question that, or, what kind of business value does it add to a user if they were to use the product to – answer that question? 

JS: So in the UI/UX field, the heuristic would be a button, or a toggle, or a switch, or a scroll, just examples. But when you think about heuristics applied to DataViz, in addition to those pieces that you might have a filter or search bar and a dashboard, but in just any general data visualization, a heuristic would be defined as all those elements on the chart space itself, the legend, the axis, all those pieces would be a heuristic. 

LM: Kind of, so, I think the heuristic is the kind of test, and those elements are what you look at to see if they have achieved that heuristic, so, like, preventing errors, let’s say, that’s one of the common ones. So you could look at all the things, whether it’s how filters work, or a particular, like, a label on, and the option on a filter. So I think what expands is we still could use a lot of those basic kind of overall tests of, like, does the content on the page achieve the functionality, does it achieve these heuristics, which are kind of like outcome, the more describing what’s a good experience. 

But then you evaluate it all a little, and I think that’s one of the challenges, which is kind of our critique frameworking was, we have a set of heuristics. And then, we also have what we call design pillars. So those are really the things that you – the actual objects on the page or [inaudible 00:18:30] look at the what of what you look at, when you are critiquing a DataViz. The heuristics are like goals, the principles of what we want to – it’s kind of like almost how we describe if it’s a good experience. 

JS: Right. 

LM: And I think the somewhat unique thing we did is that the original like Nielsen’s original 10 usability heuristics, we took those and we had an additional six, some were like a little bit invented, like, I added one for limiting distraction, which isn’t really a frugal heuristic anyway. But what we did is, so we took those, like, individual heuristics and said, like, okay – we categorized them into five broad categories, and the category is more like an outcome, and I think it helps you. So we call them human centered heuristics and outcomes, to really make the point of like, this is really describing what does a great data experience feel like for a human who’s using [inaudible 00:19:43]. So, like, if you think of something like efficiency is something we tend to think, [inaudible 00:19:51] in DataViz, we put that, what we call, productivity. So it’s kind of framing it a little bit more.

JS: Differently, right.

LM: This is what allows them the user to feel or two, and then, that rolls up to the focused and clear category. So if data experience is focused and clear, it basically allows a user to focus on the most important information, like, for completing their analysis. So the heuristics are really describing it in even more detail, like, what does it mean to be focused and clear, what allows you to be productive. It still provides flexibility is another one. And I think the other couple in there is limiting distraction and directing attention. 

JS: It’s really interesting the way you describe it, because the change of the word efficiency to productivity, I think, is really smart, because efficiency in a lot of ways sort of implies speed, like, can I get the point of this graph as fast as possible, as opposed to how does this graph actually help me do my job or make a decision, which is different. 

LM: Yeah.

JS: That’s interesting. I wanted to ask you, and then, and I want to shift gears and talk about how this launched into the book, but you framed the work on this as sort of tool agnostic, lots of people are using lots of different tools, and this could be used for any of us. And I am curious whether you think that approach was limiting, or it was freeing, because I can imagine if you’re like, let’s build this for Tableau, you might, in some sense, say, well, then you should use this filter type for this type of data, and this type of map for this type of data, even though those can be applied to different things. So I’m just curious, in retrospect, if someone said, let’s make this for Tableau, or for Excel, or for JavaScript, or Python, whatever, would you’ve been like, oh okay, yeah, that puts me in a fairly simple box, but also too constraining? 

LM: So I think it probably just made the challenge really interesting, I would say. I think it did help to move away from building, thinking about building… 

JS: Building, right. 

LM: I mean, I think, one thing is, obviously, I think, through a lot of DataViz stuff as in Tableau terms, because [inaudible 00:22:25]. So when I think about something like negative space, a lot of times, I’ll think about in terms of like, did you add padding to a container.

JS: Yeah. 

LM: But [inaudible 00:22:37] I think that can apply to, I mean, there’s ways to phrase that for like CSS.

JS: Oh sure, sure, sure, right. 

LM: You know, anything, I mean, any tool will provide a way to create negative space with some sort of feature. 

JS: Right. But it does sort of put you a little bit in a box of, am I thinking the way someone who’s, yeah, like, just the way we think about it is sort of different when we’re in our little toolbox.

LM: Yeah, totally. I do think though that what helped is, at least, on our team, and I think definitely the design team, we follow more of the design process of designing something outside the tool first, so that’s sketching, and also then using a design tool like Figma or something. And I know that’s kind of a little controversial, like, how much work do you want to put into. It’s a skill to abstract it out, and not get too detailed into, like, oh, what’s the realistic data distribution for this bar chart. 

JS: I think that’s really what the challenge is, right? Like, I don’t know what my data would look like, until I make the chart with the data, and I can go draw what I want it to look like. 

LM: Yeah.

JS: But then I throw the data on top of it, and it doesn’t work, because I have some huge outlier, and so, I need to use a log scale or something. Right? 

LM: Yeah.

JS: I mean, you’ve very kindly sent me a couple of chapters of the book, which is, I am going to use this as a segue to look through, and, like, there are some, in just the two chapters you sent me, there are a bunch of sketches. While it is a little controversial because I think there’s just the data layer, I just, and I think a lot of people, and I’d put myself in this spot, are just like a little, not embarrassed, but shy about sharing our sketches, because, like [inaudible 00:24:25] but it is such an important part of the process, just to, like, to your point from earlier, to pull yourself away from the tool, because, I’m like a Tableau not quite a newbie but, like, half a step above a newbie, like, I’ll think like, oh, I just made this thing this morning. I was like, I want to make this thing in Tableau, it should be super easy, and it’s not, because I don’t know how to do it. Right? 

And so, I think it’s just those different levels and those different steps, which brings us to your book coming out later this year. And I am increasingly thinking about this, like, new evolution of DataViz books that go beyond the 101. So we have the Bridget and Vidya book, Functional Aesthetics. We have Jen Christiansen’s book Building Science Graphics. I think your book is going to fit nicely into this new space. So your book is, Let’s Talk About Data Visualization, and it’s really focused on – I don’t want to short shrift it, but it is kind of like the more in-depth version of what we’ve been talking about, this framework, and it’s really like pushing people into a more, I don’t know, like, a more concrete way to think about critique and feedback. 

And so, before I ping you with my questions, so people can hear what I was [inaudible 00:25:48] with the blog posts that I’m publishing today, along with this post, I just want to ask you to talk a little bit about the book and what you think it’s going to provide people with who are in the DataViz field. 

LM: Yeah, definitely. So first, I’ll just say, thank you for including me in that category. I’m definitely honored, some of those are some of my new favorite books, and it’s exciting to see the field moving towards that. I think often, again, when I was talking at a conference, I’ve noticed a lot of times design gets put into the intro level stuff, and it’s really not like, I’ll give you one book example, one of my favorite books is a Primer on Visual Literacy, and that was written actually in the late 70s I think, and it’s still – I’ve reread it I think at least five times, and every time, there’s something else I take out of it, and it’s a little crazy how applicable it still is. But it’s very much not a beginner book. It’s foundational. I would say, the more you learn about design, the more you can go back and reread it, and get a lot more out of it. But it’s nice to see kind of more and more people, like, books being written on the kind of beyond the basics, but it can still be very foundational. 

JS: Right. And I think there’s always going to be, I mean, I’ll say this as an author of one of those 101 books, there will always be a space for those because there’s always going to be people coming to the field new, they haven’t really thought about how to make a bee swarm chart or something like that. Right? And so, that’s sort of a new experience, but they, like all of us, are going to grow and going to make more and more, and what is the next steps, and I think one of the other things that we don’t see a ton about in the field maybe aside from Ben Jones’ books, and maybe Andy Kirk’s book is on teams and organizations, and sort of like a larger group, which I kind of feel like your book is really going to help people with is not just, you’re not just a person on your own making stuff and putting it out, and you’re being done. You are working in a team, maybe for a boss, but we all work for a boss, you’re working for someone. Even if you’re not working for someone, you’re a freelancer, like, you are working for your audience, or you’re working for your client. So there’s always more people involved, and I think that’s where the literature really quite isn’t at now.

LM: Yeah. And so, that’s kind of my goal with the book, I would say, I was very inspired by this book that’s more in the general UX field called Discussing Design, and that highlighted some issues in how general UX field was kind of, let’s say, had room to grow in terms of doing critiques better. And so, I think that’s a good example of the fact that it is a kind of growing pain. I think it was one of the ways the UX has matured as a design field, I mean, there’s many flavors of UX, obviously. So I think one of the main Manheim things, points that they make in that book that I’m really going to try to spend some time in my book expanding on what that might look like for DataViz is the idea that we pretty much agree in the design process, if you’re following a better design process. You’re separating it out, like, defining the problem, and coming up with solutions for the problem. And yet, a lot of times, when we critique, we just jump right ahead to the solution. 

JS: Right, yeah. 

LM: So It takes, I talk about it as like a skill and like muscles that you have to build, because when you’re so used to doing it that way, it takes relearning the habit, you’ll start out by catching yourself that, oh, what I just said, really if you look out for anything, I suggest or what if you did it this way, you know, those are all solutions, and that’s fine. So I think, like, starting to learn to kind of step back and say, okay, this is a solution that’s coming to my mind, that I’m saying, what’s the problem that I’m trying to solve with it, and then, kind of, like, learning to just kind of, eventually wean yourself to at least talk about the problem first. 

This is something, I guess, I’ve kind of adjusted as a, cause I think, I probably do get to do a lot more critique in my current role than I did in previous roles, at least, my own stuff. That, like, especially the people that are on the beginning of their learning curve, and especially in design concepts, you need to give them some solutions, like, it’s too, they don’t really know how – you can help them understand the problem that you’re seeing, but, they can’t, if you say maybe the typography hierarchy is unclear, it could really help your design being much more scannable, and help people find what they need and, in the page, if you improved your visual hierarchy. But, like, what does that mean to someone who’s never looked – who doesn’t know there’s like best practices for a type ramp, and the fact that you’re probably better off to get a little bit into detail, you’re probably better off starting out in the middle, and then, going to the edges of what’s going to be the biggest, and what’s going to be the smallest, and then, kind of, feeling like that’s a very procedural, you know, it’s just, I don’t know where I got taught that honestly. 

JS: Right, you just [inaudible 00:31:56] along the way, right? 

LM: Yeah, but it makes it so much easier to do something like that. 

JS: Yeah.

LM: So I think it’s both, but I do think that whole, so I’m not saying, I guess, I’m not saying don’t ever give someone, like, this is how you should do it, but just getting more aware of kind of starting out with defining the problems you see. 

JS: Yeah. 

LM: I think also, just generally, learning to see more when we look, that’s another big piece of… 

JS: And by that, you mean, like, more of the detailed pieces of the visualization? 

LM: Yeah, so, I mean, I think some of that is, we can take from kind of art history and visual analysis, being able to kind of identify, so let’s say you start with identifying what’s the focal point, what is it you’re most attracted to. And then, kind of, looking at what are the individual design decisions that are causing, what are the elements on the page that are causing your eye to kind of be move in a certain way, or be attracted to a certain part. 

So I mean, I can give an – I think this is a kind of common example. I think it’s a pretty good, like, specific application of this. So if you think about rules that we have, that we try to follow, and whenever we see something in the wild, we are like because I follow this rule. So one, I think, fairly old, not old-old, but a few years old is the idea of how do you decide on the size for a chart. Right? And I believe it was McGill, I think, that came up with a whole measure of the banking to 45 degrees. 

JS: Yeah, right, the banking. 

LM: So that’s a very, like, specific rule, that’s really a – but if you think about it, that’s really, like, describing a solution. But a very, like, rule based one of, like, make sure it has the 45-degree angle. And so, the problem with that, obviously, is there’s going to be a lot of exceptions. Right? Doesn’t work when – it especially doesn’t work when you’re working with like a line chart that’s pretty flat. 

JS: Right. 

LM: And so, like, abstracting out a level from that would be, well, we want to size it in a way to make sure that it’s true to the data and it doesn’t distort, you know, make the data look the way that is not true to the actual true shape and meaning of data. So I think that’s definitely a step in the right direction, but, like, because I think it starts you thinking on what is the problem that you’re… 

JS: Right, you are trying to solve, right. 

LM: Yeah, like, you want to make sure that people aren’t having interpretation errors, because it’s like a weird shape. But then I think like there’s even like further, like, when I say, when I think about how do we see more when we look at it, so you could start thinking about also, and I think this is where it starts coming in that, like, we can’t really look at just one chart. So one of the big things that I think you need to consider when you’re deciding on the size is, like, what’s the relative importance of the particular, like, is it a primary chart, is it a secondary chart, and that should really drive your, how big you make one chart versus another chart. 

But then, I think this is where my art historian, internal art historian comes in, is I always really tried to consider the wider frame. So in an interactive dashboard, I think we can usually assume it’s going to be – the shape of the page is going to be a product of your screen, or whatever screen it was designed for. But you’re rarely going to have like a perfectly square page shape, right, you like rectangular or long form. Even if it’s long form, you’re only looking at it like one screen at a time. So I think, a really important thing is, let’s say, it’s your primary chart, thinking about like, what’s the shape of the frame, and is the shape, so basically, aspect ratio is the shape of the chart echoing the shape of the frame, or contrasting to it. So, like, I think if you have, speaking in really broad terms, if you have a rectangular shape of the chart, and it echoes the rectangular shape of the frame, the big frame, then it’s very – it’s a kind of like static design, it’s like, there’s not a lot of movement in the design. 

JS: Yeah.

LM: And so, that could make sense, but then, just kind of like thinking about, if you make something, and maybe it’s a chart, maybe it’s something else, like, if you make, like, if you think about maybe like a menu, that’s a tall shape, that has a kind of vertical… 

JS: Yeah, vertical.

LM: [inaudible 00:37:27] yeah, if you have one chart that’s very wide, that could lead the eye left to right, but that could also depend on other, you know, what are the shapes of the other charts, and are they competing with the kind of shape that your main chart is creating against the [inaudible 00:37:49] frames, things like that, they’re just very, like, it’s really visual analysis, which. 

JS: It is, but it’s also, I mean, one of the things that we can talk about this in a second, because I think it leads back to the post that I published today that you helped me with, I mean, I think the other piece of it is recognizing that different creators have different goals. Right? When I think about a columnist at the Times or the Post, their goal is to get eyes on the page. Right? That’s what matters. Whereas someone working inside Visa or someone who’s providing a memo to whoever their boss, their goals are just different. You are creating something for your colleague at Visa, you know they’re going to read it. The goal isn’t to get people to click on it, because the goal is to make a decision, or, as you mentioned earlier, to increase productivity, as opposed to efficiency, which I started at on my notes. 

I just think that’s really smart, I mean, there’s been for a long time, this discussion that sort of ebbs and flows on impact, and how do you measure impact, and can you even measure impact. And I think the efficiency thing sort of goes hand in hand. I think the other thing that goes hand in hand with that is speed. It’s one of the things that I just, like, there’s this obsession with, do I get the message of the graph, just like as fast as possible, as if that should be a metric. 

So I guess, the sort of next question I want to ask is, when you see people doing critique today, and we’ll keep it sort of in the Twitter world, so public critique, not within, and not within teams, but in the public sphere, what do you think is the thing most people are doing the most wrong? Is there an aspect of critique generally that you think people are just missing the point where they’re critiquing the wrong team?

LM: Yeah, so how long do you have? I mean, obviously, I am writing a book about this, because… 

JS: Right, writing a book about it, right. 

LM: Right. I think you did hit on something that’s really important is there is a big difference between critiquing, like, critiquing someone’s work, and critiquing work with someone. 

JS: Yeah. 

LM: And really the critique that I do on a regular basis at work, and that sometimes I’ve also done in with public work, like with people I collaborate with is the second, and it’s really a conversation. Sometimes people make themselves available, and you can ask clarifying questions, but really, if you’re doing the second kind, where you get to have the conversation with someone, that’s why I call my book, Let’s Talk About Data Visualization. I mean, you should start, and that should always start with just kind of like when we start on developing data visualization. We’re trying to discover and ask questions [inaudible 00:40:58] you’re trying, but you’re trying to discover what was the creator’s goals and constraints. And then really trying to understand what are the choices they made [inaudible 00:41:14] design, kind of, how those work or don’t work for what they’re trying to do. And that’s really, you do that before you go onto, like, more, like, evaluating… 

JS: Evaluating…

LM: That you think isn’t working, right? Like you have to really understand, seek to understand what it is, you know, what the current design is, I think which we probably don’t do enough of that. The thing that, I mean, I think there’s definitely, like, room for the other kind of public or more you don’t get a chance to maybe talk to the creator. But just kind of organizing, that’s a slightly different thing. It’s almost like criticism, and I don’t know if that’s what, like, if you think about our history field, like, yeah, there’s critics that go look at artwork and write out what they think of it. But that’s almost like a different goal, like, so if you’re having more of a conversation with someone, your goal is to help them figure out how to improve their data visualization. Or you could be trying to kind of evaluate your own work, but your goal is to figure out what’s not working, and how to make it better. 

JS: Right, and to help other people do better, when they’re… 

LM: Yeah.

JS: Right, yeah. 

LM: I think the one thing that really is a bit of a – it’s a pet peeve, like, the thing that I actually like is starting off things a little wrong is when someone posts all feedback welcome. And I’ll be honest, I fell into that, sometimes I still have to catch myself. And I think that’s one of the things that I really loved about the Discussing Design book, they brought [inaudible 00:43:00] really, it’s a two-way street. So if you’re not getting useful feedback, like, if the feedback you’re getting is all, like, you need to make this button bigger or wider, some of that might be coming from the, like, you’re not asking for the feedback, it’s kind of like, and the person asking for the feedback. Obviously, critique on Twitter doesn’t usually end well, someone [inaudible 00:43:29]. 

JS: Yeah, well, there’s that. 

LM: I have a whole chapter devoted to that as well. 

JS: Yeah.

LM: How can you articulate and describe what is the feedback that you’re looking for. So what’s the feedback you’re not looking for? 

JS: Right. I don’t care about the size of the buttons or the color of the buttons, I know I need to fix that. But like, so instead of all feedback welcome, do you have like a good pithy phrase for people to like, this is like the hook, so they have to wait a few months till the book comes out, but do you have that phrase in mind for people? 

LM: I probably should have a good pithy phrase. I think I, yeah. 

JS: But the point is maybe there isn’t a good pithy phrase, because… 

LM: Yeah. 

JS: Sometimes maybe you do want to know, like, are the buttons in the right spot versus the colors of the line chart, because I can’t control that, because this is what our company follows, and it’s a red line, and that can’t be changed, but the button is the button in the right spot. So maybe there isn’t a phrase, and it’s just, it’d be more focused in your soliciting. 

LM: Yeah, and obviously, this is tougher to do in Twitter, like, what we’ve done at Visa, and I’ve seen something similar in the DVS channel on critique is we have like a set of three questions that when you come in and ask for critique, you kind of fill out ahead of time to explain to people what type of feedback you’re looking for. In the book, I kind of outline, like, there’s two main questions that you as a reader should kind of insert to help people understand what type of feedback you want. The first is just kind of what’s your goals and objectives to whatever level of detail you want to get, like, what is the design trying to achieve. And the second is what are the elements that you want to have evaluated. So I think it can be helpful to kind of think about it in terms of – I use design layers in the book, so is it like something to do with the information architecture, or is it the thing that we usually tend to fix up to focus on, which is the chart design, so is it the information visualization layer. 

So I do, I guess, give some tools for describing what parts of the information product you want, like, you want to get feedback on. And it may be like a specific part of it, right? Let’s say you want help with the interactions, interaction layer, so a really broad question would be how well does the interactive features that I have, how well does that support the analytical flow, or the questions that I want to enable people to… 

JS: Right. The [inaudible 00:46:28] make you more productive. 

LM: Yes.

JS: Make you more efficient, and make you more productive. 

LM: Yes, or do they give you interesting, actionable answers. 

JS: Yeah, right.

LM: Like, a lot of times the classical example is having a million filters. They don’t really help to come to new answer.

JS: Right.

LM: Yeah, but the other one would be like maybe, do I have clear feedback, where it’s clear that when you interact with some feature, it’s clear what happened. 

JS: Yeah. 

LM: Right, so that’s like a more specific one. 

JS: Yeah.

LM: Or even, like, is this button difficult to find? 

JS: Right.

LM: Right. 

JS: Or the outcome’s obvious is what I have to do obvious right.

LM: Yeah. 

JS: So I think in the short term, before the book comes out, and you have all these checklists, by the way, everybody, for those of you listening, watching, the checklists, I don’t want to call them checklists, because they’re not really checklists, they are frameworks, and they are cues, are amazing, they’re going to be super helpful to you, so be ready. But here’s a good, in the meantime, till the book comes out, a good lesson for folks to keep in mind, like, when you are asking for feedback, be specific, and be purposeful, so that you can get the feedback that you want. And because we’ve already been talking for an hour, I want to wrap up, but I think also as the critic, to be a little bit more purposeful and thoughtful, and I like this idea of critiquing someone’s work versus critiquing with someone are two very different approaches. So Lilach, thank you so much. I mean, we could keep going, but, at some point, people are going to, like, they’re going to hit like two times speed, yeah. 

LM: [inaudible 00:48:10] because I really, I know this, kind of, started a big Twitter conversation, but I think I still think it really crystallizes what, I guess, my book tries to do and what the critique framework also tries to do. It was by, let me see, I wrote it down, it was by Dr. Cat Hicks, and she said, we try to make often, we try to make complex problems easy, rather than making it easier to work on complex problems. So I think if there’s any one thing that I think we can, like, push us forward in how we critique, is like we do need to think a little deeper. Like, I think we’ve done some initial really, we have a lot of really great work on very specific rules that kind of cover the easier things. But, and I totally get the instinct to want just a rule, like a do and a don’t. But it really is about, like, how to think a little deeper about all these things. 

JS: Yeah.

LM: And so, I do think there’s space for tools, like, I hope the critique framework is the beginning of that.

JS: Yeah, the four tools. 

LM: Yeah, that help you to think through some of those deeper things, without giving you a set of simple yes-no answers. 

JS: Right. I mean, I think that’s the challenge with the checklist, where there’s a box, where it’s like you check a box, check a box, check a box, because, as you said, sometimes they don’t apply, and some things, I would say, are more important than other things. I mean, I don’t know, like, integrity of the data is more important than the font size of your title. Right? 

LM: Right.

JS: I guess, if I had to come up with that, it’s not a pithy statement per se, but I think the message here is to think more deeply, I think on both sides is what I’m hearing from you, right? 

LM: Yeah.

JS: As a critic to think more deeply about what you are critiquing, and as the person being critiqued or soliciting critique, being more thoughtful about what you want people to say, and maybe how you respond, even to those, what I kind of call the hit and run or drive by critiques, it’s like, this is garbage, and you’re like, maybe you respond in kind of a different way to be like, well, I know that you don’t like the colors, but that’s the branded colors, and that’s what I use, but is the graph type, like, is that useful, so I like that. My wife would call it a compliment sandwich, so you give a compliment, and then, some critique, and another compliment in that. 

LM: Yeah. 

JS: Okay, so the book comes out when, like, in the fall of this year, summer? 

LM: I’m not sure that…

JS: The publishing world of – the mystery of publishing world. 

LM: Yes.

JS: Right. Okay. But in the meantime, people can find you where, on Twitter, for sure? 

LM: You will find me on Twitter. I have my regular handle, where I will be honest, I tweet about probably politics as much as DataViz. But you can also, for updates and sneak peeks about the book, you can follow DataViz Crit. 

JS: Okay, great. I’ll put all this and links to everything that we talked about on the show notes, so people can check it out, because there’s a lot here, and a lot for people to think about, and hopefully do better individuals and as teams. So this was great. Thank you so much for coming on the show, taking time out of your day, and yeah, I’ll talk to you soon. 

LM: Okay. Thanks, Jon. 

JS: Thank you. 

And thanks everyone for tuning in to this week’s episode of the show. I hope you enjoyed that. I hope you’ll start to think a little bit more in depth, a little bit more creatively, a little bit more with sophistication and purpose about your efforts in critiquing and receiving critique in your and others’ data visualizations. Be sure to check out all the links on the show notes. I’ve got links to all these different checklists and frameworks that we talked about in the interview and links to all the books that we talked about as well. There are some great new books out on the market, and I hope you’ll check them out. And, of course, don’t forget to read the blog post that is up at policyviz.com where you can sort of get my take on DataViz critique. So until next time, this has been the PolicyViz podcast. Thanks so much for listening. 

A whole team helps bring you the PolicyViz podcast. Intro and outro music is provided by the NRIs, a band based here in Northern Virginia. Audio editing is provided by Ken Skaggs. Design and promotion is created with assistance from Sharon Sotsky Remirez. And each episode is transcribed by Jenny Transcription Services. If you’d like to help support the podcast, please share and review it on iTunes, Stitcher, Spotify, YouTube, or wherever you get your podcast. The PolicyViz podcast is ad free and supported by listeners. But if you would like to help support the show financially, please visit our Winno app, PayPal page or Patreon page, all linked and available at policyviz.com.