Happy New Year, everyone! I hope you had some time to rest and relax with friends and family. Thanks for tuning into the show.
On this week’s episode, I’m excited to chat with Amelia Wattenberger, a developer and designer focused on data visualization. She’s always looking for ways to make data understandable—whether through creating dashboards, crafting explainers online, or writing Fullstack D3 and Data Visualization.
Amelia and I talk about her new book, Fullstack D3 and Data Visualization: Build beautiful data visualizations with D3, as well as her work coding and developing data visualization projects. We also talk about her installation work and all the things that go into those types of projects. I hope you enjoy the show!
Episode Notes
Amelia’s Website (check out the Photobooth and Weather Circle projects!)
Fullstack D3 and Data Visualization: Build beautiful data visualizations with D3
Support the Show
This show is completely listener-supported. There are no ads on the show notes page or in the audio. If you would like to financially support the show, please check out my Patreon page, where just for a few bucks a month, you can get a sneak peek at guests, grab stickers, or even a podcast mug. Your support helps me cover audio editing services, transcription services, and more. You can also support the show by sharing it with others and reviewing it on iTunes or your favorite podcast provider.
Transcript
Welcome back to the PolicyViz podcast. I am your host, Jon Schwabish. Hi everybody, Happy New Year! I hope you had a happy holiday season and were able to take a little bit of a break, maybe do some traveling, catch up with friends and family. And thanks for tuning back into the podcast. I’m really excited to start off 2020 with a great interview, a great discussion with Amelia Wattenberger that you’re going to hear in just a minute. Amelia has done some incredible things in data visualization. She’s created a new tool called Figma. She has just completed a book that is now out about D3. She’s done some great stuff with data visualization, installation and installation type work, all of the things that we talk about in this week’s episode. So I hope you will enjoy it and I hope you will check out a lot of the things that are coming up for me this year. I’ve got a new book coming out in early February. I have some data visualization workshops coming up in April and a bunch of other things. So please stay tuned to the podcast, to the blog and to the website. And if you’d like to support the show please head over to my Patreon page or just leave a comment in the comment section or give it a shout out on Twitter or on Facebook or leave a review on iTunes or wherever you listen to this show.
So thanks a lot for tuning in. I’m really excited for this discussion with Amelia. I reached out to her after seeing a few of the tools and projects that she was working on and she was gracious enough to agree to come on the show. So here’s my discussion with Amelia.
Jon Schwabish: Hi Amelia. How are you?
Amelia Wattenberger: I’m good. How you doing?
JS: I’m doing great. It’s Friday. So you know it’s good because it’s Friday.
AW: That’s true.
JS: What’s going on with you?
AW: Not much. We just get about two feet of snow. So winter has started.
JS: There you are. Okay, you’re already in, literally into it already.
AW: Yeah.
JS: We just started getting cold. So it’s still like the 40s. So we’re Fahrenheit for those of you who are used to some other measure of degrees. Yes. Well thanks for coming on the show. I’ve been watching your progress on various projects in awe really cause of all the amazing things that you’re doing and I want to get to all of them. And maybe we’ll start with having you just tell folks a little bit about yourself and your background and we’ll just go from there.
AW: Yeah. So I’m a web developer and designer. I work at a startup that deals with content analytics. So basically we help publishers and brands understand attention to content online. So for our main app, for example, if I were a journalist working at wall street journal, I’d log into this dashboard to understand like which of my articles people are spending the most time on or which ones are getting the most traffic from Facebook. And our other dashboard looks at aggregate attention. So I might spend a week figuring out the best way to visualize data for where people are finding articles about sports. So that looks like I’m figuring out what the users would be looking at the data and going through a ton of iterations of like sketching different visualizations and then coding them up and looking at how it works with the data and then going back to the drawing pad.
JS: And are all the dashboards that you’re building, are those all custom built by that that you are building in D3 and JavaScript?
AW: Yeah. They’re web dash words. The two of them use different front end frameworks. They all use D3 JS which is one of the main threads.
JS: Right. And we’re going to get to the book in a moment. I’ll leave it as a mystery for people who haven’t been reading the show notes of the tool we’re going to talk about. Are you pulling that data in in real time?
AW: Yeah, so actually most of our development team is more backend engineers. So it will like real hard problems here are processing all of the data and getting it so that I can look at it in real time, which is it takes a lot of work and it’s really hard to get that kind of process. So streamlined that we barely ever have downtime.
JS: Right. So can you talk a little bit about that process? I’m sure there are people who are interested in learning more about how to pull in either real time data or data that’s updating fairly frequently and then moving it into a front end dashboard either in a built in tool like Tableau or Power BI or a custom solution that way you’re doing it in like a [Indiscernible] [00:04:53] or D3?
AW: Yeah. Oh man. I [Indiscernible] [00:04:58] find out how little I pay attention to the rest of the work our company does. All I know is they have a lot of fun sounding technologies like Cassandra. And there was something called Celery for awhile, which I always got a kick out of and I wish I could tell you more honestly.
JS: Okay. But you have to then deal with all these different, so you have a backend team that’s bringing that all in, but you have to work in on the side of cleaning it and then visualizing it. So does that introduce issues and challenges that are different from a lot of your other projects?
AW: Oh, for sure. It’s like when you look at the interface, you get really only the tip of the iceberg there where with a static visualization you really can look at it and look at the data in a few different ways and say look, this part right here is interesting. And then look at that part specifically. Whereas this is kind of like one step back from that where you’re trying to build a visualization so that other people can have those insights or trying to automate those insights that someone might get from aesthetic visualization. So all of those kinds of problems. And then also thinking through like, what does this look like when there’s no data, what does this look like when the data is wrong because the tracker could be implemented incorrectly. What does it look like if the data has lagged behind or, yeah, there’s so many other things to consider when you’re visualizing data that is changing all the time and you really have no control over it. And it also makes it a lot harder to do fun things like annotations or I always see these fun graphics in journals where they’ve annotated parts of the visualization and I always get jealous because to do that kind of thing with dynamic data either it’s really hard and often it doesn’t make any sense.
JS: So I’ve asked this the following question to a few people on the show and I’m curious on your perspective, putting the tools and the technology to the side do you think it’s harder to publish a static visualization than an interactive visualization in the sense that the types of texts and annotation you need in a static data viz are different than the ones you need in an interactive data viz?
AW: I don’t know.
JS: I feel like that’s most people reaction. Like ooh.
AW: I feel like we need to trying to get me to start a fight.
JS: I’d be totally fine with that. Yeah. If you want to start or go for it?
AW: I do not. I wouldn’t say harder but they’re definitely two totally separate things like two totally separate piece that have different considerations. Like at first glance it would seem like a static visualization would be easier, right? Because it’s just one iteration of what the dynamic visualization would have to deal with. But at the same time, because you only have one iteration that iteration has to look really good and be able to explain itself. Whereas when you can make an interactive visualization, you get things like tooltips and someone could dive deeper into a specific data point if they wanted more information about it.
JS: Yeah. It’s just interesting. I think it’s always just interesting to think about the differences between the two ways we interact with data. So let’s talk about some of your recent big projects. So you have this new data tool called Figma and you also have a book that is now out in the paper version which is my preferred medium. Do you want to talk about Figma for a while? I mean where, where did it come from? What was the development like and how are you seeing people use it now?
AW: Yeah. Figma is really interesting. It’s basically, it’s a vector design tool. So if anyone’s used Adobe illustrator, it’s kind of like a strip down modern Illustrator that has like a multiplayer mode. So people use it a lot for like user interface design where multiple people could be working on the same design file at the same time or they could be getting like comments kind of like a Google Word document and like on their designs. And they recently added support for plugins. So they have this plugin API where any developer can go in and create a plugin. So I went in and made a plugin for data visualization. I think I’m calling it like DataVisor or I’ve never said that out loud. So maybe it’s a tentative data viz or.
JS: Okay. This is good. We can just figure out the pronunciation now so everybody can be clear on that. But now we have these two. Do you want to just leave that as a mystery for people?
AW: It’s really up to your own discretion.
JS: Alright.
AW: So basically what it does is you can import a Jason file or a CSV file and you can, it’ll spit out a line chart, a bar chart or a scatter plot and also has like a random data mode. My main motivation for making this is with dashboard design, the main thing that I have an issue with is creating data visualizations that look like data like if you’ve ever tried to create a line chart in a vector graphics tool you’re kind of just clicking around and these points aren’t necessarily going to be equally spaced or it’s not going to make a lot of sense. So making a, a good mockup with spark-lines has been really hard in the past. And I have colleagues that will look at a real dashboard and then grab the SVG from that page and then put it in their design tool. And I also find that there’s a big gap between product designers and developers for applications that have data in them. So designers will make something that looks nice and then hand it off to the developer and then the developer will coat up like a dashboard interface and then they’ll go back and forth a lot because what the designer has created won’t necessarily make any sense for the data that is being piped into the dashboard. So having a way for designers to have a quicker feedback loop of like, here’s the dataset, let’s look at what it actually looks like when I make it into a line chart or when we do a scatter plot. I think hopefully will help and hopefully it will also help designers think in more of like a data mindset as opposed to just making something that looks nice.
JS: Right. I mean one of the main challenges I’ve always found about using Illustrator, and I’m by no means an illustrator expert, but is always that you had to break the data away from the chart to be able to do real design and manipulation with it. So is the goal here to try to make those two things hang together more?
AW: Right. I think that would be great.
JS: Yeah. So you have these relatively standard chart types. Are you planning to try to expand that more and have you seen people trying to use the plugin so that they can accomplish these design tasks and design goals?
AW: Yeah, that’s a good question. I don’t totally know how people are using it right now. All I know is that like five and a half thousand people have downloaded it. But I don’t know if they’re using it or what they look forward to.
JS: They downloaded it. Yeah.
AW: Yeah. They have it. They can use it.
JS: Downloads are all that counts. Take it from a person who has a podcast it’s just downloads, not listening.
AW: Yeah it’s right. And there’s also a few other chart plugins, which I’m happy to see because it’s not like I get anything for someone using my Figma plugin. I have other chart types. Figma has this concept of components. So I would like to have some other way to bind data to charts other than like do you want a scatter plot or bar chart because I like to at least personally I like to approach data visualization without thinking of existing chart types and just thinking through what are the different metrics and how can we visualize each one of these, should we do color or position or size or what kind of physical attributes should we tie what metric to. So I would like a way to kind of tie components into the data that’s being imported but I haven’t exactly thought through what that would look like and it’s going to be kind of like a hairy interface design problem.
JS: That’s interesting because you seem to come from this combination of a design background and an interactive developer background with all the work that you’ve been doing in D3 and JavaScript. And so I’m curious about your thought process when it comes to creating a data visualization because it sounds like you’re coming at it from this side of what encoding should I use? Is it a bars? Is it line? Is it dot? Is it whatever. So where do you think that that thought process comes from? Like is that, I mean I don’t know if this is the right way to phrase the question, but do you think it comes more from the design side of you or does it come from more of the D3 side of you which seems to be more of a binding of data to objects?
AW: Yeah. I actually think it’s neither. It’s anything, it’s the developer side, but I think it’s also, I went to school for psychology and neuroscience. And my husband is in research so I have a pretty deep research background. And so I think it comes from the things I first learned that had to do with data viz, had to do with like perceptual studies which attributes, physical attributes are easiest to distinguish, like can we distinguish a position better than area. So that’s where I think it comes from.
JS: Interesting. Let me back up then. So you have this background in psychology and then how did you get from there to design and then to development? So I should probably should’ve started with this, but all these things linked together in such an interesting way. So I’m curious how you went through your career path to where you are now.
AW: Yeah. So when I was in college, basically both of my parents are programmers and I said, there’s no way you’ll get me to do that. I’m not sitting at a desk all day. That sounds horrible. So basically I wanted to be like a prison psychologist. I like something intense like that. So right after school ended I moved down to Texas to work in a research lab because I needed more research experience before applying to PhD programs. And while I was there, I hung out with a lot of grad students and then decided that I didn’t want to go to grad school. And so I was trying to figure out how do you use my background to do some kind of career transition. And on the side I was kind of just recreating my personal website over and over again just for fun. And somehow I got this a very similar job to the one I have now, just front end developer, but at a small enough company that I’m also doing the design and for a dashboard. So there’s a lot of data visualization work. Um so I guess the long story short is like, I have no idea how I got into this, but I really like it.
JS: And you ended up as a programmer, so your parents must be super proud and also a little vindicated at the end.
AW: I think yeah deep down they are.
JS: So tell me a little bit about the D3 work you’ve been doing in addition to the book that just came out, but how you initially to sort of go back a little bit also into your path to where you are now, how your origins into learning D3 and obviously that’s what a lot of the book I think is about, you know, learning D3 and, and what, how do you view, or how do you think about helping people learn that particular language?
AW: Yeah. This is something I’ve thought a lot about. I think I learned D3 in the same way that most people learn D3. So I was working at a company, as a developer, I had to build a dashboard. And basically if you Google like bar chart in the browser or a line chart Java script or D3 bar chart the first few results will be basically examples with code of how to do a simple chart. And so what I did and what most people do, I think is to copy and paste that code into what they’re working on and then it’s a lot of code. So you slowly change line by line until it looks kind of what you need it to look like, and then you move on because you have other stuff to do and then you have to maintain that chart. And so that code just gets messier and messier because you don’t totally understand all of what it’s doing. So after, I don’t know, eight years or something of doing that kind of stuff and slowly getting better and solely figuring out the different parts of creating a chart earlier this year I decided that I had a really hard time learning this. I don’t know if there are good resources for people who need to do this these days. And it seems like data visualization is only getting bigger. So I put together basically all this stuff I know and the basics of creating a chart with D3 into this book Fullstack D3 and Data Visualization. Honestly writing it was a really good way for me to sit down and solidify like all of the things that I’ve learned and another reason it’s so hard to learn is that data visualization is a combination of so many different fields. And like if you’re learning computer science at college, you’re not going to take a data visualization course, right? Like no one really gets taught this combination of skills, which is like web development, creating things in SVG, the fundamentals of data viz design like human perception, like user experience design. Like it’s this weird constellation and I think you can see it on your show where most people come from totally separate backgrounds. And people get to data visualization and in different ways. So having one like introduction for people who have to do something like this, I’m hoping, and it seems like it has been really useful to others.
JS: Is there and I don’t know how to word this question really. Is there a way that people need to think in JavaScript or in D3 that is distinct or different from other coding languages in the way you think about it? Like do you need to have a shift in your brain if you’re, say R programmer as a basic example like, is there something that people need to do fundamentally different to able to move from a language like R to a language like D3?
AW: I’m currently wishing that I knew R so I get better answers.
JS: Right. Yeah. So I guess it’s moving from more of like a linear coding framework, the way R data or SAS might be to what I view as my rudimentary understanding of more of a recursive language that doesn’t necessarily work in a linear way the way an R or [Indiscernible] [00:21:16] does.
AW: Yeah. Potentially with R also, are you usually creating interactive visualizations or is it usually you’ll create something static?
JS: Yeah, it’s usually, I mean, again, I’m not a pro coder in R, but like usually it’s a static graph or it’s an animated graph and then there are ways to pipe it over to interact, to create interactive graphs. But I think the primary, it’s core intention is to create static graph.
AW: Yeah. Python is like that too. I think some of the biggest differences just lie in the fact that at least what I build are interfaces for people to interact with. So instead of thinking about what chart am I going to create or what visual am I going to create, thinking through how would I use or interact with this and how to all these different, like you can have multiple charts set one acts as a control for another chart. So thinking about like interactions and how it works as kind of more of a user experience.
JS: As opposed to kind of pulling from a library of graph types.
AW: Right. Yeah. And maybe this is what you’re getting at too is with D3, D3 is probably the lowest level library that you can get for creating data visualizations and any kind of higher level chart library that you’ll find for JavaScript is probably going to be using D3 internally. So there’s a ton of libraries. And I think oftentimes when people are creating charts and other languages it’ll look something like I want to make a line chart and they use their line chart function and then they have a parameters that they assigned to it. And with D3 you’re really understanding different concepts like one of the D3 concepts that you need to learn is a scale. So how do I confer numbers from the data world to the physical world. So changing like temperature values into pixels. So how far a dot is to the right. So scale will help with that conversion. So you’ll create scales for the X dimensions and the Y dimension and then you have to create elements for each of the items in a scatterplot. And then you have to create the axes. So with that flexibility, you can basically make whatever you want, but creating a chart becomes much harder and you really have to understand a lot of different concepts and how to put them together.
JS: Right. So building on the way you were talking earlier about building off of these encodings that’s really the sort of the ground up.
AW: Yeah.
JS: Interesting. You mentioned binding things or creating these data elements in the physical world and also how you create interfaces for people to interact with. And a lot of the projects on your website seem to be not just interfacing or interacting with pieces online in a browser, but also like in the physical world. And I’m curious if you could talk a little bit about some of those projects and you know, how you navigate between these two ways of interacting with data.
AW: Yeah. A lot of my work is on the screen and while that’s great, people can only interact with stuff that I do when they’re looking at a computer screen like at their desks or something, which I find kind of frustrating. So I will take any opportunity I can to physicalize anything. So, for example, I have a laser cutter and a 3D printer in my basement and I’ve gotten to do some really fun, more like installation type work because my old company was really supportive of them. So, for example, in the entrance of the office I created a visualization that show like the clock of where we were in the workday, any announcements happening that day. And it also had a visualization of the weather. So like what’s the temperature and what’s the chance of rain over the day. And that brings up a lot of fun questions that I don’t normally get to deal with which is what hardware are you going to use, how are we going to display it? How bright is the area? So we grabbed a short 3D projector and then projected a website onto like a frosted portion of the glass in front of the office. And one nice thing about more of a physical installation than a website is that you know more about the context that people will see it so with the website, someone could be at their desk or sitting on a train or like on their phone in bed. Whereas if this is at the front of the office, you know that people are going to be at the front of their office when they’re looking at it, which is really nice. And one of the other more fun things that I got to do was we threw a big South by Southwest party with the band Spoon. So we wanted something fun and our [Indiscernible] [00:26:20] was Éleuthère [Indiscernible] [00:26:21]. So I went to play around with having an augmented reality photobooth that like found your face and then put a wrestling mask over it. I was like, yeah, this will be really easy. Right? And I realized that I had to basically learn C++ to create it, which I do not know at all. And still don’t know.
JS: Just enough though.
AW: Yeah. And anything I learned is all gone. So there is some fun problems there with like how does someone trigger taking a picture because if you look away, it would lose your face. So I got these foot panels or like buttons that you press with your foot. And then basically what I learned there is that with installations like this, people aren’t giving it their undivided attention. And also at South by Southwest parties, people get very drunk. So I kind of had to stay by that installation like all night to teach people to press this button with their foot.
JS: Yeah. See that on itself seems to be a good research project like how drunk is too drunk to be able to press the button to take the picture.
AW: Yeah. And basically another takeaway is if you’re going to do any user experience research and you really want something to be good, just test it on drunk people. Great. If they can figure it out anyone can figure it out.
JS: Anyone can figure it out, right. I mean it is interesting. We had an event here at work a few weeks ago with Alberto Cairo and I had these boards out front with stickers. It just asking some simple questions like, you know, what’s your favorite data viz tool and what graph do you like? And I feel like people engage with that more than because it was like, it was physical stickers on a physical board. It’s just different than clicking on a link on a website.
AW: Right. And they probably remember it better because they encode it in more senses. Great. And you have like more of a visceral experience with something than if you’re just looking at something on a screen.
JS: Right. So now you have all these huge projects. What, I’ll see. I’ll ask like, what do you have next? Like what’s next on the horizon?
AW: Probably just a lot of things. I’m planning a SVG video course, which I’m really excited about. So I got to do a workshop on basically the basics of SVG. And SVG is it’s basically how you would draw shapes within the browser. And I find that people really don’t understand it very well and there aren’t many good resources for knowing even like what the different shapes you can draw are and how you do it and how they interact with each other and how to make the move. So that’s something I’m really excited about.
JS: That’s great. That’s exciting. Well I’ll make sure to keep an eye out for that and I may now have to go play with Figma cause I haven’t actually played with it and now that there’s a data viz plugin, that sounds like something that would be fun to play with. So thanks for doing all that. Thanks for coming on the show. And I appreciate talking to you. I have had a lot of fun.
AW: Yeah. Thanks a lot.
JS: And thanks for tuning into this week’s episode. I hope you enjoyed the show. I hope you keep in touch. Let me know if there are folks that you’d like to hear from on the show, if there are tools that you are working on or working with and things that you might be struggling with or things that you’re trying to learn. I’m always interested in hearing about people’s processes and strategies with doing a better job communicating their work and their data. So until next time this has been the PolicyViz podcast. Thanks so much for listening.
n
