Mike Bostock is the creator of D3, a popular open-source JavaScript library for visualizing data and Observable, an open-source notebook for exploring data and thinking with code. Previously, Mike was a graphics editor for The New York Times, where he helped produce a variety of data visualizations such as Is It Better to Rent or Buy? and 512 Paths to the White House). You can see more of his work on my website and Twitter.

On this week’s episode of the podcast, I’m reposting a video conversation I had with Mike as part of the Data@Urban Digital Discussion series. We talked about the origins of D3, what Observable does and the problems it tries to solve, and answer questions from attendees. Enjoy!

Episode Notes



Data@Urban Digital Discussion

PolicyViz Data Visualization Catalog

Better Data Visualizations book website

Support the Show

This show is completely listener-supported. There are no ads on the show notes page or in the audio. If you would like to financially support the show, please check out my Patreon page, where just for a few bucks a month, you can get a sneak peek at guests, grab stickers, or even a podcast mug. Your support helps me cover audio editing services, transcription services, and more. You can also support the show by sharing it with others and reviewing it on iTunes or your favorite podcast provider.


Jon Schwabish: Welcome back to the PolicyViz podcast. I am your host, Jon Schwabish. I hope everyone is well and safe and healthy in the strange times. So I’m looking forward to this week’s episode of the PolicyViz Podcast because I am reposting one of the recent data and urban digital discussion conversations I had with Mike Bostock, the founder and creator of the D3 JavaScript library. I, myself am not a JavaScript programmer. But there were a lot of people on this conversation that had questions about how to use D3, about some of the ongoing evolution and modifications about D3. And Mike spends a bit of time talking about his new project Observable, which is, I think, has found an audience out there of users who are interested in sharing and creating open source code. So I hope you enjoy this week’s episode. Before we get into that conversation, just a couple of quick notes. First off, if you haven’t seen, I have recently posted my data visualization catalog. I’ve been collecting data visualizations for the last couple of years. I have more than 700 images in the catalog right now. I have used those for myself for inspiration and for techniques that I might like to try for other data. And I also use it, of course, for teaching in my classes, in my workshops. And I asked my Twitter network to see if they would like to take a look at that catalog, and there was an overwhelming response in the affirmative to that. So I tooled around with the Google Data Studio tool to create an interactive table where you can see both the images and the URLs and my notes. So that’s up on my website if you would like to take a look. I have placed it in a new part of the site dedicated to my forthcoming book, Better Data Visualizations that will hopefully be out this fall. So I’ll leave notes and links to that tool on the show notes page. And I’ll also link over to all the other data at Urban digital discussions that we’ve been having over the last couple of months. Most of those, if not all of those, have been recorded and posted to the urban Institute’s YouTube page. So you can go in and take a look at those. But in this week’s episode, I’ve taken that video chat, and I’ve turned it into a podcast. I think it does work as an audio. So hopefully you will enjoy it and be able to learn some things from it. So here is this week’s episode of the PolicyViz Podcast, my discussion with Mike Bostock. Good afternoon, good evening, or good morning, everybody. Come on in. Grab a cocktail if you’re on the east coast. I’m not quite there yet. I still have 29 minutes before I pour the cup first cocktail today.

Mike Bostock: [00:02:56 inaudible] brought a cocktail, and I’m on the west coast. I wasn’t told.

JS: I didn’t have to go to the liquor store today. They’ve changed the liquor store. Then you have to stand outside and give them your order, and they will go get it for you. So I got the, I said, give me the biggest bottle of bourbon and the biggest bottle of rye and just and the biggest bottle of vodka, you know, just many bottles. All right. So we’ll just let folks settle in for a minute, and then we’ll get started. And also, while we’re waiting here for everybody to settle in, I’ll just share next week’s lineup. So next week for those who are interested, three sessions next week, Tuesday at noon with Mike Corral, Jessica Holman, and Robert Casara. Academic uncertainty discussion. Wednesday at 11 with Michael Brenner from Data4Change, and then Thomas Mock Thursday at three about R. So we’ll turn back to tools. So just quickly, we’re working on getting that up. So all right. I think we’re ready to roll. Mike, thanks a lot for tuning in today. It’s good to see you.

MB: My pleasure.

JS: So, again, I don’t have an agenda or a list or anything.

MB: Good to see you see.

JS: Maybe let’s see. I don’t know. Do you want to talk about what you’re working on right now? Maybe that’s we can just start there. And just for folks who haven’t done this before, feel free to put your questions in the chat box. And then you can just unmute yourselves, and you will just build up a queue, and you can chat with Mike yourself. And we’ll just take it like that. So I don’t know, Mike, you want to start by just, you know, giving us a label and what you’re working on.

MB: Sure. Well, for the last few years, I’ve been working on building this company Observable. And it really started out, you know, coming, coming out of the work that I was doing for D3. And you know, one of the problems that keep coming back to was essentially like how difficult it is to build interactive visualizations. And, you know, seeing a lot of the people learning D3 and struggling to figure out how to put the APIs together or to, you know, really do creative stuff with all the examples that existed there. And ultimately, I kind of came to the conclusion that the biggest issue wasn’t visualization per se. It was code. It was sort of the difficulty that people run into just writing interactive programs, building interfaces. And I think part of that also comes to, you know, how much I enjoy writing code, or I guess, like, the, the joy that I get from code, the flexibility, the power that it provides. It always felt like there was this compromise between, you know, tools that were based in code that could let you do whatever you want, and tools that were sort of point and click and were easier to use, but were correspondingly, you know, much more limited in what you create. And I always, you know, I like the creative aspect of it. I don’t want to build it, you know, kind of a patronizing tool that makes all of the decisions for you, and just, I want things to be as easy to use as possible. But I don’t want to take away people’s creativity and freedom and whatever they create with it. And so that’s, you know, that’s the problem that I wanted to tackle is basically how do I let people have that expressiveness and that power that comes from code, but at the same time trying to make it more accessible. And we did have a lot of, I think, success in the D3 community with the examples where people would sort of share those examples on blocks or elsewhere on the Internet. And in a sense, like Observable was trying to kind of take that to the next level, right, thinking about how can we sort of eliminate all of the technical obstacles to getting people to start tinkering with these examples so they don’t have to, you know, get clone in the terminal and install a bunch of stuff and set up a local web server and have sort of a whole IDE set up locally. They can just get started in their browser. But more than that, I think it’s about thinking about, you know, the abstraction, the programming language. There’s a lot of complexity, I think, in sort of traditional programming languages where you have mutable state, and you have event listeners, and you’re sort of mutating things and remembering these side effects. And, you know, I’ve seen a lot of people struggle, just getting that stuff working when they’re trying to build an interactive visualization. And, you know, I would prefer people spend time thinking about, you know, is the visualization communicating? And, you know, am I learning something interesting? Am I able to, you know, express or explain whatever it is that I’m trying to do, and hopefully spend less time with sort of the machinery, the technical aspects. And so that’s, you know, essentially what we try to do with Observable is make reactivity sort of a language level feature, rather than just a library feature. And by doing that, it simplifies a lot of the code that you would need to write to make an interactive interface. And I think the most exciting thing for me is it also has this kind of social effect, where it becomes easier for you to import code that other people use and to compose it and to do interesting things. So, I mean, you can fork somebody’s notebook, of course. But you can also import the cells directly into your notebook. And then you can even rewire them. So like a lot of the D3 examples, you know, you don’t have to fork them in order to get your data into it. I mean, that’s a valid strategy. But in addition to that, you can just import them and replace the data. And I think it’s, it’s really exciting to have kind of new ways of composing together code that’s a bit more flexible and repurposable. And, you know, ultimately, our goal is to try to help the community learn from each other and share their techniques and to bring more people in that are able to do, you know, visualization effectively.

JS: Right. I’ll just ask a couple of questions until people start asking you questions, because I’m just the moderator. So I know [00:09:04 inaudible] is out there. So I’m sure he’s going to have questions. But in your view, what is the future of programming? I mean, so Observable, you’re trying to be at that, at that edge of how people are going to work together. But five years from now, do you have a vision of how people will work together on code that’ll be will be fundamentally different than how we’re doing it now?

MB: Yeah, I don’t think there’s going to be sort of a grand unification. I think that there’s or like one ring to rule them all or anything like that. I mean, I think there’s, there’s so many different applications and use cases and needs, that there’s always going to be sort of different languages, different ways of development, you know, different technologies, different styles, all that stuff. So I don’t think we’re trying to sort of replace all of the different ways that people use programming right now. I think in a way how I view Observable is more like replacing Google Docs or replacing, you know, like Dropbox Paper and stuff like that or medium like basically like, if code is this, this, this language that enables us not just to do analysis, but also to communicate more effectively, like, we need that within our communication medium, right? Like, we need that within our standard tools for working together with other people for communicating, for presenting data, all that stuff. And, you know, I certainly write a lot of blog posts and articles that involve code. And having this like medium that lets me write prose and include visuals and make them interactive at the drop of a hat, I mean, that’s, that’s what I love about it. So I think of it more as focused around communication and collaboration, rather than sort of general purpose programming although it certainly does have fuzzy lines and you can kind of extend it into a lot of different areas. In fact, what I’ve been working on this week is, you know, learning serverless computing, like the AWS stuff, and building more sort of stateful persistent really like fully fledged web applications essentially within an Observable notebook. And there was this great interactive, Chris Zubak-Skees I think is his name who made, you know, can you identify Ukraine on a map? Like there was this case where it was like Mary Louise Kelly, the NPR reporter, I was doing an interview with Mike Pompeo. And he questioned her ability to identify Ukraine on the map. And of course, famously, like she had no problem identifying Ukraine on that map. And then the question was like, how good is the general public at actually doing that? And so this reporter made an interactive on Observable, that when you clicked on the map, it recorded the location where you clicked, posting it up to AWS and storing it as a table so that you could then aggregate all of those results, and see where people were clicking.

JS: So let’s see. So Robert Casara has a question for you about video. So, Robert, if you want to just unmute yourself, go for it.

Robert Casara: Yeah. Hey, Mike.

MB: Hey, Robert.

RC: So I saw you mentioned, uh, I just say, I haven’t really done much myself on Observable, but I keep getting news. It is really awesome. And I saw you mentioned, I think it was maybe I’m making this up. But I hope, I hope I’m not that you said something about wanting to integrate video into Observable. And so I was curious what your plans were there, like, is it just going to be full on video, like sort of the traditional way? Or do you have any ideas that go beyond basically YouTube?

MB: Yeah. So we’re actually basically spinning up our dev REL team now. And one of the big initiatives is going to be a series of videos teaching people, both Observable and D3 and visualization and a variety of other things. So we’re just getting started with that. Now, I don’t know if it’s going to be like directly integrated into the platform per se. Like I don’t think we have any immediate plans to build like a MOOC or anything like that, but then Observable. But we are looking at other formats for teaching people Observable. You know, we’ve written a lot of notebooks already. And you know, that’s nice because it’s within the product. You can start tinkering with it, but, you know, different people have different styles of learning. And we want to sort of make, make it available to people who like those different styles as appropriate.

RC: Yeah. So it’s performing if it’s more like Khan Academy, where there’s more sort of like, integration between video and the content. So you have like chapters and maybe like, you have to do a thing before you can go to the next step. I mean, this is just me making stuff up here. But I was just curious if you had thought about that kind of thing?

MB: No, I don’t think so, not yet. I mean, one of the strengths of Observable is that it’s code so you can kind of do whatever you want. So, I mean, you could theoretically build that potentially a fair amount of work. But I think, you know, we’re, we’re open to ideas. I think we’re just getting started right now with the video content and making sort of interactive sessions could be, could be interesting.

JS: Yeah. So I’ll again put a call out for people to ask questions. I have another question for you. And this is, it’s going to sound like a, it might sound like an accusatory question, but it’s not. So, so what I’ve heard is that the New York Times is moving to using data wrapper for all of their, their graphics. And I know they also have an internal tool for a lot of the journalists to make, you know, more static things that they need, that they need quickly. And I’m just curious, and again, I don’t, I’m not trying to accuse the times or data or anybody of doing anything, whatever. I’m just curious about your perspective on the model of moving away from using, you know, pure JavaScript or using Observable and moving toward, more towards a tool where it has more of this drop and drag type, you know, clicking approach?

MB: Well, I would imagine that the New York Times is going to retain, like a whole variety of tools at its disposal for doing interactive, although I’m sure that there’s a lot of data like graphics and stuff like that, that will be covered well by Data Wrapper.

JS: Yeah.

MB: So I think for us, you know, I don’t think we’re really directly competing with Data Wrapper because it is more of a development environment rather than sort of a point and click construction of visualizations.

JS: Yeah.

MB: And also, I think we’re also not really just making a visualization that you can drop into a larger article. Like there’s a whole variety of things that you can build an Observable that are, you know, either dashboards or sort of explorable explanations, or sort of real analysis that you’re doing in Observable, rather than just sort of dropping in some data and then assigning some visual encodings and dropping out and SVG. So it’s really like I was saying earlier, sort of more about replacing, you know, how people communicate and collaborate within what further, I mean, for the business side of things, because I see the other, the chat question there. You know, it’s about like, how do teams of people, you know, within enterprise, that are, you know, all within these data driven organizations needing to make decisions and, and get insights out of their data? Like, how do they work together around that data? How do they communicate with each other? How do they collaborate on the analysis? And for doing that, you really need the flexibility of code. So there’s, there’s a lot of stuff that you can do in point and click interfaces, but ultimately, at some point, you know, whatever you’re doing, probably hasn’t been done before if it’s giving you a competitive advantage. Like that’s kind of the definition of doing science is that you’re doing something that’s new, right? Like you’re asking questions that have not been asked before, or at least have not been answered before. And so you need the creativity of code. And so it’s really about trying to bring code into the standard ways that people communicate, write stuff down, share their results.

JS: Yeah. I don’t know, if you want to talk to Martin’s question about the business model for Observable?

MB: Well, I think it’s really just about collaboration. So for us, we have, you know, two kind of audiences, I think. And the idea is to have some synergy between those. So we have people that are, you know, using Observable out on the public, you know, the open source, like myself, like doing D3 examples and just sharing techniques, sharing our love for virtualization, or, or whatever else, just making fun, pretty things. And for those people, you know, they’re, they’re welcome to use it for free. And then we also have sort of a more enterprise focused product, where when people are collaborating around private data or private analysis and they want to work together, they can sign up for a team with an Observable, and they just pay a subscription fee to go along with that.

JS: Yeah. Great. So we have two questions in the queue. I’ll let David, if David, if you want to unmute yourself. You can ask–

David: Sure, yeah. Yeah.

JS: Yeah.

D: Yeah. Hey, Mike, thanks for doing this is really interesting. And, you know, I’ve only dipped my toes into Observable but really exciting to get my feet wet. An exciting thing happening in the art community right now is our markdown is kind of taking off. It’s kind of a similar notebook technology. You’re probably familiar with it along with many other listeners here, but they’re starting to like integrate, you know, Bash, Python, SQL other languages seamlessly within the notebook. And a big dream of mine would be to do something similar within observable, where maybe, um, you know, alternating between R, Python before dipping into, you know, D3 and kind of this integrated context. I don’t know if that’s on your radar on the horizon, or I’d be curious to know.

MB: Yeah. I think that’s a pretty common question because, of course, there are a lot of different languages that people use, like we were discussing earlier. I think for us, though, JavaScript has so much strength, I think in it being the sort of de facto or lingua franca of the web, you know, it’s what every browser runs. And, therefore, you know, that’s why we put so much emphasis, you know, on JavaScript being the language and in fact, even why Observable extends the JavaScript language. So when you’re writing stuff in Observable, you’re not actually writing vanilla JavaScript. It’s an extension of the language that we’ve, you know, added these reactive features to it, the state of flow to it to simplify how you do, you know, interactive state. So I think we don’t have any immediate plans for supporting other languages. But I think what our long-term vision is that we want to be able to integrate with other systems or pipelines that people would use. So you might use R or Python to do some data analysis. And then we want to have an easy way to bring that into Observable either to just do the presentation, or to do sort of further, like downstream analysis from there. So we have like file attachments is a feature that we launched within the last year. And I think we’d also like to see sort of a programmatic API for that, so that you can easily sort of upload new data into your notebooks, or to sort of automate the display or like a dashboard or something like that. And so I think that there’s always going to be multiple languages that people use. Now, there are some interesting possibilities around web assembly, you know, Emscripten, and other tools that lets you take languages and compile them into something that can run fast in the browser. So there’s been a project at Mozilla. Iodide was a web based notebook that had Pyodide, which is basically the ability to run Pandas, Python within the browser, JavaScript. So we might see some things like that. But we do have a sort of a very strong focus on JavaScript, because we’re integrating in at the language level. And I think another aspect for us is having a single language is also good for collaboration. You know, when the notebooks are all written in the same language, it’s easier for you to sort of import and reuse techniques in other notebooks that you could find on Observable. So that’s part of it, too.

JS: Before Stuart Thompson, who, you know, I’m sure he’s just like chomping at the bit. Before we get to Stuart’s question, Martin followed up with me, and he was just to follow up on this, on this, what you’re just talking about, do you use, Martin wants to know, it’s a good question, do you use JavaScript for everything or do you also use R, Python, other languages for both, I guess, data wrangling and vis?

MB: I mean for me, I mean, yeah, I use JavaScript for pretty much everything. But of course, I’m not really doing visualization professionally these days. You know, I’m working on building Observable, and I’m making examples. And we’re doing some analysis internally, of course, at Observable, and for that, yeah, we use JavaScript sort of all the way down.

JS: Right. All right, so let’s see. Martin, I hope that was a what you’re looking for. I don’t know. So Stuart Thompson is here. He has a question for you. I don’t know if he’s–

Stuart Thompson: Yeah.

JS: Yeah.

ST: Hey, thanks for doing this, Mike. Yeah, I’m just kind of wondering like you and your role as like a data visualization person, like, do you miss having like that as your main responsibility or do you not? I mean, I do still make stuff, but yeah, I’m just kind of curious about that.

MB: Oh, yeah, absolutely. I miss it every day. I mean, it was a, it was a dream job. And I think it’s not just like, you know, getting to work on visualization that has a large audience. And they can sort of resonate with the general public. But it’s also just, you know, the people that I got to work with at the New York Times were fantastic. And I think, you know, all, all reporters, all journalists like really have a passion for, for learning and trying to absorb as much information as they can around them. And they’re also good at sort of explaining that to other people like digesting it and sharing it. And that makes them for very interesting people. Like, there’s a lot that you can learn from, and there’s a lot of like, good stories. So, so sure, I missed that. But I think one of the things that I enjoy now about the work that I do is also this sense of, you know, trying to build, you know, infrastructure or tools that will have lasting value and we’ll change sort of the set of people that can do, can do this stuff that can have sort of that deeper understanding of what’s going on around them, the quantitative understanding or the ability to explain it. And so I do, you know, I enjoy building new technologies, and I enjoy making them available to people. And it’s, it’s hard to do that when you have sort of deadlines of publishing graphics. So, you know, I’d love to do both. I always like having the mix of sort of practical applications like building graphics together with working on sort of the more abstract development of tooling. Like if you, if you just do tooling for too long, then you kind of you lose your concrete foundation. And so I think what we’ve tried to do to mitigate that with Observable is have a very strong focus on learning from our users, like building out sort of the user research group, and doing surveys and analytics. And, you know, obviously, just spending a lot of time talking to people like helping people out when they ask questions in the forum, or on Twitter, or Slack or wherever, and trying to learn from those frustrations and figure out how we can make the product easier for people to use.

ST: Cool, thanks.

JS: Right. Thanks, Stuart. I’ll put another shout out there for folks to put questions in the in the chat window. Mike, I was going to ask outside of the big news organizations like the Times, the Post, the Guardian. Do you have other, other media organizations that you think are doing really phenomenal or just good database that people should check out?

MB: Well, I think John Burn-Murdoch in the Financial Times is doing some pretty great Coronavirus visualizations on a regular basis. And of course, he’s well known for popularizing that bar chart [00:25:44 inaudible] for better or for worse.

JS: Yeah, um, yeah. Do you have any other ones? Yeah, I mean, he’s great. He might not have any, I was just curious.

MB: Yeah, I don’t know. I feel like I’ve been trying to avoid some of the news lately. I don’t know.

JS: Yeah, yeah. Yeah. Well, since there’s, there’s, there aren’t any other questions rolling in, so let me ask another one. Are you in the camp that you think everyone should learn to code?

MB: Um, I mean, I don’t know if I’m like an absolutist, or like more of a pragmatist. I think that it certainly is a valuable skill. And I think that if we can make it approachable, like if we can actually make it accessible to people, then like, that’s really what matters. So it’s sort of a function of how valuable is it combined with sort of how difficult is it for people to learn, you know, if you have something that is extremely difficult for people to learn, and telling everybody that they need to learn it like is just not practical, because there’s a lot of stuff like they may not get the value out of it that you’re expecting. So it’s important to accommodate that. So in a sense, like I think there is potential value that is not being realized today because it is too hard for people to learn. And so my focus is on trying to make it more accessible, rather than just, you know, telling people that they should do this impossible thing.

JS: Right. Is it that, is it the act of actually building something that’s important for code? Or is it being able to think in a particular way that–

MB: Its language is ultimately what it is, right? So it’s really about configuration versus composition. And it doesn’t have to be written. It’s just that written language, or just the concept of a language is inherently going to favor creative composition over something that is really just configuring something that’s been built by somebody else. So, you know, you can do plenty of creative things, like in Illustrator, or, you know, Sketch or other sort of drawing tools. And, you know, I think that’s an example of creativity, I think for computers, so it’s, it’s both the, like compositional approach, and of course, the fact that when you’re expressing language in a way that a computer can understand, you’re creating a solution that can basically be scaled up arbitrarily, right, like can be automated. And you essentially solve a problem once. And now you can solve that problem as many times as you want for no additional cost.

JS: Right. Right. Let’s see. So let’s see. Two other questions. One from Martin again. Are there any major developments for D3 on the roadmap?

MB: I mean, yeah, there’s D3 6.0, which I’ve been working on for, for quite some time, but which hasn’t quite made it out the door yet, although it has made it out in, in sort of piecemeal fashion. So if you recall for D3 4.0, the big effort there was to break D3 up from a monolithic library into about 30 different modules that people could use independently. And I think that’s had some pretty good traction now. I mean, I think there’s been a lot of modernization in the JavaScript tool chain since then, so it’s easier for people to sort of pick and choose which parts they want to use. So, for example, d3-array 2.0 was released a while back and I think that has some really great improvements to it both for the JavaScript language in terms of support for iterables, but also like the new d3 group, d3 roll up as a replacement for d3 nest, I think are fantastic. If I can toot my own horn, and then there’s, like d3-delaunay, which is based on Vladimir eigenfunctions, Dylonater, which is a really fast implementation of Delaunay triangulation, and then we implemented Voronoi tessellation on top of that, and so that’s going to replace d3 Voronoi. And then there’s been a bunch of other improvements like to d3 scale, a bunch of new scale types, trying to simplify a little bit how event handling works in d3 selection. So some of it hasn’t been done yet, which is why we don’t have sort of a pre release available for d3 6.0. But hopefully, we will get a chance to do that within the next few months. And in the meantime, you know, people can check out sort of individual modules and I would particularly recommend d3-array 2.0.

JS: Okay. There’s a lot of people who are like, truly, you know, sort of rubbing their hands to get together right now getting ready. That’s great. So Julia has a question for you about helping folks make dataviz. So, Julia, if you want to unmute yourself, you can just ask.

Julia: Hi, yeah.

JS: Are you there?

J: Yeah. I was wondering what, like what your challenges are and how you resolve them at Observable around helping people to make useful and readable data visualizations, because they figure, you know, you give somebody a tool like, it’s, it’s one thing to use the tool, but it’s another entire skill set, making data visualizations that are useful and that achieve what the person wants to achieve.

MB: Yeah, absolutely. I don’t think we’re doing enough there. I think part of the challenge is that Observable is more low level focused, right? Like we’re focused on sort of how people write code and how they put together interactive interfaces. And we’re not very opinionated about what those interfaces are. Like we’re not Vega-Lite, for example, like you can use Vega-Lite within Observable. But you can use any other tool that you want to make your visualization within Observable as well. And so there’s not really an opportunity at the platform level to guide people in the creation of visualizations, because it’s just a lower level abstraction. I think we’re guiding people to make simpler programs and sort of developing the, the good idioms for representing interactive programs within Observable and making them more broadly accessible. But I think, ultimately, the improvement to visualization practice comes from people helping other people and from discovering and sharing techniques, sort of through publishing notebooks and through collaborating with other people in the platform. So it’s not the platform itself. It’s the community of people that are using it and how they work together with each other. And we’re trying to make improvements there as well. For example, one of the things that we’re working on that should be available at any day now is sort of improving our commenting system, just so that people can talk to each other and comment and ask questions about notebooks. I think when we started out with Observable, you know, we, we came from an open source background, and we’re familiar with the maintenance burden, and some of the emotional cost that comes from dealing with random people on the Internet every day. And therefore we designed Observable to be very sort of private focused, like you can send a suggestion to somebody to like, here’s like a code change to your notebook, but you’re not allowed to comment directly on other people’s notebooks. And I think ultimately, that was too, too much, erring too much on the side of not letting people talk to each other. And so we’re trying to strike the right balance there to allow the community to flourish and collaborate with each other. And so, hopefully, you’ll see some improvement soon. And I think also, maybe I would add to that, is that we want to see some of the content on Observable to teach people how to use both visualization tools, but also the visualization practice. And, you know, we’ve seen some great stuff like Jeff Heer at UW, for example, has been publishing a bunch of notebooks on for his visualization classes that are being shared in Observable. And I would love to see more of that. So I think a combination of examples that people can use, but also sort of critical essays, tutorials, guidance, that sort of stuff is what I would love to see from the community.

J: Well, thank you.

JS: So another, just call for other questions. On that line you’re just talking about Mike, have you, have you ever considered or talked to anyone about coming up with a or developing a D3 JavaScript Observable type of environment like the Makeover Monday or the Tidy Tuesday, I don’t know, thing. That’s a program, but I don’t know what you would call it something.

MB: Yeah, I’d love to do some of that. I think, you know, as I mentioned, like our devrel team is just getting started. And I think we maybe are starting to have some of the bandwidth where we could support things like that. And I think having a regular cadence could be nice for building the community. I mean, it’s been a challenge, trying to do everything that we’re trying to do with the product and also like, do stuff that’s just sort of supporting the community. Like it was you’re trying to build the platform at the same time, but I mean, I think we’re, the team’s grown a lot. And so I think we’re, we’re hoping we can start to take on some of those additional responsibilities.

JS: Yeah, no, it’s certainly a big lift. I’m, yeah, I’m sure it’s a big lift. We don’t need to stretch this out if other folks don’t have questions, but I’ll ask, I’ll ask one more. And then if, if we’re done, we’re done. And we can all go to cocktails early. I was wondering, I’ve been in an ongoing discussion. And I’ve done one of these sessions on teaching databases to kids. And I’m curious, have you thought about or had experience? I know you have kids of your own, but I’m just curious, have you thought about or tried to do teach code or teach JavaScript to kids? And how do you approach that? Isn’t it, is it different than any, you know, teaching anybody else how to code?

MB: I mean, I only have like n equals one sample size on that, because I have only tried to start teaching my seven year old daughter how to code in Observable. And she was super excited about it. I mean, basically, you just put some emojis on there and some sliders and anything like, you know, colorful and moving was [00:35:47 inaudible] like, anyway, she was she was really excited about it. And it is, I don’t know, it’s fun. I mean, I’m always trying to, you know, teach people d3. And like each time you kind of go lower down the stack, it’s like it’s harder and harder because you’re having to sort of really surface all of those things that you’ve internalized and assumed and essentially, like, forgotten about you, you now process them on a purely subconscious level. And so explaining, like kind of basic things about, you know, what is a primitive type, you know, like, what is the variable is, is challenging if you don’t have experience doing that. I would love to do more of that. I think we are planning on doing some, you know, introductions to JavaScript on Observable, because I think, you know, it’s a great platform for learning because it is so focused on letting people start tinkering as soon as possible. And I don’t know if we’ll go like down to this seven year old level necessarily, but I think, ultimately, I do want, you know, to make JavaScript and programming more accessible to people, not just visualization. And I would like us to start exploring more stuff like that.

JS: Right. So I think we have maybe I mean, we have about 20 minutes. But there is one question and maybe this will, this will wrap us up from Sara, who wants to know, if someone wanted to get started into contributing to d3, how should they get started?

MB: That’s a good question. I think D3 is not really that friendly for other contributors. And I think part of it is just because there’s a desire to keep it small, comparatively. And also, there’s a big cost whenever like, something is broken, you know, because then you have to go and fix it, and everybody’s mad at you. So I think honestly, the best way for people to contribute is really through examples and tutorials, where you’re not necessarily making a new feature, but maybe you’re sort of advocating for a new feature. And you are sort of demonstrating how to do that with the existing API. Or maybe you’re sort of building your own sort of library or an extension. I think d3 is very modular. So there’s been a lot of success, I think for other modules that add to this sort of core set of d3, you know, like d3 legend, d3 annotation, for example. And just a variety of things like there. So I think that’s, that’s easier because you won’t be sort of gated on me approving your PR being merged into the library. But I think there’s still plenty of value that you can bring to the community by either teaching other people how to use the d3 API and some new way or, you know, advocating for some sort of improvement to the library.

JS: Right. Right. Great. Sarah, I hope that was helpful. Um, Ankur has a question for you on reasonable charts.

Ankur: Yes. Specifically, there’s been a whole tutorial that Mike had written how to build reusable charts. Has there been any new updates or recommendations that you would suggest?

MB: I mean, no, not really. I think, I tend to be fairly agnostic about how people use,

how people kind of assemble d3 into sort of applications or frameworks. So d3 is not a framework. It’s a collection of sort of related tools like the low level modular library. And I think there’s a lot of different ways that you can apply those tools in whatever application you’re building. So, for instance, you know, if you’re using React to build your React application, you know, there’s a pretty well established pattern, they’re building react components. And that’s probably what I would recommend using if you’re using React. If you’re using View or if you’re using Spelt or if you’re using Angular, you know, there’s different ways of building components in those systems. And I want D3 to be agnostic and to work well with all of the systems. So I don’t really have a stance or an approach that I would recommend over the others. It would depend on the other frameworks or developing style that you prefer.

A: Thanks, Mike.

MB: My pleasure.

JS: Um, let’s see, Ben, I don’t know if Ben, you want to unmute. Ben has a question for you on data literacy.

Ben: Hey, yeah, Mike, so this is just a general question. But the TAO center for data, for data journalism rather, has done like extensive work on, you know, algorithmic accountability, and then also kind of the convergence of platforms and publishers. I’m just wondering if you have, you know, any, you know, I guess, tips of the trade, you know, for data journalists who were thinking about dataviz and kind of ethical responsibilities, you know, whether it’s in the newsroom and thinking about audiences, just more broadly, I guess?

MB: I think that covers a lot of ground. So let’s see, what would I say? I think one of the, one of my sort of takeaways from working at the New York Times is just like how important it is to surface all of the assumptions that go into the design of a visualization. And that means the importance of the annotation layer, and any sort of keys or legends or guides that sort of explained what the visualization means. The problem as a designer is that you’ve internalized sort of what your visualization means, because you’ve been working with the data, you understand what the data means. And you’ve wrote the code that assigned to those visual encodings. And very easy, surprisingly easy to forget to explicitly say, you know, what the x and y axis means or what the color means, or what size means and stuff like that. And one of the sort of rules of thumb that I found good working on published graphics is like, you know, if you were to just explain your visualization verbally to somebody else, like would you have something interesting to say or not? Like if there’s nothing that you as the author of the visualization can say, it’s like an interesting take away from the graphic, then that graphic probably shouldn’t be published. Because it’s really your responsibility as, as the editor, as the author of that graphic, to have sort of extracted at least a couple sort of interesting insights. And you’re then presenting them in a visual way, because that might be a more effective way of communicating it. And there may be other things that the reader can sort of pull out of the graphic that you don’t include in that initial set. But if there’s really nothing interesting that you can say out of the graphic, then you probably haven’t done quite enough work yourself to sort of pull out what’s interesting. And the great thing about having an explicit annotation layer so that it’s essentially forcing you to do that work upfront, and to highlight at least a couple of things that are good in that graphic. I mean, sure.

JS: Yeah, that was, that was interesting. Um, I was going to ask, since, since your time at the, at the Times, I have observed this anecdotally, but I’m wondering whether you share it that there’s been a move away from, from interactivity for sort of the everyday graph and whether you think that’s a good thing or a bad thing, or just a thing.

MB: I mean, sure, there was a, there was a little bit of hubbub. Gregor Aisch wrote a great article on In Defense of Interactive Graphics. So I recommend everyone read that if you’re interested in the topic. I mean, I think it’s like any bit of new technology where things can be overused at the beginning. And then, you know, the pendulum swings back a little bit, and then people find sort of more refined ways to apply that technology. So I don’t think it’s really a question of like, is interactivity good or bad? Like it can definitely be misused, and it can be used well. And I think you have to understand sort of what the value is that the interactivity is providing, and also the costs that are associated with making a graphic interactive, because there are costs. There’s cost to any sort of design trade off. So I think in particular, the, I think the main risks that you have with interactivity is that it can lead to a potential sort of abrogation of responsibility, where like, as the editor, you know, you really should be digesting and refining and presenting this information that sort of front loads the most important thing for the reader. And if you have interactivity, there’s kind of a temptation where, like, you can show everything to everyone. And like, you don’t have to make that decision because the readers can find it. And the problem is like it’s work for the readers to find it, like you’re just shifting that responsibility from you as the editor to the reader. And so if there’s nothing sort of immediately a value to the reader, then the fact that it’s interactive is doesn’t matter, because people aren’t going to do that work for you, and they’re not going to get anything interesting out of it. So I think you have to look at it as kind of a complementary tool or ability where you have to, you want to have a good static graphic that communicate something and then in addition to that, you can have something that that lets the, you know, the reader pull out something that’s more of a personalized insight, or a more detailed stuff if they’re willing to do the work to engage with the graphic.

JS: Yeah. Yeah.

MB: I mean, there’s kind of there’s a different thing, though. I mean, about animation, right, these days, and I don’t know exactly how I feel about that where I think a lot of the things that get people’s attention on social media are things that are animated. And is that really a better visualization? Or is it just something that’s where it’s able to capture people’s attention more effectively? I don’t know.

JS: Well, so is it, so I guess the question is, is it better for that particular platform? Or because I feel like you could create a static graph or an interactive graph, you know, you could you could create static or interactive for the New York Times site, but you might create even something different for the for Twitter. I’m thinking of like the when the Times did those, those little racing, swimming animated gifts for the, for the Olympics, must have been 2016 or something like that. Like those little, those little animations didn’t show up on the New York Times website. They were only on Twitter.

MB: Yeah, I don’t know.

JS: So Sarah, Sarah has a good question related to this is your thoughts on scrollytelling?

MB: Well, I’ve written an article some years ago about how to scroll that tries to provide more specific guidance on sort of what good scrolling behavior should be, and how you can sort of still do things on scroll, but not frustrate the user who may have expectations on how the page should react to scrolling. So I think like any other technique, it should be used judiciously. I think the main advantage that you get with scrolling over clicking is that it’s sort the most common and accessible affordance, like it’s particularly like for, for journalism, you know, you’re, you’re telling a story, usually, at the very least, like there should be one sort of one dimensional path or narrative through the article or through the story that you’re trying to tell. And scrolling typically is also one dimensional. And that’s therefore that’s an advantage. Like, it’s similar to just reading static text. But now like you can have other things that are happening alongside of it, that lets you do something potentially more interesting. But, I mean, I don’t tend to use it very much on Observable I think, but I don’t know. I mean, I don’t have any sort of explicit guidelines as to like when it’s appropriate to use it or when it’s not. I just kind of know when it’s done poorly, it can be frustrating.

JS: This is a like, taking advantage for a second, but like when it comes to scrollytelling and any sort of other interactivity in particular with Observable, how are you, and how are you thinking about needs for accessibility for people like scrollytelling is a good example. Because there are people, when you have these two different things that are moving not in sequence, or one is showing on top of another one that can trigger, you know, different types of migraines and reactions and things like that. So I’m just curious how you’re thinking about vision issues, or cognitive disabilities or any of that.

MB: I mean, I think that’s a challenge for the visualization space as a whole. I mean, it turns out that by for technical reasons, like coincidentally, like you can’t actually implement scrollytelling within Observable. I mean, there’s probably a way around it. But generally speaking, because of the way that code gets run in the sandboxed iframe, it’s much harder to do sort of typical on scroll, fancy effects. And so I haven’t really seen it done on Observable, although it may be possible if you’re willing to jump through a few hoops. There’s certainly some things that you can do with intersection observers. But I think the other primary thing that where I’ve seen people make some progress in terms of making visualizations more accessible are sort of color deficiency issues and trying to both come up with good defaults for colors, both like for ordinal, or categorical colors, and for sort of continuous or sequential color schemes. And then like tools that lets you sort of either try to evaluate contrast levels or simulate different types of color deficiencies. So, I think early on, you know, we added Cynthia Brewer’s color schemes to d3 as a d3 scale chromatic. And we also eliminated the category 20 color schemes, which was somewhat controversial, but I think was the right call, because they were essentially paired color schemes, which tended to create sort of a false association between the light variant and the dark variant of the same hue. You’d have a light blue and a dark blue that would seem to be related when, of course, they’re categorical, and they probably weren’t. And also, it’s just difficult to distinguish 20 categorical colors. And so we just got rid of those. And we switched to 10. And I think that’s hopefully encouraged people to switch to using like an other category if they have more than 10 categories that they want to include in their visualization. And I’ve seen some sort of interesting new work on like perceptual color spaces, some of which has been done in Observable. So there there’s like a JZ, AZ BZ color scheme, basically, like they’re just designed to sort of give you, you know, distance in color space equates to sort of perceptual distance so that like if two color coordinates are similar, you’re likely to perceive those as similar colors. If they’re very different, you perceive them as different colors. And so you can start to quantify, like, in this color space, how different two colors are, and therefore, measure like contrast ratios.

JS: Got you.

MB: And one of the interesting things is some of the color metrics that we use, for example, there’s a WCAG contrast ratio uses, I think it’s, well, I don’t remember off the top of my head, but it’s definitely not one of these like perceptual color schemes. And I don’t think they even use like linear light color scheme. So it’s just like RGB values. And so it’s pretty heavily distorted and there are some criticism that they should be using sort of a more perceptually-based color scheme, or color space for doing this analysis. On the other hand, they’re pretty rudimentary tools for evaluating sort of what contrast ratios are. So I don’t know if, if that’s more of a pedantic argument or if it would make a practical improvement.

JS: Right. Right. I think we have one more from, from folks. Ivan has a question for you on using Observable for hiring. So Ivan, if you’re there.

Ivan: Hey there. Thank you for your amazing tools. I love them both. Probably you have much more. The question is, I know a lot of companies use GitHub for hiring people. And so I’m curious to know, do they use Observable for that?

MB: Well, I don’t know for certain, but it’s definitely I think one of our hopes is that, you know, people can kind of build up a visualization portfolio and do that more easily in Observable than they could somewhere else, and then use that, you know, in their CV or their resume to sort of give demonstration of someone’s abilities. I mean, certainly, like in our own hiring, of course, like we look to see what Observable notebook says somebody’s written. Do you like some pair programming or interviewing stuff that are based in within Observable notebooks? But I don’t know if we’ve quite reached the mainstreamness yet to be competing with GitHub as the sort of like de facto portfolio, but maybe eventually.

JS: Maybe yeah, one day, one day. So one more question from Martin, which I think is a good way to wrap up, which is, what’s your favorite Observable notebook?

MB: I, then I refuse to answer this. I don’t know. It would not be fair.

JS: No, no comment on that one.

MB: I think, I mean, I have been really impressed, I think, by the work that’s been done on like analyzing the Coronavirus or COVID-19 data recently. I mean, there’s just been literally hundreds of notebooks that people have published, that are looking at the data and visualizing it in different ways, and sort of creating sort of more localized views, you know, like somebody wants, like New York Times does a great graphic for sort of the US, but like if you’re in another part of the world, and you have data, like you can just fork that and replace it and create a sort of more localized view as well. So that’s been cool. And I’ve definitely learned from some of the other more analysis focused things like you know, people looking at creating the simulations of the Coronavirus, you know, like these SIR and SEIR epidemiological models.

JS: Yeah.

MB: But, of course, I think I would caveat that with, it can be challenging and dangerous to potentially try to draw too much from the simulations that people create if you don’t have a background in epidemiology. So they can be sort of intellectually interesting, but I would be careful about trying to draw too many conclusions from them.

JS: Right. Right. Cool. That’s great. Thanks, man.

MB: Thank you. My pleasure. Thanks for all the questions.

JS: Yeah, everybody, that was great. Let me just quickly share next week if you’re interested. If you’re interested in chatting next week, on Tuesday, Michael Corral, Jessica Holman and Robert Casara from noon to one Eastern Time, Michael Brenner for Data4Change on Wednesday from 11 to 12 Eastern Time, and on Thursday, Tom Mock, who runs the Tidy Tuesday our project and works at our studio be here from three to four on Thursday. And that’s all I’ve got for now. That’s probably enough. So Thanks, Mike. Thanks, everybody, for tuning in. Go enjoy your evening cocktails. And yeah, talk to y’all soon. Thanks again, Mike. That was great. Cheers. Thank you.

And thanks, everyone, for tuning in to this week’s episode. I hope you enjoyed it. I hope you learned a bit about d3 and about Observable. If you’re interested in supporting the show, please share it with your friends and family and neighbors. Or you can support it financially. Go over to my Patreon page, where I have various levels of support where you can help me afford the cost of web hosting, podcast hosting, and all the tools and tricks that I need to bring this show to you every other week. So I hope you enjoyed this week’s episode of the show. Until next time, this has been the PolicyViz podcast. Thanks so much for listening.