In this episode, I sit down with Alvitta Ottley and Paul Parsons to recap everything that happened at the 2025 IEEE VIS Conference in Vienna. We talk about our experiences co-organizing the VisCom workshop, the surprising attendance, and the standout keynote from Moritz Stefaner. Alvitta shares insights on accessibility research and the surge of LLM-focused visualization papers, while Paul walks us through his award-winning work on design cognition and how practitioners develop ideas. We also reflect on the evolving identity of the visualization field, from methodological rigor to the role of practitioners, interdisciplinarity, and ethical tensions. It’s a wide-ranging, candid conversation about where visualization research is headed — and what we hope to see next year in Boston.
Resources
Check out the VIS website.
Guest Bio
Alvitta Ottley is an Associate Professor in the Computer Science & Engineering Department at Washington University in St. Louis, Missouri, USA. She is also the director of the Visual Interface and Behavior Exploration (VIBE) Lab and hold a courtesy appointment in the Psychological and Brain Sciences Department. Her research uses interdisciplinary approaches to solve problems such as how best to display information for effective decision-making and how to design human-in-the-loop visual analytics interfaces that are more attuned to how people think.
Paul Parsons is an associate professor in the School of Applied and Creative Computing. His research is in the interdisciplinary field of human-computer interaction, with a focus on the design of interactive visualization interfaces and their use in complex sociotechnical settings. His work has been recognized with an NSF CAREER award and several best paper awards. He has been PI or co-PI on more than $2.2M in external funding, including NSF and NASA projects, and his research has been published in leading venues such as IEEE VIS, ACM CHI, and top-tier journals. He leads the Design, Visualization, & Cognition (DVC) Lab, investigating how designers think, create, and collaborate across domains involving uncertainty, complexity, and ambiguity. His work spans topics such as design cognition, design practice, user experience, judgment and decision-making, and joint cognitive systems.
Different Ways to Listen to the Show!
New Ways to Support the Show!
With more than 250 guests and 10 seasons of episodes, the PolicyViz Podcast is one of the longest-running data visualization podcasts around. You can support the show by downloading and listening, following the work of my guests, and sharing the show with your networks. If you’re interested in financially supporting the show, you can sign up for my Patreon platform, make a one-time payment via PayPal, or shop one of the show’s sponsors.
Transcript
00:00.54
Jon
All right. All right. Here we go. i don’t know. Maybe it’s lot of more enthusiastic start. All right. Alvina, Paul, good to see you again.
00:07.06
Avitta Ottley
thank
00:07.84
Jon
it’s been a few weeks only. How are you?
00:10.20
Paul
Very good.
00:11.13
Avitta Ottley
Pretty good.
00:11.18
Paul
Good to
00:11.26
Jon
Yeah.
00:11.61
Avitta Ottley
Thanks for inviting me.
00:13.14
Jon
Well, you know, I think we did with Robert Kassar a few years ago, we did a vis recap. So I felt like you were the natural bridge to the next one years, years later. Um,
00:22.83
Avitta Ottley
Well, how did you do that? Yeah.
00:24.86
Jon
What, ah okay. So just for folks who who don’t know, let’s do quick introductions. Elvita and then Paul. Elvita, what’s your, yeah, whatever you want to say about yourself.
00:33.65
Avitta Ottley
Oh.
00:34.10
Jon
Yeah.
00:34.78
Avitta Ottley
Hi everyone, my name is Alvita Otley and i am an associate professor of computer science at Washington University in St. Louis. Wow, that’s a lot. My work specializes in data visualization, particularly i’m interested in designing for everyday people. I’m interested in visualization literacy and interested in complex situations where perhaps a human can’t go through all of the data space. so thinking about how do you embed AI into those contexts to support decision making and just processing the data.
01:15.11
Jon
Okay. Paul, over to you
01:17.27
Paul
All right. Hi, I’m Paul Parsons. I am an associate professor at Purdue University. um And my research is in design practice, design cognition. And as it pertains to Viz, I’ve been studying how data Viz practitioners work, what their processes look like, and how they think, how they come up with ideas.
01:41.81
Paul
um And that’s the design cognition piece of it. um In terms of Viz, I’ve been, i don’t know how many times I’ve been, but I’ve been going ah off and on for at least more than a decade, 10 to 15 years somewhere by now.
01:59.13
Jon
Which is strange because you’re only like what, 27, 26?
02:01.92
Paul
Yeah, 25.
02:02.27
Avitta Ottley
Thank you.
02:02.42
Paul
Yeah. twenty five
02:03.64
Jon
Yeah, Right, right. Yeah, yeah. Okay. It’s crazy. Yeah.
02:06.33
Paul
yeah
02:06.48
Jon
um Okay. ah All right. So Viz 2025, one of what? One of like the three premier visualization InfoViz conferences, pretty much that kind of sound right?
02:19.89
Jon
Okay. um On the academic side.
02:20.53
Paul
Sure, paper, marijuana.
02:22.17
Jon
Yeah. ah this year’s was held in Vienna. I’m just trying to give folks like lay of the land. So this year is held in Vienna, Austria. Next year will be in Boston, Massachusetts in the States. um And at least as far as I have been going, which is not as frequently as you, Paul, because I’m only 18.
02:38.61
Avitta Ottley
Thank you.
02:40.80
Jon
So, you know, I don’t have much, much time. um ah It’s, it’s traditionally right. Been two, the first two days are these kind of workshops that tend to be more,
02:53.08
Jon
testing things a little bit more, ah maybe practitioner focus, a little bit more, maybe actionable or not actionable, in like ah a less actionable in like a research sense, more like hands-on sort of like exploring different processes and experiments and stuff like that. And then the next, what, four or five days depending is kind of your your more typical traditional academic conference with concurrent sessions and keynotes and there’s a best paper and there’s a big dinner and all that. That sound about right?
03:23.06
Paul
Yeah.
03:23.29
Avitta Ottley
Yes.
03:23.80
Jon
Okay, so this year we have two days of workshops.
03:23.83
Avitta Ottley
Yeah.
03:26.40
Jon
Let’s start there. Why don’t we talk about ours first? So so the three of us co-organized the VizCom workshop, which is trying to get at sort of bringing together practitioners and um and the and the academics and the researchers.
03:40.30
Jon
um You know, ah Alveda got there late because you reflect problems. So we’ll come back to you because you kind of swooped in at the end to take all the credit. No, just kidding.
03:49.97
Avitta Ottley
Yes, the
03:50.15
Jon
um You did keep us on track. I will say that. I will say that. Okay. So um Paul, what like what did you think? I mean, we spent a lot of time pulling this all together. Like what what were your impressions as it is it you know showed up that day?
04:05.05
Paul
Yeah, yeah. I mean, um overall, i I think I was pretty happy with how things went. We had a pretty full room, I’d say, for the whole time, I think, even post-coffee break.
04:18.38
Paul
I think at one point I counted somewhere around a hundred i you know give or take ten or so So it was pretty good attendance. The talks were were all good. The keynote was fantastic. I’m sure we’ll come back to that.
04:33.73
Paul
But even from just the administration side, everything was smooth. People kept the time. And I think there was good audience engagement. So think that’s really what we want.
04:45.13
Paul
you know And know I think we checked those boxes pretty well.
04:45.50
Jon
Yeah.
04:46.14
Avitta Ottley
Yeah. Yes.
04:49.02
Jon
Yeah. I mean, I was, I was shocked that people stayed on time. I mean, that was the part that, I mean, rarely.
04:54.21
Paul
Rarely happens, but it went well, yeah.
04:55.70
Jon
Yeah. um Let’s start at the end and then come back to the, to the keynote. Elvita, you, you kind of came in like, I think kind of halfway through like straight from the airport. So pretty tired. Like how did you, what was your sense of the room when you walked in sort of like halfway through this whole, I mean, half day session. pretty long.
05:14.26
Avitta Ottley
Yeah, i I mean, I was pleasantly surprised. I didn’t, i actually don’t know what I expected in terms of a attendance. It’s really hard to predict because the Viz conference, they have several workshops going on at the same time.
05:31.09
Avitta Ottley
So there might have been seven or so, six or seven
05:35.70
Jon
Yeah. Yeah.
05:38.17
Avitta Ottley
concurrent sessions. And so I was really glad to see that it was well attended and the audience, they were asking lots of questions. So it’s as good as it can be. Yeah.
05:52.57
Jon
Yeah. Um, so let’s go back to the keynote was from more Stefan, or, who we kind of gave free reign to, to kind of like, I think we, we reached out to him originally cause he had done some work with the world health organization on their design system, but we kind of give him free reign. don’t know, Paul, do you want to maybe just your reactions to his, to his talk?
06:13.07
Paul
Well, sure. I mean, i I enjoyed it. I thought it was, a in terms of the delivery, it was it was good, of course, and engaging. um But I think also he was he was ah addressing some issues that I think are quite relevant at the intersection of researcher practitioner kind of dialogues.
06:37.07
Paul
um And yeah, just really talking about his, i think his process and some of the things that he thinks about when he’s working on these sometimes pretty complicated projects.
06:49.62
Paul
um I think, yeah, it was, it was just a really nice overview. He had some sort of slides that pulled together quite a few ideas, including ai you know, AI’s role and in design. um And yeah, I think the the reception was quite good. I heard a lot of people talking about it. There were questions in the room and I heard people talking in the hallways.
07:15.11
Paul
So yeah, I think it was it was quite successful.
07:18.06
Jon
Yeah, no, i I agree. And I think he did a really nice job of doing what we hoped he would do, which is talk about his kind of practice and also kind of lead the researchers in the room to see like, what do we still need evidence for when it comes to effectively communicating you know data or dashboards or design practices.
07:40.54
Jon
Um, and he is just kind of like, he’s got that, that kind of casual speaking, uh, emotion thing that I think people just kind of naturally kind of connect with. So I thought it was like really well, uh, well received. Um, okay. so that’s Viscom obviously full throated success. Uh, it will have to be repeated every year with, you know, full funding from the organizers. Okay. Um,
08:06.46
Jon
what ah do you Do you have another workshop that you attended monday ah Sunday or Monday that you want to want to mention?
08:15.20
Avitta Ottley
I didn’t do that workshop so Paul?
08:17.21
Jon
You didn’t go to any other ones?
08:19.20
Avitta Ottley
No, mean, I got there late for Vizcom and the other workshops that i were interested that I was interested in were all on Sunday.
08:24.66
Jon
Right.
08:28.96
Avitta Ottley
So
08:30.40
Jon
Oh, man. you missed them.
08:31.86
Paul
Terrible.
08:32.44
Avitta Ottley
just had lots of meetings on Monday.
08:36.38
Jon
Paul, do you say see anything else? I know like the big one that everybody went to is the Gen AI one.
08:40.75
Paul
I did not go to that. I went to Alt-Viz on Sunday morning.
08:42.01
Jon
I didn’t either. but oh yeah, okay.
08:44.45
Avitta Ottley
Yeah, I want to
08:45.59
Paul
I had a paper and and then, of course, I was there. I didn’t go to any on Monday because I was just hit so hard with the jet lag and I needed that day to recover, I think.
08:53.98
Jon
Okay.
08:56.44
Jon
yeah Okay.
08:57.50
Paul
But I went to Alt-Viz.
08:57.82
Jon
well Yeah, I went to AltViz, which is kind of like the fun… crazy viz short paper one, which was, which is kind of fun, lighthearted. The one that I spent all day, the second day was at um input viz, um which I enjoyed, except that I kind of felt like we spent a lot of time on like a specific project, which I think was really helpful for the person whose project it was, but for everybody else in the room, I think it just kind of bogged down, but the latter half of that day was great.
09:13.41
Paul
Mm-hmm.
09:28.51
Jon
It was a lot of like, what is the future of this part of the field? What needs to be explored? What research needs to happen? So I really enjoyed that part and sort of like a actionable, like here’s a Google like signup sheet.
09:41.70
Jon
If you want to be a part of like a paper, or if you want to try some experiments, like sign up on all these different areas. so I thought that was, that was really great. Okay. Um,
09:48.95
Avitta Ottley
you
09:49.00
Paul
what’s the just What’s the gist InputViz? I don’t know if the listeners will know what the
09:52.89
Jon
Yeah, that’s that’s ah that’s a good point. So I attended and I presented a paper on the data physicalization stuff that I’ve been doing. So someone you know puts a post-it note on a board or puts a thing on a hoop or whatever it is.
10:06.43
Jon
InputVis is a little bit more general than that in what a user does or gets when they do something in a visualization. So when they… you know, the sort of, you know, uh, the, you draw it series from the New York times came up a couple times where there’s like a line chart and it stops and then use the user draw what you think the trend looks like.
10:25.49
Avitta Ottley
Yeah.
10:26.10
Jon
Or the other big one that showed up was, um, uh, this is an old one from the New York times did this like scatterplot of your feelings when Osama bin Laden was, um was killed. And the New York times had this thing where you could put a dot on, I don’t remember what the two axes are, but your feeling and whatever it was pride or something like that. And you could, you know, write your emotion on there. um And so those sorts of things. And, you know, I think a lot of it, a lot of people in the room were doing stuff like wearable technology, which obviously is, is sort of a big part of this. um
11:00.89
Jon
you know There wasn’t a ton of focus on like your kind of standard dashboarding, um which I think is kind of more in the practitioner space, but it had some of these other kind of more experimental um pieces. and Obviously, I was focusing on the physical part, but there are lots of people doing lots of kind of neat stuff. so Um, so that was fun.
11:19.29
Jon
Um, okay. Um, but then I left, um, I took the day off on Tuesday to explore Vienna, which, uh, was super great. Um, and then did hit of course the party in the nighttime, but, but during the day I took off. all right. So, um, Tuesday, Wednesday, Thursday, was there a Friday as well?
11:36.83
Paul
Friday morning, yeah.
11:36.89
Avitta Ottley
Yes. and are
11:37.66
Jon
Well, man. Okay. So, um, you know, what did you, what did you like most or, you know, what, what caught your attention?
11:48.25
Jon
and I don’t know, Avita, if you want to start.
11:50.76
Avitta Ottley
Sure. On Tuesday. So that’s when there’s scientific portion, the real papers, the real portion where you have paper presentations start. and I, for the first time was very excited about the best papers.
12:06.30
Avitta Ottley
So one of the feet One of the best papers were like looking at…
12:15.90
Avitta Ottley
So one of the best papers was looking at… What’s the name of that thing? Real visualization? Yes, okay.
12:25.86
Paul
What is
12:25.95
Jon
All
12:29.18
Avitta Ottley
Yeah, they did some real visualization. ah
12:31.93
Paul
Yeah, I know. I don’t remember what it’s called, but
12:38.01
Avitta Ottley
Okay. I could start.
12:40.78
Jon
right. All right.
12:41.37
Avitta Ottley
Okay.
12:41.74
Jon
well
12:43.21
Avitta Ottley
Yeah, okay.
12:43.52
Jon
Yeah. Yeah, that’s all good.
12:44.41
Avitta Ottley
I mean, this is the point, right?
12:45.26
Jon
Yeah. Yeah. yeah
12:48.89
Avitta Ottley
So one of the best papers is one by Paul and he can talk about it later, but I also really liked one that looked at data visualization for blind and visually impaired folks. So what they did was they had braille paper and they printed visualizations on braille paper. And ultimately, the main research goal they had was to ask, if you print these out and have people answer questions, will we see similar accuracy levels as when you have a sighted person looking at a a display, a visualization on a display?
13:31.99
Avitta Ottley
So they found that their results are comparable, which means that printing out visualizations on braille paper could be a good way of communicating data for people who are visually impaired.
13:46.04
Avitta Ottley
And it was low tech. So I really liked that work. And in their paper, they had lots of just like qualitative feedback where people people where their participants talked about how they feel left out from the world of visualization.
14:08.79
Avitta Ottley
So I do think that thinking about the limitations of visualizations, who are left out and who are included, it’ just it was a really great way to kick off the conference and really great way for us to start thinking about accessibility.
14:23.22
Jon
Yeah. Yeah. That’s great. um Paul, do you want to talk about your, your work?
00:02.83
Paul
Good. All right. Yeah. So um I had a best paper in the first best paper session.
00:11.63
Avitta Ottley
Congratulations.
00:12.92
Paul
and the um yeah, so the the paper is about data viz practitioners and how they how they do their work. And this is coming from a ah year multi year project that I’ve been doing along with one of my PhD students, Prakash.
00:31.19
Paul
And we had a pretty robust sort of data collection that we did across a design challenge and then diary entries and interviews. And what we were interested in was how designers were understanding what the nature of the problem is that they were solving and then how they went from that to then actually generating solutions. And i think there’s ah there’s a sort of assumption in the literature, if you look at the the the models and the
01:05.27
Paul
the frameworks that exist about design. There’s ah there’s an assumption that things happen somewhat in ah in a stepwise fashion, even though, of course, there’s iteration.
01:16.63
Paul
But what I mean is that we first sort of understand the nature of the problem, which includes things like um understanding the data, you know the data types, sort of the the nature of the data, characterizing the domain,
01:30.87
Paul
So understanding something about the nature of how people are working in this space, um understanding what tasks need to be done. So we might talk about that in terms of like task abstractions or task typologies and things.
01:46.30
Paul
And then once we have a sense of that, then we can start moving on to the solution. So then we like you know we’re picking chart types, we’re thinking about encodings and interactivity and so on.
01:54.22
Jon
Mm-hmm.
01:55.86
Paul
And what we’re whatre what I’m arguing in the paper, what we’re what we’re finding in the data is that it’s not really happening quite like that. um Instead, what we’re seeing is that there’s a what we’re calling, we’re borrowing this term from the design theory literature,
02:11.36
Paul
co-evolving problem and solution spaces. So people are simultaneously figuring out what the real nature of the problem is that they’re addressing and generating solutions. And they’re they’re going back and forth. And so they might be moving into the solution space and using that as a way to probe their their understanding of what the problem is that they’re working on and they’re sort of going back and forth.
02:33.85
Paul
And so that’s that’s the gist of what we’re arguing, but there’s there’s a bit more of a, I think, sort of a critique in some way of some of the philosophical kind of assumptions that are common in in the literature.
02:37.83
Jon
Yeah.
02:50.62
Paul
um But also I think sort of an an extension of some of the models that exist there. There’s a bit more of a, contribution of sort of ah underlying mechanisms of what’s going on when designers are thinking about how they do what they do. So yeah, that’s sort of the gist of
03:08.88
Jon
Now, do you get to wear a medal the rest of the week? Like ah like a…
03:13.21
Paul
No, no, no, nothing that ah extravagant.
03:17.69
Jon
it it it is it’s interesting to me a little bit. Like, I’ve just… In the social science conference that I’ve gone to, I’ve never seen a be like They don’t have, like, a best paper. They have… They’ll sometimes do, like, poster contests, which…
03:30.54
Jon
I’ve actually only judged one. And so I don’t know, like most time if it’s content or design related, like how they’re judged, but it tends to be mostly grad students anyways.
03:36.54
Paul
and
03:38.30
Jon
So, you know, I think that’s a good. good part of it. But um the best paper thing always sort of strikes me as a ah strange competition within this academic thing.
03:48.86
Jon
But yeah.
03:49.98
Paul
It is. Okay.
03:51.51
Jon
um Okay. um What? Okay. So we’ve talked about really two, two of the kind of, we’ll call them best papers since that’s what they were, they were deemed. um What else did you see that you really liked? Alveda was there, were there other papers that you saw?
04:06.89
Avitta Ottley
Yes. So, of course, there were several papers on LLMs and GPT.
04:13.05
Jon
Yeah, yeah. Yeah.
04:13.88
Avitta Ottley
And I did really like some of them. I could tell that the field will be saturated in the next coming years.
04:21.03
Jon
Mm-hmm.
04:22.88
Paul
Mm-hmm.
04:22.87
Avitta Ottley
But some of the ones that immediately come to mind was one paper by Nicholas Amquist and his group. And we’re looking at
04:31.13
Jon
he
04:33.53
Avitta Ottley
using LLMs to automate the design automate design feedback. So giving designers feedback on their visualizations uses using LLMs.
04:44.13
Avitta Ottley
i
04:44.39
Jon
Yeah.
04:45.08
Avitta Ottley
I definitely love that. That is something that we’re working on in our lab, but you’re doing it a different way. So that stood out. I also liked another paper that I saw that oddly they were looking at uh, whether or not, visualization can.
05:06.71
Avitta Ottley
Okay. I’m going to pause because there was another one that I should have talked about that. fed well
05:16.79
Avitta Ottley
There was another one by Namu Kim and his group, and they were also looking at visualization advice and they were comparing GPT to human advice.
05:27.99
Avitta Ottley
they got that human advice from the VizGuys, their website. So VizGuys, for those who don’t know, it is a website where you can get feedback on your visualization design from other practitioners.
05:42.62
Avitta Ottley
And what they did is that they got that feedback and they compare that feedback to some of the LLM feedback. They found that LLMs gave very similar ones and similar in terms of just the the style the length and the knowledge that is embedded.
06:00.50
Avitta Ottley
They found that humans though, when they were giving feedback, they were more likely to use references or give some theoretical grounding. And that is something that LLMs missed.
06:13.08
Jon
Right.
06:13.08
Avitta Ottley
So I do like this idea of using LLMs to help critique visualizations. I have some overall concerns with that because it’s really hard to validate that.
06:25.75
Avitta Ottley
And then the final one was again about LLMs and i liked it for surprisingly not what the author is intended. So with this one, they were trying to see if visualizations can help LLMs understand data.
06:42.30
Avitta Ottley
So they gave LLMs information in a table format and they gave LLMs information and typically it’s scatter plot. And The task that they had was ask if there’s anything noteworthy or any trends LLMs should identify if there is anything, any patterns there.
07:04.12
Avitta Ottley
What they found is that visualizations did help. So LLMs were significantly better at identifying trends, patterns, and so forth if they were given data visualization versus if they were given a table.
07:19.03
Avitta Ottley
ah and To me, that’s not very surprising. ah And it would be the same for humans as well, because one of if you give someone a table and ask them to see patterns, yes, LLMs could in theory, try to visualize it or run machine learning.
07:38.39
Jon
yeah.
07:39.03
Avitta Ottley
But the issues issue that I had with this study is that you had already selected the visualization, which is a task in and of itself. How do you play this?
07:47.53
Jon
yeah.
07:48.70
Avitta Ottley
So, the authors already decided that a scatterplot is the best way to see these patterns. And so that is just a step further.
07:57.84
Jon
yeah.
07:57.85
Avitta Ottley
But I really liked it though, because it starts, I think there is a possibility to use GPT to understand the affordances of different visualization designs.
08:10.46
Avitta Ottley
And so given X different visualizations and you ask LLMs, do you see any patterns and so forth?
08:10.57
Jon
Yeah.
08:18.29
Avitta Ottley
I think it would be a really cool way to test what is the first thing people tend to see, even though LLMs are not people. And there are issues about using it as proxies for people.
08:27.16
Jon
Yeah.
08:30.74
Avitta Ottley
But I do see so that there are some interesting questions that we can start asking with LLMs at scale and then validate it with humans.
08:38.87
Jon
Yeah. I guess I am a little surprised on that finding because it would seem to me that if you fed a table into an LLM, it would be able to do the statistical analysis, which which we know is insufficient, but to at least get a sense of patterns, I’m surprised.
08:56.18
Avitta Ottley
but which to what but which statistics to run. so that is a thing. Like even though there are a variety of statistical analysis they could run, one of the hard part of that problem is figuring out which one to run.
09:01.85
Jon
Yeah.
09:10.84
Jon
youhe
09:11.45
Avitta Ottley
And I imagine that LLMs have limitations on how many calls it can make. And so it might choose one,
09:16.66
Jon
e
09:18.23
Avitta Ottley
and that one might not have been the ah the ideal one.
09:22.29
Jon
Right. I’m curious because I saw so much, at least on the program of of LLMs. And um I wonder whether you think… Uh, it’s too early to do some of this work or if this is, or it’s, it, this is, you know, it’s just kind of at the ground floor and this is when work should get started. But like, whether you think the exponential curve in how they work and they’re learning is going to change everything you just saw couple weeks ago. Does that make sense?
09:53.69
Avitta Ottley
Yes.
09:54.42
Jon
Yeah.
09:54.46
Avitta Ottley
And the question the answer to that is yes and no. ah it is It is too early.
09:57.75
Jon
Okay.
10:00.14
Avitta Ottley
And i particularly don’t, I’m not working too much in this space for exactly what you said.
10:04.97
Jon
Mm-hmm.
10:06.62
Avitta Ottley
we had a project a few months ago that was that was presented at EuroVis.
10:06.71
Jon
Yeah.
10:14.31
Avitta Ottley
we were comparing different LLMs to figure out which one is best at visualization tasks. ah
10:21.66
Jon
Mm-hmm.
10:22.26
Avitta Ottley
At that time, Claude was best, but now we did it with GPT-40 and now there’s GPT-51.
10:29.56
Jon
Yeah.
10:29.69
Avitta Ottley
And so that results already is obsolete.
10:33.82
Jon
Mm-hmm.
10:33.94
Avitta Ottley
And so it is, i think it’s early, but I do see that it could still be beneficial. What I mean by that is we have a benchmark now of what older LLMs were doing, and now we can update that with the new LLMs.
10:52.72
Avitta Ottley
Now you have pipeline to easily test them.
10:53.04
Jon
Right.
10:55.56
Avitta Ottley
So I do think that’s a plus. I agree that some of the tools that we have right now are not as good as it could be, but having that infrastructure and having people go through that thought process, I think is going to make it easier for us to develop better things in the future.
11:03.77
Jon
who
11:13.14
Jon
Right. um Okay. ah Paul, um other things that you saw that you liked?
11:19.67
Paul
Yeah, maybe I’ll, um,
11:22.37
Jon
I’m guessing you saw a lot of AI LLM stuff too.
11:25.33
Avitta Ottley
Thank you.
11:25.49
Paul
oh yeah. I mean, it’s impossible not to, think it’s hard, it’s hard not to see those.
11:28.02
Jon
Yeah. Yeah. Yeah.
11:30.66
Paul
Um, but maybe I’ll, maybe I’ll shift the conversation a little bit to the keynote, the the keynote, which was on, uh, Tuesday and then the capstone, which was on Friday.
11:43.86
Paul
Um, and I, I, I really liked them. Um,
11:49.75
Jon
So who, who did, who did both just so folks know?
11:52.06
Paul
Okay, yeah, the keynote was from Hans Christian Hager. um and yeah And the title was the science of visual insight, transforming data visualization into a rigorous discipline.
11:57.30
Jon
Right.
12:06.84
Paul
um I liked it. i wouldn’t have I wouldn’t have done the same talk, but you know whatever. that’s you know that’s how That’s everyone’s prerogative, right?
12:12.79
Jon
Mm-hmm.
12:12.93
Avitta Ottley
Thank you.
12:15.64
Paul
um But what I thought was interesting was that it was, I think it sort of signals, a in a way maybe signals a kind of maturity of the discipline because it was really sort of almost meta theoretical in some ways and was asking like, what makes this a rigorous field? And what does it mean to be scientific?
12:36.50
Paul
um And there was a whole like discussion during the middle of the keynote about the nature of truth, which of course is a you know unanswerable sort of universally. But um the the speaker was sort of describing his foray into the various philosophies of truth and what it means and how it was a lot more complicated than he thought you know that it should be. And of course, people have been writing about these things since the beginning of time.
13:05.72
Paul
um And so, you know, the like the conclusion was that um we need to. Well, I don’t know if I’m characterizing correctly, but that we need to sort of make this notion of insight and the the field of data visualization more rigorous by considering what we mean by truth and recognizing that there are a variety of factors, then a variety of ways of seeing truth and So personally, I wouldn’t have, you know, I wouldn’t have done it that way, but I think it was, I think it raised a lot of questions. People were talking about it afterwards. um
13:43.13
Paul
It’s sort of like raised questions about core, sort of like central things about rigor, methodology, and like the scientific nature of the field.
13:48.60
Jon
Yeah.
13:50.75
Avitta Ottley
Yeah.
13:53.10
Paul
So, yeah, I, um I enjoyed that, ah you know, from that perspective, it wasn’t just like, let me tell you about some visualizations I’ve made, you know,
13:53.70
Jon
Mm-hmm.
13:58.92
Jon
Yeah.
14:01.75
Jon
Yeah, right.
14:02.55
Paul
thing. So it was much more philosophical and inward looking kind of
14:07.19
Jon
Yeah. I mean, i I obviously didn’t get to see that, but but I have been struck as someone kind of on the outside of this field, attending this for, you know, for occasionally for a while now and seeing some papers that come out with like,
14:23.32
Jon
you know, we tested this thing for six people and here’s the point estimate and here’s the standard error. And I’m like, six people, like your the standard error is plus or minus infinity.
14:34.62
Jon
And so it does, oftentimes when I read Viz papers, I feel like the field sometimes isn’t sure, like, is it a field of psychology?
14:35.00
Avitta Ottley
Thank you.
14:45.07
Jon
Is it a field of statistics and computer science? And what’s the blending of the two? So I appreciate the sort of, like I think, as you said, Paul, like the evolution of the field and so what at its core sort of is the is the science is an interesting question for sure.
15:00.98
Paul
Yeah, I think you he was, kind at least what I took away was he was kind of saying, you know, we need to move beyond this like field as a collection or like a toolbox of techniques in like a CS technical kind of way and really ask ourselves questions about like the philosophical foundations of the field sort of.
15:20.41
Jon
Yeah.
15:20.95
Paul
So that’s the part I liked, yeah.
15:22.78
Jon
um Okay, and the capstone? i don’t even know who did the capstone.
15:26.04
Paul
yeah The capstone was, I’m having to look these up because I forgot, Theo Dudinger, I think.
15:30.58
Jon
ah
15:33.98
Paul
I’m you know probably not saying them right right, but a diagrammatic approach to the world. And I really like this one. This one was quite different from the capstone, but shared some flavors, I think, that were similar.
15:46.55
Paul
And so he’s a, if I understand correctly, an architect by by training maybe, but does a lot of like really sort of graphic design, infographics kind of work.
15:59.10
Paul
And what he was doing was looking back, I think, I don’t know if it was like over the past 100 years or something, and using diagrams to communicate changes in our world, essentially.
16:13.50
Paul
And I think he was exploring themes of globalization, migration, and borders, and sort of power dynamics in terms of like globalization in the world.
16:14.42
Jon
Interesting.
16:24.36
Paul
and so There were a lot of really interesting um diagrams or like visualizations that he showed that that were, i mean, they’re really like bringing in a critical theoretical sort of angle, I think, really questioning the power dynamics and why we why are we keep people in or out of certain borders and and so on, looking at trends and globalization and what’s that what that’s doing to the environment.
16:47.93
Jon
Yeah. No.
16:52.40
Paul
So these types of questions, but really told through the story of a really wide variety of visuals.
16:59.70
Jon
yeah
17:00.38
Paul
So it was really it was really interesting.
17:01.56
Jon
no Yeah. And it sounds like a good way to end it. Right. With a sort of look back to look forward kind of thing. um So obviously we could talk about all sorts of papers, but ah let me just ask sort of like overarching. So like, what did you like most? And that could be either like the organization itself, the location, the papers, but like, was there something that you liked the most? Alvita, maybe we start with you.
17:26.62
Avitta Ottley
I don’t know there’s something like the most beyond just meeting people because For me, visualization, these conferences is the one time we all get together.
17:30.90
Jon
Yeah.
17:38.78
Avitta Ottley
But f if I were to really be critical about it, i i remember there was this one panel that really touched on something that was intriguing.
17:51.26
Avitta Ottley
It was a panel run by Lane Harrison, Mariah Meyer, Irvind, and some other folks, and it was a follow-up from a a workshop that they had this summer.
18:04.70
Avitta Ottley
And it was,
18:08.79
Avitta Ottley
i guess it doesn’t really matter what it was about, but a part of this was a conversation about ethics and visualization and really thinking about, are there times when it’s sometimes okay to reinforce biases and stereotypes in visualization design?
18:11.29
Jon
Yeah. yeah
18:17.22
Jon
Mm-hmm.
18:27.78
Jon
Mm-hmm.
18:28.38
Avitta Ottley
And off the top of your head, you would say, no, why would you do that? or Particularly you, John, I know that you’ve been thinking about this for a while, but their scenario that someone brought up is you are a doctor and you are trying to communicate something to like an older man, you know, ahead of time that they are accustomed to seeing pink and blue.
18:32.08
Jon
Mm-hmm.
18:53.56
Avitta Ottley
That is your, that is the ah world model.
18:53.98
Jon
e
18:57.85
Avitta Ottley
uh, Perhaps, you know, if you show them pink or blue, studies would suggest that it would be easier for them to understand. Now, should you color pink and blue or should you color it based on a more neutral color or try not to reinforce these stereotypes?
19:15.95
Paul
you
19:16.70
Jon
Yeah.
19:17.02
Avitta Ottley
And i do think that it’s ah it’s something that gives you pause. And I think it’s really philosophical question to ask.
19:21.75
Jon
will
19:25.99
Jon
Yeah. Yeah. I think oftentimes a lot of these conversations lose the specific context or the specific audience.
19:37.97
Jon
And in a healthcare care setting, you might do things that you wouldn’t do sort of publicly out in the world, but you’re dealing with a patient and, you know, you know, you need, maybe you need to do things or say things or show things that, you know, um that, you know, in a more, in a different context, like in the, in the news or something you would, or in an academic paper, you might not do, but yeah, that’s a, that is an interesting, um interesting topic.
19:38.43
Avitta Ottley
Yes.
20:04.36
Jon
um Paul, anything, any other like big reactions, feedback, things you like the most?
20:12.95
Paul
um
20:12.98
Jon
I guess you could say the least if you want to be the negative guy, but you already won your medal.
20:15.07
Avitta Ottley
Ha ha ha.
20:17.19
Jon
So I don’t think you want to throw shade.
20:18.39
Paul
Yeah, no, and I’m not going to say, I don’t want to say it no.
20:19.23
Jon
Yeah.
20:21.28
Paul
um Yeah, I think in terms of like a big picture view, um and perhaps I’m, you know, I’m seeing this from my own perspective, of course, but I think there’s a, I got the feeling that there’s sort of a,
20:37.03
Paul
maturation kind of going on in the field. you know And we just talked about the keynote and the capstone. I think those were kind of indicative of that going on. But just I’m seeing a bit more maybe both interdisciplinarity in some of the conversations going on, but also sort of inward reflection too about like, what are what is our theory telling us and where are we going as a field and what makes this a coherent field?
20:52.73
Jon
Mm-hmm.
21:06.84
Paul
And I think that’s more palatable to people. Those kinds of conversations are maybe more in people’s minds nowadays. Like I said, maybe I’m just projecting that, but I sort of got that sense. So I enjoyed that. I thought that’s a good.
21:22.52
Jon
I don’t want to be too much of a doubter, but I do. there’s There’s a movie like quote echo in the back of my head. That’s like, you know, when you come to the end, you always go back and look at the beginning. So, you know, I don’t know if, I mean, that may be too dark, but, um you know, I mean, there I certainly had a lot of conversations with people about funding challenges, about a lot of graduate students worried about their immigration status and where they could or could not get jobs or postdocs.
21:33.08
Paul
Yeah.
21:47.93
Jon
So, I mean, i know I know I had a lot of those conversations. um
21:51.35
Avitta Ottley
Yeah. I do have a least favorite thing though.
21:54.10
Jon
Oh, okay. Yeah. Let’s, let’s go.
21:55.51
Avitta Ottley
Now I think about it.
21:55.74
Jon
Who are we going to throw?
21:56.38
Avitta Ottley
One that is not
21:57.26
Jon
Who are we going to who we gonna throw under the bus? Yeah. Okay.
22:01.45
Avitta Ottley
hopefully is not too controversial. Well, but I agree with Paul that I see the field and even the conference maturing.
22:03.58
Jon
okay All right.
22:11.19
Avitta Ottley
So we even just had revised where this is a committee that comes together to think about how to improve the practices for the conference itself.
22:20.42
Jon
Mm-hmm.
22:20.44
Avitta Ottley
But over the years, what I’ve what I’ve seen is that even though we have a lot of workshops, still the types of workshops are different. So in the past, there were several workshops that bring in practitioners.
22:34.12
Avitta Ottley
I can think of the LDAP workshop, of course, the VisCom workshop, and i see less of that now.
22:36.51
Jon
Mm-hmm. Mm-hmm.
22:42.46
Avitta Ottley
And it is somewhat intentional and I guess also unintentional. I agree that funding and just of travel restrictions make it difficult for people to attend.
22:48.48
Paul
Thank you.
22:54.62
Avitta Ottley
But there are also some some structural changes that this itself is making. They are intentionally trying to cut down on the number of workshops. And that’s even going to be worse next year.
23:06.06
Avitta Ottley
So workshop might be one day instead of two days.
23:06.36
Jon
Right.
23:10.94
Avitta Ottley
i I understand the notion of that. But at the same time, i think it’s worthwhile to think about how to do it in a controlled fashion so that We don’t exclude all of the workshops that would bring in new ideas and fresh faces.
23:31.00
Jon
Yeah. I mean, I have some thoughts on that too, but Paul, do you have, do you have react? I mean, I know um I’m sure you talked to folks about plans for Boston, but like, yeah. What did you see? What’d you hear? what are your thoughts on the kind of theme that Alveda touched on?
23:48.09
Paul
Well, yeah so I have heard this the same thing, that the workshops are probably going down to one day. i don’t know all the reasoning behind it, so I don’t really have an informed opinion.
23:57.66
Jon
Yeah.
23:59.03
Paul
I mean, one critique I do have of the workshops is that they’re not really workshops.
24:03.27
Jon
Yeah.
24:03.67
Paul
you know They’re just like mini sessions. So… Take that for you know whatever, however you will. But I imagine there are probably ways that we could have more kind of hands on things like more open forum discussions and bring in other voices because it’s hard to do that. And unless you have a paper, you’re not really doing that in the workshops anyway.
24:28.53
Avitta Ottley
Yep.
24:28.76
Paul
So it’s not really like an open forum for shared discussion.
24:28.76
Jon
Yeah.
24:32.76
Paul
It’s just let me present another paper. So I think i you know I might add that on, but that’s like that’s a bigger structural change than just like cutting down maybe.
24:43.06
Jon
Yeah, I mean, I agree with you. It does feel, at least the ones that I attended, felt like, I mean, even VisCom is a series of papers, essentially, right? It’s not like we didn’t have a panel discussion of two academics and two practitioners sort of batting ideas around. I mean, I think the input vis, they were basically lightning talks and there was a lot of people. So the lightning talks talks took a long time, but the rest of the day was like, you’re working in groups and doing things. I mean, and I i personally coming from the practitioner side,
25:12.73
Jon
I struggle with this a little bit because on the one hand, I do want to see more interdisciplinary work collaboration going on between the academics and the practitioners. But on the other hand, um it is an academic conference.
25:25.54
Jon
At least that’s sort of like what it has been to to date. Maybe that’s not the right way, but like it’s an academic conference.
25:30.73
Avitta Ottley
It was.
25:31.26
Jon
So this is a place where people go to share their their research. But kind of point to your point, um Paul, about the about the keynote, like, what is the practice?
25:42.71
Jon
What is the science of it? And how much does, you know, how much can practitioners sort of lend their experience to it? I would also say just from the practitioner side, you know, thinking of just like a freelancer, totally on their own.
25:56.49
Jon
It’s a very expensive conference to, to attend.
25:58.62
Avitta Ottley
this
25:59.31
Jon
I mean, the, the, don’t know the prices off top my head. I mean, I only went for two days and it was, it was pretty expensive. So yeah, I mean, yeah.
26:05.51
Avitta Ottley
Probably 1200 for the week.
26:08.18
Paul
Yeah, it’s prohibitive for a lot of people, I think.
26:10.65
Jon
Yeah, right. And plus, you know, you have the travel, even if you lived in, in this case, in, even if you lived in Austria, and you know, you got to pay for the, for, you know, all the travel. So it’s not a cheap, not a cheap, um, uh, conference, but,
26:24.89
Jon
ah I have heard the same thing. They’re going to cut down the the workshops to one day and maybe even cut a day off at the end with fewer papers of that half. I guess it’s a half day is the last day is a half. So maybe cut that.
26:35.49
Paul
think we’re…
26:35.61
Avitta Ottley
Yes.
26:36.31
Jon
Yeah. um I mean, it’s ah it’s a long conference for sure. um But I always have a good time. i like always like to see the the papers. Yeah. And I had a bit of hesitation on some of these AI papers because I’m like, I don’t know if we’re like there yet, but um I’m not doing the work. So, you know, easy for me to criticize, not have to do anything.
26:59.45
Jon
Okay. So I guess last question is um For you two, um what are you working on now if you want to share? And like the the the deadline for Viz26 comes up in like three months or something like that, maybe four months, like it’s in March, I think, something like that.
27:17.71
Paul
Yeah,
27:19.00
Jon
Yeah. So um what are you working on now and what are you hoping to pull together that you can submit? If you if you have topics top of top of mind that you want to share. don’t know, Paul, you’re you’re nodding, so maybe you can go first.
27:32.57
Paul
yeah I was just nodding in agreement. Like, yeah, that’s a good question. um um
27:38.53
Jon
And you’ve got big shoes to fill. You’ve got to follow up best paper. Mm-hmm.
27:42.14
Paul
and know it’s only downhill from here. Maybe I should retire. Yeah. um
27:47.13
Jon
yeah
27:48.18
Paul
ah Well, so yeah, I mean, I’m sort of I’m continuing with the similar theme. I’ve still got quite a lot of data from that study, which was ah a kind of a massive undertaking, just getting that off the ground.
27:59.67
Paul
So something coming out of that, perhaps I don’t have anything concrete, but I’m also um running another study. where i’m I’m in the process of planning it where I’m essentially doing the same thing.
28:11.74
Paul
But the the previous one was with with expert practitioners and the next one is going to be with novices. And then I’m going to I’m going to run the same study and then try to figure out how things are different and how novices and experts are thinking.
28:24.44
Jon
Yeah.
28:28.18
Jon
Nice.
28:28.44
Paul
Whether that will get done in time for Viz, I don’t know. But I’ll probably try to pull a few things together.
28:31.86
Jon
Right.
28:34.32
Paul
So that’s kind of where I’m where i’m thinking.
28:37.11
Jon
Nice. Alveda, what do you have on your ah on your plate?
28:41.32
Avitta Ottley
That’s a hard question because I’m trying to think about what I can actually talk about. So there are papers right now that are in CHI and, you know, they want it to be anonymous.
28:47.45
Jon
um Right.
28:52.14
Avitta Ottley
So I i can’t talk about those.
28:54.07
Jon
Right.
28:54.78
Avitta Ottley
And there are ideas that are too junior for me to talk about as well.
28:54.87
Jon
Right.
29:00.63
Avitta Ottley
So, okay. So there are some things that might be in that sweet spot. So one of the projects we’ve been working on is extending the data visualization literacy test. So a few years ago in our lab, we created the mini VLAT, which is three minutes way of testing your data visualization literacy.
29:15.47
Jon
Mm-hmm.
29:21.50
Avitta Ottley
Rightly so, there are people who have seen this and identify shortcomings. It was not meant to be a practical thing. It was more of a research contribution, but it’s clear to me that there are people around the world who would love to have tests like this that they can use.
29:33.43
Jon
Yeah.
29:39.45
Avitta Ottley
and So one of the things that we’ve been doing, I’ve had a student who have been creating tests for almost every country. There are some countries that it’s really difficult to find data, but our idea is to have a test where the data sets that we use are going to be customized for that country.
29:59.67
Avitta Ottley
And of course, translating it into different language. As you can tell, this is a huge undertaking.
30:04.31
Jon
Yeah. Yeah.
30:07.02
Avitta Ottley
We’ve had some, we’re done with Ghana where we’ve had testing and had a well, we were focused on students, but there were students in Ghana.
30:07.10
Jon
yeah
30:16.02
Avitta Ottley
We had about 700 people who took these tests. So essentially what I’m doing is we’re creating these data i did a visualization literacy tests. And the idea is to kind of get an understanding of whether or not people understand visualization around the world in a way that is comparable.
30:34.68
Jon
Yeah. Yeah, that’s great. I mean, it’ll be, I mean, I’ve always been fascinated with um in particular newspapers and in Asian cultures that are way more data dense than we see here in the United States. um And so I would be curious to see how, you know, I mean, I think there’s like there’s that question on its own, but whether that sort of culture society impact people’s data viz literacy. So I think that’d be, that would be fascinating to you to see.
31:00.41
Jon
Yeah.
31:00.75
Avitta Ottley
I’m sure it’s the case and there are other countries where if you look at their newspapers, there are no visualizations at all.
31:06.43
Jon
Right? Yeah.
31:07.64
Avitta Ottley
So I do think, especially as the world is getting global and we’re thinking about sharing data around the world, just understanding what you can use where and maybe customize it for different cultures in and different countries.
31:21.20
Jon
Yeah. Well, it’s great chatting with you both. I think our last call, right, is to see any listeners want to be involved in Vizcom organizing, right? I think Alveda, you really want to cycle off, right?
31:31.01
Paul
Thank you.
31:34.04
Jon
but
31:34.74
Avitta Ottley
I do, but I think it’s more so that I want to make space for other people who want to come.
31:41.69
Jon
Yeah.
31:41.91
Avitta Ottley
And I think it’s good to have fresh ideas. And a part of the problem is us standing in the way and not giving opportunities to new people. So it’s there and I’m willing to help out in any way I can.
31:54.11
Jon
right. So you all heard it. ah That’s the call to be involved in Vizcom. There’s be tighter competition. So we’re going to need some help. um Great to see you both. ah Thanks for coming on the show. Lots of great stuff. I’ll put links to um all the papers in the show notes. Folks should check them out. um And ah you can go email Elvita and Paul. Y’all at them about things you liked that or didn’t like that you heard. um Thanks to you both for coming on the show. It’s always a lot of fun to talk to you.
32:22.14
Paul
Thanks, John.
32:22.71
Avitta Ottley
Thanks, John.
