This is the final episode of 2026! I hope you have enjoyed the show this year and also hope you have a great holiday season and happy new year. In this episode, I sit down with former Census Bureau Director Rob Santos to talk about the state of federal statistics, what’s threatening the quality and independence of federal data, and why surveys like the American Community Survey and decennial census matter more than ever. We dig into how census data are collected, how political appointees interact with career staff, and why attempts to limit data collection or redefine who gets counted can undermine everything from policy to local decision-making. Rob also reflects on his approach to diversity, communication, and public engagement while leading the Census Bureau. We close by looking ahead at what modernization should look like for federal statistical agencies in the years to come.
Guest Bio
The Honorable Robert L. Santos is a nationally renowned statistician. He served as the 26th Director of
the U.S. Census Bureau – the largest federal statistical agency — between January 2022 and February
2025.
Before the Census Bureau appointment, he spent 15 years as vice president and chief methodologist at
the Urban Institute, Washington, DC, and was executive vice president of NuStats, Austin, Texas. Santos
also held leadership positions at the University of Chicago’s NORC, the University of Michigan’s Institute
for Social Research and Temple University’s Institute for Survey Research.
Santos served as the 2021 president of the American Statistical Association (ASA), is an ASA fellow, and
recipient of ASA’s Founder’s Award. He served as president of the American Association for Public
Opinion Research (AAPOR) in 2014 and received the 2021 AAPOR Award for Exceptionally Distinguished
Achievement. He also received the 2022 Ohtli Award by the Mexican government and the Mexican
American Legal Defense and Educational Fund’s 2023 Excellence in Community Service Award.
Santos earned a BA in mathematics from Trinity University, San Antonio and an MA in statistics from
University of Michigan, Ann Arbor. In 2023, he was awarded an honorary Doctorate of Sciences by
North Carolina State University and an honorary Associate of Science from San Antonio College.
Different Ways to Listen to the Show!
New Ways to Support the Show!
With more than 250 guests and 10 seasons of episodes, the PolicyViz Podcast is one of the longest-running data visualization podcasts around. You can support the show by downloading and listening, following the work of my guests, and sharing the show with your networks. If you’re interested in financially supporting the show, you can sign up for my Patreon platform, make a one-time payment via PayPal, or shop one of the show’s sponsors.
Transcript
00:01.29
Jon
Holy moly. It’s been far too many years, Rob. i don’t even remember how many years it’s been. um Good to see you.
00:08.60
rob
Yeah, it’s great to see you, man.
00:11.46
Jon
um Okay. So we’re going to get into lots of good stuff about your role at Census. um But let’s talk about what you’re doing these days. Relaxing down on the ranch in Texas.
00:24.67
rob
Doing a lot of that and a lot of suffering, too. I mean, our nation is going through a lot.
00:28.16
Jon
Yeah.
00:30.56
Jon
Yeah.
00:31.45
rob
And so in the midst of my, quote, retirement, I’m out here at the ranch, 150 miles west Austin, Texas, enjoying the stars, wildlife, the grandkids, doing a lot ranch type of stuff.
00:50.28
rob
I don’t own animals, but I maintain wildlife and do a lot of, you know, clearing of land.
00:54.28
Jon
Yeah.
00:56.75
rob
And there’s just always something to do.
00:57.24
Jon
Yeah.
01:00.13
Jon
I bet.
01:00.86
rob
But at the same time, I’m getting calls from you know past census employees, from stakeholder groups, from folks potentially doing litigation and such.
01:12.23
Jon
Right.
01:12.34
rob
And ah you know they ask for advice and what I think and I tell them. So I’m actually fairly active. I also do a bit of public speaking. I’ve had some pretty regular engagements to do keynotes at universities and conferences.
01:28.82
rob
at a nonprofit organization, Guillermo’s, those types.
01:32.15
Jon
Right. So I do want to talk about the communication side because um you were kind of known, among other things, known for that at the Census Bureau. So I do want to talk about that. But I want to start the the conversation about um maybe giving listeners a sense of how census data are collected.
01:52.13
Jon
In a couple of the previous episodes of the show, I i talked to Erica Groshen and Bill Beach, used to run the BLS, to talk about, like, the commissioner can’t fake these job numbers, right?
02:02.22
Jon
The the way the the president sort of accused Commissioner McEnharper.
02:02.14
rob
Thank you.
02:06.88
Jon
So, can you give folks a sense of, like, how census data are collected and how, um and maybe give them reassurances that there isn’t, that there aren’t people there who can just start making things up to fake numbers and make things look better or worse under one administration versus another administration.
02:24.66
rob
ah Certainly, and I’ll ah speak to two different aspects. The first being how data are collected by sort of the scientifically rigorous process ah by the career staff.
02:38.10
rob
And secondly, how and a presidentially appointed you know appointee like the director or some of the other positions that are there. We have two other positions that were appointeeships.
02:55.03
rob
um how they interact with the Census Bureau career staff as far as data publication and what the numbers are and things of that go.
02:59.29
Jon
Mm-hmm.
03:05.10
rob
So first, with regard to actual data collection, Census Bureau and all federal statistical agencies have a really high bar of scientific integrity and independence when it comes to methodologies and how many, you know, how you knock on doors and things of that sort, where the only thing that can change that would be think from, you know, outside of a scientific perspective would be if a law was passed, like the current threatening laws on ah potential legislation on limiting the number of calls, like two calls for survey, that would be disastrous.
03:47.72
Jon
Right. right
03:50.72
rob
ah So apart from those types of interventions, the the the career staff have full control over how data collected are collected, how they are processed, and the numbers that are crunched, in including the types of estimators, how you blend things.
04:11.32
rob
And then they publish the results without any interpretation. And there are some examples I can give where ah you can actually interpret numbers different ways.
04:21.86
rob
And I talked to the Census Bureau about that. so that So that’s, I have high confidence that no appointee can embed themselves in the process of producing the numbers so that they come up differently than what they are.
04:40.43
Jon
Mm-hmm.
04:42.16
rob
um then it comes Then there’s the political appointees. And in our leadership roles, we’re responsible for the agency and all the products.
04:54.67
rob
Interestingly, in order to preserve the scientific integrity and the independence of the career staff and the data that are collected, there is something called pre-decisional stuff.
05:09.70
rob
And so, for example, the numbers that come out of the economic directorate on gross domestic product or some of the and economic indicator indicators that are collected, producer price index, I believe, is one.
05:27.01
rob
I, as director or any director, would not be able to see them until the morning that they’re released. We do not have that privilege. The same is true, but a little with a little bit more lead time for things like the results of the post-enumeration study that were released that showed undercounts of you know African Americans and Latinos and people in tribal lands and such, or the up the over and undercounts of different states.
06:00.00
rob
It was only a few days before their release, after everything was codified, tables were made, etc., that that the appointee then is briefed so that they know it’s going to be released in a couple of days.
06:15.96
rob
and because of that, there’s this notion that appointees do not have the privilege of seeing pre-decisional materials. They need to wait until they’re finalized and then they they are allowed to review.
06:32.59
rob
We did that and I think that is a great practice. I think it still continues. I’m sure it still continues. um And it’s an added protection that exists both at BLS as well as at Census Bureau and all the federal statistical agencies.
06:49.41
Jon
Yeah. um You mentioned one thing I wanted to… To give folks ah another sense of how this works, like specifically there is pending legislation now on limiting how many calls the Census Bureau can conduct during a survey.
07:03.51
Jon
You just mentioned how that would be very problematic. Can you explain to people why that would be problematic and why census shouldn’t just be allowed to, you know, keep calling people all the time?
07:14.63
rob
ah Well, first of all, because the taxpayer pays for data that we need as a nation to operate our governance and the the economy and community development, we need to make sure at the Census Bureau ah to have really good, solid, reliable, accurate data.
07:35.97
rob
And in order to do that, you simply can’t ask people to participate and then whoever self decides to respond on their own after a call or two, hey you get so little data from that, that it’s not reliable and it it will have inherent biases in it because the people that respond early on in a survey operation are very different than the people that respond later on after you keep trying to to get encouragement.
08:05.12
Jon
Mm-hmm.
08:05.93
rob
to participate and it’s ah because of that that it’s uh it’s pretty much required that in order to reach the levels of data quality that are needed for official government statistics that there needs to be repeated attempts to encourage participation and you see that in the census the american community survey and all of the other 120 plus ah surveys, both economic and as far as businesses, as well as household and population surveys that are conducted. It’s really, really important to do that.
08:43.62
rob
What would happen if you limited it to two is that our gold standard Data collections like the American Community Survey would end up with a tiny response rate of maybe 20%, I’m guessing here, instead of up in the 79, 80, 81% range.
09:06.94
Jon
There’s another discussion that is a little bit quiet the last couple of months. But but early on, the the president had suggested sort of creating an alternative decennial census or some other sort of census that wouldn’t ah include people who are in the living in the country without you know legal authorization.
09:25.41
Jon
um and And we could debate the the constitutionality of that, but neither of us are legal scholars. So I’ll ill i’ll pass that by. But just to ask, similar to the to the point of about…
09:36.15
rob
I have an opinion. in
09:38.05
Jon
I’m sure you do. Okay, so ah so I do want your opinion, but I also…
09:39.07
rob
morning
09:42.26
Jon
um Yeah, so i so I’d like to know what you think about that. But also, I mean, this is an issue that you were that you were working on when you were at Urban back in 2019, 2020 in the administration then.
09:53.75
Jon
I mean, so so maybe we could talk about the first part, which is, why is this um but potentially problematic when when collect when trying to collect ACS data, CPS data, census data?
10:06.71
rob
Yeah, ah okay, so let’s start with this notion of having different types of counts. um the The Constitution, I think, is pretty clear, you know, the 14th Amendment, that you count whole persons.
10:17.50
Jon
Mm-hmm.
10:21.13
rob
And because of that, as far as the Census Bureau was concerned, up until the day I left, that was going to be the rule.
10:32.01
rob
and in and And so it would literally take court litigation to change that.
10:32.11
Jon
right
10:40.95
rob
And I expect that if there are any attempts to change what numbers are presented to the president in 2030, that there would be a fair amount of litigation prior to that.
10:53.57
Jon
Mm-hmm. Mm-hmm.
10:53.81
rob
ah Interestingly, if the If you want a count of individuals who are only citizens or who are citizens plus documented immigrants with you know green cards or whatever, um you really have to count everyone first.
11:17.64
rob
And then you have to find the individuals who you need to excise. Because if you don’t, then it you have two sources of error.
11:28.13
rob
You have the source of error, did you really capture everybody you wanted to who were citizens and documented? and you really don’t have a grasp on that. And then you’d have the group of folks who are undocumented that you need to excise from the counts.
11:46.29
rob
And you there’s really it’s very, very difficult, if not possible, to get those counts.
11:50.23
Jon
right
11:52.61
rob
So the the scientifically, the best way to do it is to count everyone and then figure out which among those need to be excised. Which gets to the point that I was going to make here, which is no matter what, I believe that if you’re going to do a census, regardless of what any president wants is the count, you’ve got to start with counting everyone.
12:15.13
Jon
Mm-hmm now you’ve also, if I, if if I recall correctly, so I believe that this was like 2019, 2020, you did a project here that it’s not that, that the impact of not including people who are not citizens impacts the data for people who are citizens, um, that you, that you change the response rates, you change people’s trust in the data.
12:41.02
Jon
Um, So let me make this a broader question. Like, what do you think right now is the biggest threat to the federal data, federal statistical agency ecosystem?
12:52.29
rob
There are actually a couple of huge threats. The first of which is this notion of creating efficiency through destroying infrastructure.
13:05.55
rob
So we’ve seen the National Center for Education Statistics essentially being gutted. ah There have been severe cuts at the National Center for Health Statistics. There are similar cuts in other a federal statistical agencies, including a substantial brain drain from the Census Bureau as a result of pressure to reduce force by some percentage that I don’t know that occurred after I left.
13:30.75
Jon
Yeah.
13:33.39
rob
um So that that is a huge a huge risk that we are facing in order to be able to maintain the quality of data that’s needed in order to move forward and provide the nation with the data that that we deserve type of thing.
13:52.83
Jon
yeah
13:54.30
rob
So the let’s see, that’s that. ah Remind me of the question again, because there was another
14:00.76
Jon
what ah yeah know What are the threat the the threats to the um federal data for federal statistical agencies? So you mentioned the sort of infrastructure changes of taking down certain teams or data sets.
14:12.99
rob
Yeah, yeah the the second threat would be sort of legislation or executive orders that would alter statistics. so for example the secretary of Commerce advocated ah excising the, I believe it was government employees from the unemployment rate or something of that sort.
14:36.15
Jon
Right. Yeah.
14:37.75
rob
ah Those types of of fundamental changes that are politically motivated actually could do damage to our ability to understand who we are as a nation and to be able to govern and and have the economy grow.
14:53.63
Jon
Right. Do you think a lot of these attacks are kind of a fundamental Do you think it’s it’s it’s a misunderstanding of how important the data ecosystem is to good governance and good policy? Or is it just a political you know fight against you know whatever the political agenda might be? or Or again, do you think that there’s just kind of maybe not a full understanding of of what these data are useful for and and how they’re how important they are?
15:26.82
rob
Well, I hate to tell you, but I think it’s both.
15:29.37
Jon
Yeah. yeah Okay. Okay.
15:31.34
rob
I think there that there is a there is a heavy political influence on scoring sort of victories or political power and then implementing means and then implementing means that affect the ability of federal statistical agencies to govern or or to operate in the way that they need to.
16:01.68
Jon
Right.
16:03.22
rob
And that is the the issue of like shoot first and then aim later. And we’re seeing that repeatedly going on in the federal statistical system in terms of reductions in force, elimination of programs that are absolutely needed,
16:20.16
Jon
Yep.
16:20.36
rob
things of that sort. Like ah for example, the the National Crime Victimization Study that BJS does, that the Census Bureau collects.
16:26.95
Jon
right yeah
16:30.97
rob
Now, this is the only national survey that captures victimization, that is, people who are victims of crime that don’t report them to police.
16:45.38
rob
Well, it we happen to know that one of the most vulnerable groups that is subjected to the to to victimization are transgender people.
16:57.75
rob
Yet, currently, we are not collecting SOGI data on the National Crime Victimization Survey.
17:05.96
Jon
and
17:06.19
rob
So we are eliminating the ability to understand the victimization of some of the most vulnerable populations in our country. And it just makes zero sense. I don’t understand why we’re doing that.
17:21.16
Jon
Yeah, so so it’s ah it’s a really good example. so let me um let me just share sort of my the way that I think about this for people who I think for people who are not familiar with, you know, federal data or or or they think that these are these are data that are just collected for, you know,
17:39.50
Jon
researchers and academics in some in some, you know, ivory tower somewhere. But the but the the victimization survey is a good example of a data set that is used by local law enforcement around the country to figure out when they should send police out, where they should go, what are the hotspots for crime, what are the groups that are, you know, not um not likely to ah report crimes.
17:52.11
rob
Oh
18:05.04
Jon
And so, these data are used, they’re not just, I mean, I just, it frustrates me a little bit because I feel like a lot of people think that these data are just sort of academic or abstract in some sense, but they’re used by businesses and they’re used by law enforcement.
18:19.95
Jon
And I don’t know if maybe there’s a better way that more organizations and people can make that case to, you know state and federal policymakers.
18:21.70
rob
Thank you.
18:32.83
rob
Yeah, ah well, there are a couple of things that are going on here. The first is that all federal surveys, whether and data collections, whether they are done by the Census Bureau or other federal statistical agencies, are are embedded and motivated by statutes.
18:50.95
rob
Like Congress has said, we need the data in order to govern our governance, right?
18:51.70
Jon
Right. Right. right
18:55.90
rob
they They also have incredibly powerful um ability to help the economy. So the all of the there’s this huge portfolio of economic surveys and economic indicators and data that are collected.
19:14.41
rob
at the Census Bureau. In fact, most economic indicators, except for a few, the data are collected by the Census Bureau and then provided to whatever agency needs them, like whether it’s Bureau of Economic Analysis or the Bureau of Labor Statistics, as well as providing some of the Census Bureau’s generate some on their own.
19:37.53
rob
ah
19:37.66
Jon
Mm-hmm.
19:38.08
rob
And those are incredibly valuable for international trade, for internal decisions by companies. And then when you put together the the various amounts of public, like household and personal and health and all these other data together with the economic data,
19:46.17
Jon
Yeah.
20:01.83
rob
There are now tools like the the Census Business Builder that are incredibly helpful to entrepreneurs to create business cases that they can then go to the bank and say, you know here’s a proof that my business will do well in this small little, in in my little catchment area, in the my economic zone or whatever.
20:06.40
Jon
Mm-hmm.
20:14.58
Jon
Right. Right.
20:25.29
Jon
yeah
20:25.87
rob
So it interpenetrates that All of these data interpenetrate society. And we have learned over the over the last half century or more to rely on these more and more, which is why, number one, you we’ve got to keep collecting them.
20:41.90
rob
And number two, they need to be allowed to be collected accurately and reliably.
20:47.33
Jon
the Yeah. um So I want to switch gears a little bit. um When you were at Census, you were known for and a variety of different things, obviously. um i think from my perspective, the thing that I sort of saw you doing, the things that I read about you were um your efforts on diversity and inclusion within the Census Bureau staff and then more generally. And also on the communication side, um it seemed like and This is more of just a feeling of just being outside, but it seemed like, felt like you were out there a lot more than maybe previous directors.
21:25.13
Jon
um And so I was wondering if we could talk a little bit about both of those. um Maybe we start with the diversity piece in the census staff and then also just the the work that folks do, and then we can do the communication side.
21:36.65
rob
Okay.
21:37.10
Jon
And I know this is, I know this is like, it’s like 2025.
21:37.25
rob
it’s know i
21:40.30
Jon
Yeah, I know.
21:41.48
rob
Yeah, I smile when you say that because in my mind and what I brought to the table was this notion that communications and DEI and all this stuff, they’re all part of the same thing, which is pursuit of excellence and relevance in quality of data.
21:41.83
Jon
yeah
21:55.77
Jon
Yeah.
22:02.67
rob
And so for DEI, because about the time that I arrived, it became hyper-civilized, I’m sorry, hyper-politicized.
22:14.42
rob
And so the first year I was there, I was basically saying, I am the champion of DEI.
22:21.09
Jon
yeah
22:21.46
rob
and And once the political ah tags became so extreme, ah that even internally people were like, oh man, you know what is all this shit?
22:34.37
rob
Oh, forget my French.
22:36.60
Jon
Okay, it’s an adult show.
22:37.04
rob
um
22:37.96
Jon
We’re good. Yeah.
22:39.39
rob
on
22:39.56
Jon
Okay.
22:40.35
rob
I basically decided I would not use the letters DEI or say the words diversity, equity or inclusion because you don’t need them in order to in order to profess and and and talk about the value that different perspectives bring in terms of a conversation.
23:03.28
rob
So, and and I would even use my Latino-ness to, you know, I would meet and I’d be briefed by the the Census Bureau staff on data collections, on strategy, on this and that.
23:16.59
rob
And I would provide my input. i wouldn’t I wouldn’t say you need to do X. I’d say, here’s something to think about.
23:23.55
Jon
Mm-hmm. Mm-hmm.
23:23.62
rob
And I would offer a different perspective and show them that there is value in bringing other different people together, either be it internally within the Census Bureau, bringing different directorate folks from like the population estimates folks together with the privacy folks together with the ah race and ethnicity folks together with the economic folks, et cetera, to just talk about these different types of of issues that were being faced with in terms of data collection or accuracy or new products or whatever. um and And I would just, I would promote that.
24:00.22
rob
And then I’d show by example on how you could look at something, say a data send ah census bureau product and say, here’s a different way of looking at that.
24:12.75
rob
And the example that I would often give
24:12.94
Jon
No.
24:15.96
rob
is the 2018 census test. I don’t know if you remember that. There was a huge, several hundred thousand maybe, national survey that tested out the census form in two versions.
24:20.78
Jon
no
24:33.27
rob
Half the sample got the census form with the citizenship question in it and half just got the regular form without the Census Bureau. And so they went out and they did this humongous survey.
24:47.81
rob
They put the you know did the results together. They did this. It was incredibly powerful significance test to see if there was a difference between the the responses in the forms and the participation rates with the sentencing leadership question versus without it.
25:08.13
rob
And guess what happened? No difference.
25:13.79
Jon
Okay.
25:14.74
rob
No difference. Now, because they’re the Census Bureau and they’re, you know, the scientific integrity and transparency, they basically said, here’s the test. Here’s what we did. Here’s the methodology.
25:27.52
rob
There’s no statistical significance. End of story. I mean, that’s just only there’s only the the results, man, type of thing.
25:34.97
Jon
Right.
25:35.89
rob
um And the media, I don’t know if you remember, the media grabbed it and ran and said, look, there never was a problem. What do you know? What’s the issue here?
25:46.50
rob
Why are we debating this?
25:46.72
Jon
Right.
25:48.77
rob
And i at the Urban Institute, immediately wrote a blog that said, hey, time out. And I explained this to the Census Bureau.
25:54.77
Jon
Yeah.
25:57.98
rob
the the leadership there about a year or two ago, I said, look, here’s a situation where there’s no statistical significance.
26:08.73
rob
That doesn’t mean there isn’t wasn’t an impact.
26:11.75
Jon
Right.
26:11.69
rob
I said, my theory and what I believe happened is that after a year and a half of all of this intense media pressure saying, look, citizenship question’s coming, immigrant you know ah families are gonna be affected, they’re not gonna wanna participate. You pastors telling their congregations, don’t fill out the census form if it comes.
26:35.67
rob
um That impact was existed and was done. at the time that the Census Bureau did their experiment.
26:46.15
Jon
Right.
26:46.34
rob
And so the impact was distributed out to both sides of the, you know, the folks with the foreign households with the foreign houses without the form.
26:57.13
rob
So naturally, when you when you do that, who’s left?
26:57.24
Jon
Right.
27:01.07
rob
Only the people for whom it doesn’t matter.
27:04.20
Jon
Yes.
27:04.20
rob
And so that’s why that was there was no…
27:06.23
Jon
Right. No difference. Right.
27:07.76
rob
If you wanted to really see a difference, you should have done a survey prior to any of this media attention coming out.
27:12.06
Jon
Yeah.
27:15.14
rob
and then do one afterwards and do it like a difference in differences, a split, both cases.
27:19.32
Jon
Mm-hmm.
27:21.60
rob
And then you probably would have seen something. So I said, there’s it just did not surprise me at all that there was no difference because that’s what I would have expected.
27:27.88
Jon
Mm.
27:31.61
rob
And interestingly, after the leadership heard that, especially some of the branch chiefs at the lower level, Some of the researchers got together and they went back to some administrative data that they had on all of the sample and did an analysis. And I think it was a a Dr. Brown at Census and a Misty Hedgenus.
27:53.89
rob
And they determined that, in fact, there, you know, There were differences that were not able to be discerned through the census test alone.
28:06.63
Jon
and
28:07.22
rob
and And whatever came up supported, they actually told me it supported my vision of what I doing.
28:13.64
Jon
Oh, wow. Yeah.
28:15.50
rob
That was kind of validating. but
28:17.02
Jon
Yeah.
28:17.86
rob
But it it’s ah it’s a way of showing how you need diverse perspectives, like the perspective that I had put out. Because otherwise, if you’re self-contained in your little scientific bubble, you think only a certain way.
28:32.08
Jon
yeah
28:33.94
rob
Now, I took that to the next level because I knew that no census director had ever gone out and engage with the public. And we needed that after the 2020 experience in order to help restore trust among different types of communities and me being the first Latino director and such.
28:53.23
rob
And so I went everywhere that I possibly could. ended up in 30 with like 600 different types of engagements and at cities and in rural areas and tribal lands, et cetera.
29:07.23
rob
And i what I would do is I wouldn’t go and say, oh, you need to trust us. and and
29:13.67
Jon
Yeah.
29:13.81
rob
Instead, i started with the value proposition of the data, saying, we have your data. You need to benefit from it. And that was the starting point of ah basically coming to provide and then saying, we have data dissemination specialists that are here and I bring them with me.
29:31.90
rob
These guys will tell you what data you need for your community, for your problems.
29:32.13
Jon
Mm-hmm.
29:36.81
rob
And they’ll do seminars and they’ll do this and they’ll do that. And it was incredibly valuable to do that because what I would also tell the public is that, look, 2020 happened, you know, we it was COVID, but we managed to get through the 2020 decennial census.
29:57.98
rob
But please understand, we could not have achieved the level of, you know, of quality in that census without you down at the level.
30:09.26
Jon
Mm-hmm. Mm-hmm.
30:10.64
rob
And I told the Census Bureau staff, We cannot achieve the level of excellence that we need to to get the quality of data we need to without public participation, which is why engagement is so critical.
30:26.10
rob
And that’s what spend my time doing. And that kind of, I don’t know if you can see it from based on what I’ve said, but this idea of diverse perspectives, giving people, having their voice heard, not only internally, but externally with community engagement,
30:42.81
rob
that really leads to better data uh more relevant data higher quality data more accurate data
30:49.32
Jon
Yeah, absolutely. um I want to ask you one more question on the communication side, which is a bit more niche, but that’s what the that’s what this podcast is about. I’m curious about your thoughts on communicating statistical uncertainty.
31:02.75
Jon
um You go to a census report and you see there’s a table of the estimates and then there’s the 95% confidence interval and you go to see this product or that product and there’s standard errors and the variances. And I’m just curious,
31:15.54
Jon
you know do you you know how do you think about communicating you know that kind of statistical information that kind of statistical language to you know a huge country or a world that doesn’t always have that language and that and that those those tools
31:16.29
rob
Mm-hmm.
31:30.45
rob
um I did it sort of in a circumventing type of fashion. So here’s here’s how I did it.
31:36.60
Jon
hmm
31:39.03
rob
Because everyone likes to focus on the decennial census, I used that as an example. And I said, i would tell, you know you know, tribal leaders and elected officials, you know governors who didn’t like what I say, community community organizations, Congress,
31:53.65
Jon
hmm
31:59.32
rob
I would say, look, there has never, ever been decennial census that has been perfect. It’s impossible.
32:09.90
Jon
right
32:10.02
rob
and And then I’d go further and say, there’s never been a census of a nation in the history of mankind that has been perfect.
32:18.11
Jon
right
32:19.11
rob
That it’s just impossible to to achieve that.
32:19.34
Jon
yeah
32:22.13
rob
What you need them to be is good enough for the purpose that are that it is intended. And I said, for for the U.S. decennial census, we need sufficiently accurate numbers for 50 states for the two to enable to fu fulfill the constitutional mandate for reapportionment.
32:49.61
Jon
Yeah.
32:50.29
rob
I said, I am absolutely comfortable that the that based on what I’ve seen in terms of data quality and results and such, that the the apportionment for 2020 was a fair apportionment. it The level of quality achieved was acceptable.
33:10.72
rob
Having said that, there were undercounts, right? In fact, severe undercounts.
33:14.13
Jon
yeah
33:15.84
rob
And I said, I would tell them, you know, think about it. You need 15 numbers of people, population for the apportionment. When you start looking at different subgroups and little geographies and things of that sort, what’s happening is that the level of accuracy that that was achieved for those 15 numbers was the net, it was a net accuracy from balancing two sources of error, overcounts and undercounts.
33:45.89
Jon
Yeah.
33:47.46
rob
And when you put overcounts and undercounts together, so the people that we counted twice ended up and you know making it for the people that we missed and things of that sort.
33:47.80
Jon
Right. Yeah.
33:58.17
rob
So I think that that helped them understand that we live in a world of errors. I’d also say, look, We you know people don’t need perfect accuracy in order to make decisions on data. They just need decently good data for the purpose intended. I said, you know, hey, we’re all at a conference.
34:19.87
rob
You know, we want to go out to a restaurant. What do we do? Oh, we go over to one of the social media things. Oh, I can say the media companies now. You can go to Yelp.
34:29.25
Jon
ah
34:29.99
rob
and And you take ah take a look, you know, here the restaurants. and And so you know, everyone knows that you take a look at those reviews
34:39.42
Jon
Yeah.
34:39.47
rob
And at one extreme, a chunk of those reviews are from the competition, giving really bad reviews so that they could.
34:46.54
Jon
Yeah.
34:47.53
rob
And another chunk is from the employees or the staff or the owners who they want to make sure you go there.
34:52.30
Jon
yeah
34:53.67
rob
And then in the middle, there’s, you know, people just feel compelled. And so you look and you want to see a really large number. with the recognition that the most spectacular reviews are from, you know, the folks that that have a vested interest and the really bad ones are from the folks that don’t.
35:11.56
rob
And you make decisions on dirty data.
35:14.11
Jon
Mm-hmm.
35:15.52
rob
And we live with that all the time. We, you know, you know,
35:18.42
Jon
Yeah.
35:19.64
rob
So we shouldn’t be afraid of the uncertainty that are associated with official statistics. They’re inherent. They’re part of it. If you want to do a deep dive into it, you can, you know, take some statistical mumbo jumbo classes type of thing and say, oh, there’s a margin of error, plus or minus.
35:37.77
rob
But the bottom line is just understand that nothing is perfect and that it’s still good enough for you to be able to gain some insight and knowledge by looking at them.
35:42.62
Jon
Hmm.
35:49.40
Jon
Yeah. Yeah, that’s terrific. um I have one more question for you before we go. um We’ve done a lot of sort of looking back, um and I’d like to ask you to look forward, um and and we’ll try to go above the politics, as it were.
36:03.32
Jon
But what do you think the federal statistical agencies, the system, needs to do or should be thinking about to sort of modernize for the, like like you said, there’s always error.
36:05.03
rob
Thank you.
36:16.31
Jon
but But where should short modernization or reforms happen you know for the twenty first century?
36:23.04
rob
Well, they’re already happening. and I’m, you know, I wrote an article in the Journal of Official Statistics that talked about the challenges facing the Bureau of the Census, but they they really apply to all of federal statistical agencies.
36:32.04
Jon
yeah
36:41.92
rob
And i like two of I liken the current situation where some agencies are being gutted There is now, I believe in the in the fiscal year 26 budget, if it ever gets passed, there will there will be motion together of the BLS, the Bureau of Economic, BEA, Bureau of Economic Analysis, and the Census Bureau.
36:59.55
Jon
Yeah.
37:12.32
Jon
Right.
37:12.83
rob
So they’ll be combined and into one agency. uh that’s going to have some drastic impacts so so i likened what was going on to sort of a natural disaster hurricane they’re going through and it’s you know in it’s Florida, there was a huge hurricane, Mexico Beach or whatever.
37:27.42
Jon
Hmm.
37:37.31
rob
It destroyed like 80 or 90% of all the homes because they had been built in the 1960s.
37:42.56
Jon
me
37:43.27
rob
And there were a few that were standing because they had been built to F category four standards. Well, there’s going to be a rebuilding. And guess what? Folks aren’t going to rebuild back to the 1960 standards. They’re going to rebuild to the new standards.
37:57.28
Jon
Right.
37:58.00
rob
Okay, right now we’re being gutted as far as federal statistical agencies. It’s going to pass. This administration will pass. It will move on and there will be new administrations.
38:11.58
rob
There will be a new opportunity to rebuild the federal agencies. They should not be rebuilt in the way they were before. There should be a vision for should we do more combining? You know, there’s BLS, BEA, and census.
38:27.50
rob
What about agencies?
38:27.70
Jon
Mm-hmm.
38:28.90
rob
NCES no longer exists. Should that type of function be there? What about other agencies? How can, you know, should there be um a migration towards more of the Stats Canada model of a one statistical office that brings things together?
38:45.83
rob
Those types of thoughts and discussions need to be going on right now in anticipation of the building rebuilding process that’s going to come on the other side of the current natural disaster that’s going on.
38:57.89
rob
Part of that will be not just how to what does the infrastructure look like, but what are the data and what are the data products look like. there There’s no question that people are more and more reluctant to participate surveys, which brings in this notion of blending data, blending both the survey data that are collected and the administrative data that already exist, and know so that you don’t have to burden people.
39:17.42
Jon
Mm-hmm.
39:31.78
rob
Now, it’s not as simple as, oh, we do a survey and we do administrative data and let’s put the two together.
39:37.42
Jon
yeah
39:37.76
rob
it’s it needs to be It will need to be more strategic in a future modernization where you understand you basically harvest the easy data that you can, which is from administrative data.
39:51.84
rob
You see what the gaps that are missing because you know there are segments of the population that do not want. And it’s not just undocumented immigrants. It’s
40:00.46
Jon
right
40:01.07
rob
ah It’s folks in rural areas like, leave me alone and don’t tax me.
40:04.82
Jon
Mm-hmm. Great. right
40:06.68
rob
There places like that in in different urban areas and such. So there will be gaps and that’s where then you send people out to collect raw data that you otherwise would not be able to have to fill in the gaps that the administrative data have or to help inform and build better estimates.
40:26.72
rob
So I see a lot of future blending of data in order to get some of the classical survey data that congrats Congress mandates be collected.
40:38.92
rob
But the the real exciting piece for me that was ongoing at the Census Bureau, and I think in other federal statistical agencies too, but I saw it directly, is this being able to take advantage of blending together in a an enterprise-level data lake, all the different sources of data, all the different frames, like the address frame, the business frame, etc., the the geographic frame.
40:57.41
Jon
Mm-hmm.
41:03.21
Jon
Yeah.
41:06.55
rob
to to create new data products like the Community Resilience Estimates product the my community or or the or the Census Business Builder.
41:13.06
Jon
Mm-hmm.
41:17.42
rob
These are really, really exciting products that make it very easy for people with all different who are you know if you’re interested in your neighborhood or if you’re interested in your city or your county or your rural area it makes it really easy for people to come in grab data that’s both economic and public ah health and household data demographic data and be able to create insights and make assessments of like you know what are you know
41:48.42
rob
what it is it worthwhile building a school here or putting a factory here or or whatever.
41:51.74
Jon
Yeah.
41:56.42
rob
I think that that’s the future and I’m really, really excited about that.
41:56.74
Jon
Right.
42:01.23
Jon
Okay. I love ending on the optimistic note. So we’ll see what will happen. I know there’s a lot of efforts going on around the country and we’ll hopefully be able to be a part of them. um
42:11.10
rob
Cool.
42:11.55
Jon
Rob, great to see you. Thanks for coming on the show. i Always good to talk.
42:13.76
rob
and Yeah, it’s always good to talk and I just had a great time. Thank you so much and I look forward to seeing you again.
42:21.68
Jon
Awesome.
