why distrust data?

That tweet and the linked article got my attention (no trust of data by 25% of adults!) ... Still why reflect on this? ... so much else to get stuck on these days. First, I use official statistics in my work A LOT; second, I am always on the look out for new survey insights; and finally, I am a bit obsessed lately with models in which people are not acting on the same information. This level of distrust is troubling ... even though I doubt it's new or entirely about the data ... I want us to think about WHY.

I study consumer behavior as an economist, which in 2016 still means reading lots of research with dynamic optimization and Euler equations. This is a typical early morning ritual for me, that quiet time before my kids wake up when I can still imagine a world in which we know everything about everything, including ourselves, and we choose calmly and appropriately. BUT I balance out my openness to such models with a determination to also understand what people ACTUALLY do and think.

Nevertheless, I am picky about the survey insights that I absorb, pass on, and try to understand. My cognate in grad school was survey methodology and I still write survey questions in my research ... thus I understand how much responses can be manipulated, or even carelessly biased by poor methods and human nature. Also I want to know what people think, not what someone writing up the survey results wants me as a reader to think. (I'm not a fan of the tweet, by the way.) So I googled and found the survey's homepage, a Marketplace-Edison Research poll designed to measure economic anxiety. And, I found a description of the methods AND the full survey too (see page 30 for this question). It's not the micro data online, so I can't replicate the statistic in the tweet, but I could see that the "data trust" question was asked before voting intentions or political affiliation. I have learned from pollsters that asking about politics conjures up an identity that can be hard to shake in the rest of the survey. The main roadblock I see in interpreting the data distrust is this survey's short time series; it only began last fall as a quarterly survey. My hunch is that distrust of economic data is nothing new but I can't prove that here. Plus changes in attitudes are often more informative than a snapshot, since subjective questions are tricky to interpret. What does it mean to "trust data" anyway? Do you trust data?

To be clear, I am not justifying anyone's views, but I am also trying not to be judgmental. A key principle of surveying is not to make people feel bad or shameful about their views. Because, guess what, if you do, they are less likely to tell you what they think or did ... then you are fighting blind and may miss the chance to learn why we sometimes see the world differently. I am not in the 25% of adults who have "no trust at all" in economic statistics from the government. In fact, I am in a rare set of adults who spends more time on the Bureau of Economic Analysis' website sorting through spending data than on Amazon adding to it. So what's up with all this distrust? I have a few hypotheses to take to the data.

Hypothesis 1: government economic data don't match people's life

Sometimes I think the Representative Agent is a frenemy of economists. (Oh, not the Twitter persona, he's great, but the concept.) How can a simplifying assumption ... a focus on the typical or aggregate household ... be an enemy in disguise? Well, sometimes it gives theoretical models the focus they need and other times, especially in empirical work, it glosses over important details. Details, also known as people. So maybe distrust of economic data comes from not seeing your life experience in the numbers that roll across the screen. National aggregates get a lot of attention, so maybe it is minorities that end of distrusting data more, data that doesn't tell their story as loudly.

Not so, at least in terms of data about the economy, minorities are more trusting than whites. Only 15 percent of African-American have no trust at all in economic data almost half the fraction of whites. And among Hispanics, only12 percent have complete distrust of data. With whites comprising over 70 percent of all adults, they are well represented in both aggregate statistics and the distrust of them. Of course, this is just one cut of the data and not seeing your life experience in the data may raise other issues (more below). Government agencies have made a push to improve regional statistics and even make neighborhood data more readily accessible and help improve local decision making. And of course, lots of household level surveys exist too. Another reason to take distrust (or even disinterest) in government economic data seriously is that the quality of the data we have depends on people's participation in our surveys. Response rates on numerous surveys have been falling and research suggests that non response could impact official statistics, making them a less accurate reflection of life experiences.

Hypothesis 2: distrust stems from people being "hurt" by data

On the first Friday of the every month, my Twitter feed is overflowing with chatter about the latest employment report from the Bureau of Labor Statistics. That makes me weird. I firmly believe that few people absorb the government statistics in the way that I and my fellow econos do. Why should they? People confront economic data when it affects them. One example I can think of is the cost-of-living adjustment, such as for Social Security benefits. That came to mind when I looked at data distrust by age.

Older people are much more likely to distrust the data ... the exact opposite of learning over time that statistics are reliable. But maybe the fact that in three years since 2010, including this past year, there was no cost-of-living adjustment to benefits had led some seniors to "distrust" government data, like the CPI-W? Again, this hypothesis would be a lot better to test with a time series of data, comparing years with benefit increases and without. But feeling shortchanged by the data may be understandable given wide variation relative price changes, few of us exactly consume the representative basket. Alternatively, as risk aversion appears to rise with age, maybe so too does distrust? I wrote earlier that age is more than just a number, the impact of demographic change deserves more study.

Hypothesis 3: it's not the data, it is the way we use them ... the spin

I don't trust data, I trust people. And even then, trust but verify, right? Perfectly measured data (dream, dream), can still be suspect. In fact, data can codify a lot of the biases and mistakes we have made together in the past. Maybe we should also be concerned for the people who "completely trust" government economic data? (Do read Cathy O'Neil's book on Big Data and algorithms.) Yet, I suspect the distrust in the survey is not about data construction (I've never seen a protest at the ever-interesting BEA advisory committee meetings) ... or even about the government employees who construct the statistics in excruciating detail, and in line with international standards. I bet the distrust is more about how the numbers are interpreted and how they are used in policy making. Drawing conclusions from data is hard and reasonable disagreement is to be expected. As just one example, the seasonally-adjusted unemployment rate for African Americans was 8.3 in September, which is below its average of 10.8 percent over the past 20 years but is almost double the 4.4 percent unemployment rate of whites. Should we call that 8.3 (or 4.4, for that matter) victory or 'full employment'? And is the unemployment rate even the right statistic to assess? Relative to the past it may well be but the past can be an imperfect guide for the future. Every data point has its shortcoming, especially where there is no clear counterfactual or agreed upon target. My "moderate" growth could easily be your "weaker-than-expected" growth. And, of course, on top of honest disagreements about data, plenty of motivated reasoning is done with numbers. BUT when we start with the same data, there are at least some bounds on the disagreement. In contrast, when government data are wholesale rejected one quarter of adults, it's no surprise that we aren't living in the same world. And we stop trying to understand each other. I would be lost (and bored out of my mind) in my work on consumer behavior without data. You don't want me extrapolating from my tiny circle of experience ... and frankly no one should make decisions with that little information. We can learn a lot from the data, including these attitudinal surveys. And data adds accountability, including in how its collected. Even so, no one likes to feel manipulated or, worse, written off, especially with numbers.

Data can't solve problems but maybe it holds clues to a path forward ... to rebuild trust.

**Opinions here are mine and should not to be attributed to anyone with whom I work.**


economist - my views here are my own

Also on this blog

SHARE:  Email · Facebook · Google · Twitter · Tumblr · Kindle
SUBSCRIBE:  Receive an email on new posts from claudiasahm


Is it just not done to ask people why they distrust Government figures ?
2016-10-17, Stuart Gibson

The same thing happened here in Italy with Silvio Berlusconi. He got a lot of reforms but a lot of people ignored facts.
2016-10-17, pietro

No one 'trusts' data. We all have confidence intervals.
This combined with your point number 3 is the main issue I suspect.
Point number 1 is also in play, I think point 2 is essentially irrelevant, it might be true for some data, but not for data.
As far as economics goes, people intuitively understand that economics attempts to push the envelope and use data to draw conclusions that are not really addressable with the data. Economists don't even have agreement on how data is used - thinking mostly of macro. I see no reason to puzzle on this until you can get economists to all agree. I don't mean this as a challenge, just a description of the situation.
2016-10-17, Dan

The headline unemployment number is obviously false, and this affects confidence in the other numbers.
There is no particular mystery about what is going on.
2016-10-17, Dave Chapman

Because your aggregated statistics does not reflect the experience of the individuals:
"But several underlying factors also appear to have contributed to the closeness of the race. For starters, many Americans are economically worse off than they were a quarter-century ago. The median income of full-time male employees is lower than it was 42 years ago, and it is increasingly difficult for those with limited education to get a full-time job that pays decent wages.
Indeed, real (inflation-adjusted) wages at the bottom of the income distribution are roughly where they were 60 years ago. So it is no surprise that Trump finds a large, receptive audience when he says the state of the economy is rotten. But Trump is wrong both about the diagnosis and the prescription. The US economy as a whole has done well for the last six decades: GDP has increased nearly six-fold. But the fruits of that growth have gone to a relatively few at the top – people like Trump, owing partly to massive tax cuts that he would extend and deepen. "
2016-10-17, PSteele

It is not just economic data...more glaringly crime! No amount of crime data will make people belief that it is not the least safe time ever to be alive.
2016-10-20, Gman

I think it has a lot to do with distrust of government. If you asked the same question but about private sector data (or words to that effect), you would see a higher level of trust. But since there is such a knee-jerk anti-government strain in America, *government* data is by definition wrong or misleading or intentionally distorted or influenced by politics or evil. This is testable, of course. Maybe someone has that data. Question is, can you trust it? :-)
2016-10-20, S Branca

Have you ever seen John Williams' site," Shadow Government Statistics"? (http://www.shadowstats.com/). He shows how the criteria for measuring many government indicators (the unemployment rate for example) have been changed over the years so that, for example, 6% unemployment as reported today means something very different from 6% unemployment as it would have been calculated and reported, say, in 1960. In other words, over the years the way of calculating and presenting the data has been massaged to make things look better than they actually are. He explains all this using the government's own facts and figures, not arguing from any indeological standpoint. Given this, why indeed should people just "trust the data"? Please take a look!
2016-10-24, Statsman

I bet this puzzle would be a lot easier to solve if you sort the trust question responses by political party (including leaners). When Democrats hold the White House, they tend to trust what comes out of the Executive Branch while Republicans distrust it. The pattern reverses when Republicans hold the White House. As an example, look at evaluations of the economy by party ID for the 2000-2015 period. What you see is that Republicans consistently saw the economy as being in better shape than did Democrats UNTIL 2008. What you may be observing is simply a function of existing political circumstances.
2016-10-24, Martin Kobren

155 Million people working
81.8% full time workers
43 year low in unemployment claims
Real wages excluding transfer payments all time highs
Real median weekly earnings for full time workers all time highs
over 103 Trillion dollars of financial assets
U.S dollar is the reserve currency
Inflation and gas prices are low
Job openings had the highest print ever at 5,900,000
:-) If facts can't promote economic ideological views ... then you're taught not to trust it
2016-10-24, Logan Mohtashami

Hypothosis 4:
Most Americans receive government data via the media, vice directly from government (my assumption).
The media is roundly distrusted (as heard/seen by me, as reported by the media, i.e. annecdotal).
Ergo, Americans' distrust of government data stems from its associate with its typical messanger.
This is admittedly just a tweek of Hypothosis 3, but maybe its the media's spin on the data, not the government's spin, that causes the distrust? Or a combination?
2016-10-24, PJ Cummings

Can this question be answered with a simple answer?
No one individual can ever trust something they don't understand,
And, when something is not understandable then the natural reaction is to rely on perceived experts...
Then, it becomes an interesting exercise to determine who is considered <your> expert... Trump? Clinton? (both who have no discernible expertise) - Obama? Yourself, Ms Sahm? Maybe the neighborhood pastor? A Hero?
And, throwing around numbers won't help anything, the average American has a very low capacity and tolerance for numbers...
2016-10-24, Tony

PSteele - thank you for the reference to the Stiglitz article, it's quite helpful.
2016-10-24, Statsman

This shows how party ID relates to views on the economy. It's striking.
2016-10-25, Martin Kobren

Of course, there may be problems with your data about distrust levels as well. The most obvious to me is an availability bias ... the government collects lots of data, but most people don't hear (or care) about aggregate tonnage of a particular commodity. The data they do hear about (and care about enough to remember) are precisely the excerpts that are surprising or hurtful.
2016-10-28, Jim J. Jewett

Um, How about the fact the the Official Unemployment Data are obviously false?
This sort of thing tends to discredit all other government data. . .
2016-11-26, Dave D'Rave

  • Notify me upon new comments

☺ Got it