why distrust data?
48% of Trump supporters "completely distrust the economic data reported by the federal government" including unemployment, spending, jobs. https://t.co/5l9GhucBFI
— Justin Wolfers (@JustinWolfers) October 15, 2016
That tweet and the linked article got my attention (no trust of data by 25% of adults!) ... Still why reflect on this? ... so much else to get stuck on these days. First, I use official statistics in my work A LOT; second, I am always on the look out for new survey insights; and finally, I am a bit obsessed lately with models in which people are not acting on the same information. This level of distrust is troubling ... even though I doubt it's new or entirely about the data ... I want us to think about WHY.
I study consumer behavior as an economist, which in 2016 still means reading lots of research with dynamic optimization and Euler equations. This is a typical early morning ritual for me, that quiet time before my kids wake up when I can still imagine a world in which we know everything about everything, including ourselves, and we choose calmly and appropriately. BUT I balance out my openness to such models with a determination to also understand what people ACTUALLY do and think.
Nevertheless, I am picky about the survey insights that I absorb, pass on, and try to understand. My cognate in grad school was survey methodology and I still write survey questions in my research ... thus I understand how much responses can be manipulated, or even carelessly biased by poor methods and human nature. Also I want to know what people think, not what someone writing up the survey results wants me as a reader to think. (I'm not a fan of the tweet, by the way.) So I googled and found the survey's homepage, a Marketplace-Edison Research poll designed to measure economic anxiety. And, I found a description of the methods AND the full survey too (see page 30 for this question). It's not the micro data online, so I can't replicate the statistic in the tweet, but I could see that the "data trust" question was asked before voting intentions or political affiliation. I have learned from pollsters that asking about politics conjures up an identity that can be hard to shake in the rest of the survey. The main roadblock I see in interpreting the data distrust is this survey's short time series; it only began last fall as a quarterly survey. My hunch is that distrust of economic data is nothing new but I can't prove that here. Plus changes in attitudes are often more informative than a snapshot, since subjective questions are tricky to interpret. What does it mean to "trust data" anyway? Do you trust data?
To be clear, I am not justifying anyone's views, but I am also trying not to be judgmental. A key principle of surveying is not to make people feel bad or shameful about their views. Because, guess what, if you do, they are less likely to tell you what they think or did ... then you are fighting blind and may miss the chance to learn why we sometimes see the world differently. I am not in the 25% of adults who have "no trust at all" in economic statistics from the government. In fact, I am in a rare set of adults who spends more time on the Bureau of Economic Analysis' website sorting through spending data than on Amazon adding to it. So what's up with all this distrust? I have a few hypotheses to take to the data.
Hypothesis 1: government economic data don't match people's life
Sometimes I think the Representative Agent is a frenemy of economists. (Oh, not the Twitter persona, he's great, but the concept.) How can a simplifying assumption ... a focus on the typical or aggregate household ... be an enemy in disguise? Well, sometimes it gives theoretical models the focus they need and other times, especially in empirical work, it glosses over important details. Details, also known as people. So maybe distrust of economic data comes from not seeing your life experience in the numbers that roll across the screen. National aggregates get a lot of attention, so maybe it is minorities that end of distrusting data more, data that doesn't tell their story as loudly.
Not so, at least in terms of data about the economy, minorities are more trusting than whites. Only 15 percent of African-American have no trust at all in economic data almost half the fraction of whites. And among Hispanics, only12 percent have complete distrust of data. With whites comprising over 70 percent of all adults, they are well represented in both aggregate statistics and the distrust of them. Of course, this is just one cut of the data and not seeing your life experience in the data may raise other issues (more below). Government agencies have made a push to improve regional statistics and even make neighborhood data more readily accessible and help improve local decision making. And of course, lots of household level surveys exist too. Another reason to take distrust (or even disinterest) in government economic data seriously is that the quality of the data we have depends on people's participation in our surveys. Response rates on numerous surveys have been falling and research suggests that non response could impact official statistics, making them a less accurate reflection of life experiences.
Hypothesis 2: distrust stems from people being "hurt" by data
On the first Friday of the every month, my Twitter feed is overflowing with chatter about the latest employment report from the Bureau of Labor Statistics. That makes me weird. I firmly believe that few people absorb the government statistics in the way that I and my fellow econos do. Why should they? People confront economic data when it affects them. One example I can think of is the cost-of-living adjustment, such as for Social Security benefits. That came to mind when I looked at data distrust by age.
no cost-of-living adjustment to benefits had led some seniors to "distrust" government data, like the CPI-W? Again, this hypothesis would be a lot better to test with a time series of data, comparing years with benefit increases and without. But feeling shortchanged by the data may be understandable given wide variation relative price changes, few of us exactly consume the representative basket. Alternatively, as risk aversion appears to rise with age, maybe so too does distrust? I wrote earlier that age is more than just a number, the impact of demographic change deserves more study.
Hypothesis 3: it's not the data, it is the way we use them ... the spin
I don't trust data, I trust people. And even then, trust but verify, right? Perfectly measured data (dream, dream), can still be suspect. In fact, data can codify a lot of the biases and mistakes we have made together in the past. Maybe we should also be concerned for the people who "completely trust" government economic data? (Do read Cathy O'Neil's book on Big Data and algorithms.) Yet, I suspect the distrust in the survey is not about data construction (I've never seen a protest at the ever-interesting BEA advisory committee meetings) ... or even about the government employees who construct the statistics in excruciating detail, and in line with international standards. I bet the distrust is more about how the numbers are interpreted and how they are used in policy making. Drawing conclusions from data is hard and reasonable disagreement is to be expected. As just one example, the seasonally-adjusted unemployment rate for African Americans was 8.3 in September, which is below its average of 10.8 percent over the past 20 years but is almost double the 4.4 percent unemployment rate of whites. Should we call that 8.3 (or 4.4, for that matter) victory or 'full employment'? And is the unemployment rate even the right statistic to assess? Relative to the past it may well be but the past can be an imperfect guide for the future. Every data point has its shortcoming, especially where there is no clear counterfactual or agreed upon target. My "moderate" growth could easily be your "weaker-than-expected" growth. And, of course, on top of honest disagreements about data, plenty of motivated reasoning is done with numbers. BUT when we start with the same data, there are at least some bounds on the disagreement. In contrast, when government data are wholesale rejected one quarter of adults, it's no surprise that we aren't living in the same world. And we stop trying to understand each other. I would be lost (and bored out of my mind) in my work on consumer behavior without data. You don't want me extrapolating from my tiny circle of experience ... and frankly no one should make decisions with that little information. We can learn a lot from the data, including these attitudinal surveys. And data adds accountability, including in how its collected. Even so, no one likes to feel manipulated or, worse, written off, especially with numbers.
Data can't solve problems but maybe it holds clues to a path forward ... to rebuild trust.
**Opinions here are mine and should not to be attributed to anyone with whom I work.**