2016 Dealt a Blow to Polling. Did 2020 Kill It?
Less than 24 hours after the first ballot counts began to pour in on election night, the media and political classes had already declared a loser: polling.
“Devastating for my industry” was how one Republican pollster described the election results.
“Polling seems to be irrevocably broken, or at least our understanding of how seriously to take it is,” wrote Margaret Sullivan, a media columnist at The Washington Post.
“民调似乎无可挽回地崩溃了，至少我们对于民调严肃性的认知是崩溃了，”《华盛顿邮报》(Washington Post)的媒体专栏作家玛格丽特·沙利文(Margaret Sullivan)写道。
Politico Playbook went even further: “The polling industry is a wreck, and should be blown up.”
Are they right? Here’s what people are saying.
‘A profession-wide crisis’
In the wake of the 2016 election, many pollsters resisted the narrative that their profession had taken a nosedive: National polling averages projected Hillary Clinton would win the popular vote by just under four percentage points, and she ultimately won by just over two. According to FiveThirtyEight’s founder Nate Silver, that’s about as accurate as polls have been on average since 1968.
2016年大选后，许多民调专家否认他们的职业一头走向下坡路的说法：全国民调平均值预测，希拉里·克林顿(Hillary Clinton)将以略低于四个百分点的优势赢得普选，而她最终以略高于两个百分点的优势在普选中获胜。根据FiveThirtyEight博客的创始人内特·席尔瓦(Nate Silver)的说法，这和1968年以来民调的平均准确率差不多。
But American elections are decided by swing states in the Electoral College, not the popular vote, and it was at the state level that polling data missed the mark. There were several proposed reasons for this, including a failure to correct for the overrepresentation of college graduates, which pollsters hoped they had fixed this time around.
That turned out not to be the case. As Dave Wasserman, an editor at the Cook Political Report, put it, polls, especially at the district level, “have rarely led us more astray & it’s going to take a long time to unpack.”
事实证明并非如此。正如《库克政治报告》(Cook Political Report)主编戴夫·沃瑟曼(Dave Wasserman)指出的，民意调查，尤其是在地区层面上的民意调查，“让我们史无前例地误入歧途，这需要很长的时间来解读。”
• In the House, many analysts predicted Democrats would keep their majority or even expand it by five to 15 seats. By Thursday, Democrats were on track to keep control but lose six seats.
• In the Senate, Lindsey Graham of South Carolina won his race, which was widely seen as competitive, by some 13 points. Susan Collins of Maine also held on to her seat, which not a single poll since July suggested would happen, according to FiveThirtyEight.
• 在参议院，南卡罗来纳州的林赛·格雷厄姆(Lindsey Graham)以约13个百分点的优势赢得了被普遍认为竞争激烈的竞选。缅因州的苏珊·柯林斯(Susan Collins)也保住了自己的席位，根据FiveThirtyEight的调查，自7月份以来，没有任何一项民调显示会发生这种情况。
• There were significant errors in the presidential election, too: In Florida, polls put Joe Biden ahead by an average of 2.5 points; as of Thursday’s count, he lost the state by more than three. Polls also put President Trump ahead by less than a point in Ohio, which as of Thursday he won by eight.
• 总统选举中也出现了重大误差：在佛罗里达州，民调显示乔·拜登(Joe Biden)平均领先2.5个百分点；截至周四，他以三个百分点以上的劣势输掉了该州。民调还显示，特朗普总统在俄亥俄州领先不到一个百分点，截至周四，他在该州领先了八个百分点。
“No matter who wins the presidential race, it’s clear that the vast majority of the polling underestimated Trump’s support once again,” writes Nathan L. Gonzales, editor of the Inside Elections newsletter. “For months, we’ve been saying that it would take dozens of pollsters, partisan and nonpartisan, independently making the same methodological mistake for the outcome to be different than what we were projecting. And that is apparently what happened.”
“无论谁赢得总统大选，显然绝大多数民调都再次低估了特朗普的支持率，”《选举内情》(Inside Elections)时事通讯编辑内森·L·冈萨雷斯(Nathan L. Gonzales)写道。“几个月来，我们一直在说，只有几十个有党派立场的民调机构和无党派的民调机构分别同时犯同样的方法错误，才能得出与我们的预测不同的结果。现在发生的事情显然就是这样。”
A failure, or a misunderstanding?
Many pollsters believe that the general frustration with their profession owes less to a failure in polling itself than to a general misapprehension of its purpose, as well as its prominence in political discourse. As Courtney Kennedy, the director of survey research at Pew Research Center, has noted, polls can be quite useful for finding out what the entire citizenry thinks about policy issues. Even in the context of the presidential election, the national polling average may end up being off by no more than 3 to 4 percent — a mediocre margin of error, but not a terrible one by industry standards.
许多民调专家认为，人们对他们的职业普遍感到失望，与其说是因为民调本身的失败，不如说是因为人们普遍误解了民调的目的，及其它在政治话语中的重要性。正如皮尤研究中心(Pew research Center)调查研究主任考特尼·肯尼迪(Courtney Kennedy)指出的那样，民意调查对于了解国民整体对政策问题的看法非常有用。即使是在总统选举的背景下，全国民调的平均误差最终可能也不会超过3%到4%——这是一个中等程度的误差幅度，但以行业标准来看不算太糟。
The problem, Zeynep Tufekci writes in The Times, is that producers and consumers of political media have come to think of polling not as a potentially blurry reflection of public opinion, but as a means of predicting the future. Presidents are not elected by a national vote in the United States, so election modelers — which are different from pollsters — like FiveThirtyEight attempt to “forecast” elections based on measurements like polling data and certain fundamental assumptions about how the political climate works (e.g., “a better economy favors incumbents”).
There’s a case to be made that this crucial distinction between polls, which are a snapshot in time, and models, which deal in probabilities, is simply one that the public doesn’t understand or that the media has done a poor job communicating.
“The apparent results of the presidential election fall within the ‘likely range’ of outcomes projected by polls and forecasts,” writes David Byler at The Washington Post. “Those ranges exist for a reason: For all that some readers treat these mathematical analyses as dispatches from the future, that’s an impossible expectation.”
“总统选举的明显结果在民意调查和预测结果的‘可能区间’之内，”《华盛顿邮报》(The Washington Post)的戴维·拜勒(David Byler)写道。“这些区间的存在是有原因的：尽管有些读者将这些数学分析视为对未来的预测，但是这样的期望是不可能的。”
But Dr. Tufekci argues the problem runs deeper than a failure of communication. In weather forecasting, both the fundamental assumptions (the science of atmospheric dynamics) and the measurements (years of detailed data from a vast number of observation stations) are highly sophisticated. But in politics, “we simply do not have anything near that kind of knowledge or data,” she writes. “While we have some theories on what influences voters, we have no fine-grained understanding of why people vote the way they do, and what polling data we have is relatively sparse.”
The upshot, as Dr. Kennedy has written, is that national polls are much better for relaying public opinion on issues and candidates writ large than for predicting the victor in the Electoral College.
Does polling have a future?
Questions about the use of modeling aside, it remains to be seen why so many pollsters independently made the same mistake of underestimating Mr. Trump’s support nationally. “Our projections in the 2018 midterm elections, using the same methods of analysis, were accurate,” Mr. Gonzales writes. “A key question moving forward is whether public opinion polling is irreparably broken or if polling is just broken in elections with Trump on the ballot.”
One theory that gained purchase after 2016 was the so-called shy Trump voter problem: “With Trump, what’s really unique was that people were reluctant to say they supported him primarily because … people call them racist, xenophobic,” Chris Kofinis, a Democratic pollster, told the Canadian Broadcasting Corporation. “You know, think of the negative that has been applied to Trump over his last four years. People didn’t want to be associated with that.”
2016年之后，有一种理论被广泛接受，那就是所谓心怀顾忌的特朗普选民：“对特朗普来说，真正特别的是，人们不愿意说出自己支持他，主要是因为……支持者会被说成种族主义者和仇外者，”民主党民调专家克里斯·科菲尼斯(Chris Kofinis)在接受加拿大广播公司(Canadian Broadcasting Corporation)采访时表示。“想想过去四年里用在特朗普身上的负面词语。人们不想和这些扯上关系。”
But some believe the source of the error goes beyond the president. “The crisis we seem to be dealing with — and I’m not sure we entirely understand it yet tbh — is not a Shy Trump voter problem,” Derek Thompson of The Atlantic tweeted. “It’s an Invisible Republican problem. Up and down the ballot, from POTUS vote to district level, pollsters are whiffing on the G.O.P. share of the vote.”
Why? One explanation is that for years now, the rate of response to telephone surveys has been plummeting. (The issue hasn’t been solved by the rise of online polls, which tend to lean to the left of telephone surveys and are generally, though certainly not always, less reliable.) There’s reason to believe that voters without a college education and with lower levels of social trust, who were more likely to lean Republican, are especially loath to answer the phone, which poses a sampling problem that weighting has not been able to solve.
If conventional surveys are indeed becoming a less accurate instrument for gauging the views of an important subset of voters, regardless of who’s running for president, the polling industry will face existential questions about whether and how it can recalibrate.
“But the answer almost doesn’t matter, unless you’re a professional pollster,” David Graham argues in The Atlantic, “because after two huge presidential flops, pollsters have lost the confidence of the press and public.”