New analysis reveals pitfalls of presidential polling

March 12, 2021
What made the 2020 presidential polls so wildly inaccurate? (AP Photo/Rogelio V. Solis)

What made the 2020 presidential polls so wildly inaccurate? (AP Photo/Rogelio V. Solis)

Most presidential polls conducted in the immediate runup to the 2020 election overestimated support for Democratic nominee Joe Biden and were among the least accurate in the past 25 years, new research found, shedding light on how hard it has become to make accurate electoral forecasts.

According to an article published March 1 in Presidential Studies Quarterly, national preelection polls favored Biden by approximately 1.72 percentage points on average, assuming a tied election — a level of bias that exceeded polls conducted ahead of the 2016 election, in which Republican nominee Donald Trump shocked many pollsters with an unexpected victory over Hillary Clinton.

That pro-Democratic tilt was also reflected in polls projecting the national congressional vote, and in polls gauging presidential and congressional preferences in individual states, author Costas Panagopoulos found, a discovery likely to frustrate researchers and polling groups who had taken steps meant to correct for the previous cycle's mistakes.

"Overall, 2020 was a lackluster cycle for preelection polling in the U.S. general elections," Panagopoulos, a professor at Northeastern University, wrote. "Based on a variety of common metrics, polls across the board were generally less accurate than in prior cycles, and they tended to reflect systematic biases favoring Democratic candidates."

Panagopoulos, an expert on campaigns and elections, focused his analysis on 14 different polls conducted between Oct. 27 and Nov. 3, Election Day, using each polling organization's final available poll in that stretch.

Based on the A metric — a statistical tool that captures a poll's predictive accuracy and indicates which side the poll was biased toward — Panagopoulos found all but two of the polls featured at least a slight pro-Democratic tilt.

The mean A measure for the group of national presidential polls was -0.069, signaling a "fairly sizable Democratic bias," according to the study. 

Comparable presidential polls over the past seven election cycles weren't always biased on the whole toward the Democratic candidate, Panagopoulos noted, citing pro-Republican biases on average in polling from 2004 and 2012. But the 2020 cohort showed the highest average bias since the 1996 contest between incumbent Democrat Bill Clinton, Republican Bob Dole and Reform Party nominee Ross Perot, according to the research.

In the two decades of presidential election cycles from 1996 to 2016, the mean level of A sat at -0.014, indicating a "slight" pro-Democratic tilt over that period. Viewed in that context, Panagopoulos wrote, "The 2020 polls overall … were among the least accurate in the past quarter-century period."

Polls projecting each major party's overall share of the House of Representatives vote nationally were also biased toward the Democratic Party on the whole, according to Panagopoulos' research. Assuming a tied election, the mean value of A for the polls implied that they favored House Democrats by about 2.5 percentage points on average.

That indicates they were "considerably less accurate overall" compared to the 2016 election cycle, Panagopoulos wrote, in which polls favored Democrats by roughly 1.12 percentage points on average.

The study also examined 136 state-level polls gauging preferences for presidential, Senate and gubernatorial candidates. Of that sample, 38 showed statistically significant pro-Democratic biases, while none were found to be significantly tilted toward Republicans.

The mean value of A for the 136 state-level polls was -0.090, a finding Panagopoulos said suggests that "the pattern of pro-Democratic bias detected in the national presidential and congressional generic vote polls also characterizes statewide polls" taken just before Election Day in 2020.

"In fact, assuming all races were perfectly tied, the mean estimate of A would translate into a … Democratic overstatement of 2.25 percentage points on average," he wrote.

It's important not to overlook the fact that many polls still performed well, Panagopoulos said, noting, for example, that nine of the 14 national presidential polls he analyzed didn't show statistically significant biases one way or the other.

He added that many polling organizations have also taken effective steps toward rooting out sources of bias since the 2016 election cycle.

"Pollsters have been relatively successful in identifying biases and their sources in polls and addressing these effectively in their methodologies," he told The Academic Times. "For example, pollsters after 2016 paid more attention to adjusting for education, which appears to have impacted poll results in the 2016 cycle."

Still, Panagopoulos acknowledged, more work needs to be done to understand precisely how and why biases continue to crop up in polling at many different levels. The paper noted several possible explanations, including the difficulties of capturing the opinions of late-deciding voters and predicting which voters will ultimately end up casting ballots.

Polls themselves have a role to play in shaping turnout and perceptions, Panagopoulos said.

"Accuracy in polls matters for many reasons, but perhaps mainly because poll results fuel perceptions that can affect voter behavior, including whether or not voters vote or which candidates they decide to support," he said. "Polls have the capacity to shape participation."

For those reasons, it's important to keep track of how well they predict actual electoral outcomes, Panagopoulos said.

"Accuracy and bias do vary across election cycles, and it is important to track and monitor these disjunctures, to understand and explain them better, and to continually refine polling methodology," he said.

The article "Polls and Elections: Accuracy and Bias in the 2020 U.S. General Election Polls," published March 1 in Presidential Studies Quarterly, was authored by Costas Panagopoulos, Northeastern University.

Saving
We use cookies to improve your experience on our site and to show you relevant advertising.