Polling is fundamentally broken and must be fixed if Americans want to live in a healthy, functional democracy: That’s the premise of Georgetown professor Michael Bailey’s new book, Polling at a Crossroads.
“Polls are important. We simply cannot understand politics, society and the economy without them,” writes Bailey, the Colonel William J. Walsh Professor of American Government. “Getting polls right is not easy, however. Extrapolating from the responses of the increasingly rare and potentially atypical people who respond to modern polls is daunting, making polling vastly more difficult than when random sampling was viable.”
“To say that getting polling right is difficult is not to say that it is impossible. While some may believe that polls will never be able to correct for biases caused by unmeasured factors, this book shows that new theories, data and methods can substantially improve our ability to account for this kind of bias as well.”
In an election year, the impact of polling on politics, the news media and public discourse is ever-present, according to Bailey.
Polling at a Crossroads, the new book from Professor Michael Bailey.
“In many ways, polling shapes our reality at any given moment,” said Bailey, who holds appointments in the College of Arts & Sciences and the McCourt School of Public Policy. “We care about who is ahead and who is behind — and why. Campaigns and media change behavior based on polls, and our ability as social scientists to understand politics and society depends on polls.”
Polling decides how campaigns allocate resources, the markets in which they spend money on ad buys and which states candidates visit to get out the vote. When a presidential election comes down to a handful of battleground states, and in some cases just tens of thousands of votes in a country with more than 258 million voting-age adults, the accuracy of polls can have extreme impacts on the outcome of an election.
“In 2020, if 40,000 more voters in three states voted for Donald Trump, he would have shocked the world again,” writes Bailey. “The shortcomings of the 2020 polls were particularly unnerving because pollsters tried to learn from the mistakes of 2016, only to produce the least accurate polls in 40 years.”
Once the election is over, polling shapes how voters view its outcome, the legitimacy of the process and the subsequent favorability of the electee. In other words, at every moment that polls try to measure our political process, they are also shaping it.
Traditional polling is dead because the vast majority of Americans do not want to be polled. In most polls, functionally all of the people called refused to participate. The small, remaining minority of people who respond to polls are, as Bailey puts it, “atypical.”
“A few decades ago, pretty much everyone answered the phone when it rang,” said Bailey. “We didn’t have caller ID so we defaulted to answering the phone. Now that everyone has cell phones, it is honestly pretty weird to answer a phone call from an unknown source.”
To illustrate this, Bailey brings up a recent New York Times poll of six battleground states. The poll, which reported the sentiments of 4,000 respondents in some of the most contentious states in the upcoming election, is one of the larger polls conducted this year. To get those 4,000 respondents, however, the pollsters called a whopping 400,000 people — a 1 percent response rate. The poll’s margin of error, 1.5 percent in either direction, assumes a 100 percent response rate.
“Random sampling theory has traditionally anchored survey work, justifying how characteristics of relatively small groups of people will converge to the characteristics of a population as long as the small group is a random sample of the large one,” writes Bailey. “But random sampling is, for all intents and purposes, dead.”
The New York Times poll is not an anomaly, according to Bailey, but the norm for pollsters everywhere. While polls can correct their data so it’s more representative, scaling certain responses up or down based on demographic data, there’s no existing technical solution to account for nonresponse.
To correct for nonresponse, Bailey suggests finding ways to describe just what makes responders atypical. The group of voters more likely to respond differs from election to election. In 2020, Biden voters were more likely to respond to polls than Trump voters, but that may not be the case in 2024.
Bailey suggests one path forward may be conducting parallel polls with the same set of questions but different levels of incentive, enticing those who may not want to respond with monetary compensation and other rewards. By comparing the responses of those less likely to respond and those more likely to respond, pollsters might better develop weights and methodologies to account for nonresponse.
“Polling has reached a critical moment,” writes Bailey. “Polls have failed us in high profile elections and, at best, have a weak basis in the random sampling theory that historically justified polling. The goal of my book is to help us understand this moment and plot a course for where to go next.”
At the start of the spring semester, students in the Presidential Nominations Lab met with major political players, voters and journalists and watched their coursework play out in real time.
Read Full Story
Sen. Jon Ossoff (SFS’09), D-Georgia, delivered the government department’s annual Bernstein Lectures.
Read Full Story