A lot of things went wrong in 2020.
And presidential polls were no exception.
If Trump does pull it out, it would be the complete and utter destruction
of the polling industry, at least in politics.
Joe Biden was supposed to win the 2020 presidential election by eight
points, according to the polls, which were wrong.
The polls are so tight right now.
An eight-point lead for Biden in Wisconsin
The president trailing Biden by eight points.
Now, Joe Biden leading President Trump by 10 points.
He won by five points.
He was supposed to win Wisconsin by 10 points.
Instead, Biden eked out a victory there with less than one percent of the
vote between him and incumbent President Donald Trump.
The polls were very wrong in Wisconsin.
The polls also had Biden winning Florida and North Carolina.
Yup. The polls were wrong there, too.
Can we trust these polls?
Here's why and what's being done about it.
I'm looking at two key areas that I think were really important
demographically in understanding the difference between the actual
outcomes and what the polls suggested was going to happen.
And that is one the big diverse mix of Latino voters in southern
Florida and in the Rio Grande Valley, in particular in Texas, and then
white voters without a college degree in the Midwest.
Because if we look at the states where we had the biggest problem, that
those were the states, we had the biggest problem, Texas, Florida and the
Midwest. If you look at other polling, well, it was off a little bit in
places like Georgia and Arizona, even Pennsylvania.
Those were much closer and much more within the realm of our threshold
for uncertainty that we would expect in an election.
So I think we have to look at why would these certain demographics be off
in the poll, in the election, in this particular election.
But they weren't often two years ago in the midterms.
That's where we're scratching our heads.
It was a better performance for pollsters that in 2016, what about twenty
eighteen, there wasn't a presidential election then there were midterm
elections instead. The polls didn't do so poorly, not nearly as poorly as
2016 and 2020.
One big reason, there was one name missing from the ballot, Donald Trump.
And when that name gets on the ballot, it has a huge effect on both
polling and what happens on Election Day.
This is one thing that we've seen in our polling that's different for
President Trump than for any other president before.
That is that when we ask voters whether they approve or disapprove of the
job the president is doing for past presidents, been a matter of do you
agree or disagree with his basically his policies, his overall approach to
governance. With Donald Trump, that question has actually become a
reflection of people's own identities, like if you approve or disapprove
of Donald Trump and you say that publicly, that conveys to other people a
lot about who you are as a person intrinsically.
And that's much different than anything we've ever seen before.
In 2016, one theory with a catchy name emerged to explain why polls fared
so poorly, the so-called shy Trump voter, that's a voter who, because of
Trump's unending string of controversies, is shy to admit to pollsters
that they're voting for him in 2020.
That theory has been refined a bit.
It's no longer about being shy.
It comes down to trust.
The pollsters that I talked to this year talked more about kind of this
different phenomenon where it's not really that these voters are too shy
to talk about wanting to vote for Trump.
It's just that they don't want to talk to the pollsters at all.
When they look at their numbers, they notice that in some cases they're
accurately measuring where Biden ended up netting out in these early
results. So like in Wisconsin, for example, the person who leads Marquette
Law School's polls told me that Biden support in Wisconsin was 48 percent
of likely voters were going to vote for him within, you know, a plus or
minus four point four percent margin of error.
And he ended up with a little over forty-nine percent of the vote based on
these early tallies.
So that's not all that far off.
But then of likely Trump voters, that poll found that forty-three percent
of them were going to favor Trump.
And really he pulled out with nearly forty-nine percent as of the early
tally. So even if they are measuring the Biden supporters and Democrats
fairly accurately, they're clearly missing something with the Trump
supporters. The
top-rated polls on 538 all have something in common, they're all called
live-caller polls. That's a mixture between phone calls on mobiles and
landlines where pollsters actually talk to potential voters.
Getting people to talk to pollsters over the phone is proving harder and
harder, even without the trust issues that's theorized to have affected
the 2020 election.
These are the response rates for Pew polls going back to 1997, they show a
steady march downward.
There is not a great deal of standardization.
This variation is something that consumers need to be aware of.
So when I see variation, I'm talking about, of course, the length of the
survey, the amount of questions or the amount of time it takes to complete
the survey, the mode of the survey.
So are we doing face-to-face surveys, landlines, cell phones, robocalling?
Are we doing online polls variation in the times that the surveys are in
the field, which can go from, you know, one to seven days or longer.
So for those of us consuming polls, you know, we have to be aware of this
wide variation and really take it upon ourselves to click the link that
provides more information about the poll.
Take a recent Monmouth poll, for example.
The pollster spoke with two hundred and eighty-nine people on landlines
and 521 people on cell phones.
That's expensive.
This is all forcing polls to go increasingly online.
And there's a whole host of issues with trying to reach potential voters
without using live polling.
You know, a lot of young people don't have landlines.
They're more likely to respond over the Internet or to a text message.
So people who maybe have traditionally been harder to reach by the polls
are maybe going to be incorporated in them more often.
And, you know, I think different types of polls are incorporating these
methods to a varying extent.
So it's not necessarily going to be the same across the board.
Those of us who still use live interviewing for most of our polling, the
vast majority of the numbers, we're dialing our cell phone numbers with
some landline numbers thrown in there because some people still use their
landlines, but not a lot.
The question, though, that that's raised there is not the telephone
numbers, which we have for the vast majority of voters.
So that's not a problem.
Problem is, some people say, is that you're not going to get the shy Trump
voter when you're talking to a live interviewer.
But what we're finding is that folks who use noninteractive polling, such
as IVR, which is that where you punch in the number on the telephone or
online polling or text message, polling had the same problems.
So it wasn't that the shy Trump voter wasn't talking to a live interview.
It was if there is shy Trump voter out there, they simply weren't
responding at all.
So that's one of the things that we have to look at.
But I've tried some other methods as well.
And we actually did some of that using email samples from the voter lists
where we not only matched up phone numbers, but we matched up emails to
some of those voters and conducted it that way.
We found that, you know, the email samples tend to actually be slightly
more democratic than our live telephone sample.
So that raises questions about whether that technology is going to get you
there. Similarly, text message polling, we actually did a test of a text
message polling and found that more Democratic.
We didn't release it, but we just wanted to do the test and we found it
was more Democratic than our live interviewing poll.
So there's not a magic bullet out there in terms of the type of
methodology that you use.
You really have to focus on just simply, do I have the right mix of folks
who are willing to answer this?
This poll? The American Association of Public Opinion Researchers is an
organization representing pollsters and public survey professionals across
the United States. It's in the middle of researching the reasons why
polling might have missed the mark in 2020.
The AAPOR did a similar postmortem after the 2016 election
In 2016, it took several months for the main organization that reviews
these polls to come out with their postmortem about why the polls were off
that year. So it's going to take a while to understand the full impact of
the polls. I think maybe the main lesson for the public and maybe for the
pollsters to consider when they're putting out these results is just, you
know, the polls are only as good as the people who respond to them.
They're only as good as their methods.
So it may be more about having the public reframe how they're thinking
about the polls and, you know, not taking them necessarily as a predictor
until maybe we have a better way of sorting that out.
We're looking at these election polls and what went wrong with the election
polls, but I've been kind of beating this drum for the past four years and
pointing out situations where we ask what seems to be innocuous questions
about life in general, such as do you plan to take a vacation this year
where we're suddenly getting these huge partisan splits on a question that
never had anything to do with partisanship.
And that, I think, is part and parcel of, you know, our deepening
partisan tribalism. But that has gone, you know, into hyperdrive.
And, you know, so we're talking about these elections and what went wrong
in the elections. But there's a lot that I think I as a pollster, have
questions about that we've been measuring for the past few years that we
might have been off by a few points because of this huge partisan divide
where people interpret everything that we ask about through this lens of
how does it reflect on me?
How does it reflect on me as a Trump supporter, as a Trump opponent?
I'm going to answer this question in a way that makes Trump look good or
look bad has been something that we've been experiencing and quite
frankly is a lot more problematic for the polling industry than what
happened in this past election.