According to some polls, presumed Republican Presidential nominee Donald Trump is leading over current President and non-criminally charged presumed Democratic candidate Joe Biden. Well, if the scary polls are making the news, I guess it’s official: it’s election season, baby!
First and foremost, I need to come clean here: I have a real personal bias against polling in the year 2023. Mainly because I am the only person I know who answers calls from an unmarked number, and that’s because I’m an idiot who always thinks there’s an emergency on the other end of the line. (Spoilers, it rarely is. Usually, it’s a telemarketer trying to get money from me.) This is all to say that polling, in my opinion, has not figured out how to appropriately sample likely voters in a way that accurately represents the electorate.
So, before I get to the fear-mongering over the recent New York Times/Siena College poll that has Biden trailing by Trump in key battleground states, I want to share its methodology. Per The New York Times:
The New York Times/Siena College polls of 3,662 registered voters in Arizona, Georgia, Michigan, Nevada, Pennsylvania and Wisconsin were conducted in English and Spanish on cellular and landline telephones from Oct. 22 to Nov. 3, 2023.
[…]
Overall, 94 percent of respondents were reached on a cellular telephone.
First, please note that 3,662 people spread across six states is approximately 600 voters from each state. That’s like a well-attended (but not record-breaking) bar trivia night at your local Chili’s. Think about it. That’s 75 teams of 8 people each. The Taylor Swift-themed 1989 (Taylor’s Version) release dance party I went to the other night had more people than those included in the sample size for Wisconsin.
This is not to downplay the danger of a second Trump term. The thought of that is terrifying and we have to stay vigilant against it. This is merely to point out that, again, anecdotally, I am genuinely the only person I know in my vast network of fellow amateur ghost hunters, Twilight fanatics, and geriatric millennial Swifties who ever answers phone calls from unknown numbers. (This is why the CDC has included me in their polling rotation!) At the very least, polls are missing those three crucial demographics for likely voter data, and I can’t imagine that’s the only cohort they’re missing out on. So with that said, onto the fear-mongering among the three thousand-ish people who picked up the phone! Per Reuters:
Democratic President Joe Biden trails Republican frontrunner Donald Trump in five of the six most important battleground states exactly a year before the U.S. election as Americans express doubts about Biden’s age and dissatisfaction toward his handling of the economy, polls released on Sunday showed.
Additional findings from the New York Times and Siena College Polls released on Monday, however, showed that if Trump were to be convicted in criminal charges against him, some of his support in some swing states would erode by about 6% — “enough, potentially, to decide the election.”
This is about the most brainless thing I’ve ever heard. Who are these people, whose minds are so rotted by “party over country” that if Trump is convicted, they will still vote for him? Have their brains been so washed that they don’t compute one of the numerous (over 100!) charges against him is for stealing and then improperly storing our country’s secrets?!
In true New York Times fashion, they bent over backward to interview people who will give them the conservative voice they believe their paper to be so desperately lacking at all times. Here’s my favorite one, though:
Dakota Jordan, a 26-year-old also from Maricopa County, did not vote in the 2020 election. He said that he would rather not have Mr. Trump in office at all, but that “given the choices,” he would vote for him over Mr. Biden, absent a criminal conviction. “If he was convicted, there’s absolutely no way — I can’t elect a criminal as my leader,” he said.
I’m sorry, but what?! You’re interviewing someone who literally didn’t vote in the last election about who he plans to vote for in this election? How in the world is he considered a likely voter?! Bless that man’s complete and utter lack of shame in talking about the election and his opinion on who he might hypothetically vote for next year, when he didn’t participate in the last one. That feels distinctly American, actually.
Here’s another concerning poll from ABC News, which says Americans overwhelmingly think the country is headed in the wrong direction:
A year before the presidential election, three-quarters of Americans (76%) believe the country is headed in the wrong direction and the leading Democratic and Republican candidates are viewed broadly unfavorably, according to a new ABC News/Ipsos poll. Only 23% of Americans think the country is headed in the right direction.
[…]
Similarly, by a 25-point margin (29% to 4%), Americans would be more likely to vote for the Democratic candidate if someone other than Biden is the party’s nominee — with 55% saying it would make no difference. The margin is somewhat higher, 35 points, among both Democrats and independents.
On the surface, this looks very bad for the Biden camp, doesn’t it?! However, let’s look at that poll’s methodology, shall we? Per ABC News Politics:
This poll is based on a nationally representative probability sample of 949 adults age 18 or older with oversamples among Black and Hispanic respondents.
[…]
The survey was conducted using KnowledgePanel, the largest and most well-established online probability-based panel that is representative of the adult US population.
[…]
KnowledgePanel members receive a per survey incentive,usually the equivalent of $1 (though for some it is $2) in points, that can be redeemed for cash or prizes.
OK, so this poll was less than 1,000 online monetized poll-takers. I’m sorry, but what? How does this mean anything other than approximately 1,000 online monetized poll-takers have some hot takes on the presidential election, which they were paid (poorly) for?! At least one source agrees with me on all of this nonsense. Here’s a write-up from Fortune explaining why calls and online polls are just junk now:
Pew has documented that telephone response rates have fallen below 9% which is not considered close to valid measurement in any social science fields. Online surveying can be more problematic as there there is no national list of email addresses from which people could be sampled. Thus there is no systematic way to collect a traditional probability sample of the general population relying the internet.
Additionally, here’s what the Fortune article had to say about sample size, and why most polls don’t get that right, either:
With the exception of Edelman, the response sample sizes are often far too small with most polls surveying less than 1,000 people–sometimes only a few hundred. Making things worse is the narrow overspecification asking for more than what the data can give. A sub-category with seven respondents gives nothing but noise.
Remember, our polls had sample sizes of approximately 600 per state for the New York Times poll, and less than 1,000 for the ABC News poll. How do these mainstream publications publish these results with their whole chests and feel proud about it?
I feel the need to reiterate once more: this is not meant to downplay the existential threat of a second Trump term. He is right up there with climate change. However, this is meant to throw as much water on the fear-mongering headlines for polls that, let’s be honest, have not accurately predicted federal election trends in quite some time. Remember the promise of a Red tsunami in 2022? That was basically just a trickle from a knocked-over glass of cherry Kool-Aid with no sugar. Sure, the Republicans took the House (and that’s been a mess!) but the Democrats kept the Senate. It could have been much worse, and would have been, if you had believed the polls and the media headlines.
Now, do I think the media will ever stop fear-mongering in headlines over junk science polls? No. They love the traffic too much. However, the times they have a-changed, so I think it’s important for citizens to be literate about polls—especially when they are being touted as definitive political forecasts—and look at the methodologies. When you do that, just ask yourself a simple question for the polls that rely on phone sampling: do you, or people you trust, pick up calls from unknown numbers? I think you’ll have your answer, then, as to how much weight you should give phone sampling.
As for me, my phone is ringing. It could be an emergency; it could be a poll; or it could be that guy from the Arizona Fraternal Order of Police who calls me monthly despite me telling him to stop. I don’t know. I have to go answer to find out.
(featured image: David Dee Delgado / Chip Somodevilla, Getty Images)
