The Unscientific Nature of Poll Reporting

I’ve said before that one of the biggest areas in which media bias can manifest itself unintentionally is in poll report. Polling is a science involving various sampling techniques, question strategies, and methods of contacting respondents. As such, most all polls can tell us something about how the electorate will vote in upcoming elections. However, when reporting poll findings, journalists (especially TV journalists) often fail to keep results in their proper scientific context, thereby gleaning the wrong message from the data, usually a more favorable one. My argument is best made by looking at several recent polls and what you aren’t hearing from journalists.

The first is a poll showing that only 38% of Americans think Bush is beatable. Polls like this tell us absolutely nothing about how Americans will vote. They only serve to gauge a general public opinion and become important only in the context of who it is that thinks he is beatable and the affect their opinion will have on voter turnout. Without this supplemental data it’s a pretty irrelevant stat. In other words, it should mean something to campaign strategists but very little to election watchers. Yet, Foxnews and CNN have both cited the poll without giving it any sort of context.

As I was going to bed last night, I heard a similar poll being reported as news on one of the local stations. The poll found that two years after 9/11, +70% of the public believes there will be another terror attack. Again, this poll doesn’t tell us all that much about an actual threat, but only about the level of fear. To the untrained ear, this one might even cause fear; “If 70% percent of the country is afraid, maybe I should be as well”.

Another poll getting a lot of underserved press is the Time/CNN survey asking: “If George W. Bush runs for reelection in 2004, would you say you will definitely vote for him, might vote for or against him, or will you definitely vote against him?” To which 41% answered "definitely against", while only 29% responded "definitely for". On the surface this poll appears to show some serious slippage in Bush's support. But since only hardcore partisans generally answer "definitely" before the exact candidates are known, we can that this poll is really only reflecting a fairly normal party alignment. Since FDR, Dems have had a major advantage in the number of people identifying themselves with their party. At one point there were five Democrats to every three Republicans. While this Democratic hegemony has begun to crack over the last several decades they still have a decisive advantage.

There is also something a little vague about the way the question is asked. Notice the options are definitely for, maybe, and definitely against. "Maybe" is a very broad category hiding tons of nuance. For instance someone could be tentatively for Bush, but unwilling to say definitely, which is to say never.These people might be classified as leaning toward Bush. There is also sure to be a number of those maybes leaning to a Democrat. But suppose for argument’s sake that of the 25% maybes, 20% are Bush leaners will only 5% are Democrat leaners. Well the election results look much different. If the leaners vote the way they lean Bush could easily best his opponent by 4% or 5%. So the definitely for or against data only tells us the level of base loyalty that either side has.

Incidentally, other unreported polls contradict this poll’s finding. In a Ipsos-Reid/Cook Political Report Poll (via Polling that asks a near identical question, 38% responded that they would definitely vote for Bush, while only 36% responded they'd "definitely vote for someone else". The discrepancy between the two polls highlights an equally important question about these sorts of Bush vs. Someone else polls. What they can do is expose an incumbent's weakness. What they can't do is predict election outcomes. When someone responds to such polls they very possibly imagine their ideal candidate, one who satisfies all their ideological criteria. Yet, elections are not between one real and one ideal candidate. They are between two very real and thereby flawed candidates. what's telling is that in the very same Time/CNN poll discussed above where only 29% of respondents declared they would "definitely" vote for Bush, head-to-head matchups were decidedly for Bush: Bush gets 50% to Kerry's 45%, 50% to Lieberman's 44%, 52% to Dean's 42% (how's that for a "frontrunner"), and 53% against Gephardt's 42%.

A final example. Last week a CBS News Poll reportedly found that only 35% of registered voters could actually name a candidate running for the Democratic nomination. The poll’s finding was greeted by shock and awe from journalists and pundits. The student of politics, however, is not the least bit surprised. Why? Because it is generally well known that only about 25% of registered voters participate in primaries. Considering this fact, 35% is not actually that bad, especially not this early in the game.

Polls are very important tools for politicians and strategists who look at them in a whole nexus of polls and historical trends. However, they are unfortunately very popular tools for journalists (specifically TV). They make an easy story, much easier than actually covering the substance of a campaign. Often called “horse race journalism”, modern media coverage is generally about who is winning and not necessarily about what anyone is saying. Polls have to be reported within the context of general political analysis by experts, not flashed on CNN Headline News with little to no support information. Polls have affects on the way voters turn out and vote and must be treated with responsibility and intelligence.

Posted by Mike Van Winkle at September 14, 2003 3:58 PM | TrackBack (1)