September 14, 2003
The Unscientific Nature of Poll Reporting
I’ve said before that one of the biggest areas in which media bias can manifest itself unintentionally is in poll report. Polling is a science involving various sampling techniques, question strategies, and methods of contacting respondents. As such, most all polls can tell us something about how the electorate will vote in upcoming elections. However, when reporting poll findings, journalists (especially TV journalists) often fail to keep results in their proper scientific context, thereby gleaning the wrong message from the data, usually a more favorable one. My argument is best made by looking at several recent polls and what you aren’t hearing from journalists.
The first is a poll showing that only 38% of Americans think Bush is beatable. Polls like this tell us absolutely nothing about how Americans will vote. They only serve to gauge a general public opinion and become important only in the context of who it is that thinks he is beatable and the affect their opinion will have on voter turnout. Without this supplemental data it’s a pretty irrelevant stat. In other words, it should mean something to campaign strategists but very little to election watchers. Yet, Foxnews and CNN have both cited the poll without giving it any sort of context.
As I was going to bed last night, I heard a similar poll being reported as news on one of the local stations. The poll found that two years after 9/11, +70% of the public believes there will be another terror attack. Again, this poll doesn’t tell us all that much about an actual threat, but only about the level of fear. To the untrained ear, this one might even cause fear; “If 70% percent of the country is afraid, maybe I should be as well”.
Another poll getting a lot of underserved press is the Time/CNN survey asking: “If George W. Bush runs for reelection in 2004, would you say you will definitely vote for him, might vote for or against him, or will you definitely vote against him?” To which 41% answered "definitely against", while only 29% responded "definitely for". On the surface this poll appears to show some serious slippage in Bush's support. But since only hardcore partisans generally answer "definitely" before the exact candidates are known, we can that this poll is really only reflecting a fairly normal party alignment. Since FDR, Dems have had a major advantage in the number of people identifying themselves with their party. At one point there were five Democrats to every three Republicans. While this Democratic hegemony has begun to crack over the last several decades they still have a decisive advantage.
There is also something a little vague about the way the question is asked. Notice the options are definitely for, maybe, and definitely against. "Maybe" is a very broad category hiding tons of nuance. For instance someone could be tentatively for Bush, but unwilling to say definitely, which is to say never.These people might be classified as leaning toward Bush. There is also sure to be a number of those maybes leaning to a Democrat. But suppose for argument’s sake that of the 25% maybes, 20% are Bush leaners will only 5% are Democrat leaners. Well the election results look much different. If the leaners vote the way they lean Bush could easily best his opponent by 4% or 5%. So the definitely for or against data only tells us the level of base loyalty that either side has.
Incidentally, other unreported polls contradict this poll’s finding. In a Ipsos-Reid/Cook Political Report Poll (via Polling Report.com) that asks a near identical question, 38% responded that they would definitely vote for Bush, while only 36% responded they'd "definitely vote for someone else". The discrepancy between the two polls highlights an equally important question about these sorts of Bush vs. Someone else polls. What they can do is expose an incumbent's weakness. What they can't do is predict election outcomes. When someone responds to such polls they very possibly imagine their ideal candidate, one who satisfies all their ideological criteria. Yet, elections are not between one real and one ideal candidate. They are between two very real and thereby flawed candidates. what's telling is that in the very same Time/CNN poll discussed above where only 29% of respondents declared they would "definitely" vote for Bush, head-to-head matchups were decidedly for Bush: Bush gets 50% to Kerry's 45%, 50% to Lieberman's 44%, 52% to Dean's 42% (how's that for a "frontrunner"), and 53% against Gephardt's 42%.
A final example. Last week a CBS News Poll reportedly found that only 35% of registered voters could actually name a candidate running for the Democratic nomination. The poll’s finding was greeted by shock and awe from journalists and pundits. The student of politics, however, is not the least bit surprised. Why? Because it is generally well known that only about 25% of registered voters participate in primaries. Considering this fact, 35% is not actually that bad, especially not this early in the game.
Polls are very important tools for politicians and strategists who look at them in a whole nexus of polls and historical trends. However, they are unfortunately very popular tools for journalists (specifically TV). They make an easy story, much easier than actually covering the substance of a campaign. Often called “horse race journalism”, modern media coverage is generally about who is winning and not necessarily about what anyone is saying. Polls have to be reported within the context of general political analysis by experts, not flashed on CNN Headline News with little to no support information. Polls have affects on the way voters turn out and vote and must be treated with responsibility and intelligence.
Posted by Mike Van Winkle at September 14, 2003 03:58 PM | TrackBack (1)Your title “The Unscientific Nature of Poll Reporting” vague and misleading when compared to the actual content of your post.
You’re dead on when saying that polls are often misrepresented by the mainstream media. You point out some great examples of how journalists can mislead the public with polls.
However, your title would be better served by saying that polling rarely presented by mainstream media in black and white without a twist. The way it sounds, to me, is that the science of polling is flawed, which is incorrect. Polling, a fancy name for statistical sampling, is on solid ground in that aspect.
Other than that the article brings up good points about how polling is reported by the media.
Posted by: joestump at September 14, 2003 04:41 PMSo to sum things up a biased polls are anything that shows that not everyone like Bush? What gets me is that Neo-cons use the myth of the “liberal media” anything polls and opinions are not in favor of “Our Great President”.
All I have to say is this …. Where did all the landslide posts go? Maybe I need to start reading democrats watchblog for them.
Posted by: Glenn Brown at September 15, 2003 02:45 PMListen I could of made similar arguements about the pathetic poll reporting on democratic presidential candidates. The fact that national polls are site when only state polls really matter….and so on. The fact that you gleened a partisan message from a fairly non-partisan article disappoints me and makes me wish I’d never posted it.
get over yourselves
Good topic.
I blogged recently about polls in a post entitled: When Polls Go Bad.
A poll showed 71% of Russians fear America’s next target for war will be their country and 36% of Russians have a favorable view of the United States.
That means 7% are waiting for the bombs to fall on them, but like us anyway. Go figure.
Posted by: CJ at September 16, 2003 12:55 AMBut still Please…. Could we have another landslide post? I am really looking forward to another landslide post.
Posted by: Glenn Brown at September 16, 2003 10:31 AMEverything is biased so media bias is a reality. There is no such thing as an unbiased person. Just read Jakes site “mental ground zero” for an example of pure opinion. If fox news is biased to the right then CNN ABC NBC and CBS can be called biased to the left. Why do liberals see no bias in the (mainstream)media (it is)while constantly claiming that Fox news is biased (it is)? I don’t know? It is beyond me.
Posted by: Pete at September 16, 2003 03:41 PMYes it is Peter when I talk about politics it is pure opinion. My opinion. Just like Rush, Oreilly ,The Wall Street Journal, New York Post, NewsMax, Drudge Report, Ann Coulter, Scarborough and Paula Zahn.
It is not news it is editorial.
Polls are not God but they do provide a good snap shot of the greater trends.
Posted by: Jake of 8bitjoystick.com at September 16, 2003 05:20 PMPete- I would disagree, and place Fox FAR right of center, CBS a little less biased but still right of center, NBC and CNN a little left of center, and ABC a little further left(but still not half as far as Fox is on the other end).
Agreed, everyone is biased, but i think most of the prominent media outlets that you note recognize their bias and try pretty hard to not let it leak into the coverage too much (with the exception of Fox). So in the end, most media hovers near the center.
But thats just my (biased) opinion..
Posted by: nobody at September 17, 2003 01:37 PMHAHAHAHAA!
OK….
I am actually a little speachless here and this is exactly what I am talking about. While the left likes to talk about the “corporate media” out of one side of the mouth, they then claim that Fox is the biased news agency. Priceless if you ask me.
Of course you would think that.
Keep it up and keep losing seats in the house and senate….and try to float loser candidates for the executive office. We are loving this….well at least I am.
Hey! I have an idea! Lets try to start a liberal talk radio network! Yeah, that will be awesome!
Too bad mainstream media is already flooded with liberals.
Therein lies the reason that Rush Limbaugh exists….you people just don’t get it. We are in control of ALL three houses of government because of your inability to see the truth.
Oh wait! I forgot that the rest of the world is crazy….and you are the sane ones….
Paranoid schizo’s think that too but that doesn’t win elections, as has been shown recently.
Pete- Majority does not equal a politically centerist position as you seem to be implying.
Regardless of how many listeners Rush has, he is still extreme right, and extremely offensive. The reason he has so many listeners is because he doesn’t really dissect politics or ask tough questions, he just spews simple formulated opinions with a nice big confident radio voice to convince people that he must be right. People seem to like people with strong unwaivering opinions regardless of whether they are right or wrong. Liberal news sources tend to focus more on the content and less on the delivery, which is less effective for the majority of americans.
I’m sure if liberals/democrats want to stay in the game they will have to play by GOP rules to an extent and build some overly aggressive opinion casters (we can see this is already working with Dean’s “Anti-Bush” campaign), partisan think-tanks, lobby-control technicians, billioniares willing to funnel big bucks into smear campaigns, etc, etc… It’s a sad state that we will have to resort to such techniques, but GOP set the new hardball standards, and they seem to be working..
I can’t believe someone just suggested that CBS News… headed by Dan Rather… is a little right of center. Is this the introduction to the opposite sketches?
Posted by: CJ at September 19, 2003 12:46 AMRegarding CBS, here are some findings from a study recently conducted by the Center for Media and Public Affairs (the results were made public a few weeks ago). The study looked at evening news coverage of the war, and how many positive/negative comments about the war were made during the coverage.
Examining 1,131 stories on the CBS, NBC and ABC evening newscasts, as well as Fox’s “Special Report With Brit Hume,” the study concluded:
“CBS Evening News” — 74 percent positive on-air comments about the conduct of the war. (On CBS, Dan Rather said: “Facts on the ground indicate that overall, from a military standpoint, the invasion continues to go well.”)
Fox’s “Special Report” — 60 percent positive. [“Special Report” aired the least combat footage and 47 percent fewer images of civilian casualties.]
“NBC Nightly News” — 53 percent positive.
ABC’s “World News Tonight” “was the most antiwar,” — with 34 percent of the on-air comments rated positive.
War coverage by CNN and MSNBC was not included for budgetary reasons, noted the report.
Uh…yeah, I am believing this (not)
Look, JUST TURN ON THE TV!
Count how many positive war stories there are and tell me that the media isn’t reporting the whole story. My friend Matt Adams just returned from there and could NOT believe his eyes at what the media is reporting. It is ridiculous. Someone find for me, a mainstream press story that talks about the progress made in this war. I have only been able to find one story so far that reports positive news.
Even Fox is reporting the doom and gloom daily. I do not need to research it…i turn on the T.V. every single morning and the story is…
“One more soldier killed in Iraq blah blah…ambush blah blah”
When was the last time that a war happened and EVERY SINGLE TIME someone was killed it was reported.
Just imagine this happening in WW2…
Instead of war correspondents saying things like
“and today the U.S. dealt another decisive blow to the nazi war machine” etc…
You would get…
“General Patton, some critics are suggesting that this war is becoming a QUAGMIRE and that we are getting BOGGED DOWN….etc.
“I do not need to research it…i turn on the T.V. every single morning and the story is…
“One more soldier killed in Iraq blah blah…ambush blah blah”
You’re right this is much more accurate source for evaluating media than an organized study.
By the way, the media covers soldier dying because 1)thats whats happening, and 2)news always covers death/crime/etc because it traditionally gets higher ratings. Be that as it may, the US media has been seriously downplaying US casualties regardless of what you think. I recently read a letter (don’t have time to hunt for it now) by a woman who lost a son in Iraq and was seriously pissed that her son was shot in guerrialla type confrontation and was near-death in the hospital (he might have died, I don’t remember) and press release to US Media cited it as a non-combat minor injury.. There are plenty of instances like this if you compare daily US press releases on injuries and casualties with those of any and every international source..
Posted by: nobody at September 19, 2003 12:06 PM“There are plenty of instances like this if you compare daily US press releases on injuries and casualties with those of any and every international source..”
Oh you mean like the BBC (what a joke) or al jazeera (another joke).
All we get is negative news on Iraq and so if there is some type of conspiracy to cover up casualties as you suggest then why don’t you find the truth and print it here but be sure to confirm it because i don’t believe in kooky leftist conspiracy theorists.
Of course it will never happen because there is no conspiracy. Just more tripe from leftist KOOKS with no way to back up thier “information” but wierdo political agenda driven websites. I will give you the email address of a young kid that i know who just returned from Baghdad…he will tell you what is really happening over there.
So, If I read you right- All journalism at home and abroad - is a joke. And if we want real information, we should rely on you and your friends who we can email. Great. Just checking.
Nope, I am only saying that you should spend a little more time talking to people who have been there and a little less time reading internet links and articles by the vehemently anti-American BBC stooges and NPR.
So…do you wan’t to talk to my friend and hear the truth or would you rather read articles by politically motivated rags?
It is an open opportunity to see things from a different perspective.
It is a challenge as well.
Perhaps you would like to personally email me? I would find it to be a welcome change. As long as you don’t make racist comments to me or call me an idiotic grunt as in this statement by an editor from this site (who didn’t want to say this in public for obvious reasons)when we got into a fight about the war in Iraq and the living conditions there, which he felt were inadequate…
“It was not my desire to sleep in foxholes, eat cold food, watch children go hungry, crawl in the mud, or get shot at; I figured I would leave that to the stupid people among us. Thank God there are plenty of you to go around Pete. I choose a higher calling, namely, military intelligence.”
Heh….isn’t that funny?
As long as you do not say anything like that, I will have no problem giving you some personal opinions from the men on the ground.
Give me a buzz if you wish. I am more than willing to hook you up with some guys that will tell you the truth.
