Professor Pollkatz on Polling Firm Bias

A few weeks ago I ran across a website that attempted to measure the "bias" of pollsters, and then I posted an article about it. Well, something must have been in the air that week, because shortly after I found that website, which measured polling bias during Bush's tenure, Professor Pollkatz went ahead and measured polling bias during Clinton's second tenure. He (at least think Pollkatz is a he) doesn't post the actual results, but his write-up is interesting: I, like many others, have long suspected that some pollsters may have their thumb on the scale, favoring Bush, in their polls. This would be foolish, of course, because the real bread and butter for the pollsters is in market research, where accuracy is more important than politics. Even so, that was my suspicion.

I was mistaken. I finally rolled up my sleeves and tested the hypothesis directly; not only were none of the results significant, none of them even suggested the swings I expected.

Pollingreport.com posts several "approval" polls from Bill Clinton's second term. I used this data to construct a "Clinton Index," similar to my Bush Index. To my wonder and amazement, I discovered that the rankings of the individual pollsters were not much different: Fox and Zogby at the extremes, everybody else clustered around one or two points. Most important, none of the ten pollsters in the database exhibited a significant, or even conspicuous, swing from Clinton low to Bush high (or vice versa).

So, except for Fox and Zobgy, polling firms did not appear to lean either for or against Clinton. Then again, Zogby was a lot closer to the final result on Election Day in 2000 than most other firms. I am going to once again seriously think about removing all Fox polls from my calculations.

Pollkatz has lots of other stuff on his site that would cause any political junie to salivate. Check it out.

Tags: Media (all tags)

Comments

3 Comments

How much deviation from average for Fox?
Is it consistent?

If it is, you could consider adjusting Fox by that factor as an alternative to dropping them.

by JimPortlandOR 2004-07-12 12:25PM | 0 recs
Re: Why stop there?
The formatting was screwy with the last two sentences, so I cut them.
by Chris Bowers 2004-07-12 12:26PM | 0 recs
Re: Why stop there?
To my wonder and amazement, I discovered that the rankings of the individual pollsters were not much different: Fox and Zogby at the extremes, everybody else clustered around one or two points. Fox and Zogby were the furthest away from the mean--everyone one else was within two points. He notes that neither Fox nor Zogby appear to be dishonest, just outliers. I wouldn't be omitting Fox or Zogby because they are biased, but because they have been outliers for almost a decade. That's a little long to still be cutting them slack.
by Chris Bowers 2004-07-12 12:29PM | 0 recs

Diaries

Advertise Blogads