MyDD :: Gallup's Insanity Continues

Gallup's Insanity Continues

by Chris Bowers, Tue Sep 28, 2004 at 02:07:59 PM EST

Okay, so Rasmussen today has Bush up 47.9-46.3 today, using a voter turnout model that is 39-D-36R-25 I/O. IBD / CSM, which does not weight by Party ID, comes out with a poll today that shows Bush and Kerry tied at 44 (link). Further, in the Gallup poll released yesterday, Bush and Kerry's support by party ID was nearly identical to 2000 levels. However, despite all this, Bush leads Kerry by thirteen according to Gallup because, apparently, the Republican convention caused the greatest shift in Party ID since the New Deal, or maybe even the Civil War.

Ruy Teixeira has turned this all into what he calls, The How Can Gallup... Game. Everyone can play. Here are some of his personal entries:

How can Gallup......have Bush up by 13 nationwide, when he's only up by 2 points among Florida RVs?--and according to their own poll!

How can Gallup.....have Bush up by 13 nationwide, when he's only up by 2 points among Nevada RVs?--again, according to their own poll.(...)

How can Gallup.....have Bush up by 13, when he's only leading among independents by 2 (and that was exactly Bush's margin among independents in 2000 when, as you recall, he did not win the popular vote by 13 points)?

Not only that, Bush's current margin among Republicans in the Gallup poll is not too far from his 2000 margin (93-6 now vs. 91-8 then) and Kerry's margin among Democrats is identical in magnitude to Gore's (85-10 now vs. 86-11 then).

Actually, this one is kind of easy. The only way you can produce a 13 point Bush lead with these internals is if you have quite a few more Republicans than Democrats in the sample--my guess is 7-8 points more. If you re-weight their sample to the 2000 exit poll party ID distribution (and I kind of have to do this, just to drive certain pollsters and their acolytes into a frenzy), you wind up with a modest Bush lead of 2 points.

Here's my daily entry into the game:

"How can Gallup so regularly be far off the mark in their final polls of campaigns when compared to other firms and still be considered a credible pollster?"

Tags: General 2008 (all tags)

Permalink :: 16 Comments :: Post a Comment

Galluping away (none / 0)

Just to throw a softball question out here, is it possible that somewhere around 10% of the American electorate have switched their party identification (either party registration or self-ID) to Republican since, say, the GOP convention? What if millions of Democrats tuned into Zell's speech, and that drove them into the waiting arms of the Republican Party? Is that possible?

by leftish on Tue Sep 28, 2004 at 02:52:53 PM EST

No, not likely. (none / 0)

Ruy Teixeira made a great argument. 9/11 shifted voter ID a measly few percentages points for a short time, after which it went back to normal. Is it likely the Repub convention would have a bigger effect on voter ID than 9/11?

by Teaser on Tue Sep 28, 2004 at 03:18:25 PM EST
[ Parent ]

Re: Galluping away (none / 0)

I kind of think that if millions of Democrats had converted because of the RepubCon, somebody on this site would have met some of them. Examples, anybody?
Later,
Alex

Hillary Clinton or Barack Obama for President! Beat McCain!

by Alex on Tue Sep 28, 2004 at 03:19:09 PM EST
[ Parent ]

Re: Galluping away (none / 0)

The "Mystery Pollster", Mark Blumenthal (who, BTW, polls for Democrats), it is not only possible that such a large shift occurs, but it in fact HAS OCCURRED in the last three of the last five elections. He cites a Pew report:

"Pew also reports data from three "panel" studies - from 1988, 1992 and 2000 -- that interviewed respondents from the same random sample twice: once several weeks or months before the general election and again a few weeks after.
· In 2000, 18% of respondents changed their answer between September and November
· In 1992, 26% changed their party identification between June and November
· In 1988, 16% of the sample changed its party identification in just two weeks, shifting from a 5% Democratic edge (35% to 30%) to an even division (33% to 33%). Note the similarity in both time and magnitude to the shifts seen this month."

http://mysterypollster.typepad.com/main/2004/09/my_post_on_weig.html

Given this info, plus the fact that both the new ABC/WaPo and new Pew surveys give Bush a distinct lead, I'm of the opinion that Bush currently has at least a 5 or 6 point lead. I'm afraid that trying to adjust for Party ID is just not an accurate way of looking at it.

IMO, Kerry needs to make sure he really does well in the debates...

by BigModerate on Tue Sep 28, 2004 at 04:40:14 PM EST
[ Parent ]

Re: Galluping away (none / 0)

Well perhaps.

BTW, I do think Bush is leading right now, but I think it is considerably closer than the Gallup poll suggests: averaging all the polls taken since 9/20, including today's Rasmussen, I come out with Bush 48, Kerry 44. Basically the problem with Gallup is that because they are oversampling Republicans, they are putting Bush's horserace number above 50.

Ben P

by Ben P on Tue Sep 28, 2004 at 05:39:20 PM EST
[ Parent ]

Gallup (none / 0)

I think Rove must be holding Gallup's children hostage.

by herodotus on Tue Sep 28, 2004 at 03:05:31 PM EST

Re: your entry (none / 0)

Perhaps the people who consider them credible are using Bush's definition of credibility: as long as Gallup is consistent, it's credible, regardless of whether it's consistent message ("Bush has a double-digit lead") bears any relationship to reality.
Later,
Alex

Hillary Clinton or Barack Obama for President! Beat McCain!

by Alex on Tue Sep 28, 2004 at 03:16:25 PM EST

gallup numbers (none / 0)

I did a Monte Carlo spreadsheet to figure out the likeliest true numbers in the Gallup poll. The bottom line: If we use Rasmussen's 36%R-39%D-25%I weights, I get Kerry 50.1%, Bush 46.0%.

Quick explanation:
Someone, myDD or Kos, said Gallup sampled 328 Gops, 236 Dems and 189 Inds. If so, then the percentages are 43.55%R, 31.34%D, and 25%I, resp., and Gallup's data must be consistent with:

Bush% = 52% = 43.55*(%Gops for Bush) + 31.34*(%Dems for Bush) + 25*(%Inds for Bush)

Kerry% = 44% = 43.55*(%Gops for Kerry) + 31.34*(%Dems for Kerry) + 25*(%Inds for Kerry)

I generated random numbers (on Excel) to plug in for Gops for Bush, etc., selecting those trials that gave me a Bush% of 51.5% - 52.5% (52% is the result of rounding) and Kerry% 43.5% - 44.5%, and then averaged same to determine Gops for Bush, etc. Several samples of 20,000 trials gave results similar to one decimal point.

My estimates:

Gops for Bush 87.1%
Dems for Bush 8.5%
Inds for Bush 45.6%

Gops for Kerry 8.9%
Dems for Kerry 89.2%
Inds for Kerry 48.4%

Plugging these estimates into Rasmussen's weights yields Kerry 50.1%, Bush 46.0%. Not statistically significant, but a far cry from the 52/44 spread announced by Gallup.

by drlimerick on Tue Sep 28, 2004 at 03:22:45 PM EST

In Tab Bias (none / 0)

What I don't get is that they have a 12% difference between dems and gopers for respondents. Are we to believe that they are that good at weighing responses that they can overcome such a disparity. I mean, if you get dems that are pro bush in a sample that does not accurately reflect the real world, well that bias would be amplified in weighing, right. And given this 12% difference one would have to assume that, given that dems still out number gopers, this is a distinct possibility.

by Loganpoppy on Tue Sep 28, 2004 at 03:55:07 PM EST

Preaching to the choir again... (none / 0)

Selectively reporting the facts doesn't do your side any good, Chris. You gleefully point out the 45-45 outlier poll as somehow correct while skewering the Gallup outlier as incorrect, but you manage to ignore the 7 or so other polls (www.realclearpolitics.com) released since the weekend that have Bush up by an average of 6-8 points. But I'm glad that you're doing this, because the more that the liberals convince themselves that Kerry is actually ahead and Bush is actually behind, the better it is for Bush to preserve his solid lead until Nov 2.

by merzbow on Tue Sep 28, 2004 at 04:42:20 PM EST

Re: Preaching to the choir again... (none / 0)

We'll see . . . glad to see your overconfident.

Also, we have already forced CNN to dedicate a segment of Inside Politics to "Gallup bias" and used it as an effort to try to refute our charges. Score one for the liberal blogosphere. USA Today is apparently got a story in the pipeline to the same effect. Score us another one.

As for the horserace, if one is to go to pollingreport, take all the polls taken since 9/20 and average them together (and throw in today's Ras poll), one gets a result of Bush 48, Kerry 44. I guess you could call that a solid Bush lead. However, an incumbent under 50 should concern you just a touch. Not to say Bush is doomed or anything - indeed if I were forced to say who would win the election today, I would say Bush. But the election isn't being held today and the polling industry is in an awful lot of flux, which you would know if you were out phonebanking like me and the other Dems here are. 25% response rate these days. Thus the inconsistency between the polls, as the various agencies are forced to use mathematical formulas to somehow extrapolate from their raw data. Anyway, we shall see . . .

by Ben P on Tue Sep 28, 2004 at 05:45:46 PM EST
[ Parent ]

Re: Preaching to the choir again... (none / 0)

More on why party ID does change significantly over short periods of time:

http://people-press.org/commentary/display.php3?AnalysisID=97

This is a report by Pew, and nobody has claimed they are biased. Now matter how much people may deny it, a double-digit change in party ID over a period of months is NOT historically rare. In fact, Ruy Teixeira was claiming the same thing a few months ago when the Dems were ahead, and conveniently changed his opinion once the Republican convention proved his original opinion was correct (http://mysterypollster.typepad.com/). The liberal blogs are consistently opportunistic when it comes to the facts about polling. It's too bad Ruy's forum is now moderated and routinely filters out posters who disagree with him (we'll see if my latest try gets through). Then again, maybe it's good for Republicans, for the reasons mentioned in my last post.

by merzbow on Tue Sep 28, 2004 at 06:02:01 PM EST
[ Parent ]

Re: Preaching to the choir again... (none / 0)

Again, we'll see. Ruy is a pollyanna, no doubt. But to assume that Gallup is nailing this election strikes me as eqully so, only in the other direction. Gallup's short term track record is not very good. It hasn't come close to accurately calling an election since 1984, and was wildly erratic to boot in 2000. As to the CBS and ABC/WaPo, I think those polls are more realistic, and are not really any diff't from IBD, Rasmussen, Zogby, etc. if you factor in margin of error.

I tend to think the whole field of phone polling is in crisis. Why? Because people aren't answering their phones any more. Most outfitsIf you pushed, me I'd tell you the current state of the race is the average of all the polls - 48 to 44, Bush to Kerry. But I would be somewhat uncomfortable doing even that. Not enough people answer landline phones anymore - pollsters report 30%. So you have to do a lot of extrapolation of one kind or another to get results - thus the large degree of difference between various companies.

by Ben P on Tue Sep 28, 2004 at 09:17:19 PM EST
[ Parent ]

Re: Preaching to the choir again... (none / 0)

Preaching to the choir?? Hey, you're preaching to yourself.

I looked at the Pew link. Look at that first graphic. How much did the Dem-GOP difference change in those periods? Individuals moved around, sure. But the overall numbers stood rock-steady. Except for the 1988 numbers, where the change was within the MOE.

Second graphic: winter/spring of 2003, GOP party ID bouncing between 30-33%. In spring/summer of 2004, bouncing between 31-34%. Yes, party ID can change by a point over 15 months. This isn't contested. And there's noise - partly poll MOE, partly momentary event-driven fluctuations. News? No.

Third graphic: GOP post-convention uptick. Yep, there's that noise again.

OK, that about wraps it up for Pew.

And even if we bought into everything Pew was saying, the GOP still wouldn't have a 43-31% edge in party ID.

George Gallup's an evangelical Christian and big GOP donor. He's got an axe to grind. That happens with people. But pollsters, like mathematicians, are supposed to put that aside when practicing their profession, otherwise they produce shoddy work. Gallup can't seem to do that.

You're welcome to believe him anyway. Suits me.

by RT on Tue Sep 28, 2004 at 09:43:44 PM EST
[ Parent ]

What about the Pew poll? (none / 0)

Since only two weeks ago we were touting the results of the Pew poll that showed the race tied, we have an obligation to report that the latest Pew poll has Bush up by 8. It just isn't intellectually honest to use the Pew poll one week to make the race look closer than the other polls say it is and then ignore the next when its results aren't nearly as comforting.

by smartliberal on Tue Sep 28, 2004 at 11:32:00 PM EST

Re: What about the Pew poll? (none / 0)

I think all polls, favorable or unfavorable, should be critically examined. Polls that have lopsided samples in ways that could affect their outcomes should be rejected, or if possible have their results interpreted in terms of their samples, regardless of whether their results are favorable or not.

by RT on Wed Sep 29, 2004 at 11:36:16 AM EST
[ Parent ]

You are not logged in.

In order to post a comment, you must be logged in. If you have a member account, please log in to comment.

If not, you can make an account right here. It's quick and free.