Yet More Reason to Discount Zogby Polling Emerges

In the past I have been rather skeptical when reading polling results published by Zogby International. Much, but not all, of this sentiment is connected to Zogby's incorrect November 2004 prediction that John Kerry would defeat George W. Bush in that month's presidential election. Additional reports, such as this one from Pollster.com's Mark Blumenthal that show Zogby running afoul of commonly held polling standards, have not instilled much confidence in the results coming out of the organization. Yet if there were any inkling in my mind that I could seriously read polling from Zogby it was put to rest by an piece on Hillary-haters in Friday's issue of The Washington Post by Dana Milbank.

According to Milbank, the normal crew of conservatives whose hatred for the Clinton family has not subsided in the more than six years since they left the White House trotted out polling today showing that a significant portion of the public believes that a Hillary Clinton administration would be corrupt. Of course such a position is a minority, according to the poll in question, but a somewhat significant one.

But looking at the details of the survey, Milbank emerges rather skeptical -- and for good reason.

The poll Fitton commissioned, actually five questions added to a nationwide poll by Zogby International, was rather loaded in its language. "Some people believe that the Bill Clinton administration was corrupt," one question begins. In another question about Hillary Clinton, every answer included the word "corrupt," and the question was not asked about other candidates so that a comparison could be made.

The pollster, John Zogby, defended the questions as "balanced" -- a label Fitton made no attempt to earn. As he presented the results yesterday, he announced that Bill Clinton's financial conflicts of interest "make the issues of Halliburton and Dick Cheney . . . pale in comparison." [emphasis added]

The fact that Zogby would accept a commission to poll-test attacks on a political candidate of any stripe begins to call into question his stature as a supposedly non-partisan pollster. But even leaving this broader complaint aside, because there are certainly some partisan pollsters who are able to produce solid non-partisan numbers (say, like Democrat Peter Hart and Republican Bill McInturff, who team up to do surveys for NBC News and The Wall Street Journal), the apparently unbalanced wording of the polling conducted by Zogby International belies the notion that the organization is serious about coming up with results that actually reflect the views of the American public rather than just the views of those who paid for its services. To harp on one example, beginning a question on the scruples of a politician by saying that some people believe his or her spouse was corrupt inserts such a bias to void the results of the question -- and perhaps even the questions that follow. Simply put, the questions in the poll were not, as Zogby insists, "balanced."

You can take a look at the poll (.pdf) if you'd like. But suffice it to say that it merely underscores the need to take polling from Zogby International with a huge grain of salt -- and perhaps even a whole box full of large grains of salt.

Update [2007-4-6 14:41:27 by Jonathan Singer]: Mark Blumenthal adds more on the poll and takes the further step of comparing it with some unbiased polls -- for a surprising result.



Display:


Re: Yet More Reason .. (none / 0)

Yay!! This is why I really like Mydd so much. My job as a researcher requires I keep accurate statistics. Zogby is a very odd little instrument these days. If they tighten up their polling they will have a good poll, but IMHO especially their internet polls become an exercise in mailing list mechanics and pre-sharpened axes to grind.
wow this was a good piece thanks jon
.. and when I win the lottery, gonna donate half my money to the city so they have to name a school or a park after me - camper van beethoven
by heyAnita on Fri Apr 06, 2007 at 03:58:41 AM EST

Re: Yet More Reason to Discount Zogby Polling Emer (none / 0)

Zogby is in it for the money. I have always discounted his polls. But MYDD and the MSM  sanctioned a poll from NH a fews days ago with a 5.3% margin of error without telling its readers that such polls are not reliable.

I never trust Zogby, it used solely by Washington times, so go figure.


by mdiogu on Fri Apr 06, 2007 at 05:10:59 AM EST

Zogby is also used by the Wall St. Journal. NT (none / 0)

NT


by EricJaffa on Fri Apr 06, 2007 at 06:47:21 AM EST
[ Parent ]

What I think is really nice (none / 0)

What I think is really nice, in particular, is not so much that Zogby is in it for the money - mdiogu - but rather that Jon wrote an article on how they're operating and we had a chance to look into the poll instrument itself. To me, the internals of a poll are really important -

I deal normally with low N-space and so much of my stats are wobbly and I am gambling all of my research sometimes on very thin trendline

I was aware that Zogby sends out email broadcasts to targetted mail list respondents - the questions are somewhat innocuous but not overly so and don't have a tendency to disperse the aim of the poll evenly through the questionnaire - often times his polls sort of read like a discussion between himself and the polled about his general thought - again I say this as a very faint conclusion.  As always, they pay you off to take the poll by offering you some kind of free download or something but once you've done that a couple of times, the only reason you would be responding to that poll would be imho if you've got an axe to grind - or you want to voice some kind of opinion and if the sample space is partisan, that opinion will pick up the bias.

I don't know if Zogby has enough really good instruments to even out this problem - but imho the internet polling he does is where his numbers tend to start becoming a little bit off base. For me, its the internet polling that I tend to throw away.  I wish I knew more about the other pollsters.

I wonder if http://www.electoral-vote.com will pick up on this article jon wrote? Someone should tell that poor expatriate about Zogby so he can add a weight


.. and when I win the lottery, gonna donate half my money to the city so they have to name a school or a park after me - camper van beethoven
by heyAnita on Fri Apr 06, 2007 at 07:10:04 AM EST

Re: Yet More Reason to Discount Zogby Polling Emer (none / 0)

Skepticism of Zogby polls originates in 2004, when those polls incorrectly predicted a victory for Kerry. Let's see: pre-election polls all showed a victory for Kerry; exit polls confirmed this; even Brit Hume of Faux News conceded the victory to Kerry based on exit polls.

Also, given the documented cases of tabulation fraud and voter suppression, as well as the consistency in which the exit poll/final totals discrepancy favored Bush, I think the claim that 'Zogby got it wrong' is a little loose.

More importantly, the claim that Kerry lost the election is even looser.


by scudbucket on Fri Apr 06, 2007 at 11:48:56 AM EST

maybe zogby is doing us a favor (none / 0)

the negative feelings about Hillary among the "general election  universe" are real, I live in Nevada but am from New Jersey so I can get a feel for how Hillary is percieved differently outside the norhteast, she does have alot people who would consider votingb democratic
for someone else, who simply don't like her. If she's nominated those vpoters are gone and if the Republicans nominate someone likeable such as
a Fred Thompson then the race would nopt be about the war or healthcare, but about Hillary and how polarizing she is, and we lose.
Obama! because 51% isn't enough!
by nevadadem on Fri Apr 06, 2007 at 12:16:13 PM EST

Re: Wrong to Discount Zogby on 2004 (none / 0)

As an election integrity activist in California, and an active reader of mydd, I came to respect your tendency to ignore voter fraud issues as I came to appreciate the effectiveness of your focus on building a progressive force in the Democratic Party.  I think that you have been enormously effective at mydd at turning things around and creating an effective and dependable progressive voice in this country. That is of course huge.
My explanation to myself is that in order to build a new positive progressive force, you had to set aside charges of electoral fraud that were negative framing and confusing to deal with.
However, when you look at polls that called a Kerry victory, you are cheating yourself and all of us when you discount them.  Drinking the Kool Aid on this issue will keep you making assumptions based on false conclusions.  
Recent events in Ohio give us hope that we will have more proof of malfeasance in the near future, although there is plenty of proof already that over a hundred thousand votes were not even counted. Two people were sentenced to prison for rigging the recount.
And of course malfeasance was not limited to Ohio.
There is no question but that John Kerry won the election in 2004.  If you have other criticisms of Zogby, they may stand on their merits.  But of course he was right on 2004.
by syolles on Fri Apr 06, 2007 at 12:39:00 PM EST

Re: Yet More Reason to Discount Zogby Polling Emer (none / 0)

You know, it would actually be really interesting to see a poll that asked "To what extent do you think [X candidate] would be corrupt?" for the entire field of viable Presidential candidates.

I can't imagine the utility of asking that question for just a single candidate, though. And even though I'm no pollster, even I can see the "would Hillary Clinton be corrupt?" question would not provide meaningful results if you put it right after the "SOME SAY Hillary Clinton is corrupt-- do you agree?" question as this poll did.


by Silent sound on Fri Apr 06, 2007 at 05:02:14 PM EST

Newsmax (none / 0)

I won't believe any poll commissioned by an ultra-conservative website known for spreading lies.
Zogby may feel compelled to give Newsmax sexed up information that appeal to the right-wing masses.
by kingsbridge77 on Tue Oct 02, 2007 at 08:33:30 AM EST


You are not logged in.

In order to post a comment, you must be logged in. If you have a member account, please log in to comment.

If not, you can make an account right here. It's quick and free.