Last week, we spent a good deal of time on MyDD arguing over the Iraq withdrawal plans of several Democratic candidates for president.
The crux of the argument had to do with several candidates claiming, or at least frequently implying, that they would withdraw "all" American troops from Iraq, while simultaneously offering policy proposals detailing the various tasks American troops in Iraq would perform after "all" troops had been withdrawn. Implying that you will withdraw all troops, when you policy proposals state otherwise, is a dishonest charade, and I hope that last week's discussion was a step toward ending that charade.
This week, I have spent much of my time focusing on another dishonest charade surrounding the 2008 presidential election:
national polls that imply they are surveying the primary / caucus electorate but are actually surveying all self-identifying and leaning members of a political party. The primary / caucus electorate is a far smaller and narrower universe than all self-identifying and leaning members of a political party, and the two universes are often quite different from one another. However, the implication in report after report on national polls is that the two universes are identical.
Consider the write-up in the Washington Post on the latest national poll it released:
Former New York mayor Rudolph W. Giuliani's lead over his Republican presidential rivals has narrowed considerably, while Sen. Hillary Rodham Clinton (N.Y.) has maintained her advantage in the race for the Democratic nomination, according to a new Washington Post-ABC News poll.
The clear implication at the start of this story is that it will discuss a poll that measures the opinion of the Democratic and Republican primary / caucus electorates in 2008. Notice how Clinton's lead in this poll is interpreted as an "advantage in the race for the Democratic nomination," and how "Giuliani's lead over his Republican presidential rivals has narrowed considerably." This article is clearly claiming, or at least strongly implying, to be discussing a poll of the Democratic and Republican primary / caucus electorates. After all, it is only the primary / caucus electorates that will determine the nominee, not all self-identifying and leaning Democrats and Republicans. Thus, unless you are measuring those electorates, then it is would irresponsible to claim you have data arguing the Giuliani's lead has increased, or that Clinton has an advantage in the race for the Democratic nomination, right?
However, as we go through the article, it is quite clear that the poll is just measuring all self-identifying and leaning Democrats and Republicans. The word "primary" never even appears in the article, and the word "voter" only appears once. However, when describing poll respondents, the word "Democrat" appears in the article six times, and the word "Republican" appears nine times. Despite the implication at the start of the piece, the writers of the article seem to go to great lengths in order to make it clear that the poll is measuring "all Democrats" and "all Republicans" rather than members of the Democratic or Republican primary electorates.
This problem is not isolated to the Washington Post, or to ABC News / Washington Post polls. About a dozen times every month, a major news organization will commission a poll of all Democratic and all Republican self-identifiers and leaners on the 2008 presidential election. Every single time, it will be accompanied by press releases, wire articles and television segments that directly imply the poll in question measures the progress of the campaigns for the Democratic and Republican nominations. However, when one examines the article in details, there are never any references to primary voters or caucus goers, simply references to Democrats and Republicans. I'd be willing to bet some money that the reason for this is because if the articles implied that these national polls were measuring primary voters and caucus goers, then their in-house or commissioned pollsters would find themselves in the midst of an ethical problem. The following is from
the Code of Ethics of the American Association of Public Opinion Research (PDF, emphasis mine):
1. We shall recommend and employ only those tools and methods of analysis that, in our professional judgment, are well suited to the research problem at hand.
2. We shall not knowingly select research tools and methods of analysis that yield misleading conclusions.
3. We shall not knowingly make interpretations of research results that are inconsistent with the data available, nor shall we tacitly permit such interpretations.
4. We shall not knowingly imply that interpretations should be accorded greater confidence than the data actually warrant.
All of the articles on national Democratic and Republican preference polls seem to take great pains to never actually state that they are measuring either the Democratic or primary / caucus electorate. The problem I have is that even though these polls and these articles never actually
state they are measuring the Democratic or Republican primary caucus electorate, there is a consistent
implication that that is exactly what these polls are doing. It is, in my view, at least a form of tacit permission to news organizations to make claims inconsistent with the data available in these polls.
Something needs to be done about this. Right now, I'm not sure what needs to be done, but I wouldn't mind hearing your ideas. It may not seem like the most important campaign to take in regards to the 2008 presidential race, and it might not even be a partisan problem. However, I don't think it can be denied that polls are used as a key tool in forging national campaign narratives, or that perceived candidate poll standing is an important factor for all presidential campaigns in the scramble for resources, positive attention, and simple credibility. Thus, if the polls, and the reporting on those polls, are flawed, it will have a negative impact on the 2008 election. Even if, in the long run, it ends up helping Democrats, it is still a problem that needs to be fixed.
