A Small Victory on Polling Transparency

In a victory for transparent polling advocates everywhere, it appears that both Pew and Gallup will begin publishing party ID data along with their survey reports. Mystery Pollster received the following email from Frank Newport of Gallup: As far as I know, Gallup has no history over the last 70 years of routinely posting the party ID composition of each survey we conduct, just as we routinely don't report ideology and a lot of other measures regularly asked in each survey. As noted, we send the party ID composition percentages to anyone who is interested (actually, we really don't get that many requests for them). But since this seems to be an area in which there is perhaps bourgeoning interest, we'll probably start posting them on our website for each survey, along with rolling trends and some explanations of how Gallup measures party ID and what it's significance is [emphasis added]. And received the following email from Pew: Given the evolution of the dialogue on the subject - for which MysteryPollster deserves a lot of credit -- and the greater understanding among political observers regarding the perils of weighting party ID to an arbitrary parameter (clearly illustrated by the party ID distribution on Election Day 2004), we will begin posting party ID and its trend in our toplines in future survey releases [emphasis added]. The main issue that I always had with the lack of Party ID weighting had more to do with the way it impacted the media campaign narrative than with the actual issue over whether or not partisan self-identification should be a weighted demographic in actual surveys, an issue which I realize is a contentious source of debate in the professional polling world. . From the perspective of the campaign narrative, the tendency of commercial polling outfits to not weight by party ID seemed to be the main factor that guaranteed they would be mentioned more frequently by commercial media outlets. The less factors that are weighted in a poll, the more dramatic the top-sheet results of the poll are likely to be, and thus the more likely a polling firm that did not weight by party ID was to be mentioned by commercial news outlets, especially periodicals, which absolutely crave dramatic poll swings.

This resulted in a very poor campaign narrative during the Presidential general, which I believe inaccurately stated that Bush had a sizable lead in September and that Kerry had a sizable lead in mid-October as the commercial media seized upon the polling information most likely to support either claim. As far as I could tell, neither was the case. I argued that weighting by party ID revealed a more stable campaign narrative that itself was consistent with the total depth and breadth of all polling available. It is the responsibility of the media to provide the public with an accurate snapshot of the Presidential horserace, and I believed that reporting Party ID numbers, registered voter numbers, and the totality of all polling information was necessarily to achieve that desired accuracy.

The ultimate goal is not necessarily for there to be more uniform polling that all weights by party ID, but for there to be better reporting on polls in the media at large. This requires pollsters to be more transparent about their methodology and survey internals, rather than to change their methodologies. By releasing their party ID figures along with their survey reports, Pew and Gallup have done just that. Now the ball is in the media's court. Good for Pew and Gallup. Thank you to Ruy Teixeira, Mystery Pollster, Steve Soto and DemfromCT for their work on this topic. A better informed public has always been the goal, and their efforts have helped to achieve this.

This is a step in the right direction. Now, if only we could get polling firms to publish registered voter numbers along with likely voters until the last two weeks of an election, and to report on the breadth of all polling data available. Of course, the latter might require commercial media outlets to end their exclusive contracts with commercial polling outfits, something that will be far ore difficult for either side to accept. Still, if it results in better reporting and a more nuanced, accurate campaign narrative, it will be for the good of the nation. We need to report all of the polls, all of the internals, and all of the caveats. We have come this far, so there is no reason to stop now.

Tags: Media (all tags)


1 Comment

sick of exit polls
but still there's more... this is a critique of the Freeman paper's statistics by Rick Brady and can be found here.

I agree the bigger story is the push towards transparency, one step at a time. Thanks for your efforts.

by DemFromCT 2005-03-20 03:46AM | 0 recs


Advertise Blogads