In their most recent survey of candidate preference in the Presidential election, the highly respected Harris poll, which has nailed the final percentage for each candidate
within one percent for three consecutive elections, described their methodology as follows
:The Harris Poll® was conducted online within the United States between September 20 and 26, 2004 among a nationwide cross section of 2,301 adults (aged 18 and over), 1,777 of whom are likely voters. Figures for age, sex, race, education, region and household income were weighted where necessary to bring them into line with their actual proportions in the population. Propensity score weighting was also used to adjust for respondents' propensity to be online.
Harris does not weight by Party self-identification, but does weight according to many other widely available demographics, such as age, income and race. Similarly, in their recent press release arguing against weighting by Party ID, the Pew Research Center for People and the Press
wrote the following
:Given that it [Party self-identification] is an attitude and not a personal characteristic, it is not at all comparable to race, ethnicity, gender, age, education, or other demographic markers that are routinely used to check on the representativeness of surveys.
The debate over whether or not pollsters should weight by Party self-identification continues, but one thing that polling firms do not debate is that characteristics such as age, region, race, education, income, and gender are demographics and not attitudes. Further, considering that they are demographics, in order for a survey of candidate preference to be accurate, that survey must have representative samples of age, education, gender, income, race and region in order to be representative of the electorate. No one would disagree with this.
No one, that is, except for Gallup:
Gallup has been very forthcoming in responding to my requests for information, and they obviously stand behind their methodology and assumptions. But take a look at the demographic breakdown of this week's sample, and ask yourself how likely is this to replicate itself on Election Day, given the increases in registration this year?
Total Weighted Sample: 557 Likely Voters
(2000 exit poll actual results in parentheses)
By Political Ideology:
Conservative: 41% (29%)
Moderate: 41% (50%)
Liberal: 18% (20%)
GOP: 39% (35%)
Dem: 35% (39%)
Ind: 25% (27%)
Over $75,000: 32% (28%)
$50-75,000: 16% (25%)
$30-50,000: 26% (24%)
Under $20K: 9%
And if this wasn't bad enough,
White: 85% (81%)
NonWhite: 15% (19%)
Black: (a subset of NonWhite) 8% (10%)
Not only does Gallup not weight by Party self-identification, a characteristic that is debatably a demographic, they do not weight by age, race and income either, characteristics that are unquestionably demographics.
Even without the Party ID debate, Gallup is clearly using an unrepresentative sample. To use such shoddy methodology, have such a tight connection to major media outlets, and simultaneously show Bush ahead by more than any other poll, is nothing short of an irresponsible in-kind contribution to the Bush campaign.
Shame on you Gallup. Shame on any news outlet that still uses their polls.