Polling Project: First Numbers

The poll is finished. The methodology can be found here. We will release the results in eight separate bursts, starting today and finishing up on Tuesday morning. The releases will follow the chronological ordering of the questions. There are twenty-two questions in the poll, plus a number of demographic questions. The release will cover questions 1-5.

The reason we are not releasing all of our information at once is because we want to give proper attention to all of the questions in the poll. Of course, in the interests of full disclosure, absolutely everything in the poll, including the raw data, the methodology, the entire questionnaire, and all of the results, will be available to the public on Tuesday morning. This will be the most public, open-source poll of all-time--Chris

If you will forgive me for being a tease, our first release will focus on perhaps the most boring questions in the entire poll:

Wright Consulting. 1004 Registerd Voters. January 16-26. Margin of Error,

1. Are you registered to vote in elections in your local area?
Yes: 100.0%

2. And are you registered to vote as a Democrat, a Republican, an Independent, a member of some other party or are you registered but with no party preference?

Democrat 33.1%
Republican 29.5%
Ind/Oth/No Party 37.5%

3. Generally speaking, in your opinion is the United States headed pretty much in the right direction these days or is it headed off on the wrong track?

Right Direction 36.9%
Wrong Track 48.3%
Not Sure/DK/ref 14.8%

4. And do you approve or disapprove of the way George W. Bush is handling the job of President?

Approve 42.9%
Disapprove 50.4%
Not Sure/DK/ref 6.7%
5. How would you describe the availability of good jobs at decent wages in your local area? Would you say such jobs are widely available, available but not easy to find, rarely available or not available at all?

Widely Available 17.6%
Available, not easy 37.1%
Rarely Available 25.3%
Not available 9.4%
Not Sure/DK/ref 10.6% Not as groundbreaking as we had promised, is it? Well, for these questions, that's good. There are many times when the results of these questions, especially question #4, can result in a poll becoming big news. When Bush plunged into the 30's according to most polls, that was big news. When Bush "recovered" into the low forties, that was big news. However, the absolute last thing I wanted was for our answers to these questions to be anything but ordinary. As much as I may want Bush's approval rating to be 30%, the last thing I wanted was for our poll to show him with such an approval rating. I did not want that because we all know that any poll showing Bush with a 30% (or a 55%) approval rating is inaccurate. In our first entry into the polling world, we simply can't make news with outlying results: we have to make news with accuracy.

I believe we have indeed achieved accuracy. These numbers are very much in line with the recent results of other polling firms. (Source for ARG. Source for Rasmussen. Source for all other polls).

Organization, Date: Approve / Disapprove
Rasmussen, 1/24-1/26: 46 / 53
Fox /Opinion, 1/24-1/25: 41 / 51
ARG, 1/18-1/22: 38 / 58
LA Times / Bloomberg, 1/22-1/25: 43 / 54
CBS / NYT, 1/20-1/25: 42 / 51
Gallup / CNN, 1/20-1/22: 43 / 54
Hotline, 1/12-1/15: 46 / 53
Mean: 42.7 / 53.4
Median: 43 / 53
MyDD / Wright, 1/16-1/26: 43 / 50

Our "direction of the country" numbers are also in line with those of other polling firms (source):

Organization, Date: Right Direction / Wrong Track
Gallup, 1/9-1/12: 36 / 61
Pew, 1/4-1/8: 34 / 61 (satisfied / dissatisfied)
AP, 1/3-1/5: 32 / 65
MyDD / Wright, 1/16-1/26: 37 / 48

Clearly, when it comes to the most commonly asked political polling questions, our survey sample is very much in line with those of other polling firms. This is a good, solid sample. Given this, the results that we will release over the next few days can be accepted with much greater confidence. Perhaps best of all, considering that one of the charges we are most likely to face is that we are partisans, I would also like to note that our results are actually slightly more favorable to Bush and the conservative viewpoint than they are to the Democratic and progressive viewpoint. In our poll, Bush's job disapproval rating is most similar to that produced by, of all places, Fox News. Our approve / disapprove margin is most similar to Republican polling firm Rasmussen. Our approve numbers are precisely at the mean and the median in of the last seven public polls. Our "wrong track" direction of the country numbers are much more favorable to the Bush administration viewpoint than those of other polling firms. Even our "right direction" numbers are slightly higher than those of other recent public surveys.

Accuracy was always the key in this project. In deciding to move forward with our historic polling project, collectively we had a number of different goals and motivations. We wanted to commission this poll because there were a host of questions we wanted to ask. We wanted to ask questions about Iraq withdrawal that no other polling firm was asking, questions about possible investigations of President Bush that polls operated by commercial news outlets seemed unable to ask, and questions about warrant-less domestic spying because, at the time, few other organizations were doing the same and we were not happy with the questions they were asking. We also wanted to run this survey because we know that polls commissioned by commercial news organizations do not just report on the state of public opinion, but because they also help shape public opinion as well. We wanted to challenge the notion that the creation of conventional wisdom is entirely the domain of a small handful of pundits and talking heads who work for large news organizations. We believe it is possible for the grassroots and the netroots to alter conventional wisdom as well.

However, above all else, we wanted a poll that would be an accurate survey of national public opinion on a variety of topics. Without accuracy and rigor, all of our other goals would be impossible. If our numbers are not accurate, our results, no matter how well intended, would become worthless. Without accuracy, there was no reason for anyone, either inside or outside of the blogosphere, to take us seriously. Without accuracy, there was no way that our questions, no matter how original and insightful, would be repeated in variation by other polling outfits. Without accuracy, all we would have done is waste the nearly $17K that the MyDD community, and the wider netroots community, amazingly donated to the poll.

With all of this in mind, my biggest worry when the poll was in the field was that we would come out with outlying answers to frequently asked questions. No matter how good your pollster is, and Joel Wright is very good, there is always a chance you will end up with an outlying poll. That is why, at the start of the poll, we included two common poll questions in order to establish whether or not our sample was in line with those of other polling outfits in recent weeks. To my great relief, we came in right at some very ordinary, average results.

In closing, I would like to point out the results to question number five, for two reasons.

First, it is interesting that only 18% of the country think that good jobs at good wages are "widely available." This is an important result that Democrats should use whenever they are faced with claims from Republicans that the job market is fantastic. People just don't see it that way.

Second, I want to note that the rest of the poll is filled with questions of this nature. We are laying out the most mundane, the most ordinary results first. However, the rest of the poll is chock full of juicy results that we will release between now and Tuesday morning. The first key is showing that this is an accurate poll. From that point, we can start to explore further wonders that unfold later in the questionnaire.

Thank you to everyone who made this happen. We are boldly going where no blog has gone before, and the readers of this blog made it all possible.

Tags: Blogosphere, Bush, Economy, Media, polls (all tags)

Comments

29 Comments

Re: Polling Project: First Numbers

All polls try to take complex social issues and squeeze them into a simple yes/no answer. It's not the methodology of selecting the sample or the questions that needs to be questioned, but, rather, whether polls actually provide any useful information.

Is the country headed in the right/wrong direction, for example. What is a person supposed to answer if they think foreign policy is a disaster, but domestic policy is great, or vice versa? How about if they like the farm policy, but hate the import policy?

Is Bush doing a good job? Same thing, one may like his religious pandering, but dislike his tax breaks. How are they to answer?

I hope you have fun with your adventures into sociology, but I doubt anything useful will come of this. Take a look at PIPA for more meaningful (although still flawed) ways to gather data.

Finally, it is well known that when asked for their opinions people will respond even when they have no idea of the issues involved. Just yesterday there was data released showing that about half the population doesn't even know which party controls congress.

Sorry to be so negative, but only detailed interviews which get past people's glib answers can reveal what their positions really are.

by rdf 2006-01-27 08:55AM | 0 recs
Re: Polling Project: First Numbers

Very astute comment, but IMO doesn't go far enough. What if one thinks the country is headed in the wrong direction because of pork-barrel politics, politicization of Supreme Court nominations, and lack of will to protect the country from further terrorist attacks? According to the interpretation presented here, and common elsewhere, a person who answers "wrong track" for those reasons is counted as "unfavorable" with respect to Bush. Most of those positions could be either bipartisan or blaming one party more than the other, and certainly the third would be less likely to come from a person who would prefer to see Ted Kennedy in charge.

While this poll may be in the mainstream in its choice of interpretive framework, it is precisely the use of the data to attempt to shape public opinion that leads over time to invalid data. Validity in this context means measuring what you intend to measure. When the data is collected in this fashion and overinterpreted, self-aware respondents will begin to answers pollsters strategically: They will not provide their true opinion if they know their response will be used to argue that "people" have some other opinion. They will answer in terms of the way they expect the data to be interpreted. Accurate measurement becomes impossible.

There was a heyday of powerful predictiveness in US opinion polls, but that day I believe may be past.

by beachrat 2006-01-30 12:27PM | 0 recs
Re: Polling Project: First Numbers
43% approval is a failing grade even on the most generous of curves.
-Synthesis
by synthesis 2006-01-27 08:56AM | 0 recs
Re: Polling Project: First Numbers

So is 51%, but it still wins you an election. ;o)

by Jefe Le Gran 2006-01-27 01:16PM | 0 recs
Re: Polling Project: First Numbers

Hah!  In my Signals Analysis class on the first test of the semester, 43% was good for a 'B'!

Nevertheless, W is a failure in all the ways that count.

by teknofyl 2006-01-27 07:48PM | 0 recs
How does Party ID compare?

Do most other polls show an R/D/I split along these same lines?

by Steve M 2006-01-27 09:17AM | 0 recs
Re: How does Party ID compare?

Yes, most do. They vary to some degree, Gallup tends to report more Repubicans, for example. Our key benchmark was the Pew Center's 2004 analysis of voter identification which is mentioned in the methods statement post. Our numbers are exactly in line with theirs on this issue.

by Sun Tzu 2006-01-28 05:19AM | 0 recs
Re: Polling Project: First Numbers

The big number here is the one about jobs and wages.  The Wal-Mart economy brings everyday low wages along with its prices, if it brings wages at all.  And a huge divide in income distribution.  Each store carries with it one well paying job (the store manager) a handful of reasonable jobs (some of the managers), and hundreds of lousy jobs.  Meanwhile the manufacturing gets shipped to rural China.

See the LA Times series on Wal-Mart from a few years ago if you don't think this is true.

by David Kowalski 2006-01-27 09:35AM | 0 recs
Re: Jobs and Wages

My instinctive reaction to these numbers (may be nonsense) is that respondents are congratulating themselves on being winners. Most people are employed, even when employment numbers stink. Therefore, most people pat themselves on the back by putting themselves mentally in the "available but hard to find" category.

Will be very interested in more findings. I usually think the interesting information is less in the data than in the interpretaton.

by janinsanfran 2006-01-27 10:46AM | 0 recs
Re: Polling Project: First Numbers

Why ask registration?  A lot of states don't require you to register with a party to vote in the primary, and some have open primaries, and in places that are one-party-dominated some people register with the dominant party so they can have some voice in picking candidates.  As a result, party registration tends to be a much less reliable indicator of partisan preference than the more-standard partisan identification question.

I'm curious to know the rationale behind asking about respondents' registration.  Do you have an inkling that registration results are more meaningful than they've been given credit for being?

by kilb 2006-01-27 09:55AM | 0 recs
Re: Polling Project: First Numbers

Don't they have to ask registration to show that the poll is not tilted to dems or reps?  If all respondants were from one party the poll numbers would be skewed to the right or left and thus meaningless as far as the whole country goes.

by mpower1952 2006-01-27 08:07PM | 0 recs
Re: Polling Project: First Numbers

You could acheive the same thing by asking partisan identification -- in fact, that's what most polls do.  The reason they don't ask for registration is that the rules of registration vary widely from state to state, so it is not an equally valid measure of partisan loyalties in different parts of the country.

by kilb 2006-01-28 04:15AM | 0 recs
In Missouri

Respondents don't differentiate between their self identification as to party preference and if they actually had to register to vote with a party preference.

Since I've lived in Missouri (a fair amount of time) voters do not register by party - believe me, if we did it'd make organizing and GOTV a hell of a lot easier.

It's almost become a running gag for us when we do voter ID calls. We'll ask the party ID question and the person on the other end of the line with a voter history going back to the beginning of time (or not) will indignantly reply, "Why, I've been a registered XXXXXXXX all my life!" We never bother to reply, "You can't be, voters in Missouri don't register by party" - it'd just piss them off.    

by Michael Bersin 2006-01-28 04:39AM | 0 recs
Re: Polling Project: First Numbers

What you are doing here is fantastic. I am so impressed by your committment to total transparency in this. I kicked in $20 for the poll a while back, and I am proud to have done so.

Congratulations. What is so valuable here is the discussion.

by sdedeo 2006-01-27 10:15AM | 0 recs
Re: Polling Project: First Numbers

I can't wait to see the rest of the numbers!

I too was concerned that the benchmark question results might be off... but it looks like it is right on.  Awesome start.

by KansasNate 2006-01-27 11:41AM | 0 recs
Re: Polling Project: First Numbers

Good job guys!

I like that we are taking responsibility and acting and not just whining on blogs.

I have been incredibly impressed about how the liberal blogs took charge this week and caused action to happen. From no longer allowing the traditional media shills from getting away easily with right wing stenograhpy to the filibuster campaign. And now this independent and transparent poll is all moving in the right direction.

We have a tremendous opportunity. The liberal base is energized. We have proven that we can compete financially through small donations. The target ought to be how we can positively impact this years mid-term congressional elections. We need to be focused and organize first to elect liberal fighters with spine as the Dems candidates in the primary and then help in creating the momentum and tilt towards the Dems in Nov. We have the opportunity to win a majority in the House. Then we can take care of business and take down the Bush-Cheney-Repub culture of corruption & incompetence.

Thanks Matt, Chris, Jane, Kos, Digby, et al and all those in the liberal blogosphere. We are maturing and we are creating a new force in American politics. I am very optimistic and hopeful.

by ab initio 2006-01-27 12:18PM | 0 recs
Re: Polling Project: First Numbers

Thank  you for this.  You have done well with my $50.00 and when you want to follow up I will pony up again.

At last, someone credible is asking about the whole herd of elephants in the living room.

Keep it up.

Dipsop

by dipsop 2006-01-27 12:36PM | 0 recs
Re: Polling Project: First Numbers

When you are all done analyzing/publishing/promoting the results, is there any chance that a spreadsheet of the data will be made available?  I ask this because I find the "bar graph" only approach to be limiting (and I'm a stats nerd).  Regardless, good job, and I'm glad I donated.

by mfeld356 2006-01-27 01:10PM | 0 recs
Re: Polling Project: First Numbers

I'd actually like to get the raw data and run some crosstabs via SPSS.

by Michael Bersin 2006-01-28 04:41AM | 0 recs
Re: Polling Project: First Numbers
The answer is yes, the original dataset will be made publicly available for free. And pretty soon. Chris and I discussed this specifically yesterday with the Mystery Pollster. He asked the same question.

It'll take me some time to prepare documentation for those without SPSS or similar statistical software. That's a very important issue because the sample is balanced, weighted, to reflect population parameters. The weights must be turned on, filtering the data, in order for accurate numbers to be generated. I don't use Excel much because of SPSS, so I'm unsure if it can generate statistics through a weighting variable. Perhaps others know?

I'll be working on documentation and my own deeper analyses  after next week. I think certainly by the end of February, no problem.

by Sun Tzu 2006-01-28 05:12AM | 0 recs
Re: Polling Project: First Numbers
Super! Thanks again for all the hard work.
by mfeld356 2006-01-28 11:29AM | 0 recs
Good job, guys. n/t

by NeuvoLiberal 2006-01-27 03:18PM | 0 recs
I'd Like To See The Cross-Tabs

between Party ID and the job availability question.

Once again, I think the "reality-based community" effect will be found rearing its anti-Loch Nessian Monster head.

by Paul Rosenberg 2006-01-27 03:26PM | 0 recs
Re: I'd Like To See The Cross-Tabs

That would be an interesting point. This is gonna be great.

I am so proud of my contribution to this poll. I bet they ask most of the questions I think of that the national pollsters never ask.

And the explanations of the questions so far are so interesting. (Yes I lead a boring life) Can't wait to get to the good stuff.

by mpower1952 2006-01-27 08:12PM | 0 recs
Re: I'd Like To See The Cross-Tabs

Be sure and check back later this morning. I'm going to write up the analysis of the released questions with further detail, including party, and have that for you soon. Actually, it's what I'm doing next. Thanks!

by Sun Tzu 2006-01-28 05:15AM | 0 recs
Re: Polling Project: First Numbers

I'm not a regular MyDD regular.  I rarely stop by this place except through links from other sites.  After this inspired poll initiative, all that's changed.

by pseudo999 2006-01-28 01:38AM | 0 recs
Re: Polling jobs

Regarding question number 5. This is a prime example of why polls can be so misleading. You ask that question of all respondents, assuming they are qualified to answer the question. I'd venture to guess that a majority of people you call (Republican or Democrat) who are already employed and have been for some time, have absolutely NO IDEA what the availability of jobs or the job market is like in their area.

I would have asked prior to this question something to the effect of, have you been employed at your current job for more than 6 months (or something like that) and then the responses to the jobs availability question might have been more on the mark, if you took the responses from ONLY people who are looking for a job currently or have looked for one in the past 6 months... or something along those lines.

I truly believe the actual facts regarding job availability in this poll are very misleading and do not accurately reflect the actual job market in that area. But you're right... it's a great stat for Democrats. Too bad it's not accurate. (This would be in line with many of the anti-Bush remarks made by Democrats today, though.)

by oneteam 2006-01-29 06:51PM | 0 recs
Re: Polling Project: First Numbers

First, nice work regarding being meticulous and professional on this.

I'd like to offer two cents on the job numbers as well. Objectively, the US is creating more jobs at higher wages than any Western economy.

I don't doubt that the above numbers represent people's feeling about jobs, but by any historical measure the current situation is extraordinary for job seekers -- whether or not they actually feel that way.

Still, it's 55% positive versus 36% negative. If Democrats want to make it an issue, it will be one of perception, not reality, and the facts point to good conditions.

The negative sentiment may be more of a reflection of our high level privilege than anything...

by ORinSF 2006-01-29 07:53PM | 0 recs
My compliments on your Poll! Thoughts.

My compliments on your polling efforts, I look forward to your results.  I posted this comment and wanted to offer it to you and your readers as interest.  Cheers...

From Miguel Lecuona - Texas Pundit
www.txpundit.blogspot.com

Polling vs Elections: Why the Polls Fall Short

Most polls do not translate very well to presidential elections, in my opinion. Even well-intended polls by partisans, with sound methodology, leave lots of guesswork when attempting to project results to actual elections.

Let's agree that this poll and the methodology are accurate for what they are asking and to whom they ask the questions. Let's also assume that there is no media bias or partisan polling bias in the results. I know -- stay with me anyway!

The issue is projecting the results of this poll (or any poll) into a reliable election prediction. Todays do not speak to voter turnout, or to the changing voter demographics that will exist nearly 3 years from now.

Also, I would be concerned that a sample of 1004 voters is not very translatable to Electoral College results.

For example, Dems are heavily concentrated in major Northeast and California Urban centers, often carrying those cities by as much as 75% and creating big state-specific landslides. The poll says that, nationally 33% of all people are Democrats. If that's true, then this base is not widely or evenly distributed across the country, making it less impactful to an Electoral College majority. As soon as the Dems achieve more than 51% in a state, the rest of the votes are "wasted", so excessive margins indicate the base is highly concentrated. OK for polls, not good for Elec. College.

Looking closer at the samples, they are deemed representative of the party affiliations... if that is the case, then the following holds true:

Poll 1004 Voters
Dems 33%
Reps 29%
Indep 37%

2004 Election results: Bush 51%, Kerry 48%.

Assuming both parties held their bases, this implies that Indeps voted 22 for Reps vs 15 for Dems.

Are they really "independent" if they vote 3-2 for Bush? If they were Independent, one might expect a closer split. But if Independents continue to vote Republican, the distinction is not very meaningful.

I will look forward to seeing the rest of the questions, and their answers. But the best polls are the actual election results.

Miguel Lecuona - Texas Pundit

by miguel lecuona 2006-01-29 09:24PM | 0 recs

Diaries

Advertise Blogads