MyDD :: 'Liberal Media' Myth Goes 'Scientific'

'Liberal Media' Myth Goes 'Scientific'

by Paul Rosenberg, Thu Dec 22, 2005 at 01:36:03 PM EST

Nothing is more crucial to the right wing than the myth of "liberal media bias." It is, quite simply, a one-size-fits-all, best-defense-is-good-offense response to anything that threatens to get in their way. Witness Bush's attempt to shift blame to the New York Times for (very belatedly) spilling the beans on his wanton lawbreaking with countless illegal wiretaps.

The problem is, the myth, for all its propaganda power, is just that--a myth. Which is why there's a crying need to dress it up as "fact." Conservatives have been trying to do this for quite some time, with unconvincing results, to say the least. But now they've upped their ante with a new study promoted by UCLA's media office, in a press release ("Media Bias Is Real, Finds UCLA Political Scientist") that could have been written by a rightwing think tank. And the study itself, it appears, virtually was, despite the fact that one of two co-authors--Tim Groseclose--is a UCLA professor. It's already getting out there in the rightwing blogosphere, so let's do some debunking, shall we?

UCLA's press release begins:

While the editorial page of The Wall Street Journal is conservative, the newspaper's news pages are liberal, even more liberal than The New York Times. The Drudge Report may have a right-wing reputation, but it leans left. Coverage by public television and radio is conservative compared to the rest of the mainstream media. Meanwhile, almost all major media outlets tilt to the left.

These are just a few of the surprising findings from a UCLA-led study, which is believed to be the first successful attempt at objectively quantifying bias in a range of media outlets and ranking them accordingly.

The "fact" that the Drudge Report "leans left" is not just a "surprising finding," though. It's a flashing red light that says, "Warning! Bad Data Ahead!" For any study that kicks out a result like this must surely have something fundamentally wrong with it.

The other "surprising findings," in contrast, are not so surprising to serious media critics--notably those on the left. It's long been known that The Wall Street Journal does more hard-hitting reporting than The New York Times--the non-ideological translation of the UCLA press release--and FAIR (Fairness and Accuracy In Reporting) has long pointed out that public broadcasting is quite corporate-friendly and citizen/activist averse--particularly in its flagship programming. (See for example, back in 1993, "New Study Reveals Public TV Bias And Debunks Conservative Myths".) So what we have here--at least as advertised--is a mix of bad data signalling a flawed methodology, and an ideologically slanted report of stuff that left/liberal media critics have been saying for over a decade.

Big news, no?

In fact, there are a number of anomalies that stick out, indicating that something is seriously wrong with the methodology. For example, the ACLU and the NRA have very similar scores--49.8 and 45.9--both centrist, though slightly to the right. The authors take note of the ACLU's position, and explain why--its opposition to campaign finance restrictions was cited repeatedly by Congressional opponents:

"In fact, slightly more than one-eight of all ACLU citations in Congress were due to one person alone, Mitch McConnell (R.-KY), perhaps the chief critic of McCain-Feingold."

It does not seem to occur to the authors that this may be indicative of a much more pervasive problem with their basic, untested assumption that politicians cite sources for ideological reasons, rather than pragmatic ones. And they did not seem to be troubled by NRA's position at all, focusing instead on the RAND Corporation as a second anomaly that needed special attention.

There's also the little matter of calling organizations like the NRA and the ACLU "think tanks." They're not. Putting advocacy organizations and think tanks into the same dataset creates a classic example of comparing apples and oranges.

A prominent critic of an earlier incarnation of this paper, Geoff Nunberg, pointed out (among other things) that the list of groups used is deeply problematic:

Start with the list of groups from which G & M drew their initial sample. They describe this simply as a list of "the most prominent think tanks," but that isn't quite accurate. In fact their list was drawn from the 200 links included on the site wheretodoresearch.com (which actually describes it merely as a list of "major think tanks and policy groups"). The list was compiled by one Saguee Saraf, a free-lance researcher with a masters degree in history who lists among his achievements that he was named Man of the Year by the Cheshire (Connecticut) Republican Town Committee.

Saraf gives no indication of how his list was compiled, or what criteria were used -- nor, more to the point, do Groseclose and Milyo say why they consider the list authoritative. In fact its contents are a jumble of think tanks, lobbying groups, trade associations, and advocacy groups, assembled in a catch-as-catch-can manner....

Then, too, Groseclose and Milyo's survey of the citations of groups in the Congressional Record shows some results that would most kindly be described as puzzling. In their list of the "twenty think tanks most cited by members of Congress," for example, they list in 13th place the Alexis de Tocqueville Institution (which they refer to as the "Alexis de Tocqueville Institute"), which comes in ahead of Common Cause (14th), the Family Research Council (16th), and the Economic Policy Institute (19th), not to mention a number of much better-known groups that appear on Saraf's list but not in G & M's top 20, like the NRA and the Hoover Institution.

FAIR itself currently has a front-paged a comment (permalink here) on this report, with an excerpt from an article in FAIR's newsletter, Extra!, which comments on an earlier incarnation of the paper. The bolded text is where they really poke the hole in the center of this whole charade:

A study being published in the Quarterly Journal of Economics has been getting some attention for its supposed quantification of bias in the media. An earlier incarnation of the study was discussed in the May/June 2005 issue of Extra!:

The report used a peculiar Rube Goldberg-like method to calculate media bias from think tank citations: Taking the Americans for Democratic Action ratings of congressional voting records as its yardstick, it assumed that media outlets have ideologies similar to those of members of Congress who cited the same think tanks that the media outlets did.

This approach is based on the problematic notion that politicians cite the think tanks that they most agree with rather than the ones whose citation will be the most politically effective; a problem the researchers acknowledge when they attempt to explain away some curious anomalies that their method produces. (The National Rifle Association comes out as a centrist group; the Rand Corporation turns out to be left-leaning.)

If the authors truly wanted to rank media outlets on the ADA scale, the simpler method would be to look at the ADA ratings of congressmembers quoted by those news outlets. One suspects that the authors avoided this obvious approach because the results would have been less to their liking: Studies in Extra! have repeatedly found various media outlets quote Republicans more often than Democrats, by ratios ranging from 3 to 2 on NPR (5-6/04) to 3 to 1 on nightly network news (5-6/02) to a startling 5 to 1 on Fox News Special Report (7-8/04). Fox News, according to Groseclose and Milyo's method, is a "centrist" news outlet.

The above is an excerpt from an article by Michael Dolny, "Right, Center Think Tanks Still Most Quoted: Study of cites debunks 'liberal media' claims". Dolny, a sociology professor, has been doing these studies annually since 1996, when he was an intern at FAIR. Unlike Groseclose and Milyo, he actually restricts himself to just looking at think tanks. The article explains:

The study counts citations of the 25 most prominent think tanks of right, center and left, using the Nexis news media database. Citations are counted in what Nexis designates to be major newspapers, as well as in Nexis' transcripts file, which includes the major broadcast and cable news outlets. Because stories included in the Nexis database change over time, figures for previous years are recalculated for comparison purposes rather than taken from previous editions of the study.

Conservative or right-leaning think tanks garnered 50 percent of citations among the 25 most-cited think tanks, the same percentage as last year, and near their 10-year average of 51 percent of citations. Centrist think tanks declined slightly this year, garnering 33 percent of the citations, compared to 37 percent last year and 36 percent as their 10-year average. Progressive or left-leaning think tanks had the greatest percentage increase this year, receiving 16 percent of citations, up from last year's 13 percent and their 10-year average of 14 percent.

This is what's really happening in the media, when it comes to citing think-tank sources--an overwhelming predominance of conservatives. Dolny's work is not nearly as complicated as Groseclose & Milyo's. But, then it doesn't have to be. It's not trying to stand reality on its head.

Groseclose & Milyo published in an economic journal precisely because it uses economic models that come out of an ideological project to reinterpret the world in terms of conservative economic ideology. While there's no direct link between the economic ideology and the outcome of their research, it is not surprising that the two go together. They reflect a common attitude of beginning with an ideological outlook, and then looking for data to support it. There is nothing wrong with this per se. But if it's not accompanied by a willingness to take criticism seriously, and respond to it fairly, then the end result is not good science, and arguably, even, not science at all.

Tags: Media Bias, Pseudo-Science (all tags)

Permalink :: 7 Comments :: Post a Comment

p.s. re Drudge (none / 0)

The authors "explain" Drudge's liberal rating thus:

Another result, which appears anomalous, is not so anomalous upon further examination. This is the estimate for the Drudge Report, which at 60.4, places it approximately in the middle of our mix of media outlets and approximately as liberal as a typical Southern Democrat, such as John Breaux (D-LA). We should emphasize that this estimate reflects both the news flashes that Matt Drudge reports and the news stories to which his site links on other web sites. In fact, of the entire 311 think-tank citations we found in the Drudge Report, only five came from reports written by Matt Drudge. Thus, for all intents and purposes, our estimate for the Drudge Report refers only to the articles to which the Report links on other web sites. Although the conventional wisdom often asserts that the Drudge Report is relatively conservative, we believe that the conventional wisdom would also assert that--if confined only to the news stories to which the Report links on other web sites--this set would have a slant approximately equal to the average slant of all media outlets, since, after all, it is comprised of stories from a broad mix of such outlets.

The authors think that this explains away the anomaly, but it actually serves to make the anomly prototypical: If a known conservtive activist has no trouble linking to articles that show a "liberal bias" then the "liberal bias" is simply an illusion, and pattern of citations that Groseclose & Milyo have detected do not reliably correlate with political outlook. On average, other considerations will be far more important than "ideology" defined by the Groseclose/Milyo measure, not just for Drudge, but for everyone in general.

This is not to say that there can't be unconscious acceptance of a contrary bias. This certainly happens, and has even been documented by researchers who study these things a lot more rigorously than Groseclose & Milyo do.

But if the "bias" doesn't bother Drudge, then it's awfully hard to see why it should bother anyone else, except, of course, for those who see him as a hapless Clinton/Communist/Al Qaeda stooge.

by Paul Rosenberg on Thu Dec 22, 2005 at 01:39:59 PM EST

So . . . (none / 0)

. . . they're saying that Drudge "leans left" because Drudge links to news organizations which (as "everyone" "knows") are left-leaning?

Is this another case of the facts being biased against the right?

Yeah, I'm cynical.

by catastrophile on Thu Dec 22, 2005 at 03:58:16 PM EST
[ Parent ]

Biased Facts (none / 0)

Well, as Brian Eno and David Byrne once wrote:

Facts are simple and facts are straight
Facts are lazy and facts are late
Facts all come with points of view
Facts don't do what I want them to
Facts just twist the truth around
Facts are living turned inside out
Facts are getting the best of them
Facts are nothing on the face of things
Facts don't stain the furniture
Facts go out and slam the door
Facts are written all over your face
Facts continue to change their shape

I guess that qualifies.

by Paul Rosenberg on Fri Dec 23, 2005 at 01:33:50 AM EST
[ Parent ]

Re: Biased Facts (none / 0)

that's why I stopped making sense a long time ago

DAGGER

by goplies on Fri Dec 23, 2005 at 06:43:08 AM EST
[ Parent ]

the left (none / 0)

was successful at infiltrating the drudge report but it was too late in the election cycle to do any good.

DAGGER

by goplies on Fri Dec 23, 2005 at 06:36:44 AM EST

biggest flaw... (none / 0)

The biggest flaw in the study methodology is that they assume that if someone cites a source, then they're agreeing with that source.

When it comes to politicians, it's certainly plausible that someone would repeatedly cite a source they disagree with. (Rush Limbaugh, for example, goes on and on and on about PETA all the time. Because he hates them.)

When it comes to news media, it's also plausible that someone would repeatedly cite a source because that source is quotable, or works hard at media placement, or has a unique point of view, or because that media person is lazy and has a small rolodex.

Just because you quote someone doesn't mean you agree with them. And that's the fatal flaw in the study.

by karichisholm on Sat Dec 24, 2005 at 05:47:39 AM EST

It's how conservatives do research (none / 0)

You start with the conclusion you want to reach, then structure your methodology to bring about that conclusion.

When I was short for time in University, and assuming I already had an opinion on a topic for a paper that was due, I would write the damn paper, then go through and find sources to back what I had already said. For a research paper, this is obviously not very scholarly as I should have read a variety of sources, thought about the subject and drawn conclusions to then write the paper with. What this study does is closer to my flawed method (which got decent grades, but I obviously wouldn't want to defend those papers now).

It's why the methodology on so much conservative "research" is so convoluted. They avoid actually lying (usually) but where the obvious and straightforward methodology does not suit their aims, they takes twists. They know that the media tends to only report on a study's conclusions and never really analyzes method.

I mean, who the hell sits down to study media bias and says "We'll arbitrarily define a set of groups as left or right, and then count the number of times each media source quotes them." As the poster above notes, conservative talking heads get a great deal of milage in bashing the ACLU and MoveOn. Hell, DailyKos bashes Bush all the time, I wonder if this study would call Kos right wing?

Any competent, honest researcher would also stop when they found their methodology had reached such surprising conclusions as "Drudge is leftist" and the "ACLU leans right" - surprising conclusions do not mean research is necessarily flawed, but you have to justify these answers vis-a-vis your methodology a lot better than to say (paraphrasing them) "people think drudge is right wing, but that's just a misperception, look our method shows he leans left!" You can't reference your own methodology to show that your methodology is valid.

Obviously the real test of a media source's bias is to ask: If I took a source 100% at their word, and accepted all the things they told me, where would I fall on the spectrum?" If these idiots can actually believe that someone doing that for Drudge would come away voting Democratic, I've got some prime land in Florida for them.

by scientician on Sat Dec 24, 2005 at 02:03:41 PM EST

You are not logged in.

In order to post a comment, you must be logged in. If you have a member account, please log in to comment.

If not, you can make an account right here. It's quick and free.