National Journal: Don’t Shoot The Pollsters

National Journal: Don’t Shoot The Pollsters
Reaction To NBC’s Polling On The Public Option Misses The Whole Story
by Mark Blumenthal
Monday, Aug. 31, 2009

(Click the link to read the entire article.  Below are some of the key paragraphs.)

Like predictions? Here’s an easy one: When the political winds shift and polls show bad news for a political candidate or cause, pollsters will become the object of partisan attack.

It happened just last week when a new NBC News poll found considerable public skepticism about Democratic proposals for health care reform. The office of House Speaker Nancy Pelosi put out a statement condemning the poll under the headline, “Bad Analysis, Worse Question,” and the Huffington Post’s Sam Stein noted the “longstanding ties to the health insurance industry” of Bill McInturff, the Republican pollster who conducts the NBC News/Wall Street Journal poll along with Democrat Peter Hart.

In June, the NBC/Wall Street Journal poll devoted 14 questions to health reform, three to the proposed public option. The first question asked respondents to rate how “important” it is to “give people a choice of both a public plan administered by the federal government and a private plan for their health insurance.” More than three-quarters considered this choice “extremely important” (41 percent) or “quite important” (35 percent).

They followed up with a question that presented two sentences arguing for the public option and two sentences arguing against, and asked respondents which they agreed with more. A narrower plurality (47 percent to 42 percent) expressed support for the public plan.

According to McInturff, he argued at the time that first question “doesn’t measure support for the program, it simply measures Americans’ desire to have more options,” while the follow-up would better reflect the political “dialogue” that Americans would hear over the summer.

In their July poll, Hart and McInturff dropped the question about choice, opting instead to ask a simpler favor-or-oppose question about the public option that they intended to track, along with other more general measures, for the remainder of the debate. They found 46 percent of adults in favor and 44 percent opposed to “creating a public health care plan administered by the federal government that would compete directly with private health insurance companies.”

They repeated the same question two weeks ago and found 43 percent in favor and 47 percent opposed, which NBC News described as a “shift” from the July results. The critical firestorm ensued, focusing on two complaints: First, that the observed change since July was not big enough to attain statistical significance and second, as Stein put it, that “the pollsters dropped the word ‘choice’ in their July and August polls.”

Critics were right to question the characterization of the difference between the July and August results as a “shift,” since it was not large enough to attain statistical significance. Still, that conclusion was a very minor piece of NBC’s reporting, and other surveys have tracked significant increases in general opposition to reform proposals over the summer months. Also, the most recent ABC News/Washington Post survey showed a 10 percentage point drop (from 62 percent to 52 percent) in support for “having the government create a new health insurance plan to compete with private health insurance plans” between June and August.

But the fury over “dropping the word ‘choice'” has less merit. First, the pollsters did not so much drop one question as add another. They did not intend for their first question to serve as a measure of support for the public option. Their reporting never portrayed it as such, nor did it treat the new question as comparable to the old. Both ABC News polling director Gary Langer and blogger Nate Silver agree that the initial “importance” question was an inferior choice to measure support for the public option.

What I find unreasonable and unsupported are the attacks on McInturff’s character, and the insinuation that he “biased the results” to serve a client, America’s Health Insurance Plans, that opposes the public option. Among other things, this smear implies that Peter Hart — who, as Stein recounts, serves numerous labor clients that support the public option — was somehow asleep at the switch.

The NBC/Wall Street Journal poll works precisely because Hart and McInturff bring the experience of working on behalf of political campaigns and advocacy groups.

Public Opinion Strategies