Polling Problems

Our chapter on public opinion discusses the challenges facing pollsters.  Several recent articles touch on this subject.

At National Journal, Steven Shepard looks at declining response rates (see a May post) and increasing numbers of cell-only households

Former Democratic pollster Mark Blumenthal, cocreator of the site Pollster.com, examined a number of Gallup polls earlier this year. Blumenthal found that Gallup registers lower approval ratings and ballot-test performances for President Obama than other survey houses—differences that are small but statistically significant.

...

Gallup Editor in Chief Frank Newport defended his methodology in the article and in an interview withNational Journal. “One of the biggest changes in recent years, even as our response rates go down, we have millions of people, tens of millions who are desperate to give their opinion,” Newport said.

Incoming AAPOR President Paul Lavrakas said in a telephone interview with NJ that phone polling is “nowhere near dead, nor do I expect it to be dead for several decades. I don’t see that anytime soon we’re going to stop using random-digit-dial surveys in America,” he added.
Lavrakas also said that one reason telephone polling about elections remains fairly accurate is that the same individuals who are more willing to be surveyed also vote more often.

John Harwood writes at The New York Times:

Bill McInturff, a Republican pollster, and Peter Hart, his Democratic counterpart, who conduct the NBC News/Wall Street Journal poll, proved the point in their latest poll, conducted July 18-22, when they increased the proportion of respondents who rely exclusively on cellphones to 30 percent from 25 percent. To home in on them, the pollsters ended calls answered on cellphones if the respondents said they also had land lines.

Their findings affirmed arguments that “cell only” Americans have significantly different, and more Democratic, political views than those with land lines. Over all, the poll showedMr. Obama leading Mr. Romney by 49 percent to 43 percent — providing a confidence-boosting talking point for Democrats and provoking sharp criticism from Republicans.

Scott Rasmussen, who owns an independent polling firm, approaches the “cell only” problem differently, as he must by law. His Rasmussen Reports conducts surveys through automated dialing, which under Federal Communications Commission rules is permitted for land lines but not cellphones.

So in Mr. Rasmussen’s polls, online interviews account for 15 percent to 20 percent of each survey, which he figures helps him reach the same kinds of voters, especially younger ones, in the “cell only” category. The result he reported the morning of July 25, a few hours after the NBC News/Wall Street Journal poll was released, was strikingly different: Mr. Romney had 47 percent, and Mr. Obama 44 percent.

“Nobody has the answers,” Mr. Rasmussen said of different approaches to the issue. “We’re all experimenting with the same thing. How do you reach people in a way they communicate?”

At The Hill, David Hill adds:

The unique challenges of 2012 don’t stop with cellphones. We’ve always known that the hardest thing to do in polling is predict whether a respondent will actually vote or not. Frankly, asking voters directly is the least effective means of making that judgment. Past research has demonstrated that the best ways to judge likely turnout are based on facts gleaned from a voter’s recorded history. How long has someone been registered to vote at his or her current address? (The longer, the better, for turnout.) And did he or she vote in the last election of the same type, in this case the 2008 election? A potential problem using these criteria in 2012 is that 2008 brought out hordes of new voters who seemed mostly motivated by anti-Bush sentiment and the pro-change imagery of Barack Obama.

Is there anything to give us assurance that the new voters of 2008 will return in 2012, like the models say they should?

I am starting to see uneven evidence for this. In some states, like California, they might. In other states, say, Iowa, they might not. So a lot of pollsters could get fooled on turnout.