The Perils of the Poll Position

Tweet

Everyone’s a poll cat in this turbocharged election year, trying to divine an outcome we won’t realize until late Nov. 4 — if then.

All through the interminable series of primary elections, it was who’s ahead, who’s behind, who’s gaining, who’s slipping — and why.

A dozen or more polling firms — several associated with major networks, newspapers or magazines — were pleased to answer those questions for you, day by day, week by week, with dubious accuracy.

Now that the two multicandidate fields are down to their “presumptive” nominees, it’s head-to-head combat for the next four months, with new polls popping up every day.

We must approach all this with extreme caution, right down to the final three or four weeks of the campaign — and even then remain wary.

First of all, polling this early — the classic “snapshot in time” — is rarely predictive. At this writing, the conventions have not taken place, the vice presidential choices have not been made and the all-important media blitzes are barely under way.

Historically, we’ve seen roller coaster swings in the polls — as many as 16 points in either direction — from preconvention weeks well into September. Occasionally, a clear trend or pattern emerges but not often.

Even as the campaigns take shape, accurate reading is very difficult because of inherent problems involving sample size and margins of error, determination of “likely” versus “registered” voters and the almost hopeless task of estimating turnout.

Take the first issue: margin of error.

We know that the size of the random sample more or less determines relative accuracy. The major national polls sample about 1,500 voters, giving them an error margin of about 3 percent. Some sample 1,000, with accuracy of 4 percent. If the sample drops to 100, the margin of error grows to 10 percent.

Polling firms freely give you the sample size and the margin of error. So what’s the problem? The margin of error itself.

For example: Most polls in June and early July gave Barack Obama a lead of about 6 points. With a 3 percent error, that means he might be as many as 9 points ahead or as few as 3, which is quite a spread — from a horserace to a landslide. Did you need a poll to tell you that?

But it gets worse.

The poll might purport to show how John McCain is doing among, say, Latinos — who represent as little as 6 or 8 percent of the sample based on registration and expected turnout. The margin of error then rises to a larger number than the population proportion. It’s absolutely useless information (although some pollsters take a separate sampling of ethnic groups or demographic niches).

Nevertheless, in early June, several cable channels raved for days about a problem Obama allegedly had with suburban women. One problem: The subsample, or cell, was so small the margin of error was 10 percent — but no one bothered to mention that.

More vexing is the designation of “likely” versus “registered” voters.

Some polling firms give results for registered, others for “likely” and some for both. The media may report both but focus on one or the other for “news value.” It depends on how the editor decides to play the race — narrow or wide.

But hardly ever do the pollsters tell us how they determined “likely” voters — what they call a screen. Some firms have tighter screens than others — with varying assumptions about how different groups will turn out.

Screens are considered proprietary information, so they rarely disclose the details, although they all try to make assumptions based on voting history.

This led to massive error in the past. One determinant of whether you are likely to vote is if you voted in a recent election. But what if you’re a new registrant: Are you more or less likely to turn out? Recent history says more, but some firms assume less.

Historically, Democrats polled better among all registered voters than among tightly screened “likelys” because African Americans, Latinos and younger voters, which lean Democratic, under-performed in turnout. Suddenly, they are over-performing!

What pollster could have anticipated the massive registration and turnout of those demographic groups in this year’s primaries?

This leads to further cautions:

First, it is far, far too early to make a good estimate of who the likely voters may be, so no screen, no matter how well tested, can be considered trustworthy.

Second, organizations such as CNN and Web sites that present an average or “poll of polls” are mixing apples, oranges and bananas — registered and likely voters alike, both live and automated telephone polls plus dicey Internet polls.

Averaging polls, with all their variables, conflicting sample sizes and techniques, sounds like a good way to level out error, but it is fraught with potential miscues.

As a gross example, take the February California primary election, where one pollster found Hillary Clinton ahead by 10 points while another said Obama was winning by the same amount. Average them and you had a dead-even race — but, as we now know, Clinton won by 10 points.

The most famous “averaging” Web site, Real Clear Politics, in 2000 showed a final winning margin of more than 3 percent for George W. Bush — but, as we know, Al Gore won the popular vote by more than a half-million, or 0.51 percent.

Which leads to yet another caution:

National polls at best give us a hint of the popular vote, but the lesson of 2000 is the most votes do not always win. Gore won the popular vote but lost the Electoral College by double digits. On the other hand, if John Kerry had carried Ohio in 2004, he would have lost the popular vote by some 3 percent but won the electoral vote.

Therefore, state-by-state polls give us a better picture, though most of those state polls use smaller samples than the national polls and are subject to similar problems. We saw huge variations and little accuracy among the polls during that exhausting series of primaries.

So, be a poll watcher if you will but remember we’re in the very early innings of a long game. Add many grains of salt to your political diet.

Sign up for our free e-newsletter.

Are you on Facebook? Click here to become our fan.

Add our news to your site.

Tweet