1. News & Issues

Discuss in my forum

Polls and the "Margin of Wrongness"

How Accurate are State Polls?


How accurate are state polls? That's the question we will answer today, taking a look back at how the polls fared in the 2008 and 2010 election cycles. Heading into election day, polls show Obama slightly ahead in enough states to win the Electoral College handily. But does history predict a different outcome?

The "Margin of Wrongness"

Polls have what is called the margin of error, a "scientific" measure that concludes how likely any given poll's results will be within a certain percentage. A poll of 500 people will have a wider margin of error (and will be regarded as less accurate) than one with 900 people. For state-wide polling, that is the average range of poll respondents used to reflect the opinions of several million people.

For people who are into politics and polls, Real Clear Politics collects all sorts of political polls and tosses them together to create an average. In theory, the method of using many state polls would create a larger sample, and the average would then create a more accurate reflection of the state of the race. But it doesn't always work out that well as different polls use different sampling methodologies, have partisan tilts, or are simply just large outliers that can drastically alter the "average." While the margin of error likes to predict how close a race will be within a certain degree of likelihood, what I like to call the Margin of Wrongness actually see how far off the results are. Let's just say reality is often outside the Margin of Error.

How Close are State Polls?

There are a few way to answer this question. If success is measured by predicting the eventual winner of a state, they are usually pretty decent. But their success in predicting winners is usually limited to easy wins, where polls show one candidate or another with a very large lead. For instance, in 2008, polls showed Obama would win Nevada by 6.5 points, but he won by almost double that (12.4). They also showed he would win Iowa by 15.3 points, but that was well off as he won by 9.3. Both victories were outside both the statistical Margin of Error as well as the Margin of Wrongness, but they still "predicted" the winner. By that metric, they are okay overall, though not so great in close races. However, if success is determined by how closely they pick the margin, the picture gets a bit murkier.

When we take a look at 2008's closer presidential state races, and there were not many, the predictions are not very solid. That year, only Florida, Ohio, Indiana, Missouri, and North Carolina had the margin between Obama and McCain within just a few points. The polls incorrectly picked the winner in Indiana and North Carolina, and they were just barely saved in a third, Missouri. Overall, polls in the current 11 swing states were off by about 3 points in 2008. It didn't matter in the blowout states, which were enough to give Obama the presidency alone, but the results were essentially 50-50 in the close races.

If you were expecting a rebound year in 2010, you would be wrong. The margin accuracy in senate races that year actually got worse, up to being 4.3 points off. Once again, being off 4.3 points did not matter very much in the blowout races. But the RCP average, and pollsters in general, whiffed on the close races once again. In Nevada, Sharron Angle was supposed to unseat Democratic Senate Majority Leader Harry Reid by 3 points. She lost by 6. Likewise, every final poll had Ken Buck winning in Colorado by an average of 3 points, he lost by 1. In Alaska, Joe Miller was predicted to win (though polling more infrequent), but lost to a write-in candidate. In Washington, it was supposed to be a fraction of a point win for the incumbent, but she won by 4 points. Again, polls picked the wrong winner in about half of the closer, MoW cases. In 2010 Governors races, they unanimously picked the wrong winner in Illinois and Connecticut, and in Minnesota, a 5.2 point poll win was actually a .5 point win. Likewise, the polls were off by more than 4 points on the margin overall.

2012 Swings States within the Margin of Wrongness

Final national polls have, for the most part, seen a tie in the popular vote. This is already a big change from 2008 when polls were showing that Barack Obama was on a clear - and easy - path to victory. But state polls have given a slight edge to President Obama, and the poll averages predict he will win all but three states he carried in 2008: Florida, North Carolina, and Indiana. Indiana is not considered a swing state this year, and only a liberal pollster is keeping north Carolina within the MoW. Overall, Obama has slight leads in 9 of the 11 battleground states. Whether or not a national lead tie and a blowout win in the Electoral College is possible or even likely will be known on Tuesday, November 6.

But 2012 is very different from 2008 with these state polls. Although Obama was expected to win the Electoral college based on polls in both 2008 and now, the expected margins couldn't be more different than what we see today. Recall that in 2008, Obama was beyond the MoW in 8 of the 11 current battlegrounds: Colorado, Iowa, Michigan, Nevada, New Hampshire, Pennsylvania, Virginia, and Wisconsin. So even if the polls were off 3-4 points (and more than half of them were) it did not matter because Obama was so far ahead.

But this year, Obama is not beyond the Margin of Wrongness in a single battleground. All 11 battleground states fall below the 2009 and 2010 MoW, and 8 of 11 fall below the 2008 MoW. Remember, polls showing a margin of victory below 3-4 points over the last 4 years were just as likely to pick the wrong winner as the right winner, even if not a single poll projected the eventual winner. Judging from the last 2 election cycles, it is actually more likely that Obama will not win all 9 of the states he is currently within the MoW, than that he will win them. If Romney were to hold Florida (up 2 currently), North Carolina (almost out of the MoW at +4), and take the tied Virginia, Romney really would need just 2 or 3 of the remaining 8 states within the MoW. Over the past two election cycles, history would hint that he would take at least 4 of them.

©2014 About.com. All rights reserved.