Why Russian Pollsters So Poorly Predict

Those who didn't follow the U.S. presidential election closely might be surprised to learn exactly who won one of the most exciting and memorable victories of all time, defeating a powerful, experienced opponent who is accustomed to holding the advantage.

But this election was actually won by analysts who use modern methods of data analysis to predict the results of elections. It was also won by political consultants who use those findings to fashion their strategies. The losers in this election were other analysts who spoke on television and wrote newspaper columns and who recklessly based their predictions on their unreliable "sixth senses."

Predicting an election outcome is more than just correctly guessing the winner or even guessing the correct point spread between the various candidates. For a prediction method to be successful, its methodology must be understandable, it must be open to independent verification, and it should systematically produce accurate predictions. For several months prior to the U.S. election, a handful of experts analyzed voter data almost as soon as it came in, making a special effort to avoid the errors typically made when people unfamiliar with mathematical statistics interpret the results.

It is interesting that Republican candidate Mitt Romney's team of analysts fared far worse at this task than those working for the general public. The analysts employed by U.S. President Barack Obama not only disclosed more information about their strategies over the course of the campaign and more actively shared the details of their predictions, but they also were ultimately more accurate than Romney's as well.

This should be a lesson for Russian politicians, too. They pay a lot of attention to closed surveys and studies, including those conducted by security services and private firms. But the quality of closed research is significantly lower than that of open surveys, even when both types are conducted by the same firm.

In general, Russian polling firms produce research of very low quality. In December, both Kremlin-linked and independent pollsters were correct in the sense that they predicted the official tally in advance. Embarrassingly, hard statistical analysis in two papers already accepted for publication by the Proceedings of the National Academy of Sciences demonstrated that the official tally was way off the real returns.

It is difficult to imagine just how thoroughly U.S. specialists will analyze data for economic or other purposes. Similarly, highly competitive election campaigns, which need every possible vote, use the most modern models and techniques available. There are many independent firms that work to achieve high-quality predictions in political contests to later win lucrative commercial orders from business clients.

This isn't done in Russia, but not because domestic businesspeople are ignorant. In fact, experience shows that Russians are quick to take techniques and methodologies developed abroad and adapt them for domestic use whenever the need arises. The reason is simply that Russian businesses have higher profit margins because of low competition and rarely need to rely on ultra-precise data analysis. But that is only for the time being. When competition increases in Russia, domestic businesses will be thankful that modern methods of statistical analysis were perfected during the 2012 U.S. presidential election.

Related articles:

• Obama Says He's Willing to Visit Russia
• Estonia Opts for Online Voting; How About the U.S.?
• Duma Lawmaker Slams U.S. Vote as 'Unfair'