The West Needs to Measure Russian Election Meddling. Here’s How

An article over the weekend in The Sunday Times says that a parliamentary report in the U.K. has concluded that it was impossible to quantify the impact of Russian efforts to influence the 2016 Brexit referendum. That largely matches research on the 2016 U.S. election by the eminent communications expert Kathleen Hall Jamieson, who has nevertheless argued that Russia played an important role in President Donald Trump’s victory.

This doesn’t mean, however, that foreign propaganda’s impact couldn’t be measured quite precisely in the future — if, that is, anyone really wanted to quantify it.

The U.K. Parliament’s Intelligence and Security Committee oversees the British intelligence services, and its report on Russian interference uncovered by The Sunday Times would be the rough equivalent of the U.S. intelligence community’s 2017 assessment of Russian activity in the run-up to the 2016 presidential contest. The British government, however, has refused to release the report before the parliamentary election scheduled for Dec. 12, and has been widely criticized for it, including by Hillary Clinton, the 2016 loser in the U.S. The critics wonder if the government is trying to protect information concerning wealthy donors to the pro-Brexit campaign and the ruling Conservative Party.

The leak in The Sunday Times doesn’t reveal any new information on that matter, but it shows that the report tried to assess the influence of the Russian propaganda outlets RT and Sputnik. The Intelligence and Security Committee calculated that 260 anti-European Union articles published by the two outlets in the six months before the Brexit referendum were retweeted so widely that they could have been seen up to 134 million times, about three times the combined number of Twitter impressions generated by the two biggest pro-Brexit campaign groups, Vote Leave and Leave.eu.

But did people who saw the material change their mind about how they should vote on Brexit, or did those who agreed with the anti-EU propaganda turn out in greater numbers than they would have done otherwise? Those are the billion-dollar questions for those trying to figure out whether Russia helped cause the Brexit mess or put Trump in the White House.

Jamieson, the University of Pennsylvania professor for whom the study of media inputs into voting behavior is an academic specialty, made a valiant attempt to answer these questions in her 2018 book, “Cyberwar: How Russian Hackers and Trolls Helped Elect a President: What We Don't, Can't, and Do Know.” The point Jamieson made in that work was that based on previous studies, the Russian trolls and hackers probably did affect the outcome, including by mobilizing potential Trump voters, discouraging liberal voters who weren’t keen on Clinton and shifting traditional media’s agenda in the final phase of the campaign.

She also wrote, however, that quantifying the impact of the Russian activity was impossible in the absence of “real-time, rolling cross-sectional polling data tied to media messaging and exposure in each of the three decisive states,” Michigan, Pennsylvania and Wisconsin. Even with such real-time panels, she wrote, it would be hard to separate the effect of specific Russian propaganda efforts because of the difficulty of finding a control group not exposed to them. Jamieson recalled how she and her colleagues analyzed the impact of individual ads in the 2000 election by comparing data from battleground and non-battleground states. That was no longer possible in 2016:

Because we lack a reliable way to locate either internet advertising and messaging or those exposed to it, and, in the case of media coverage of the hacked content, the entire nation was exposed to the resulting reporting, our 2000 model no longer works.

None of these obstacles would be insurmountable, though, if dedicated researchers, or, indeed, governments, now that foreign interference — be it by Russia, Iran or the U.S. — is in everyone’s sights and considered to be an important threat to democracy. 

A representative panel of both voters and nonvoters would be needed, citizens who would be willing to allow their social-media feeds to be tracked. Given what researchers already know about the propaganda networks run by various states and contractors, it wouldn’t be difficult to document the spread of propaganda and the panel members’ exposure to it. Panelists could be polled at regular intervals to check how the content has affected them. By the end of the project, researchers wouldn’t even need to know how they voted — it would be enough to establish that they’d gone to the polls.

Let’s face it: Thanks to social media, it’s much easier for governments and private influencers to deliver propaganda to any corner of the world, regardless of what restrictions are placed on political advertising. RT and Sputnik don’t need to buy ads to generate tens of millions of impressions. Before any vote that has a bearing on Russian interests — and that means most European and U.S. national elections, as well as many in Asia and Africa — the Russian propaganda machine is going to manufacture lots of authentic-looking content, which will be spread by both paid trolls and true believers. 

That means opportunities will arise to measure the effect of these influence operations. In fact, many such opportunities already have been missed: Elections have come and gone without a serious effort to figure out exactly how Russian internet-based meddling has affected results.

In one recent analysis of 20 recent elections, Lucan Ahmad Way and Adam Casey of the University of Toronto wrote that “almost all cases of success by candidates whose policies dovetailed with Russian interference efforts can be explained by the actions of powerful domestic actors.” But separating the Russia effect from those actions would only be possible with the kind of real-time panels mentioned by Jamieson.

A tantalizing example of the kind of results they could produce came in a paper published last year by Leonid Peisakhin and Arturas Rozenas of New York University. They analyzed the impact of Russian television broadcasts on voting in Ukrainian elections, relying both on data on where these broadcasts could be received in Ukraine and on polling. Peisakhin and Rozenas found that watching Russian TV mainly strengthened existing pro-Russian attitudes rather than altering beliefs — but by doing so, it also mobilized support for pro-Russian candidates and parties at the ballot box.

Data-based research like this is what’s needed in Western societies to figure out the best responses to Russian propaganda and trolling. Do counter-propaganda efforts work? What would be effective in mobilizing voters who aren’t receptive to the Russian messages? Does it even make sense to push back against the Russian propaganda messaging, as an entire academic and media industry that has emerged since 2016 has maintained — or is the real impact of that messaging negligible? All these questions, bafflingly, remain unanswered three years after Trump’s victory and Brexit, and they can’t be answered on the basis of the limited data available about those two momentous events. What’s known about the suppressed U.K. report is further evidence of that.

This article was first published in Bloomberg.