Russia’s Claims of Sputnik V's 97.6% Effectiveness Under the Spotlight

Russia is once again facing questions over its coronavirus-related data after it claimed its Sputnik V vaccine has 97.6% effectiveness based on an analysis of infections among vaccine recipients in Russia.

The Russian Direct Investment Fund (RDIF), which is funding and marketing the vaccine, said Monday it had analyzed the number of coronavirus cases among people who had received the Sputnik V vaccine and the wider population, and had found the jab to be 97.6% effective — notably higher than the 91.6% efficacy reported in peer-reviewed Phase 3 clinical trials.

RDIF said it analyzed coronavirus infections among 3.8 million vaccine recipients which had occurred at least two weeks after a person had received their second dose of Sputnik V. This was then compared with the overall number of infections across Russia during the same period, between early January until the end of March. 

They found an infection rate of 0.027% among vaccinated patients, compared with 1.1% in the wider population — equivalent to effectiveness in preventing infection of 97.6%.

“This data confirms that Sputnik V demonstrates one of the best protection rates against coronavirus among all vaccines,” Kirill Dmitriev, RDIF said in a statement.

However, the findings have raised questions about the methodology used to arrive at the record high figure and seem to contradict previous official statements on how many vaccinated Russians have contracted Covid-19.

‘Embellishment’

An infection rate of 0.027% among the 3.8 million vaccinated patients implies around 1,020 individual cases across Russia during the monitored period. It appears the calculation has been based on taking the number of reported infections among people who have been vaccinated, and then dividing this by the total number of people who had completed the vaccination cycle by 31 March. 

If this methodology has been used, it could be misleading, experts warn, since not all 3.8 million people were protected for the full period under analysis. According to Russia’s patchwork vaccination statistics, more than half of the 3.8 million did not complete their vaccination cycle until the final two weeks of the period under observation. This reduces the actual timeframe in which somebody who is classified as “vaccinated” could have been infected with the virus, and can thus inflate efficacy, if calculations are not adjusted to take this factor into account.

For instance, it appears a person who received their second dose of the vaccine on March 16 would only qualify as vaccinated for one day of the observed period, which runs until March 31 — thus significantly reducing their opportunity to contract the disease and be included as a case among the vaccinated group. A controlled study would have taken a fixed group of vaccinated people — for example, the estimated 741,000 who would qualify as vaccinated at the end of February — and studied them over a set period of time against another unvaccinated group of people with a similar demographic profile.

After accounting for this, analysis by The Moscow Times and independent statistician Alexander Dragan, who has been closely tracking Russia’s vaccination statistics, suggests the real world effectiveness could be to 86-90% — slightly below the results Sputnik V’s Phase 3 clinical trials, which were published in The Lancet and showed an efficacy of 91.6%. 

This figure is based on calculating the infection rate after adjusting for how many days a person classified as vaccinated within the three-month period analyzed by RDIF. 

In The Moscow Times analysis, those vaccinated on the first day of Russia’s mass vaccination campaign received a weighting of 82 days, as they were vaccinated for the full 82 days in the period studied by RDIF, whereas those vaccinated on the final day would receive a weighting of just one day, as they were only protected for one day. This adjusts for the fact that those who received their vaccination toward the end of the studied period had significantly less time in which they could be exposed to the virus.

Based on Dragan’s estimates of the pace of vaccinations across Russia, that formula would give an efficacy estimate of between 86-90%. 

“This is still a very good result,” Dragan told The Moscow Times. “I don't understand why this embellishment is necessary — 85% and above is a really great number, especially when it’s based on such a large sample.”

'Very weak'

Other health experts and statisticians also questioned the RDIF claims.

“The whole idea of estimating the protective efficacy of a vaccine from this type of cohort study is very weak,” Vasiliy Vlassov, an epidemiologist at Moscow’s Higher School of Economics, and former adviser to the Russian Health Ministry and World Health Organization (WHO) told The Moscow Times.

“Even a useless vaccine would show some efficacy [due to] the cohort design,” and the “healthy user effect,” he added. “There is no possibility of controlling for that — it’s why we need randomized clinical trials.”

Statistician Mikhail Tamm, who also reviewed The Moscow Times’ analysis, similarly questioned the RDIF’s claims of such high effectiveness based on the study presented. He highlighted unknown factors such as testing levels, variance in vaccination and infection rates across different regions and likely different demographics among the vaccinated and unvaccinated groups as just some aspects that could have a notable impact on the vaccine’s actual effectiveness.

“It is very difficult to estimate the true rate without much more data and much more complicated calculations,” he said. “It could quite easily be around 80%, and it could quite easily be, say, 92%. But with the data we have, [86-90%] is definitely better than the one obtained by RDIF.”

Even at the lower end of those estimates, Sputnik V would still come out as one of the most effective coronavirus vaccines in the world.

Contradictory results

Health experts said the premise of RDIF's claim — that real world effectiveness of the vaccine would be higher than efficacy seen in clinical trials — was also unusual.

“Because the real world generally involves populations with health conditions that might impact how well a vaccine works,” said Judy Twigg, a professor at Virginia Commonwealth University, specializing in Russian healthcare policy. She also pointed to virus mutations which develop after the completion of clinical trials as a factor which can reduce the effectiveness of vaccines in the real world, compared to that observed in clinical trials.

RDIF did not respond to requests by the Moscow Times to share more details of the methodology used and the underlying data. In its published statement, the fund said it would publish full results in a peer-reviewed journal in May, when it is also expected to release its own research into the effectiveness of Sputnik V against new mutations of the coronavirus.

The analysis is based on data held in a Health Ministry confidential database tracking coronavirus patients and vaccine recipients. Russia has previously cited the confidential nature of the database as precluding sharing regular daily updates on the number of vaccines the country has administered.

But even without the underlying data, other official statistics also seem to counter the claims of 97.6% effectiveness published by RDIF on Monday. 

For instance, at the end of March, Moscow authorities said there had been “around 1,000” cases of vaccinated people catching the virus in Moscow alone — based on assessing those infected two weeks after receiving their second shot, the same criteria used by RDIF.

The RDIF statement Monday implies around 1,020 infections across the whole of Russia during the same period. That would suggest Moscow accounted for almost all the nationwide Covid-19 cases among vaccinated people reported by RDIF, despite only being home to around 640,000 of the 3.8 million. In other terms, the infection rate implied by the Moscow authorities’ figures is around 0.16%, Dragan said — or six times the nationwide level recorded by RDIF over the same period.

Separate analysis on the use of Sputnik V in Argentina, conducted by biologist Kirill Skripkin, implied effectiveness in the real world of around 80%, although this could be higher due to the country prioritizing jabs for medical personnel, who are more likely to come into contact with the virus.

Russia has been accused of launching an aggressive vaccine diplomacy drive around Sputnik V, which has been authorized for use in more than 60 countries. The vaccine developers recently criticized a U.S. study which showed the jab could be less effective against new strains of the virus, before accepting its findings and pointing out Sputnik V was more effective against new variants of concern than its rival vaccines. 

It also accused Slovakia of “sabotage” and a “disinformation campaign” after the EU member said a batch of 20,000 jabs it received from Russia differed from those currently under review by the European Medicines Agency (EMA).

For some, that campaign is backfiring, distracting from what many see as an undoubtedly impressive scientific achievement.

“It's unfortunate that they're undermining confidence in what appears to be a world-class vaccine, through a now-consistent pattern of exaggerated claims and incomplete data to support those claims,” said Twigg.

“What we see is the worst example of the manipulation of science for the purposes of marketing,” added Vlassov.