A recent study by Public Health England (PHE), of ‘real world’ data related to efficacy of the AstraZeneca and the Pfizer-BioNTech vaccines against two variants of SARS CoV-2, has attracted much attention among policymakers and the media. Genomic analyses of viruses isolated from persons vaccinated during the rollout of mass vaccination formed the basis for analyses. Efficacy of protection from the two vaccines against symptomatic Covid-19 was studied. In another study component, patterns of variants isolated from vaccinated and unvaccinated persons in the community were compared.
The manuscript has been placed in the public domain as a pre-print, i.e., it is yet to be peer reviewed. However, the reassuring message that the vaccines work against the two most-widely prevalent variants in UK, attracted attention. The variants studied were B.1.1.7 and B.1.617.2. As they are also the most prevalent variants in India, the study is very relevant to our country. For those confused by these numbers, B1.1.1.7 was first reported from Kent in UK and B.1.617.2 was first reported from Maharashtra.
The paper merits scrutiny because of the statistical fog created around the reported efficacy estimates, due to a small sample size in the AstraZeneca vaccine group. This is particularly problematic when estimates are presented of efficacy against the B.1.617.2 variant, which entered the UK later than the B.1.1.7 variant. The latter emerged earlier within that country. The sample size is larger for the Pfizer-BioNTech vaccine as it got regulatory approval earlier than the AstraZeneca vaccine in the UK.
The distinction between a point estimate of efficacy and a 95% confidence interval around that estimate becomes very important, while examining the reported results. A point estimate is the result observed in the study. Statistically, this result lies in a band of possible true values that extend on both sides of the observed value. That band gives the confidence that the true value lies somewhere within its two extremes, even if the observed point estimate doesn’t itself represent the true value. Since an observation may arise by chance alone, it is necessary to estimate this band of uncertainty. It suggests that if the study was repeated 100 times, 95% of the values will fall within that band. Larger the sample size of the study, narrower the confidence interval and lower the uncertainty.
The PHE study reveals the problems of wide confidence intervals associated with the small sample size of 1,054 people infected by B.1.617.2, as compared to the larger sample size of 11,621 infected by B.1.1.7. While headline messages give only the point estimates, it is essential to examine whether the uncertainty band (95% confidence interval) is narrow or wide. Else, we can generate misleadingly attractive point estimates even from a sample size of 10.
In persons receiving one dose of AstraZeneca vaccine, efficacy of 33.5% was reported against the B.1.617 variant. The 95% confidence interval around this estimate is 20.6-44.3%. Even the optimistic pole of this band (44.3%) is below the WHO prescribed protective threshold of 50% efficacy. In contrast, a single dose of the AstraZeneca jab had a point estimate of 51.1% efficacy against B.1.1.1.7, with a 95% confidence interval of 47.3-54.7%. Not outstanding, but also not disheartening.
Delayed spacing of the second dose, upto 12 weeks, was justified when we were dealing with the wild virus and a single dose provided >50% immunity upto three months. With the variants now dominant, we must consider a shorter dosing interval; 33% protection against B.1.617.2 is inadequate.
When we look at those who received two doses of this vaccine, we have a point estimate of 66.1% efficacy against B.1.1.7, with a comforting 95% confidence interval of 54-75%. However, against B.1.617.2, the doses are reported to show an efficacy of 59.5%, with an astonishingly wide 95% confidence interval of 28.9-77.3%. This puzzling level of uncertainty arises because of a small sample size. While 59.5% appears plausible (better than the 33.5% with a single dose), the long stretch of the uncertainty represented by the wide confidence interval makes it difficult to award a statistically defensible pass mark. The Pfizer-BioNTech vaccine fares better, both in higher point estimates of efficacy and narrower 95% confidence intervals. Against the B.1.617.2, two doses exhibit an observed point efficacy of 87.9% (with a 95% confidence interval of 78.2-93.2%}. Against B1.1.1.7, it is even better, with an efficacy of 93.4%, lying within a 95% confidence interval of 90.4-95.5%. Another study, reported in the same paper, showed that the B1.617.2 was more frequently associated with infection among vaccinated persons (40% more than B.1.1.1.7), while the distribution of the variants was balanced among the unvaccinated.
Taken together, the PHE studies reveal that while both variants appear to be subdued by the vaccines, the hardier B.1.617.2 variant is less responsive to both vaccines than B1.1.1.7 and that a single dose of AstraZeneca does not offer sufficient protection against B1.617.2. Even two doses of that vaccine are unable to convincingly demonstrate high efficacy against this variant, because of a small sample size, despite a reassuring point estimate.
These uncertainties ought to be resolved in India, where we have large numbers of vaccinations ongoing and a high number of infections, both among the vaccinated and the unvaccinated. By matching these data to genomic testing (which needs to be stepped up), we can provide better answers than PHE. Answers which are burdened with less uncertainty and inspire more confidence.
The author, a cardiologist and epidemiologist, is president, Public Health Foundation of India.
Views are personal.
