Research by the University of Waterloo’s School of Public Health and Health Systems says there is mandatory data regarding the efficacy of Covid-19 vaccines that is not being reported by manufacturers, which can mislead consumers and distort the interpretation of its efficacy in terms of infecion prevention.
“There may be much more complexity to the ‘95% effective’ announcement than meets the eye, or perhaps not. Only full transparency and rigorous scrutiny of the data will allow for informed decision making. The data must be made public.” These indicators have been omitted by official bodies against the express indication of regulators.
The document says that relative risk reduction and absolute risk reduction measures in the evaluation of clinical trial data are poorly understood by healthcare professionals and the public. “The absence of reported absolute risk reduction in COVID-19 vaccine clinical trials can lead to outcome reporting bias that affects the interpretation of vaccine efficacy.“
The article uses clinical epidemiological techniques to critically analyse efficacy reports from clinical trials of the Pfzier/BioNTech and Modern Covid-19 mRNA vaccines.
Based on data reported by the manufacturer of the Pfzier/BioNTech BNT162b2 vaccine, this critical evaluation shows:
- Reduction of relative risk, 95,1%; 95% CI, 90,0% a 97,6%; p = 0,016
- Reduction of absolute risk, 0,7%; 95% CI, 0,59% a 0,83%; p
For the Modern mRNA-1273 vaccine, the evaluation shows:
- Reduction of relative risko, 94,1%; 95% CI, 89,1% a 96,8%; p = 0,004
- Reduction of absolute risk, 1,1%; 95% CI, 0,97% a 1,32%; p
According to the published data, the unreported absolute risk reduction measures of 0.7% and 1.1% for Pfzier/BioNTech and Moderna vaccines, respectively, are much lower than the reported relative risk reduction measures.
As also noted in the BMJ, Pfizer/BioNTech and Moderna reported the relative risk reduction of their vaccines, but not the corresponding absolute risk reduction, which “appears to be less than 1%“. Absolute risk reduction (ARR) and relative risk reduction (RRR) are measures of treatment effectiveness reported in randomised clinical trials. As the ARR and RRR can be dramatically different in the same trial, both values must be reported to avoid biasing the results.
The public data, without the absolute risk reduction figures, were reviewed and approved by the list of members who sit on the Vaccines and Related Biological Products Advisory Committee (VRBPAC) of the U.S. Food and Drug Administration (FDA) for clearance.
“Ironically, the omission of absolute risk reduction measures from the data analyzed by VRBPAC ignores FDA guidelines for communicating evidence-based risks and benefits to the public.”
– The FDA’s directions to information providers say:
“Provide absolute risks, not just relative risks. Patients are unduly influenced when risk information is presented using a relative risk approach; this may result in limited decisions. Therefore, an absolute risk format should be used.”
“Manufacturers did not report absolute risk reduction measures in publicly disclosed documents. The U.S. FDA’s Risk and Benefit Advisory Committee (VRBPAC) did not follow published FDA guidelines for communicating risks and benefits to the public, and the committee did not report absolute risk reduction measures when authorizing BNT162b2 and mRNA-1273 vaccines for use in an emergency setting. Such examples of reporting mislead consumers, distort interpretation of the efficacy of the Covid-19 mRNA vaccine, and violate the ethical and legal obligations of informed consent,” says the paper published by the researchers.
The New England Journal of Medicine also published clinical trial data on safety and efficacy of the BNT162b2 vaccine and the mRNA-1273 vaccine, but with no mention of absolute risk reduction measures.
Why is this indicator relevant?
A 2018 review of 52 randomised flu vaccine trials studying more than 80,000 healthy adults shows consistent efficacy with a 40% to 60% reduction in flu reported by the Centers for Disease Control and Prevention (CDC). However, a critical evaluation of the data points to an overall ARR of only 1.4% – a vital clinical information missing from the CDC report. An absolute risk reduction of 1.4% works out to a number needed to vaccinate (NNV) of approximately 72 people, meaning 72 individuals need to be vaccinated to reduce one case of influenza.
The NNVs for the Pfzier-BioNTech and Moderna vaccines are 142 (95% CI 122 to 170) and 88 (95% CI 76 to 104), respectively, to reduce one infection,.
The researchers argue that the omission of absolute public health risk reduction results and clinical reports of vaccine effectiveness “is an example of outcome reporting bias, which ignores unfavourable results and misleads the public impression and scientific understanding of a treatment’s efficacy and benefits“.
“In addition, the ethical and legal obligation of informed consent requires that patients be educated about the risks and benefits of a health procedure or intervention.”
A limitation of this paper is that it only critically evaluated the efficacy of the mRNA vaccine in healthy individuals who were randomly placed into two groups under strictly controlled conditions. “The critical evaluation did not include the safety and efficacy of the vaccine in a general population that includes unhealthy people and lacks control over variable factors.”
This assessment is part of a special edition on the pandemic outbreak of the coronavirus.
