A Lecture in an Epidemiology Course (Winter 2040)
Good morning,
Twenty years ago, the world faced a viral pandemic called the covid pandemic (then, COVID-19), which mainly affected the elderly and was blown out of proportion. The virus was created in a laboratory as part of foolish and dangerous gain-of-function research.
Many of you were too young to recall the details but one significant event was the development of an mRNA vaccine, now called gene therapy. Not only was it developed quickly but it was also tested quickly and claimed to have been highly effective against death from covid, based on what they then called “real-world studies”. There were no randomized trials with a mortality endpoint.
As we now know, the new gene therapy was far from highly effective. “Real-world studies” were bias-laden observational cohorts, and effectiveness was temporary and mediocre at best. If many lives were saved by those injections, they were saved in hypothetical models, not in mortality statistics.
Twenty years later we are still studying the long-term morbidity and mortality consequences of disseminated lipid nanoparticles (the mRNA carriers), self-manufactured toxic spike protein and aberrant proteins in various tissues, elevated levels of IgG4 antibodies after repeated injections, and the integration of foreign DNA fragments into the genome.
Today, we will examine the first study that reported effectiveness against death from covid of 84%, or 72%, or 62%, or 44% — following the first injection — and learn a few lessons.
Relying on data from the largest health care organization in Israel, the article was submitted and published online in February 2021, only two months after the beginning of the vaccination campaign.
First lesson: You should always ignore the name of the journal, the names of the authors, and the phrase “peer-reviewed”. Neither is an indicator of valid results. Biases in observational studies are tricky to detect and remove, and at that time few researchers understood the significance of the healthy vaccinee phenomenon (a type of confounding bias) and differential misclassification of the cause of death (a type of information bias). Both are common knowledge today among epidemiologists, thanks to slowly released data on non-covid death by vaccination status, and reviews of death certificates from that time vis-à-vis linked hospital records.
Second lesson: Never trust a study that shows estimates of effectiveness against death that range from 44% (lower 95% confidence interval: -36%) to 84% (upper 95% confidence interval: 100%) — within a maximum follow-up of about one month. The inference is overly sensitive to analytical decisions and the typical reason is sparse data.
There were only 41 reported covid deaths in the large cohort (about 600,000 matched pairs), or 59 in another analysis, and many of them were not death from covid as we’ll see later. That other endpoints were common does not matter. No endpoint may substitute for death.
You are probably surprised that the authors estimated effectiveness on the basis of such a small number of deaths and thereby affected public health policy for billions. That was unheard of before the covid pandemic and is unheard of these days. But you need to understand the authors’ mindset in the context of that time. Excellent researchers and the mainstream media were heavily biased towards anything that inflated both the significance of the pandemic and the effect of a new vaccine. It was acceptable to publish favorable results from sparse data.
Third lesson: When overwhelmed by numbers, models, tables, graphs, supplementary material, and sophisticated analytical decisions, check what you find in a simple computation. I am not saying that a “crude” analysis cannot be misleading, but you may sometimes find it informative enough. A simple analysis of the mortality data is what we will do next.
Let me remind you, first, that any causal inference is derived from assumptions, some of which are trivial (e.g., integrity of data files); others are more complicated. The question at hand is this: Under reasonable assumptions, is the data compatible with near-null effectiveness against death, rather than 44% to 84%?
The answer is “yes”.
I will make two assumptions:
1. No covid death could have been prevented within the first two weeks of an injection, so any observed benefit of the first dose before day 14 is completely explained by bias.
2. The biases that operated in the first two weeks continued to operate at later follow-up intervals.
The authors accepted the first assumption. Their estimates of effectiveness in the main analyses excluded the first 13 days of follow-up. They wrote:
“The period immediately after the first dose, when immunity is gradually building, was excluded in the main analyses because the risk ratio is expected to be close to 1 during this period.”
Two graphs of cumulative covid mortality were presented: one in the main article (left); another in a supplementary appendix (right). Below each graph, I computed the risk ratio of death in three consecutive two-week intervals.
Skipping the first interval, vaccine effectiveness (one minus the risk ratio) ranges from 44% to 76%, similar to the range of estimates that was reported by the authors (44% to 84%). In this case, a simple analysis of the sparse data largely agrees with sophisticated analyses. It was good enough.
Unlike the authors, however, I did not discard the data from the first two weeks as “a temporary increase in events among unvaccinated”, which was no more than wishful thinking. Rather, I assumed that biases that were operating at that time did not miraculously vanish.
Whichever they were, their collective magnitude may be estimated by the bias factor — the multiplier that restores the expected null effect (risk ratio=1) in the first two weeks. It was 3 (left table) or 2.3 (right table).
As you can see above, applying the bias factor correction to estimates of the risk ratio in the next two-week intervals has eliminated the pseudo benefit of initiating the two-dose vaccination protocol. We observe a typical random spread around a near-null parameter: 0.72, 1, 1.2, 1.3. And if we correct the authors’ estimates for a bias factor of 3, we get the following spread: 0.48, 0.84, 1.1, 1.7.
Which biases were at fault and what evidence do we have to infer their persistent existence?
There were at least two: misclassification of the cause of death, and the healthy vaccinee phenomenon.
Broadly speaking, misclassification means that some covid deaths were mistakenly classified as non-covid deaths, and some non-covid deaths were mistakenly classified as covid deaths. We’ll focus on the latter case, which was far more common.
At that time, it was natural and financially rewarding to attribute deaths to covid, correctly and incorrectly. In Israel, for example, half of reported covid deaths during the vaccination campaign did not contribute to excess mortality, which means that those people would have died regardless of their positive PCR test. They did not die from covid, and a covid vaccine could not have saved them.
It follows that about 20 of 41 deaths in the study (or 30 out of 59) were not due to covid. If so, the study has estimated the magnitude of biases (the pseudo effect on non-covid death), as much as it has estimated effectiveness (against covid death)…
That many reported covid deaths were not caused by the virus is also evident from the distribution of the time to death in the study. The median was only 11 days after a positive PCR test (top figure), shorter than the typical distribution after the onset of symptoms (bottom figure) — a median of 19 days — even if testing happened 1–3 days after symptom onset. In other words, the distribution was shifted to the left, as compared with what we expect to see in the case of true covid deaths.
Why was it shifted? Because many deaths had other causes. These were deaths of patients who were hospitalized for various reasons and had an incidental, positive PCR test on admission. Keep in mind that at least 50% of the infections were asymptomatic, and the vaccination campaign coincided with a winter covid wave.
So, we have clear evidence of misclassification of the cause of death, but it was worse. Misclassification was differential, meaning “dependent on vaccination status”.
Misclassification was differential because PCR testing was not applied uniformly. Vaccinated people were less likely to be tested than their unvaccinated counterparts, for two plausible reasons: First, some physicians and some vaccinated people might have attributed covid symptoms to “reactogenicity” — covid-like symptoms following vaccination — so PCR testing was not done. Second, and more important, it was assumed that the gene therapy was highly effective, so why bother to do a PCR test in vaccinated. Furthermore, such testing was openly discouraged.
Differential misclassification of infection status was carried forward to other endpoints, including death. Even though covid deaths at that time were over-recorded overall, those of vaccinated were less likely to be recorded than those of unvaccinated. I know, it’s a little complicated. Anyway, the result of testing bias is obvious: a lower rate of reported covid death in vaccinated people — pseudo effectiveness.
Are you asking about all-cause deaths in the study?
The data was available to the authors but was not reported. Actually, non-covid deaths have been consistently hidden in most papers from that time. Covid vaccine research was heavily biased, consciously or subconsciously. I know, it is difficult to believe.
Differential misclassification of the cause of death was combined with another strong bias, widely appreciated these days: the healthy vaccinee phenomenon. People who were vaccinated were healthier than their unvaccinated counterparts, and standard methods of adjustment failed to fully remove that bias.
Back then many researchers dismissed the bias as a temporary distortion: people who were sick delayed vaccination until they recovered, and those with short life expectancy were not vaccinated.
That was true, of course, but the healthy vaccinee phenomenon is broad and prolonged. For various psychosocial reasons, people who were vaccinated, against flu or covid, were healthier to begin with. As a result, they were less likely to die from covid and from non-covid causes, both of which have made up those 41 or 59 deaths in the study. The healthy vaccinee phenomenon, coupled with differential misclassification, easily explain the “effect” on death. Neither bias disappeared after 13 days of follow-up.
Misclassification was rarely mentioned at that time, but everyone at least paid lip service to the possibility of confounding by unmeasured health characteristics. And there were other sources of misleading inference, which we will not discuss today. A perfect storm of biases was operating in that study and in countless “real-world studies” afterwards. In fact, the healthy vaccinee phenomenon alone was sufficient to create the illusion of effective vaccine and booster doses in the frail elderly.
Are you wondering whether any of that was exposed or suspected in “real time”?
Yes, it was. But not in biomedical journals or in mainstream media. Those who tried to criticize the novel gene therapy, for which a Noble prize was hastily awarded, were called anti-vaxxers. Doubting the safety of the injections was patronizingly labeled “vaccine hesitancy”. Most of the world was brainwashed.
Powerful forces have derailed the normal course of biomedical science, and it took many years to get us back to where we are now. Perhaps that’s the most important lesson for you today. “The science is settled” is always fake news. Do not let anyone censor again a scientific exchange.
Let me end today’s lecture with an insightful quote of Karl Popper, a 20th century philosopher of science, with my additions in brackets.
“There are all kinds of sources of our knowledge; but none has authority… The fundamental mistake made by the philosophical theory of the ultimate sources of our knowledge is that it does not distinguish clearly enough between questions of origin [e.g., data analysts from Harvard wrote so in The New England Journal of Medicine] and questions of validity [Did their study indeed show protection against death?].”
