Healthy vaccinee bias: loud and clear in an ONS analysis (UK)
The Office for National Statistics (ONS) in the UK published data on Covid and non-Covid mortality, by vaccination status, which many readers might interpret as evidence for vaccine efficiency against death. That’s not new. Over the past two years we have seen similar comparisons of vaccinated and unvaccinated, from many countries, often interpreted as vaccine efficiency studies — explicitly or implicitly.
These interpretations are mistaken.
Let’s see why, in the ONS data, and what we can really learn from the graph below.
There are many lines in the graph, but for the time being focus on just two:
Dashed orange line (top): age-standardized non-COVID death rate in unvaccinated
Dashed blue line (below): age-standardized non-COVID death rate in ever vaccinated
Do you see something peculiar?
The (age-standardized) death rate from causes other than Covid is lower in ever-vaccinated than in unvaccinated — throughout.
Do you think that Covid vaccines protect against death from diseases such as heart attack, stroke, or cancer? Of course not. So why is the death rate from non-Covid causes lower in those who were vaccinated against Covid than in those who were not?
It is simple. Vaccinated are healthier than unvaccinated, on average, so their mortality rate is lower. This observation was made long ago in US studies of the flu vaccine (here and here), and more recently in a US study of Covid vaccine recipients (here and here).
So, when we try to compare the death rates from Covid in ever vaccinated vs. unvaccinated, we have a problem. At least part of any observed benefit is not benefit. Vaccinated people are expected to have lower Covid mortality than their unvaccinated counterparts — even if the vaccine has no effect at all — because they are healthier to begin with. They are less likely to die from “everything”, including Covid.
We cannot infer about the efficiency of Covid vaccines from a naïve comparison of Covid mortality in vaccinated and unvaccinated. These comparisons are contaminated by the “healthy vaccinee bias”, which is a type of “confounding bias”. (The latter is taught in epidemiology, biostatistics, and research methods.)
Is the ONS graph completely useless?
Not at all. We can use the graph to demonstrate what happens after crude (rough) correction for the healthy vaccinee bias.
To form a reasonable comparison of Covid mortality in vaccinated and unvaccinated, we first need to estimate the magnitude of the “healthy vaccinee bias”. Then, we can make a bias-corrected comparison.
How do we estimate the bias, say, at some time point?
If vaccinated were as unhealthy as unvaccinated, on average, their non-Covid mortality rate should have been the same. Therefore, the difference in non-Covid mortality between vaccinated and unvaccinated is an estimate of the bias (when estimating vaccine efficiency against Covid death.) For example,
Non-Covid mortality in unvaccinated: 1,500 per 100,000
Non-Covid mortality in ever vaccinated: 1,000 per 100,000
The bias when estimating vaccine efficiency: 500 per 100,000
In this example, an approximate correction for the “healthy vaccinee bias” requires us to raise the Covid mortality rate of the ever-vaccinated by 500 per 100,000. That’s their expected rate of Covid mortality, if they were “just as unhealthy as the unvaccinated”.
Of course, that’s not precise, rigorous correction. There are formal research methods to handle confounding bias (to deconfound). Nonetheless, it is a first-order correction of an otherwise biased comparison.
To summarize, to learn something about the vaccine efficiency we need to compare two rates:
O: Observed Covid mortality in unvaccinated
E: Expected Covid mortality in vaccinated, if they were as unhealthy as unvaccinated.
In the graphs below, I show graphical estimates of the bias (bidirectional arrows) at some time points and use those estimates to draw a rough line of E.
Compare the new, raised, solid blue line (expected Covid mortality in ever vaccinated) to the solid orange line (observed Covid mortality in unvaccinated). We do not see striking benefit of being ever vaccinated, contrary to what might have been implied from naïve, biased comparison. Any possible benefit is modest and confined to three months.
In the next pair of graphs, I show the same bias-corrected results for the booster dose. Not only do we not see any benefit, but the opposite is observed.
Last, but not least.
The ideas I wrote here are included in a thorough analysis of the ONS data by Will Jones, editor of the Daily Sceptic (my italics):
“This means that the unvaccinated are dying of non-Covid causes at a rate up to 50% higher than the vaccinated. Since the vaccines cannot elevate the death rate of the unvaccinated, and assuming they are not miracle drugs that ‘cure all that ails you’, this is evidence of heavy bias. It may be a healthy vaccinee bias or a population estimate issue (as Igor Chudov argues), but once again it is an artefact confounding comparisons between vaccinated and unvaccinated. Since the gap between vaccinated and unvaccinated Covid deaths (solid orange and blue lines) is on a similar scale, this raises questions about how much of that difference is also bias.”
Will provides a link to a tweet by Dr. Clare Craig, in which she wrote (my italics):
“Bias in non-covid mortality is ~= alledged covid mortality benefit from Vx.”
Indeed.
