COVID vaccine research in Israel: no troubling questions please

Imagine you were reading a scientific article on an important topic in a prestigious journal. Two of the article’s co-authors are not only scientists but also methodologists — that is, they study how to analyze data and tell others how data should be analyzed.

As you are reading the article, you realize that the authors did not abide by their own methodological standards. So you write a letter to the editor, asking the authors to do just that and come back with answers.

“To the [Lancet] Editor:

The study by Barda and colleagues of a third dose of the BNT162b2 vaccine[1] excluded events that occurred during the first six days of follow up. The authors designed the study “to emulate a target trial”, but the methodological article they cite[2] (written by a co-author of Barda’s article) does not call for exclusion of early events. On the contrary, “With observational data, the best way to emulate time zero of the target trial is to define time zero to be the time when an eligible individual initiates a treatment strategy.”[2] What are the estimates when all events are included?

Although the study groups were carefully matched, residual confounding bias can never be excluded in observational research. One method to detect such bias employs “negative controls”, as eloquently explained by another co-author of Barda’s article.[3] In brief, researchers estimate the effect of the intervention on an outcome for which the effect is expected to be null. If the estimate is not null, residual confounding bias might also be present for the outcome of interest. For instance, “influenza vaccination was also “protective” against injury or trauma hospitalization…interpreted as evidence that some of the protection observed for pneumonia/influenza hospitalization or mortality was due to inadequately controlled confounding.”[3]

It is easy to apply the method to the study of Barda and colleagues:[1] What is the estimated effect of a third dose of the vaccine on non-COVID death, counting deaths as of day one? Is it null as expected? Of note, in a recent analysis COVID-19 vaccines showed unexpected “protection” against non-COVID mortality.[4]

I declare no competing interests.

Eyal Shahar, MD, MPH

Professor Emeritus of Public Health

References:

1. Barda N, Dagan N, Cohen C et al. Effectiveness of a third dose of the BNT162b2 mRNA COVID-19 vaccine for preventing severe outcomes in Israel: an observational study. Lancet, October 29, 2021 DOI:https://doi.org/10.1016/S0140-6736(21)02249-2

2. Hernán MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. Am J Epidemiol 2016; 183(8): 758–64

3. Lipsitch M, Tchetgen Tchetgen E, Cohen T. Negative controls: a tool for detecting confounding and bias in observational studies. Epidemiology. 2010;21(3):383–388

4. Xu S, Huang R, Sy LS, et al. COVID-19 Vaccination and Non–COVID-19 Mortality Risk — Seven Integrated Health Care Organizations, United States, December 14, 2020–July 31, 2021. MMWR Morb Mortal Wkly Rep 2021;70:1520–1524.”

Almost three months later you get a boilerplate response:

Manuscript number: THELANCET-D-21–07968
Title: Letter to the editor

Dear Dr. Shahar,

Thank you for submitting your Letter to The Lancet. Having discussed your Letter with the Editor-in-Chief, and weighing it up against other submissions we have under consideration, I am sorry to say that we are unable to accept it for publication. Please be assured that your Letter has been carefully read and discussed by the Editors. Thank you for your interest in​The Lancet, I hope this decision does not deter you from considering us again in the future.

Yours sincerely

Josefine Gibson
Senior Editor
The Lancet

I am assured that the science around COVID has been corrupted to an unbelievable extent. Anything that might threaten the “safe and effective vaccine” narrative will be suppressed as much as possible.

Official Israeli data raises more questions

The Lancet did not allow to probe into Barda’s article on the third dose of the Pfizer vaccine, but recently published mortality data by Israel Central Bureau of Statistics (CBS) provide interesting insight on what has happened at that time. It also reveal the quality of official Israeli data on COVID mortality, which is the source of the mortality outcome in the Lancet study and some other studies.

Stay with me while we go through the numbers.

The data below tell us about excess mortality in Israel during various periods, as estimated by CBS, as well as the corresponding number of reported COVID deaths (Table). Anything in red is my addition. The table in blue translates and magnifies the data that are contained in the red rectangle.

Israel started the vaccination campaign in December 2020. Therefore, the CBS data for July through October 2020 is a pre-vaccination summer/fall COVID wave (henceforth, “first summer wave”)

The third dose was administered as of mid July 2021 during a second summer/fall COVID wave. The Lancet study spanned about two months (July 30, 2021 through Sept 23, 2021), so most of the deaths in the study cohort are nested in the CBS period July through October 2021 (henceforth, “second summer wave”).

Compare the estimate of excess deaths in the first summer wave (pre-vaccination) with the estimate in the second (“booster” dose administered). They are virtually the same: 2,400 excess deaths (17%).

Two doses, six months or so earlier, did not help to reduce excess deaths in the second summer wave. But neither did the third dose! So how would we reconcile these data with Barda and colleagues reporting 81% third-dose effectiveness for COVID death? How could the booster have had such a strong effect, yet excess mortality remained unchanged?

Well, it turned out that the number of COVID deaths was reduced. In the first summer wave, Israel ministry of health reported 2,231 COVID deaths, just 182 deaths less than the number of excess deaths (2,413). In the second summer wave only 1,677 COVID deaths were counted — 741 less than the number of excess deaths (2,418).

So the CBS data show reduced COVID deaths in the second summer wave, as compared with the first, but no change in their estimate of excess deaths. That’s VERY interesting, to say the least. Let’s explore some explanations.

Did they under-count COVID deaths in the second summer wave? Have they been consistently missing COVID deaths in Israel?

Thankfully, the answer to the last question is staring at us from the table.

Look at the row of data in-between the two summer waves: December 2020 through March 2021. That’s the winter during which the Israeli population was first vaccinated. In that 4-month period, CBS estimated only 1,641 excess deaths, but there were twice as many reported COVID deaths (3,298). For every COVID death there was another so-called COVID death that did not account for excess mortality. The person would have died, regardless of their positive PCR test (“death with COVID”). That’s not a minor mismatch between reported COVID deaths and excess deaths. That’s a red flag that cast doubt on the quality of official COVID mortality statistics in Israel. There was at least one period of extensive over-counting of COVID deaths.

Did they somehow start under-counting COVID deaths four months later, in those who received the third dose and died? If they did, the estimate of vaccine effectiveness in the Lancet article is biased upward. If they did not, what make up the difference between excess deaths and COVID deaths? What has caused 741 non-COVID excess deaths in the second summer wave? That’s almost one-third of the excess deaths.

Having no data-driven answers, I will end with my speculation.

The vaccine effectiveness of the booster was nowhere near 81%. There are under-counted COVID deaths among booster recipients, but not enough to make up the gap between COVID deaths and excess deaths. The remainder are probably the continued effects of fearmongering AND booster-related deaths. Based on some sources, I estimate the latter to be in the single-digit range per 100,000 recipients. Perhaps 200–400 booster-associated deaths.

The CBS table contains one more interesting line: data for the two years of the pandemic. They estimate 10,527 excess deaths between March 2020 and April 2022, and 10,680 reported COVID deaths in that period — a remarkable match. Only 153 COVID deaths were “death with COVID” and no death was associated with fearmongering and disrupted life.

If these were numbers on the spreadsheet of an accountant, no one would have trusted the perfect balance after two years of such bookkeeping.