Dear Norman Fenton, Martin Neil, Clare Craig and Scott McLachlan,
ONS Deaths by Vaccination Status statistics
Thank you for contacting us with your concerns about ONS’s Deaths by Vaccination Status publication and for your patience as we have carried out our investigations. It has taken us some time to respond, for which I apologise. This is because these are important issues and we wanted to consider them carefully. I have set out our thinking on the issues below.
Our first consideration is the purpose of the ONS statistics. The Deaths by Vaccination Status publication aims to provide a broad overview of the situation regarding deaths and COVID-19 vaccinations. The analysis was developed in response to user requests during the pandemic to understand the vaccination status of people who die from COVID-19.
As ONS makes clear in its publication, this analysis is not intended to be, nor is it appropriate for, understanding vaccine effectiveness. Analyses about vaccine effectiveness and safety are covered by the weekly COVID-19 vaccine surveillance report, published by UK Health Security Agency (UKHSA). We do consider that it will be important for ONS to reiterate what its Deaths by Vaccination status report is for, and is not for, in future publications.
Your paper on this topic recognised that the sample analysed in the Deaths by Vaccination Status publication is not a whole-population sample. We agree and think this is an important point. To summarise, the publication uses data from the Public Health Data Asset (PHDA), which combines data from the 2011 census and the General Practice Extraction Service (GPES). For an individual to be included in the PHDA, they must have responded to the 2011 census and be presently registered with a GP. Approximately 79% of the population fall into this category. Those missing from the PHDA dataset are therefore not missing at random, and they are more likely to fall under one or more of the following categories:
- Younger in age
- Born outside of the UK
- Unvaccinated (as it is more difficult to obtain a COVID-19 vaccination without being registered with a GP)
We consider that it is therefore likely that the sample used in the Deaths by Vaccination Status publication is not representative of the general population. Those who are missing are, we think, more likely to be younger and unvaccinated. This is also acknowledged by ONS in its Deaths by Vaccination Status publications.
ONS is working to address some of the sampling issues present in the first six iterations of the publication. As stated in the notice at the top of the most recent publication, there is a delay in publishing the next edition. This is because ONS requires further data on subsequent booster vaccinations, and also because it is waiting for data from the 2021 Census. These data will substantially increase the sample size, meaning that the sampling frame will be much more representative of the general population. We will monitor how ONS delivers and communicates these changes as part of our ongoing review work.
In your letter to us, you state that there is a ‘gross underestimation of the population proportion unvaccinated’ present in the Deaths by Vaccination Status publication. In our view, although the number of people in each vaccination category is used in the publication, it is only used to determine the age-standardised mortality rate (ASMR) of each group. The number of people in each group is not and should not be used as a measure of vaccine uptake in the UK. This is because the sample in this publication is not random: as indicated above, those missing from the sampling frame are more likely to unvaccinated. We will look to ONS to make this distinction clearer.
In your paper you also state that the number of deaths reported within the first 21 days of receiving the first dose of the Covid-19 vaccination is below that which could be expected in the general population, and that this difference is evidence of systematic undercounting of deaths by ONS. Our view of this question starts with the consideration that the population included is not representative of the general population, and it is therefore likely that any pattern you observe is attributable to the ‘healthy vaccinee’ effect. This happens when people who are ill (either due to COVID-19 or another relevant illness) are likely to delay vaccination. The result of this effect is a lower-than-average mortality rate within the first 21 days of receiving a vaccination. This effect is described by ONS in their Deaths by Vaccination Status publication.
Turning to the underlying data recording, we do not consider that there is evidence to indicate that ONS has systematically undercounted deaths within the first two weeks of receiving the COVID-19 vaccination. ONS have confirmed with us that they do receive data in these instances, and that the individual would fall into the ‘vaccinated’ category. We have asked that this be made clear in future iterations of this publication.
ONS has previously published information on the number of people who died after receiving the COVID-19 vaccination (up to January 2022, there were 24 deaths registered in the UK with ICD-10 code U12, COVID-19 vaccines causing adverse effects, on the death certificate). When we met with ONS, it confirmed that it is working on a publication that focuses on deaths after the COVID-19 vaccination, which will be published this year.
Overall, then, our view is that the Deaths by Vaccination Status publication does not provide information on vaccine effectiveness or vaccine safety, and should not be used in this way. We would advise use of the weekly COVID-19 vaccine surveillance report, published by UK Health Security Agency (UKHSA), instead. This looks at a range of measures of vaccine effectiveness, including effectiveness against infection, transmission, hospitalisation and mortality. This UKHSA publication also provides a list of other publications written by UKHSA and its collaborators, which look at the effectiveness of the vaccines.
Thank you again for contacting us about this issue.
Yours sincerely,
Ed Humpherson
Director General for Regulation