Dear Nick,
England and Wales Mortality Statistics produced by the Office for National Statistics (ONS)
The COVID-19 pandemic brought with it an increased public interest in mortality statistics. We received enquiries about mortality statistics from a range of users, from politicians to the general public. Casework enquiries on mortality statistics typically focused on specific decisions made in producing the statistics, such as those to remove 2020 data from the five-year average, and the rationale behind the reporting of mortality statistics by the date the death is registered, rather than the date of death. When we initially began responding to queries in this area, we found that there was a paucity of information on steps taken to assure the quality of mortality statistics. Because of this, we decided to take a more detailed look into mortality data, with a focus on compliance with the Quality pillar of our Code of Practice for Statistics.
ONS mortality data are used in a broad range of statistics, including weekly, monthly and annual counts of deaths. Data are also used in mortality publications on specific topics, such as those on avoidable deaths and excess winter mortality, and deaths related to drugs and alcohol. For this review, we focused on the data sources, methods used, and the quality assurance processes.
We would like to acknowledge the responsiveness of the team responsible for mortality publications during the pandemic. It has been really valuable to a wide range of users to have COVID-19 specific mortality statistics that were of the same standard as more long-running statistics.
The quality of mortality statistics
We consider that the mortality statistics produced by ONS are of good quality. Our review identified several strengths in these statistics:
- The detailed documentation of the methods used in the creation of these statistics, as shown in the user guide to mortality statistics.
- The communication of the strengths and limitations of these statistics, specifically on the impact of registration delays, as outlined in the Impact of registration delays We understand that registration delays occur as a result of the death registration process, outwith the control of ONS, and this publication helpfully explains this limitation and its impact on the data to users.
- The strong working relationships with other organisations, including the General Register Office (GRO) and the statistics agencies in the devolved administrations
We also found that the processes used to assure the quality of the mortality data and statistics are comprehensive and effective. Death registration data are provided to ONS from the GRO once a death has been registered. These data undergo extensive checking at various points during the analytical process, including a series of manual and automated checks of the data from the GRO and the quality assurance of statistical outputs. These processes are mapped out in a flow chart accompanying the user guide. Through discussions with ONS and reviewing the work going on behind the scenes, we are satisfied that the checks performed are sufficient to assure the quality of mortality data. The following are key examples of the quality assurance arrangements:
- Extensive checking of mortality data occurs when they are received from the GRO. All areas of the data are checked, including the place and cause of death and the occupation of the deceased
- The dual running of statistical analyses: two analysts run separate analyses on the same dataset. The analyses are then compared, aiming to ensure that the analysis and the outputs are the same.
- All bulletins, tables and figures are produced by one analyst then checked by at least one other analyst. Once outputs have been published, there is a further round of checks.
Communication of the quality assurance processes
We do not consider that there is sufficient description of your quality assurance procedures in the public domain. It would provide more visible reassurances to users for ONS to publish more information on the steps taken to assure the quality of mortality data.
While we are satisfied with the depth of the description of the methods used in the Quality and Methodology Information (QMI) that accompanies these datasets, we consider that more detail included around the quality assurance processes (including the criteria used for checking the data at each point of the analytical process) is vital to ensure public confidence in these statistics.
Reporting mortality data by the date of registration as opposed to the date of death
In 2006, ONS decided to move from publishing mortality data by the date of occurrence to the date of registration. This enabled a static record of the deaths registered up to a certain date to be created, and at the time, this resulted in more timely reporting of deaths. However, during the COVID-19 pandemic, we have seen more-timely reporting of deaths data elsewhere, such as those reported in UK Health Security Agency’s Coronavirus dashboard.
ONS should ensure that when mortality data are reported, it is made clear that it is done so by the date that the death was registered, not the date that the death occurred. Although this is made clear in the majority of publications, we found the distinction was not made in some sections of the previously produced Coronavirus (COVID-19) latest insights, which may have led to confusion for users.
There should be clearer communication around the rationale for reporting mortality figures by the date that the death is registered, not the date that the death occurred, and the effect that this has on the use of the statistics. Through our engagement with you, we learned of the reason for reporting deaths in this way. However, this information is not easy to access. We would like to see this information published alongside the quality and methodology information accompanying the statistics. This would help to maintain public trust in the methodology used in these statistics.
User need for data on deaths by date of occurrence
Through enquiries as part of our casework, we have been made aware of a user need for mortality data published by date of occurrence. We consider your engagement with users may evidence a wider need for data to be reported in this way, and you should engage in such a way to understand the strength of that need.
Deaths registered by coroners
Informed by the user needs, ONS should investigate the feasibility of more-timely mortality measures, specifically around the reporting of the number of deaths occurring before they have been formally registered. We know that registration delays are more likely to occur when the death is registered by a coroner than when registered by a doctor. This change could result in the provision of more timely data, particularly for death registrations by a coroner. We would be keen to work with ONS to engage with the coroner’s system to develop possible solutions to this issue.
ONS’s review of methods
We understand that in response to OSR’s review of excess deaths reporting, your Methodology and Quality Division will be conducting a review of methods used in mortality statistics. This is something we welcome and should provide ONS with the opportunity to enhance the value of mortality statistics; we look forward to hearing the outcomes of the review.
I would like to thank your team for its positive engagement on this review, including Julie Stanborough, James Tucker and Sarah Caul.
Yours sincerely
Ed Humpherson
Director General for Regulation