Detailed findings

Introduction

1.1 ONS started publishing weekly provisional statistics on deaths of people who lived in registered care homes in response to urgent user need in 2020. The COVID-19 pandemic had quickly affected elderly people, and those in residential accommodation. Health agencies and local authorities responsible for public health needed information to help them manage the impacts of the illness.

1.2 ONS has published the annual statistics twice, in the autumn of 2020 and again in 2021. The statistics are based on deaths data from different sources – the General Register Office (GRO), the Care Quality Commission (CQC) and the Care Inspectorate Wales (CIW). The statistics focus on deaths of people whose address was a registered care home in England and Wales. ONS has produced several breakdowns of the data – by sex, age, local authority and cause of death – providing users with a range of statistics. In the statistics, ONS also presents age-specific mortality rates^[1] (ASMRs) of care home residents.

1.3 The accompanying commentary in the first publication was more comprehensive and included more analysis and discussion than the latest publication. Worthy of note is the innovation and collaboration, both within and outside ONS, that the statistics team undertook for the speedy development of these statistics for the public good. For example, ONS has built strong working relationships with CQC and CIW. The team plans to continue to collaborate with these organisations on future analyses about other topics in the care sector, for example, to estimate the proportion of privately or publicly funded residents in care homes.

1.4 Social care is devolved across the UK. National Records of Scotland publishes deaths in care homes as part of its weekly Deaths involving coronavirus (COVID-19) in Scotland statistics; and the Northern Ireland Statistics and Research Agency publishes COVID-19 related deaths to care home residents in Northern Ireland as part of its weekly deaths statistics. To assist users, ONS has developed an interactive dashboard that brings together the official statistics relating to adult social care, which includes data on deaths in the care sector, from the four UK administrations. By accessing the linked 4 nations matrix file, users can see how the statistics are constructed and whether they can be compared.

1.5 ONS has a range of development plans for statistics about adult social care more generally. As part of developing these statistics, ONS is planning to explore statistics about the following, all of which would add value to the adult social care landscape:

• unpaid carers
• self-funded care home residents
• recipients of home care
• improving care home population estimates
• workforce statistics

[1] An age-specific rate is the incidence or mortality rate for a specified age group, in which the numerator and denominator refer to the same age group; it is expressed as the number of deaths per 100,000 population at risk.

Expanding the user base to develop the statistics

1.6 During the speedy timelines of the COVID-19 pandemic, ONS urgently developed weekly datasets of provisional statistics for an expert set of users that detail whether a death involved coronavirus. This demonstrates ONS’s motivation to improve statistics and data in line with the Code of Practice for the public good. The weekly data have been very relevant and timely for central government health and care officials, care regulators (CIW and CQC) and local authorities managing the public health emergency.

1.7 The team outlines longer-term plans for its statistics, for example, by posting updates in blogs (Shining a light on what we know about adult social care and Improving access to data on adult social care) as well as speaking about the statistics at conferences. Despite this outreach, we found that these statistics had a small user group outside of the core policy officials and data suppliers who had helped develop the statistics. The users who had helped develop the statistics told us that ONS had engaged well with them and that the statistics were useful for them.

1.8 Some key stakeholders whose jobs involve analysing and commenting on the adult social care sector told us they had not come across the statistics at all. Other users commented that navigating some of the accompanying documentation could be difficult and these users also felt that understanding key data definitions and changes to definitions could be a challenge.

1.9 ONS has released Deaths in the Care Sector annually as an experimental output. Experimental statistics are ‘newly developed or innovative official statistics undergoing evaluation. They are developed under the guidance of the Head of Profession for Statistics and are published to involve users and stakeholders in the assessment of their suitability and quality at an early stage’. The limited user engagement we found could indicate that ONS has not yet fully explored the public value of the statistics, that the statistics are still in quite early stages of development and that the statistics have not reached their full potential to improve understanding of the adult care sector. For example, the inclusion of deaths of people in receipt of domiciliary care, as well as those who lived in a care home, would be a helpful development. The team should build on its initial work and engage with a wider range of stakeholders to identify and get consensus for which adult social care statistics will be most helpful for ONS to develop in the future.

Maximising coherence and understanding

1.10 ONS publishes a large suite of mortality statistics. Users highlighted that they were unsure where these statistics on deaths in the care sector fitted within the wider mortality statistics landscape, given that ONS already published information about where people had died, including the number of deaths of care home residents.

1.11 As part of its new analyses about the adult social care sector, ONS has published an analysis of life expectancy of care home residents using pre-pandemic data. Users we spoke to as part of this assessment told us that this information would be very useful if ONS could include data from the years of the pandemic. To meet users’ needs and maximise coherence between the statistics covering the pandemic, ONS should consider publishing supplementary analyses and comparisons of life expectancy of care home residents including the years of the pandemic.

1.12 Statistics should be presented in a clear, unambiguous way. However, the output title suggests that the contents would include statistics on the deaths of all people receiving social care, whether that is domiciliary care or some form of residential care. Users commented that the title of the publication could mislead because the publication covers only care homes and not the rest of the care sector. ONS told us it plans to change the title of the next release to Deaths of Care Home Residents, England and Wales, which will reflect the content more accurately.

1.13 Users valued the commentary and discussion provided in the first publication of these statistics. However, as noted above, commentary was much shorter in the more recent publication and users felt they would have benefited from more. The reasons they gave for this are that the information lacked a wider context and they were unable to use the data to evaluate the quality of care given or place the impact of the pandemic within the context of other causes of death of residents of care homes.

1.14 In the first output, ONS had included breakdowns by some demographic characteristics which are not recorded on the death certificate. These included ethnicity, disability and religion. The information was retrieved by linking death records to the Census 2011 data. Users queried whether there were further opportunities for data linkage with other sources, noting that a more granular presentation of the statistics, for example, by type of care home, and more visualisation to describe the data, would be helpful.

Explaining data limitations and quality assurance

1.15 The Code of Practice notes that opportunities for data sharing and data linkage should be taken whenever possible. As noted above, ONS has taken advantage of the Census to retrieve information about some personal characteristics and re-used data to create the statistics. Internationally recognised standards, classifications, definitions, and methods are applied to the mortality data, which is good practice.

1.16 ONS has not published some important metadata about these statistics – information about quality assurance processes, risks and mitigations for data quality, and methodological details. Users are therefore unaware of the strengths and limitations of the data. ONS is preparing this documentation and should ensure it fully explains uncertainty, or other limitations, and the mitigations applied to maximise data quality for users.

Estimating the care home population

1.17 Expert users raised concerns with us about the methods used to estimate the care home population which is used in the calculation of the age-specific mortality rate (ASMR) and that the estimates of the care home population might not be accurate enough for their purposes. ASMRs are expressed as the number of deaths per 100,000 population at risk. The calculation of an ASMR is widely used across ONS mortality statistics, but the number of care home residents to use as the denominator has been estimated by applying a care home resident proportion to the latest population projections.

1.18 The impact of this uncertainty has not been explained and expert users have expressed reservations about trusting the data because of this. ONS has told us that it plans to use data from the Census 2021 to estimate the care home resident population and is exploring alternate data sources to provide estimates of the care home resident population in subsequent years. Following any methodology changes, ONS should ensure that it explains the comparability of the estimates over time. ONS should also ensure that users are aware of the limitations of the current methods it has applied and outline plans for options to obtain a more accurate care home population estimate.

1.19 Independent measures, such as internal and external audit, peer review and National Statistics Quality Reviews can be helpful to evaluate the effectiveness of the statistical processes and quality. As noted above, the development of experimental statistics should include a review of the quality and ONS has not yet carried out any peer review or external audit on these statistics. As it develops the statistics, and to give an assurance of quality for users, ONS should implement a form of external assurance, such as a peer review, for these statistics.