Rapid Review of NHS England’s statistics on daily Covid-19 deaths in hospitals

Dear Mark,

CORONAVIRUS (COVID-19) DAILY DEATHS

I am writing to endorse the approach you have taken to develop the daily deaths statistics that NHS England has published in response to the coronavirus (COVID-19) pandemic. The statistics report on the deaths of patients who have died in hospitals in England and had either tested positive for COVID-19; or where COVID-19 was mentioned on the death certificate. This information has proved critical for decision-makers and scientists, for example, in managing resource throughout the pandemic and reviewing lockdown measures in England.

My team has conducted a rapid regulatory review of these statistics. We have reviewed the extent to which they have been produced in accordance with the Code of Practice’s Trustworthiness, Quality and Value pillars, while taking account of the pressures that NHS England has faced to deliver timely statistics about a rapidly evolving national emergency. I really appreciate your colleagues’ positive engagement throughout this process. A summary of our findings, including recommendations, is set out below.

Value

  • We welcome the rapid development of these statistics so they could meet the national demands for mortality data and inform decision-making, for example through their use in the UK Government’s COVID-19 daily briefings. These statistics have proven to be invaluable to both policy colleagues, the media and the general public as a timely, robust source of information on hospital deaths in England.
  • NHS England has striven to improve and innovate these statistics since their first publication, both in response to national data requirements and to enquiries from the media and general public. For example, NHS England improved the presentation of its data and statistics by using charts and graphs to illustrate findings. NHS England has also started publishing detailed weekly tables which include breakdowns by ethnicity, gender, age groups and pre-existing conditions, which reflects a positive response to user demand. Whilst we consider these statistics and data to be incredibly valuable given their high level of insight, no time series data are available in this form, only overall totals. We ask you to talk with your users and reflect on how time series data might add value for users of the data about ethnicity, gender, age groups and pre-existing conditions.
  • Whilst the COVID-19 daily deaths publication is easily accessible through NHS England’s website; it is only available in xlsx format which limits its ability to comprehensively meet user needs. For example, this format is not machine-readable which limits users in producing their own analysis of the data and statistics. We encourage you to reflect on the accessibility of the data and statistics to ensure they meet a wide range of user needs.
  • We also welcome NHS England’s collaboration with the University of Bristol and its collection of data on the deaths of people with learning disabilities or autism which is due to be published shortly. This will offer additional, valuable insight into the impact of COVID-19 on these groups and help inform decision making and public debate.
  • A potential source of confusion for users has been around understanding the impact of any lags in recording deaths on the statistics. We appreciate that the data are presented on the basis of the date of death rather than the date of recording, and that in your charts you illustrate clearly the impact of any later recording on the data series. As mentioned earlier, the availability of the data behind the charts, in machine-readable form would aid accessibility and transparency in order to aid further analysis and insight.

Quality

  • In March 2020, NHS England moved quickly, from its Incident Control Centre collecting operational incident response data about COVID-19 deaths in hospitals through its network of regional contacts, to establish the COVID-19 Patient Notification System. Developing this standardised collection tool so quickly was an important and impressive endeavour, as was establishing a dedicated team to work closely with yourself, as Head of Operational Information.
  • It is very important that the strengths and limitations of these statistics are transparent, and that users can be assured of their quality. We appreciate the headline data notes that your team has published alongside the statistics, though we consider more information about the quality of the breakdowns by ethnicity, gender, age groups and pre-existing conditions, which evolved later, should also be included to provide users with further reassurances around these data.
  • We welcome the recent move to publish the data collection user guide which includes information about how the data are collected and validated. It was good to hear from your team about how it works with its national and regional contacts to assure the quality of the data, and that the team sense checks with other sources. We have encouraged your team to reflect on their own quality documentation in line with the Government Statistical Service guidance on urgent quality assurance of data.
  • During the pandemic, the data landscape has evolved and become more complex, bringing with it the greater risk of presenting a confusing picture for users. The key example being the early shift in focus from examining hospital deaths to Department for Health and Social Care (DHSC) starting to publish daily statistics on deaths in all settings. We welcome the recent information published alongside the statistics that provides some clarity about the different methodologies used to report on COVID-19 deaths by the DHSC and the Office for National Statistics (ONS). We think it would also be helpful if you could briefly explain for users what statistics are available about COVID-19 deaths in hospitals for the other countries of the UK.
  • As England has moved through the first wave of the pandemic, questions naturally arise about the future of these statistics – for example, “for how long will daily reporting continue to make sense?” and “what potential lessons can be learned that will inform how to operate during a potential second wave?” We were very encouraged to learn that your team has held a pause and review with stakeholders, including representatives of those in the supply chain, to reflect and learn lessons. Involving users and partners closely in decisions on future changes will be important to the development of these statistics.

Trustworthiness

  • NHS England’s orderly and timely release of these data has been invaluable in allowing users to act quickly and has reflected the pace of change during the COVID-19 pandemic. Where occasional errors have arisen in the data, these have been reported quickly and transparently and the team assured us that lessons have been learned.
  • We welcome that NHS England has published a pre-release access list alongside these statistics. We understand that the list of those with early access has reduced significantly in number since the broader DHSC statistics became the headline government figures. For clarity, we would advise that you update the published pre-release access list with the now reduced list and indicate how far in advance access is granted.

We look forward to seeing these statistics continue to develop. Based on the findings of our review you can include a statement in your release such as “These statistics have been produced quickly in response to developing world events. The Office for Statistics Regulation, on behalf of the UK Statistics Authority, has reviewed them against several key aspects of the Code of Practice for Statistics and regards them as consistent with the Code’s pillars of Trustworthiness, Quality and Value.”

I am copying this letter to Nalyni Shanmugathasan and Paul Shafee, the responsible analysts at NHS England and NHS Improvement, and Stephen Balchin, Head of Profession for Statistics at DHSC.

Yours sincerely

Ed Humpherson
Director General for Regulation

Adult Social Care Statistics in England Report – Letter to Mark Svenson, NHS England

Dear Mark,

Today, we have published the finding from our review of Adult Social Care statistics in England. The need for good data to support delivery of adult social care should not be underestimated. While there is rightly a focus on delivery of social care, a scarcity of funding has led to under investment in data and analysis, making it harder for individuals and organisations to make informed decisions. This needs to be addressed if social care is going to evolve to support a changing society and meet the increasing demands expected over coming years. Data matters in solving problems, supporting efficiency and improving outcomes.

Our review identified important improvements needed covering: leadership and collaboration; data gaps; and existing official statistics. We would like to see stronger leadership and collaboration across government to support better data on adult social care and consider NHS England should be collaborating with others to shape these improvements.

As part of our review of existing official statistics, we considered the quality and value of official statistics about adult social care against the standards set out in the Code of Practice for Statistics. It highlighted improvements around accessibility, coherence, quality, timeliness and granularity of the data. The review included two outputs published by NHS England (Delayed Transfers of Care and Better Care Fund Quarterly Reporting), immediate actions related to these outputs are outlined in the Annex to this letter.

Improved statistics are essential to support policy makers who are developing proposals to reform the funding and delivery of adult social care as well as individuals who will be able to hold government to account and make better informed decisions about issues which impact the lives of themselves and their families.

We will continue to work with a range of organisations to make the case for improvements to social care statistics. Specifically, my health and social care lead will liaise with you regarding progress towards these recommendations.

Yours sincerely

Mary Gregory
Deputy Director for Regulation

 

Related Links

Report on Adult Social Care statistics in England

Statements on NHS funding

Dear Mark

I am writing to you regarding the announcement made on 5th August 2019 on the provision of £1.8 billion funding for the NHS. This figure consisted of two distinct parts: £850 million of new funding to upgrade outdated facilities and equipment, and a £1 billion increase to NHS capital expenditure limits.

As I am sure you are aware there has been a significant level of debate as to whether the £1 billion increase in capital expenditure should be considered new resource or should be considered cash already held by Health trusts which they have now been given permission to spend as a result of increased limits on capital expenditure. However, further statements and responses from the Department of Health and Social Care (DHSC) do not seem to have fully answered these questions.

DHSC’s press release on 5th August stated that the whole £1.8 billion was new money, a claim that has been reiterated by Health Ministers. On 17th August, NHS Chief Financial Officer Julian Kelly wrote to NHS trusts informing them that the £1 billion increase in DHSC’s baseline capital expenditure limit means they can revert to their original capital plans which are funded by their own income and reserves.

Full Fact requested more information from DHSC on this matter, and on 29th August DHSC responded stating that “The NHS has an extra £1.8 billion of new funding to invest in projects, money which the NHS did not previously have to spend…This includes £850 million which will directly fund 20 new hospital upgrades—money that had not been previously allocated.”

So far, we have been unable to identify an authoritative official statement describing how the £1.8 billion is funded, and the mechanisms that led the £1 billion capital expenditure to be withheld and then subsequently released to trusts. I encourage DHSC to release such a statement as soon as possible in order to enhance transparency and support public understanding.

I am sure that you will agree that the relative complexity of the funding structures underpinning this announcement need to be carefully explained.

I am copying this letter to Full Fact.

Yours sincerely

Ed Humpherson
Director General for Regulation

Letter to Mark Svenson on winter A&E performance in England

Dear Mark

Thank you for the opportunity to discuss the concerns raised by the Health Service Journal about comments made at NHS England and Improvement’s joint Board meeting on 28 March about winter performance in NHS hospitals in England. It has been reported that coding issues with same day emergency care (SDEC) patients were cited as a reason why four-hour A&E waiting time performance did not improve this winter, despite the more favourable weather conditions and lower flu and norovirus prevalence than in the preceding winter.

We have worked with you previously to highlight the importance of maintaining the trustworthiness, quality and value of A&E performance measures in England. Any suggestion that data integrity is compromising the statistics’ ability to accurately capture service performance is always concerning.

You have told us that SDEC is a new way of delivering care to patients whose needs are not always best met by a traditional A&E attendance. You outlined that analysis of the Admitted Patient Care dataset highlights that a number of trusts report patients with conditions amenable to same day emergency care as being admitted to hospital. These patients therefore appear in the non-elective admissions statistics where they are recorded as admissions of zero days’ length. In recent months you have started to show the zero and 1+ lengths of stay non-elective admissions separately to highlight the differential growth rates of these two elements. This separate presentation allows users to estimate the impact of SDEC patients on non-elective admission rates.

We understand that the impact of SDEC treatment on A&E performance is harder to quantify. However, you think it unlikely that they account for this year’s performance in the way reported. In some cases, a patient can have an A&E attendance recorded if they present at an A&E before they are redirected to SDEC. Typically, these types of attendances are short, so these patients would be recorded as having been seen within four hours, whereas they were actually redirected to an alternative form of care. The number of SDEC patients is currently small compared to the volume of patients seen in A&E, and the subset of SDEC patients who first present at A&E is smaller still. For these reasons, you are confident that the statistical impact of recording SDEC patients as short A&E admissions is likely to be very minimal. You have provided assurances that appropriate steps are being taken to adapt systems to improve the recording of SDEC patient journeys.

As the statistician responsible for guaranteeing the integrity of NHS England’s statistics, we recommend you seek assurance on:

  • The number of SDEC admissions versus the number of A&E attendances;
  • The number of SDEC admissions who are also recorded as an A&E attendance.

The introduction of new ways of delivering care, and of measuring performance, brings numerous challenges for statistics production. It is important to raise awareness of these challenges and to be transparent about the steps being taken to maintain the trustworthiness, quality and value of NHS England’s statistics. This is particularly important if there is any perception that measures have been changed in order to mask performance failures. We recommend you publish details of your work to safeguard the integrity of these statistics, for example via blogs or special articles in the coming months.

Yours sincerely,

Ed Humpherson
Director General for Regulation

 

Related Links:

Ed Humpherson to Mark Svenson, November 2018

Ed Humpherson to Mark Svenson, May 2018