Dear Eugene
Northern Ireland (NI) Outpatient Waiting Time Statistics
I am writing to you following our compliance check of the quarterly publication Northern Ireland Outpatient Waiting Time Statistics published by the Department of Health (NI), against the Code of Practice for Statistics.
In 2020, following a public consultation in 2019, you decided to change the data collection method used to produce these statistics. You have replaced the aggregate manual returns from the Health and Social Care Trusts with patient level data from the Patient Administration System (PAS) that are accessed via the HSC Data Warehouse by the statistical team. Following our review, we are content that the Northern Ireland Outpatient Waiting Time Statistics should maintain the National Statistics designation. We found a number of positives during our review to highlight here:
- Users had the opportunity to feedback on the proposed methodology changes via the public consultation, supporting independent and transparent decision making that is in line with the Code of Practice for Statistics. All users were supportive of the changes and a summary of the responses is published. The collection of data is now less burdensome on data suppliers and the patient-level dataset provides more opportunity for data linkage projects and more insightful analysis.
- You have a good working relationship with the HSC Trusts as part of the data quality assurance process – both informally and formally. There is a governance structure in place where any issues with data quality can be raised formally with the data suppliers and this is attended by the principal statistician.
- The statistics are used as a key metric in Northern Ireland to measure hospital performance against ministerial targets. Given the importance and longevity of these statistics, it is good to see that they are presented in an impartial and objective way. For example, tables 2a, 2b, 3a and 3b in Appendix 1 of the bulletin provide figures for the number of patients waiting, disaggregated by HSC Trust and specialty. The granular presentation in these tables allows users to see the distribution of waiting times, in a clear and transparent way.
- We welcome the statisticians’ plans to innovate and develop these statistics in 2022. The views of a broad range of users should be central to this and despite ongoing resource challenges, we are pleased that the statisticians have agreed to consider ways to engage with users more proactively. Positive developments include the implementation of Reproducible Analytical Pipelines (RAP) to streamline the production of these statistics, publication of additional variables such as an age breakdown of patients waiting in response to user needs and a review of the publication and supporting documentation.
We also identified some ways in which the value and quality of the statistics could be enhanced:
- Within the publication, the statistics about completed waits in Independent Sector hospitals and for those in Day Case Procedure Centres are not National Statistics. We found the presentation and explanation of this unclear in the bulletin and more could be done to improve the way this is communicated to users. The similarities and differences between each of the data sources used to produce these statistics should be explored further through an assessment of risk to quality using the QAAD toolkit.
- As part of the review of the publication in 2022, the supporting quality and background information should be improved to make the material more accessible and adhere to relevant guidelines to ensure the needs of different users are met. There is a range of guidance available in communicating statistics and making publications more accessible via the GSS Policy and Guidance Hub.
- Explanations in the bulletin around the impact of the Covid-19 pandemic on the statistics is limited. To enhance the value of the statistics, the scale of the impact on data quality could be made clearer to users and more guidance could be provided around appropriate interpretation of the figures given the impact on outpatient activity in hospitals, particularly in any trend analysis and comparisons made.
We appreciate your teams’ positive engagement with us during the process and the enthusiasm for the continued development of the statistics. I hope that you consider the wider findings of this review across other National Statistics produced by the Department, such as the annual outpatient activity publication.
Following your planned internal review of the statistics in the new year, we would like to meet again with the team in April 2022. Please do not hesitate to get in touch if you would like to discuss any aspects of this letter in the meantime.
I am copying this letter to Siobhan Carey, Chief Statistician for Northern Ireland.
Yours sincerely
Mark Pont
Assessment Programme Lead