Quality
Improving quality information
22. During the course of the assessment, PHS has improved its signposting to quality information and links to sources under the section on Metadata. We highlighted to PHS that some documentation such as the A&E Datamart User Guide had legacy branding or older revision dates. We heard from some users that this could sometimes be confusing to users and could lead them to navigating to redundant pages. PHS has now adopted an ongoing proactive approach to reviewing legacy documents posted on its website.
23. Data suppliers (NHS Boards) that we heard from are confident about the data extraction and submission process as the hospital data collection systems have been operational for a number of years. PHS provides detailed guidance to data suppliers, who run reports to identify data anomalies and sense check data prior to sending it to PHS. Some NHS Boards check these anomalies with clinical staff to assure their validity. Currently there is one scheduled meeting a year between data suppliers and PHS. We heard from PHS and NHS Boards that this frequency is adequate although we support PHS’s commitment to reviewing this schedule to check whether more frequent meetings would be helpful.
24. The new platform originally had a section on data quality which stated that ‘aggregate returns are subject to only basic quality assurance checks. NHS Boards are required to confirm to PHS that the statistics are accurate.’We considered that this did not sound reassuring to users and did not reflect the robust quality checks that PHS carries out. We suggested that PHS should be more explicit in how it assures itself that the data it collates are of sound quality. In response to this, PHS reworded the section to say ‘PHS works closely with colleagues in the NHS Boards to improve the validation and accuracy of the data and to ensure that the appropriate data standards are understood and applied by all sites. This takes place on a continuous basis, as queries or potential issues arise.’
25. We welcome the approach that PHS has recently undertaken to quality assure its administrative data (QAAD). By applying the QAAD principles, the Content Review Board within PHS has deemed that the A&E dataset meets level A2 enhanced assurance. This means that PHS has evaluated its own administrative data quality assurance arrangements against a medium level of public interest in the statistics and published a fuller description of the assurance.
26. PHS has made a concerted effort to improve its communication about the differences between the weekly and monthly statistics. It has published improved information on what sites are included in the monthly statistics and explanations that some smaller sites are not included in the weekly statistics. We noted initially that monthly calculations were based on the date and time of arrival of the patient, whereas for weekly statistics the date and time of discharge was used. Since 2 May 2023, PHS has publicly stated that the two calculations align and that both use the date and time of arrival of the patient. We welcome this move to be consistent, particularly as some users were still not aware that there were differences between the two sets of statistics. We expect PHS to review and consult further on this issue with users, to ensure that the publication adequately explains the differences between the two sets of statistics throughout the platform. This could form part of the user engagement that PHS has committed to carrying out during the summer/autumn 2023.
27. PHS has published a revisions statement explaining that given the dynamic nature of the A&E dataset, figures may be subject to change in future releases. It is good that PHS has also committed to considering how to improve its communication of uncertainties in the data.