Value

Meeting user needs

9. As a topic of high public interest, we found a varied user base for both the weekly and monthly statistics. We identified users from Scottish Government, other public sector bodies, the Scottish Parliament, health think-tanks, and the media. Examples of uses of the statistics included Audit Scotland who use the monthly statistics to scrutinise NHS Boards’ performance and in a report about NHS recovery. We also heard that the media use the statistics to assess trends in performance across hospitals. Use of the statistics also extends beyond Scotland, with organisations including health think-tanks using them to compare A&E performance across the UK.

10. Overall, the statistics were considered very useful, with users appreciating the timeliness and granularity as well as the variety of formats that are used for publication. There was also positive feedback on the new platform and how PHS outlines in the overview that there are some differences between the monthly and weekly statistics. In this section, PHS defines the different categories of emergency departments in Scotland and explains that data for some of the smaller sites are collected only monthly. There is also a helpful glossary which explains some of the technical terminology for users who are unfamiliar with this. However, despite all of these efforts, we heard from some users that they were still not aware that there were differences between the two sets of statistics.

11. The new platform should also help to encourage a wider user base with the functionality to extract granular data at hospital or board level allowing users to interpret trends. This ultimately should aid the local implementation of initiatives to tackle A&E waiting times and support patients to access the right care at the right time.

12. PHS presents the statistics in a variety of formats, supporting different user needs and preferences. These formats include interactive charts, csv files, Excel tables and open data. The interactive charts were particularly useful to users we spoke to, with some finding the ability to select individual hospitals and see their performance over time helpful. We also heard that the open data are used for carrying out further analysis and creating new data visualisations.

13. PHS provides contextual information on performance monitoring and on the Redesign of Urgent Care in the overview page of the new platform. The ‘main points’ pages provide summaries of the latest information, which are updated monthly and weekly. In its newly published workplan, PHS is planning further developments such as including more detailed commentary to put the figures into context (for example, whether or why the latest number of attendances is different from the previous week’s figures).

14. PHS engaged with its key stakeholders (Scottish Government and NHS Boards) and acted on their feedback during the development of its new platform. The teams across Scottish Government that we interviewed spoke highly of their regular engagement with PHS. The platform also has an online feedback form asking questions about the format of the statistics and the ease of locating required information. However, given the high public interest, PHS’s user engagement is quite limited in terms of user types. Currently, user engagement beyond Scottish Government and NHS Boards does not form a regular part of the production process. Following our feedback, PHS is planning to extend its user engagement and has set out a draft engagement plan up to December 2023 to consult with a range of groups. This is under PHS internal review and will be published in due course. PHS has added some further detail on user engagement about these statistics in its section on Regular User Engagement.

Innovation and improvement

15. PHS demonstrates a strong commitment to ongoing innovation and improvement in several ways. This includes the development of the new platform, as discussed above, provision of open data and development of traditional production processes to follow Reproducible Analytical Pipeline (RAP) principles.

16. PHS follows RAP principles and uses an automated process to produce the month and weekly data. All code is subject to version control and stored on a private Github repository. The PHS data science team publishes R coding standards which are followed closely. We support PHS’s plans to further enhance these processes by introducing automated testing to the code.

17. We understand that PHS plans to review unscheduled care patient pathways to determine variations in data collections across the NHS Boards. NHS Boards vary in how unscheduled care services are configured and PHS conducted a mapping exercise to help understand the variations across Scotland. An expert group is expected to be established in late summer 2023 to discuss definitions around the four-hour performance standard and agree any recommendations that would support improved quality and consistency with the national data returns.We welcome PHS’s desire to publish these recommendations later in 2023.

18. PHS now publishes weekly open data alongside the monthly open data. This enhances the value of the statistics for users who wish to carry out their own analysis.

Comparability of A&E statistics

19. To aid user understanding and support appropriate use of the statistics, PHS has included a section on comparability across sites in Scotland. This explains some nuances in service delivery and site differences, which can impact comparability. PHS explains that although A&E data should include trolleyed areas of assessment units, this is often not distinguishable in the raw data and therefore not always possible to separately identify that this is the case in the data outputs.

20. Hospital waiting times comparisons across the four nations often feature high in the media and in public debate. Unsurprisingly, we heard that some users wish to compare A&E statistics across the home nations. PHS includes information about the limitations of doing this in the comparability section, which are largely due to differences in service delivery and definitions in the four nations. For example, there are differences between Scotland and England in how the four-hour standard is defined i.e. in Scotland if a person is taken to A&E by ambulance the clock for the four-hour standard starts when the ambulance arrives at the A&E facility and registers with the A&E department. In England for similar cases the clock starts when handover to the A&E department occurs or 15 minutes after the ambulance arrives. PHS also points users towards to the NHS Digital annual comparison across the four nations which it considers the best source on comparable A&E data. However, further guidance on the differences in waiting times statistics across the UK would help users to make appropriate comparisons where possible. This is particularly important when the four nations’ waiting times are compared publicly, for example in the media or parliament.

21. The UK Health Statistics Steering Group (UKHSSG) is responsible for improving the coherence and accessibility of health and social care statistics across the UK. Within the UKHSSG, theme groups have been established which recognise the importance of UK wide coherence including one on cross-UK performance statistics. This group aims to provide meaningful comparisons of secondary care waiting times and performance statistics across the UK. Staff from PHS participate in this group and there are links to the Government Statistical Service page which provides further information about the comparability of waiting times statistics across the four nations.