Findings

Statistical governance and pre-release access

2.1 Overall we found there to be effective processes in place for statistical governance. The team told us that the Head of Profession is responsive and gives it the support it needs. Although some issues remain from the merger between NHS England and NHS Digital in 2023, the team has found effective ways to navigate these. The team manages both the pre-release access list and the risks associated with pre-release well.

2.2 Following the merger between NHS England and NHS Digital, and an interim period with two joint Heads of Profession, there is now one Head of Profession, who is one of the Deputy Directors of Analysis for NHS England. The Head of Profession is supported by a Statistical Governance team, who provides advice and guidance to statistics teams across NHS England.

2.3 Despite the merger being considered complete, the team told us that there are still some areas where the former NHS England and NHS Digital are not yet fully integrated. Although the team feels fully supported by the Head of Profession, there are some legacy systems that the Statistical Governance team does not have access to, meaning that it can be harder to deal with any issues that might arise. The team has developed a process for highlighting any problems with the Head of Profession and the Statistical Governance team and feels confident that it can manage this situation until it is resolved.

2.4 A core principle of the Code of Practice of Statistics is that statistics should be equally available to all, and not given to some people before others. To preserve this principle, the circulation of statistics in their final form ahead of publication should be restricted, in line with the rules on pre-release access set out in legislation.

2.5 NHS England publishes a pre-release access list for the cancer waiting times statistics release that details the departments and job titles of individuals who are granted early access. Generally, the pre-release access list is in excess of 100 people. While this is a large number, these individuals are distributed across Number 10, the Department for Health and Social Care and NHS England. Furthermore, NHS England’s practice of releasing statistics on several topics at the same time as part of its “Superstats Thursday” means that the overall pre-release access list is longer to cover these many topics.

2.6 We feel that the team has given due consideration to both the pre-release access list and practices surrounding pre-release access and is taking the necessary steps to mitigate any risks. As well as outlining its overall rationale for the pre-release access list, the team in NHS England has provided us with its justification for everyone on the list. The team also recently carried out a comprehensive review of the list and removed individuals who no longer had a sound rationale to receive pre-release to the statistics. It continues to monitor the list every month.

2.7 The team provides training to all those who receive pre-release access to the statistics to reduce any risks associated with early sight of the information, and caveats are included on the release at the time of sharing it.

Engagement with users

2.8 NHS England engages effectively and regularly with a range of users of the statistics via the Cancer Waiting Times User Group, the Cancer Data and Analytics Advisory Group (CDAAG) and the National Cancer Board (NCB). These groups give the larger cancer charities, such as Macmillan and Cancer Research UK, as well as members of NHS trusts and integrated care boards, opportunities to raise points related to the cancer waiting times publication. The users that we spoke to were mostly happy with the engagement that they had with the team and found members of the team to be responsive and helpful.

2.9 NHS England made several changes to the cancer waiting times statistics following the introduction of the new standards in October 2023. These changes were widely consulted on, and although the statistics were not the main focus of the consultation, consideration was given to how the statistics would change as a result of the standards changing. Users of the statistics were widely represented in the respondents to the consultation, and NHS England addressed feedback requesting more-granular breakdowns of tumour type for a range of cancers for the 31- and 62-day standards. These breakdowns are now included in the monthly statistical releases. NHS England also kept OSR updated about the changes to the cancer waiting times standards and the impact on the statistics.

2.10 NHS England did not include any proposals for cancer waiting times statistics in the recent England-wide consultation on health and social care statistical outputs. Despite this, the team informed us that it reviewed the main messages from the overall consultation and considered how these might apply to the cancer waiting times statistics. Based on this, the team identified areas for improvement of the statistics related to topics such as the availability of more-granular data, the presentation of the statistics, and the ease of accessing and using the data, much of which overlaps with the findings of this assessment. The team intends to consider this alongside the findings from our user engagement activities.

Keeping users informed of development plans for the statistics

2.11 Producers of statistics should be open about plans and priorities for their statistics and about progress towards enacting these plans. During the assessment, the team told us about various plans for future data developments and improvements to the statistics. These included improvements to data submission and validation processes and investigating possibilities in relation to users’ requests for more-granular data. The team intends to discuss these plans with a sample of users, including their Patient and Public Voice group. Currently, however, there is no publicly available information about these plans. Publishing and maintaining a development plan would inform users of planned changes to the statistics and allow them to provide feedback on these plans. Enabling user input in this way could help NHS England further improve the statistics.

Communicating quality and methods

2.12 Users told us that they were happy with the quality of the cancer waiting times statistics and had confidence in both the underlying data and the published information. However, there is currently very little published information on data quality and the methods used to produce the statistics. Although the users we spoke to were, for the most part, knowledgeable about the data, they highlighted the importance of clearly communicating this information for less experienced users.

2.13 One example is that there is currently no published information on when cancer registrations might not be included in the Cancer Waiting Times dataset, or the estimated overall coverage of the Cancer Waiting Times dataset. During our assessment, the team explained that fundamental differences between the Cancer Registration Dataset managed by the National Disease Registration Service and the Cancer Waiting Times dataset mean that only around 80% of patients appear in both datasets. For example, when looking at the faster diagnosis standard, patients whose cancer is diagnosed following presentation at the accident and emergency department are more likely to appear in the Cancer Registration Dataset than in the Cancer Waiting Times dataset. Although there is detailed guidance for those collecting and submitting data on who should be included or excluded from it, a high-level explanation for users of the statistics would be helpful.

2.14 Another example is that although the collection of cancer waiting times data is mandated, the team told us that there can be issues which impact the data to be submitted, meaning that providers may make a partial or missing submission. In either case, the team contacts the relevant providers immediately to help resolve the issue. Any cases of missing or partial returns, and the estimated impact on the overall national activity volumes, are highlighted in the PDF summary of the main findings. This information is helpful and the impact of missing or partial returns on the overall data is low. As many users told us that they do not use the PDF, however, it would be useful for it to be included alongside other quality and methods information that the team publishes.

2.15 Some of the more-granular breakdowns in the Excel workbooks contain small numbers of people, which increases the risk of an individual being identified in the data. In response to questions that we and users raised about this, the team shared a document outlining its approach to disclosure control (the processes in place to protect the confidentiality of individuals) with us. It was evident from this document that a great deal of thought and consideration had gone into this process, and that the team had taken the necessary steps to satisfy itself that the risk of identifying any individuals within the data was low. Again, this information should be shared publicly so that users can both see the approach taken to considering disclosure control and be reassured that the risk of disclosure is low.

2.16 There is no published information on coherence and the cross-UK comparability of the statistics, although this topic is covered in the article “Waiting times for cancer treatment across the UK” published by the GSS Coherence Team in 2024. To support appropriate use of the statistics, we would like to see a link to this article and some explanatory text and links to other countries’ outputs included in NHS England’s own release.

2.17 During our assessment, we discussed the need to publish more information on quality and methods with the team, who agreed that this would be beneficial. The team shared comprehensive quality information with us, which should be made publicly available so that all users can reassure themselves about the quality of the statistics.

Presentation and accessibility of the statistics

2.18 There are limitations of the NHS England website that are outside of the team’s control. Despite this, there are still opportunities for the team to improve the presentation and accessibility of the statistics. Although the data are available in a variety of formats, the information is generally not presented in a manner that is accessible to a broad range of users. Most of the users we spoke to were experienced and so knew how to access the data they wanted. Many of them still expressed frustrations though and felt that less experienced users could struggle with the data as they are currently presented. The team was keen to engage with users further to identify how best to present the data to meet their needs.

2.19 The cancer waiting times release comprises a PDF summary and data available in both Excel and CSV format. This approach is consistent across NHS England, and we recognise that the team is limited in what improvements it can make to the release while it remains on the NHS England website. The team initially told us that there were plans in place for a unified content management system based on the existing legacy NHS Digital platform, which would allow for publishing in HTML and dashboard formats. This became uncertain in light of the announcement of the reorganisation, when the team reported that it was too early to know what the future platform for publications will be. The team remains committed to developing more visual displays of the data, however, and has told us it will continue to explore ways to do this.

2.20 The majority of users we spoke to were experienced with these statistics, and for the most part they downloaded data from the Excel or CSV files and imported them into their own dashboards or workbooks. They then carried out analyses such as trends over time and comparisons with similar trusts or England as a whole.

2.21 Some users told us that they found it time-consuming to download the data and carry out their own analyses each month. They felt that if there were a dashboard or other interactive presentation of the data within the release, it would save them a lot of time in having to compile and analyse the data themselves. In turn, the duplication of effort across several individuals would be significantly reduced. However, some users told us that they needed to be able to analyse the data themselves, so it is important for NHS England to retain the flexibility for users to do both.

2.22 Most users did not find the PDF helpful in its current format, saying that it was dry and lacking in detail. They felt that more information could be added to either the PDF or contained within a dashboard to provide more insight into the story of cancer waiting times.

2.23 It is important that the data are accessible to as wide a range of users as possible and not just analysts or those who are experienced with these statistics. Many users felt that although they were familiar with the layout of the publication and understood how to find the information they wanted, it had taken them many months to get to that stage. They felt that it could be confusing for those who were new to the data to find what they needed. Similarly, they were concerned that less-familiar users might miss important information due to the presentation of various pieces of data across different pages and files. Users also felt that while the data might be presented in a way that analysts can understand, non-analysts might struggle to extract data of interest to them.

Enhancing insights

2.24 Although NHS England has responded to the request for more-granular breakdowns of tumour type identified via its consultation, users told us about several requirements they have in relation to the data that are not being met at the moment. The team needs to consider what data they can make available in relation to these requirements, to enhance the insight the statistics provide.

2.25 A common theme identified from speaking to users was that they would like data on the faster diagnosis standard, with a breakdown for those who had cancer diagnosed versus those who had it ruled out. They explained that cancer is often ruled out more quickly than it is ruled in, so presenting a combined figure can give the appearance of better performance for those diagnosed with cancer than it is in reality.

2.26 In the past, NHS England has received requests for these data from various organisations and has published them as supplementary information. Users would like to see this breakdown included each month as part of the main release. The team in NHS England has committed to reviewing what is published as supplementary information with a view to identifying what can be regularly added to the release.

2.27 Other user needs that are not currently being met in relation to the data on cancer waiting times included more-granular breakdowns, for example for specific cancer types, and information on health inequalities associated with these statistics. The team told us that it needs to explore what is possible in relation to more-granular breakdowns due to the likelihood of small numbers and possible disclosure issues. Potential solutions included publishing these breakdowns at the national level only or publishing annually rather than monthly. Users indicated that either of these solutions could be acceptable.

2.28 In relation to providing information on health inequalities, the team told us that this could be more difficult to achieve in the short term due to the need to link the cancer waiting times data to other datasets containing information such as demographic variables. Linking these datasets would be necessary to provide the insight that users require on health inequalities as suitable variables are not available in the cancer waiting times dataset. The team committed to engaging with users to further understand their needs, with a view to developing longer-term aims for health inequalities breakdowns. These longer-term plans should be included in the development plans that NHS England publishes to satisfy Requirement 1.

2.29 One final item that several users would like to see alongside the main release is management information on the backlog of patients waiting longer than 62 days from an urgent referral for suspected cancer. NHS England explained to us that this would not be possible as the management information is derived from a different data source and is not subject to the same quality assurance procedures as the main cancer waiting times data. For users’ benefit, we encourage NHS England to explain to users both directly and via its website why this information is published separately and the difference between the two data sources, and to improve the signposting between the main release and the management information.

Reproducible analytical pipelines (RAPs)

2.30 The team told us that it has undertaken extensive work in the last year to automate the production of the cancer waiting times publication using RAP principles. The team felt that it is still fairly early in this journey compared to other teams across the organisation, and the code is still in its infancy. However, the team plans to improve and develop the existing code and explore making it publicly available.

2.31 There is also a large and growing RAP community within NHS England, which offers staff the opportunity to learn more technical skills, and members of the team are active within this community. We encourage the team to continue its work in this area and to draw on the expertise across NHS England where necessary to facilitate ongoing developments for the cancer waiting times statistics.