Executive Summary

Drawing on our work as the UK’s statistics regulator, this report shares our views on the performance of the statistical system and the challenges facing it, highlights areas of progress and innovation, and sets out our recommendations for advancing the system.

The UK’s statistical system includes those who collect, produce, disseminate and regulate official statistics, alongside central bodies that set strategic direction. We want this report to support the system in continuing to strive for excellence and working to address challenges collaboratively.

Our standalone guide, The Office for Statistics Regulation’s Guide to the UK Statistical System, provides an overview for readers who wish to know more about the different components of the statistical system in the UK.

The statistical system has benefited from the publication, in 2024, of two wide-ranging reviews:

• The Independent Review of the UK Statistics Authority (UKSA), commissioned by the Cabinet Office as part of its Public Bodies Reviews program and conducted by Professor Denise Lievesley CBE, focused on the governance, accountability, efficacy and efficiency of the UKSA.
• The Public Administration and Constitutional Affairs Committee (PACAC), responsible for overseeing the work of the UKSA on behalf of Parliament, carried out an inquiry exploring the changing statistics and data landscape, and the UK’s readiness to respond to it. Its report, Transforming the UK’s Evidence Base, was published in May 2024.

We have referenced specific findings from these reviews where these are relevant in this report.

The current landscape

Producers have continued to see increased demands for new statistics and insight, but with less focus on a single high-profile issue than we have seen in previous years. In the main, producers have been able to respond to these demands. However, financial and resource pressures and recruitment challenges have meant that several producers have had to cut back existing outputs to invest in new priorities. This mismatch between the demands for data and the supply of resources has placed the system under strain, meaning that it is increasingly difficult to meet user needs. It is important that core statistics be sufficiently resourced and funded to serve the public good.

Increased demand and financial constraints will remain important pressures in the year ahead. To be at its best, it is important that the Government Statistical Service (GSS) build on the development of its GSS Vision and GSS Strategic Delivery Plan and key appointments. In face of financial constraints, demand for central GSS support provided by the Office for National Statistics (ONS) outstrips supply. In our view, some services are delivered more cost effectively by the GSS centrally than by individual producer organisations. Therefore, the system should consider how it can best deliver a central service that promotes and supports the profession.

User engagement is vital if the statistical system is to produce evidence that meets users’ needs and answers their questions; it should underpin everything the system does. Through our regulatory work, we have seen many examples of good engagement, but users have continued to tell us that they would like to see statistics producers engage more widely, listen more (including to experts who can support new developments) and be more open and less defensive when facing criticism. We recommend that the system invest more heavily and strategically in its approach to engagement. The Statistical Assembly stemming from the Lievesley Independent review of the UK Statistics Authority will be key to making this a reality.

Ensuring quality

Long-standing challenges with household survey response rates became more critical, with impacts such as the suspension of UK survey-based labour market estimates in October 2023 due to falling response rates in the ONS Labour Force Survey, which covers England, Scotland and Wales. Surveys will remain essential in some areas, but it has become more challenging and expensive to maintain quality. This underlines the importance of different collection approaches and nudge techniques to deliver the best-quality estimates. Where appropriate, administrative data can also provide part of the solution.

Our economic statistics-focused Spotlight on Quality programme has identified the need for the statistical system to do more to ensure it is using suitable data sources to measure the modern economy and is enhancing the communication of economic statistics so that users fully understand the quality of the data, any sources of uncertainty and the impact of the trade-off between balancing timeliness and accuracy on revisions performance.

Improvement and innovation

We have seen a range of innovative transformation programmes across the statistical system. To maximise the benefits, we want to see more join-up and coherence across developments. Overarching plans and priorities should be set with, and communicated to, users of statistics.

Users have demanded increased comparability of statistics across the four nations in topics like health, and we see producers taking a leading role in improvements in comparability and coherence – helping to make it clearer to users when statistics can and cannot be looked at together. It is important that the system, co-ordinated by ONS, build in user needs for UK-wide coherence from the outset, working in partnership across the UK.

Significant progress is still needed in overcoming many of the remaining, often longstanding, barriers to data sharing and linkage. Despite the value of sharing and linking data being widely recognised, and pockets of innovative and ambitious work, there remain areas of challenge. These include the public’s attitude to, and confidence in, data sharing and culture and processes in government.

The Lievesley Independent review of the UK Statistics Authority and the PACAC Transforming the UK’s Evidence Base report highlight the need to overcome the systemic barriers to data sharing between departments. Unless significant changes are implemented, we are concerned the progress that has been made could be lost. We want to see all the parts of the system working together to address these challenges, to reach a place where sharing and linking datasets, and using them for research and evaluation, is the norm across the UK statistical system, rather than the exception.

With developments in artificial intelligence, it is vital that the statistical system be equipped to maximise the opportunities and address the challenges that will arise in this area. The statistical system, including OSR as its regulator, should show strong leadership in the AI era by setting relevant standards based on the Code of Practice for Statistics. This includes being transparent about the real and perceived risks of using AI in official statistics and how these are addressed, to build public confidence.

Effective communication

We have seen producers effectively communicating key messages from rich and complex datasets through a variety of means that are tailored to different audiences. Census outputs in particular were not just published as data tables and bulletins, but through innovative interactive maps, tools and games which allow users to better engage with and understand data related to their own lives and geography. However, more needs to be done to embed high-quality communication around quality, uncertainty and revisions when data are published and used, including sharing best practice.

It is important to take an open, clear and accessible approach to the release and use of data and statistics – we call this ‘intelligent transparency’, the principles of which are now well embedded across the analytical professions. We see government departments increasingly following these principles and making underlying analysis available when statements using unpublished data are made. However, concerns are still being raised with us in relation to ministers and other government officials quoting unpublished figures or figures that lack context in the public domain. We want intelligent transparency to be the default approach to releasing and using data and statistics across government. Transparency and good use of data were particularly important in the General Election campaign, and our dedicated election pages on our website supported these approaches.