The introduction of new data sources can have both positive and negative impacts on statistical quality and producers need to manage and mitigate any associated risks.

The transformation of UK economic statistics is an example of where a producer has managed quality risks in its efforts to improve the statistics. As part of its continued ambition to improve UK consumer price statistics, ONS has introduced new data sources to improve its measurement of prices and is developing experimental measures of inflation to reflect the lived experience of households using scanner and web-scraped data.

As of March 2023, rail fare transaction data for Great Britain is now included in the Consumer Prices Index, with plans to incorporate new data and methods on second-hand car indices in spring 2024, though these were also originally planned for spring 2023. For second-hand cars, the data and methods are more complex. When incorporating new data sources, there is a need to carefully assess the accuracy and credibility of the information they provide. Given the high-profile nature of the consumer price statistics, and the need to have full confidence in the systems and quality assurance of data being incorporated into them, the team took the decision to delay inclusion of the second-hand cars data.

Different data structures, definitions and formats can create challenges and lead to errors in statistical outputs. In 2022, the implementation of updated Standard Occupational Classification (SOC) from SOC10 to SOC20 led to an error in some occupational data derived from ONS surveys. Subsequent analysis identified the scale and impact of the error and led to a large-scale re-coding exercise of affected data.

Sufficient data infrastructure, processes and systems are needed to support successful delivery of statistics across the UK. There is a diverse landscape of bespoke systems used in statistical production, ranging from modern to traditional legacy systems. Legacy systems (outdated software which is still in use) are often unequipped to deal with new sources of data and can be a contributing factor in statistical errors.

Producers are facing increasing challenges when using data collected from surveys. Declining response rates, sample biases, and data privacy concerns can have a significant impact on the quality of statistics, with increased reliance on statistical methods to ensure accurate and reliable insights. In addition, we are seeing a shift in approach to gathering social survey data, with face-to-face interviewing being increasingly replaced by digital collection methods. Recent examples include the ONS Labour Force Survey, the Natural England People and Nature Survey for England and the ONS COVID-19 Infection Survey.

Response rates, especially to social surveys, are falling. We have explored some of the issues around this in our public good research. We heard that an important factor is people not understanding why providing information is important. Low response rates can cause more variability and increased reliance on weighting methods to improve the accuracy of survey estimates. Low response rates in face-to-face interviews, and the resulting impact on quality, was a factor in ONS requesting the temporary suspension of National Statistics status for the estimates from the Crime Survey for England and Wales.

There are a number of projects underway to respond to this challenge and improve the quality of data derived from social surveys. In some cases, transformation projects are underway (for example, the Labour Force Survey and household financial statistics) to improve statistical quality. It is important to manage and mitigate any risks during any transition.

Administrative data are, by definition, data that are primarily collected for administrative or operational purposes. The use of such data in the production of official statistics can lead to challenges such as a lack of data completeness, variation in definitions, validity, accuracy and consistency.

In our regulatory work we continue to highlight the fundamental need for producers to understand any quality issues in administrative data and publish quality information. Strengths and limitations of the statistics and data should be clearly explained to support appropriate use and mitigate against the risk of misuse. As an example, in January, our assessment of Scottish prison population statistics, asked the Scottish Government to publish more-detailed information about its quality assurance approach.

The Northern Ireland Statistics and Research Agency (NISRA) carry out an annual quality audit providing an overview of quality management and performance against the NISRA Business Plan quality target, and includes an indicator around Quality Assurance of Administrative Data (QAAD) documentation. This organisation wide focus is helping to identify where improvements can be made, and efforts should be targeted.

The user demand for UK comparable statistics has continued over the past year. With differing government targets, policy measures and policies across the UK, it is inherently challenging for data and statistics to be directly comparable. As an example, for devolved matters such as health or education statistics are tailored to the needs of the individual nation, meaning that the same concept could be defined and measured in four different ways.

The 2021 Censuses of England and Wales, and Northern Ireland and Scotland’s Census 2022 are an example of the complexity of producing UK comparable statistics. The Census offices (National Records of Scotland, ONS and NISRA) agreed the conduct of the censuses in the UK, which included aspects where the Censuses would aim to achieve harmonisation.

Each office developed and implemented its own Census plans and made decisions about live census operations. In England, Wales and Northern Ireland the census was undertaken in 2021 which was during the COVID-19 pandemic. In Scotland, the Census collection was moved to 2022 at which point pandemic restrictions had been lifted. There were also differences in some of the questions asked in the censuses depending on the data needs of the respective country. For example, the Censuses for Scotland, England and Wales, asked voluntary questions about gender identity or trans status, whereas Northern Ireland did not. And there were differences between Scotland and England and Wales in how these questions were asked. The censuses highlight the complexities of producing comparable data across the UK.

Whilst it is not always possible to produce comparable statistics due to differing policies and services, these series can be very beneficial to users of statistics. As highlighted in our Lessons learned for health and social statistics from the COVID-19 pandemic: 2022 update, users of health and social care statistics have a strong interest in comparable UK-wide data. We have endorsed the approach taken by the Scottish Ambulance Service (SAS) in developing new operational statistics on unscheduled care in response to high public interest on the topic, particularly on the issue of ambulance response times. To help with the comparability of data across the UK, SAS is due to make changes to its methodology for response times to bring it in line with methods used across the rest of the UK.

The ONS is a key body in the statistical system for delivering initiatives across the UK. It is important that, in this leadership role, ONS ensures it consistently engages with the devolved administrations and any other relevant government departments. This is particularly important where the ONS is leading on transforming statistics on topic areas which have a UK wide importance, such as the future of the England and Wales Census.

It is important to note that the GSS has a statistical coherence work programme. Examples of recent work on improving coherence include the creation of the Health Statistics Leadership Forum, the prototype UK Climate Change Statistics Portal, and fuel poverty, with a recent article published by the GSS coherence team outlining the similarities and differences in the way fuel poverty is measured across the UK.