Office for Statistics Regulation Annual Report 2021/22

Our performance

Delivery of the key commitments set out in our 2021/22 Business Plan.

How statistics are produced

We uphold trustworthiness, quality and value of statistics and data used as evidence.

2021/22 Highlights

Our review of the ONS population estimates and projections focused on the statistics in relation to the Code of Practice (May 2021)
Following an intervention in 2018, the Police Funding for England and Wales statistics were given NS designation (July 2021)
The state of the UK’s statistical system 2020/21 report highlighted the impact the pandemic had on the statistical system (July 2021)
We published our Guidance for Models which helps to support designing, developing and using models (October 2021)
We published our review of climate change statistics and particularly focused on their accessibility and coherence (October 2021)
We completed the phase 2 assessment reports for the 2021 Census for England and Wales and Northern Ireland (November 2021)
A blog explored what we mean by effective analytical leadership and how TQV supports this (January 2022)
The review of transport accessibility statistics allowed us to explore a data gap in transport statistics (February 2022)
The review of migration statistics produced by ONS helped us ensure that migration statistics are trusted, high quality and of value (March 2022)

Outputs

Year ending 31 March 2022
Assessment Reports	12 reports published (compared with 5 the previous year)
Assessments closed*	8 closed
Designations as National Statistics	7 designations, of which: 3 confirmations of existing statistics and 5 new (compares with 3 new designations the previous year)
De-designations	None
Compliance Checks and Rapid Reviews	44 closed (compared with 47 the previous year)
Systemic Reviews	11 reviews published (compared with 7 the previous year)

*An Assessment is closed once we have reached a final decision on designation; the publication of the Assessment report is an interim stage. 

Outcomes

Evolving statistical production
Consider implications of decisions made during pandemic – balancing quality with having what we need at the right time – and what this means for our judgement of quality of statistics, including contributing fully to the review of health and social care statistics commissioned by the Public Administration and Constitutional Affairs Committee (PACAC)	Partially delivered: In October 2021 we published our review of health and social care statistics during the pandemic, identifying ten key lessons. We promoted this with a well-attended seminar and have published guest blogs on the topic, including from Sir David Spiegelhalter. We are seeing examples though our regulatory work of producers revaluating what they can produce, and we challenge where they are not, but our work on considering quality has been more limited than we’d planned, with more to do.
Review impact of pandemic on data collection, including for the UK National Accounts	Partially delivered: We have considered this in our regulatory work – for example, in our Compliance Check of the effect of the pandemic and Brexit on trade statistics we found that ONS compensated well for lack of usual data sources, improving imputation methods and working with others to develop and improve alternative data collections. This is an area where we need to build on what we already know and further embed our thinking into our regulatory guidance.
Examine specific quality concerns, for example – we will undertake an assessment of the Living Costs and Food Survey following concerns over the survey’s ability to deliver good quality estimates	Fully delivered: We published our assessment of the Living Costs and Food Survey in July 2021 and ONS committed publicly to deliver on several streams of work to increase the quality and public value of the survey. They are reporting progress each quarter. We continue to pursue requirements from our assessments where we have highlighted quality concerns and want to see more progress – for example, Business Demography.
Continue our assessment of Census 2021	Fully delivered: Following our assessment reports in November 2021, we were able to confirm NS designation of the 2021 Census in Northern Ireland in May 2022, and of the 2021 Census in England and Wales in June 2022.

Granularity of statistics
Review regional statistics to measure changes in regional performance. The Treasury Select Committee has expressed an interest in the regional GDP estimates being assessed for potential National Statistics designation during 2021-22	Fully delivered: We have been working in the public interest on regional disparities and encouraging a statistical system that is more responsive to regional and local demands. We have reviewed key ONS surveys that feed into economics statistics (Annual Business Survey, Business Demography, Annual Purchases Survey); focused on driving improvements in sub-UK economic statistics (ONS Small Area Income Estimates, NISRA BESES statistics, ONS FDI statistics, ONS Regional HFCE statistics, HMT CRA statistics, ONS GVA statistics) and provided evidence to the Treasury Committee’s inquiry on regional imbalances in the UK economy. We will continue to review new statistics and data sources from the Department for Levelling Up, Housing and Communities, Office for National Statistics and other data providers as they are developed to ensure that evidence and evaluation is at the forefront of pushing the plans forward.
Establish a framework, strategy and priorities for our role in regulating statistics on inequalities. We will also explore topics that support inclusivity in society – for example, accessibility of transport networks, shining a light on where the granularity of data do not allow us to understand the lived experience	Partially delivered: We have established a cross-cutting Equalities Forum within OSR to draw insights from our regulatory work and identify priorities for regulatory activity. In February 2022 we published a review of transport accessibility statistics, recommending that statistics are improved to better reflect the lived experiences of disabled people, ensuring that everyone who should be is counted and we are looking to our recommendations to drive positive change. We continue to intervene where we identify concerns – for example, writing ONS in March 2022 about the potential for their 2019 publication ‘What is the difference between sex and gender?’ to cause confusion, leading to work to provide greater clarity – and to speak publicly – for example, providing oral evidence to Parliament on the role OSR can play in supporting the Inclusive Data Taskforce recommendations from September 2021.

Granularity of statistics

Review regional statistics to measure changes in regional performance. The Treasury Select Committee has expressed an interest in the regional GDP estimates being assessed for potential National Statistics designation during 2021-22

Fully delivered: We have been working in the public interest on regional disparities and encouraging a statistical system that is more responsive to regional and local demands. We have reviewed key ONS surveys that feed into economics statistics (Annual Business Survey, Business Demography, Annual Purchases Survey); focused on driving improvements in sub-UK economic statistics (ONS Small Area Income Estimates, NISRA BESES statistics, ONS FDI statistics, ONS Regional HFCE statistics, HMT CRA statistics, ONS GVA statistics) and provided evidence to the Treasury Committee’s inquiry on regional imbalances in the UK economy. We will continue to review new statistics and data sources from the Department for Levelling Up, Housing and Communities, Office for National Statistics and other data providers as they are developed to ensure that evidence and evaluation is at the forefront of pushing the plans forward.

Establish a framework, strategy and priorities for our role in regulating statistics on inequalities. We will also explore topics that support inclusivity in society – for example, accessibility of transport networks, shining a light on where the granularity of data do not allow us to understand the lived experience

Partially delivered: We have established a cross-cutting Equalities Forum within OSR to draw insights from our regulatory work and identify priorities for regulatory activity. In February 2022 we published a review of transport accessibility statistics, recommending that statistics are improved to better reflect the lived experiences of disabled people, ensuring that everyone who should be is counted and we are looking to our recommendations to drive positive change. We continue to intervene where we identify concerns – for example, writing ONS in March 2022 about the potential for their 2019 publication ‘What is the difference between sex and gender?’ to cause confusion, leading to work to provide greater clarity – and to speak publicly – for example, providing oral evidence to Parliament on the role OSR can play in supporting the Inclusive Data Taskforce recommendations from September 2021.

Data gaps
Monitor new COVID-19 data requirements and producer plans for meeting those needs: for example, on survival rates, long-Covid, impacts on births and maternity provision, divorce and families, adult social care, suicides, and mental health. We will look at how far there will be a sufficient evidence base to inform social and economic policy around the long-term consequences of the pandemic, including the impact of COVID-19 on children and young people into the future	Partially delivered: In March 2022 we published a review of children and young people statistics: Visibility, Vulnerability and Voice: The importance of including children and young people in official statistics, supported by a public event. Our report explored how well the needs are being met, particularly given the impact of the pandemic on children and young people. We will continue to extend this thinking to other groups and issues.
Understand how well the current statistics and data meet the current and future needs of society and are delivering public value	Partially delivered: This is an ongoing goal. We have maintained reasonable contact with senior decision makers in Government, especially chief scientific advisors and directors of analysis. This provides good intelligence on areas of emerging interest, including crime and AI in government. We have provided thought leadership through various channels, including a blog by our Director General on the importance of evaluation, and holding our own public events as well as contributing to others, in the UK and internationally. Our new programme of events for Analysts in Government should continue to support a widened dialogue around public value.

Data gaps

Monitor new COVID-19 data requirements and producer plans for meeting those needs: for example, on survival rates, long-Covid, impacts on births and maternity provision, divorce and families, adult social care, suicides, and mental health. We will look at how far there will be a sufficient evidence base to inform social and economic policy around the long-term consequences of the pandemic, including the impact of COVID-19 on children and young people into the future

Partially delivered: In March 2022 we published a review of children and young people statistics: Visibility, Vulnerability and Voice: The importance of including children and young people in official statistics, supported by a public event. Our report explored how well the needs are being met, particularly given the impact of the pandemic on children and young people. We will continue to extend this thinking to other groups and issues.

Understand how well the current statistics and data meet the current and future needs of society and are delivering public value

Partially delivered: This is an ongoing goal. We have maintained reasonable contact with senior decision makers in Government, especially chief scientific advisors and directors of analysis. This provides good intelligence on areas of emerging interest, including crime and AI in government. We have provided thought leadership through various channels, including a blog by our Director General on the importance of evaluation, and holding our own public events as well as contributing to others, in the UK and internationally. Our new programme of events for Analysts in Government should continue to support a widened dialogue around public value.

How statistics are used

We protect the role of statistics in public debate.

2020/21 Highlights

We published our annual review of UKSA casework in 2020/21 which shows we completed 323 cases that year (September 2021)
A blog on communicating data explained the importance of clearly explaining any decisions made and context around the data (November 2021)
We welcomed a blog by ONS clarifying the data from one of its outputs after it was being misused (January 2022)
Our regulatory guidance for the transparent release and use of statistics and data sets out our expectations for both producers and users of statistics (February 2022)
The Chair of the UKSA wrote to Alistair Carmichael MP around the use of official crime statistics (February 2022)
The Chair of the UKSA also wrote to the Prime Minister around the number of people in work (February 2022)

Outputs

Year ending 31 March 2022
Cases opened	241 (compares with 323 the previous year)
Time to close a case (days – median)	10 (equal to the previous year)
Time to close a case (days – mean)	18 (compares with 14 the previous year)
Cases still open at year end	8

Outcomes

Respondents in the Public Confidence in Official Statistics 2021 survey were asked to what extent they agree with the statement “it is important for there to be a body such as the UK Statistics Authority to speak out against the misuse of statistics”. The majority (96%) of respondents able to express a view agreed with this statement, with a similar number (94%) agreeing that it is important to have a body who can ensure that official statistics are produced free from political interference. While we are cautious about putting too much weight on these two questions in the survey, these findings may at the very least indicate the public value the independent production of statistics, as well as challenges to the misuse of statistics.

We develop a better understanding of the public good of statistics

2020/21 highlights

Our review of mental health statistics in Northern Ireland helped us understand the public value of mental health statistics across the UK (September 2021)
We have reviewed the National statistics designation and explained how to make them useful (October 2021)
Our blog on what we mean by ‘statistics that serve the public good’ explores our vision (January 2022)
We changed the Code of Practice to allow release at times other than 9.30am where it is shown to better serve the public good (March 2022)

Outputs

Leading a better understanding of the public good is the foundation of our vision. We cannot deliver on our vision of statistics that serve the public without a proper understanding of what this means. Following a phase of wide consultation with analysts and academics – and publication of our findings – we are now focused on understanding the public perspective, with questions on the Public Confidence in Official Statistics Survey due for analysis in summer 2022 and working with ADR UK to procure engagement with the public on perceptions of the public good.

The Authority Board has endorsed the conclusions of two important reviews in the last couple of months – our policy consultation on allowing producers to release statistics at other times than 9.30am under certain conditions, and our review of the NS designation, where we are now in the planning phase for implementation.

There are three dimensions to our data and methods programme: applying data science techniques to support our regulators, including development of a dashboard and using web scraping to support assessments; data and methods in government; and AI policy. As part of consolidating our role in data, we have released the first iteration of guidance on how the Code of Practice can support good practice in designing, developing, and using models, which we plan to develop further in the coming year.

Outcomes

Consolidating our role in data
Review the impact of data linkage on the statistical system to understand how common and effective progress has been since our report in 2019. We are aware of some good examples of progress but we need to get a sense of the landscape	Not delivered: Extensive engagement with key players in the data landscape have given us a much better picture of the current issues and concerns. We’re not yet delivering direct outcomes from this work but we have a stronger platform now and we have identified this as a priority for 2022/23.
Follow up on the findings of our review of the approach to developing statistical models designed for awarding 2020 exam results, which identifies lessons for public bodies considering the use of statistical models to support decisions and publish our guidance on statistical models	Fully delivered: In October 2021 we published guidance on how the principles in the Code can support good practice in designing, developing and using models. In March 2022 we tested the lessons that came out of the guidance and of our review of the statistical models designed to award grades in 2020. We applied our lessons learnt to QCovid® – a different sort of algorithm – this provided assurance about the use of the model and identified further lessons which will continue to inform updates to our guidance planned for 2022/23.
Follow up on our Reproducible Analytical Pipelines (RAP) Review	Fully delivered: We are regularly engaging with producers about RAP – for example, producing a video for ONS to use in their training course – and we have started to publish Code Case Studies based on producer experiences. Recent guest blogs confirm how we are helping to facilitate RAP where we can and we have embedded our thinking into how we regulate and our published reports, drawing out recognised good practice and where there is scope to do more.

Public good
Conduct primary research with the public to understand definitions of public good	Fully delivered: When assessing the value of statistics in OSR, two of the key factors we consider are relevance to users and accessibility. The findings from PCOS 2021, published in April 2022, gave much cause for celebration on these measures, while also raising important questions to explore further in our Research Programme on the Public Good of Statistics. We are following up with a series of workshops with the public.
Complete our review of the National Statistics designation, presenting recommendations to the Authority Board and developing and user testing proposed solutions	Fully delivered: We completed our review and we are reframing the designation to focus on its characteristic as an independent review that confirms the statistics comply with the Code by showing trustworthiness, quality, and value. Independent review was one of the benefits of the designation that producers described, providing a different, external perspective on the statistics. It also meets one of the concerns of users, to ensure producers are held to account. The project to refresh the designation is underway.

Public good

Conduct primary research with the public to understand definitions of public good

Fully delivered: When assessing the value of statistics in OSR, two of the key factors we consider are relevance to users and accessibility. The findings from PCOS 2021, published in April 2022, gave much cause for celebration on these measures, while also raising important questions to explore further in our Research Programme on the Public Good of Statistics. We are following up with a series of workshops with the public.

Complete our review of the National Statistics designation, presenting recommendations to the Authority Board and developing and user testing proposed solutions

Fully delivered: We completed our review and we are reframing the designation to focus on its characteristic as an independent review that confirms the statistics comply with the Code by showing trustworthiness, quality, and value. Independent review was one of the benefits of the designation that producers described, providing a different, external perspective on the statistics. It also meets one of the concerns of users, to ensure producers are held to account. The project to refresh the designation is underway.

« Previous