Introduction

Official statistics are an essential public asset. The UK Statistics Authority works to promote, monitor and safeguard the production and publication of official statistics. In accordance with the statutory requirements set by the Statistics and Regulation Service Act 2007, we strive to ensure that official statistics are accessible, reliable and that they serve the public good. 

The Authority’s casework function plays an important role in building public confidence in the production and use of official statistics. The Office for Statistics Regulation (OSR) is responsible for the Authority’s casework function, undertaking monitoring to identify issues and investigating issues raised with the Authority. This report provides a summary of casework for the period from 1 April 2020 to 31 March 2021 (2020/21). 

This year, our report is split into three sections – Issues, Impact and Management. It provides a summary of the types of issues we have considered, the impact we have seen from our interventions, and some background on our processes and management approach. The report is supported by two annexes. Annex A provides a breakdown of the management information related to our casework, and Annex B provides examples of where our casework has been referenced in parliamentary debate. 

To find out more about our casework process and how to submit a concern visit our Casework page

 

About the data

  • Percentages have been rounded to the nearest whole per cent.
  • When calculating response times, cases that were closed on the same day they came in, have a 1 day response time, rather than 0.
  • Internally generated cases are issues that have been noticed by internal staff. The OSR team monitor the use of statistics in public debate including social media.
  • The categories provided in this report are provided to give an indication of the areas and issues covered by casework. There may be cases which could be classified as multiple categories, in these cases a decision has been made on which category is most relevant. In each case only one has been selected. As part of a range of process improvements OSR is looking to develop the categories and make improvements to the management information associated with Casework.
  • Casework issues are broadly split into two categories – use/misuse of statistics, and compliance with the Code of Practice for Statistics. Use/misuse is where there are concerns that statistics have been used inappropriately or incorrectly in public debate (whether deliberately or not). Code compliance assesses whether official statistics have been produced and presented in line with the Code of Practice for Statistics. OSR also considers whether official statistics meet the expectations of the code through its range of other regulatory work.
  • The time taken to complete casework can have a large level of skew and so, where possible, both the median and the arithmetic mean have been calculated. In order to offset the effects of this skewness the median is the headline measure used in discussion.
  • You can see all cases considered during 2020/21 in the 2020/21 issues log. This includes information on the date the case was opened, date closed, description and outcome for all 323 cases considered in 2020/21. Issues logs are linked via the casework page on our website.

Back to top
Download PDF version (521.01 KB)

Review of population estimates and projections produced by the Office for National Statistics

Published:
10 May 2021
Last updated:
11 May 2021

Executive Summary

Read the full report here (PDF)

 

Foreword

Estimating the UK’s population is not straightforward. It involves taking the figures from the last Census, and updating them for births, deaths, and migration. At the local level, there are complicating factors because the migration that is of interest is not just of people leaving the UK or coming to the UK, but also people moving from one place to another inside the country.

So ONS’s population estimates are challenging to produce, relying on a range of assumptions about how people move into, from and within the UK.

The ONS’s population estimates are also important. They provide insight into the size and location of the UK’s population – important in itself; but the estimates also feed into a range of other data sets, like household projections – and these in turn inform important operational and policy decisions.

In the light of these two characteristics of difficulty and importance, we conducted our review to consider whether ONS’s estimates and projections can always bear the weight that is put on them.

We found that ONS is taking a sensible approach, particularly at the national level, drawing on its own expertise and that of external experts. It conducts a very wide range of engagement activities to keep users informed about the statistics. The estimates are highly regarded, but there is a risk that ONS misses the bigger picture of what the population data inform and is not regularly sense checking what it does against local insight. Part of this sense checking involves drawing on the challenges from users in different parts of the country – in effect, for ONS to be open to the insights that come from people who say “those figures don’t reconcile with what we see in our area”. That is not to say that the insight should be taken without question. We are simply urging a creative conversation that regards this sort of feedback as useful intelligence to help sense check and quality assure the ONS estimates.

In short, then, we conclude in this review that ONS needs to build on what it does already and enhance its approach in three ways: improve methods; enhance communication; and embrace challenge.

Ed Humpherson

Director General for Regulation

Our review was initiated in response to concerns raised with us in November 2020 regarding the population projections and mid-year population estimates for Coventry. The concerns were around the perceived inaccuracies of the population estimates on which the household projections and subsequent housing need are based.

The population estimates and projections are important data with implications for many other statistics and which influence decision making by individuals, national and local governments. The projections feed into local planning decisions which can have a long-term commitment and therefore the impact of issues can have far reaching consequences.

It is not within our remit to regulate operational decisions made by government or local authorities, nor to form a judgement on decisions about government policy. Our review focused on the population estimates and projections in the context of the principles in the Code of Practice for Statistics.

What We found

ONS collaborates with a range of experts to determine the methods, data and assumptions which underpin the population estimates and projections. Its approach is generally seen as fit for purpose and is highly regarded internationally. One area of challenge has been migration, where there are limitations in the data available. ONS has sought to address this challenge by introducing some methodological fixes, such as the way students leaving university are identified. However, more needs to be done to investigate the root and scale of the issue associated with students and outward migration.

ONS has a number of methods for quality assuring the statistics, including deep dives, triangulation of data it holds and comparisons against historic data. ONS developed a range of variant projections to cater for the different uses of the data. We recommend that ONS develops case studies of how these variants are being used in practice to support their use more widely, as some users were unclear on which variants would best cater to their needs.

ONS engages regularly with experts, academia, and other users, and participates in relevant events and conferences. We found that while ONS is good at sharing its work outwardly, there is room for improvement in the way it takes on board feedback and handles challenge. We would like to see ONS be more open and responsive to issues when they first arise and view challenge as an opportunity to improve outputs and not a criticism of its approach. We recognise that ONS is balancing competing priorities, but a more open and constructive approach to responding to user feedback would create opportunities for ONS to continually improve the population estimates and projections, and ensure users feel listened to.

We have identified a number of actions for ONS to take in response to our findings. These are provided in more detail later in this report and are summarised below.

Improving methods

  • ONS needs to investigate the root and scale of the issue associated with cities with large student populations and communicate its findings publicly, to support the appropriate use of the existing data.
  • ONS needs to integrate a more flexible and responsive approach to methodological changes in its design for admin-based population estimates, working with its external partners, so that improvements are more timely.
  • ONS should collaborate with others to incorporate local insight and carry out sensitivity analysis to enhance its approach to quality assurance.

Enhancing communication

  • ONS should be open with users about its short-term solution to bridge the gap of migration data until the administrative data alternative is fit for purpose and ready to use.
  • ONS should provide more specific guidance on interpreting the levels of uncertainty associated with the population estimates and projections, to help support the appropriate use of the statistics.
  • ONS should develop case studies of where the variant projections have been used in practice and beneficial to users, to support their use more widely.

Embracing challenge

  • ONS should take a more open and constructive approach to responding to user feedback, by improving its complaints procedure and viewing challenge as an opportunity to improve the statistics and outputs.
  • ONS should collaborate with experts to frame the statistics for different audiences and scenarios.

Next steps

We expect ONS to reflect on our findings when developing its new admin-based approach to population estimates and projections. ONS should report back to us in July 2021 with its plans for addressing our recommendations. Further check points to discuss progress against plans will be arranged in the second half of the year.

To support the delivery of the recommendations, ONS should focus on determining whether the issues raised here have an impact on other official statistics. We would encourage ONS to engage with the devolved administrations, through its existing working level partnerships, to assess how the issues concerning students and emigration may impact their estimates and projections

Back to top
Download PDF version (295.59 KB)

Reproducible Analytical Pipelines: Overcoming barriers to adoption

Published:
23 March 2021
Last updated:
21 April 2021

Ensuring statistical models command public confidence

Published:
2 March 2021
Last updated:
2 March 2021

COVID-19 surveillance and registered deaths data review

Published:
22 April 2020
Last updated:
11 June 2020

NHS Digital data access update

Published:
2 April 2020
Last updated:
28 July 2020

National Statistics Designation Review – Phase 1 Exploratory Review

Published:
16 March 2020
Last updated:
28 September 2020

NHS Digital data access review update – May 2019

Published:
22 May 2019
Last updated:
21 May 2019

International perspective on criminal justice statistics

Published:
10 October 2017
Last updated:
15 October 2018