Executive Summary
Read the full report here (PDF)
Foreword
Estimating the UK’s population is not straightforward. It involves taking the figures from the last Census, and updating them for births, deaths, and migration. At the local level, there are complicating factors because the migration that is of interest is not just of people leaving the UK or coming to the UK, but also people moving from one place to another inside the country.
So ONS’s population estimates are challenging to produce, relying on a range of assumptions about how people move into, from and within the UK.
The ONS’s population estimates are also important. They provide insight into the size and location of the UK’s population – important in itself; but the estimates also feed into a range of other data sets, like household projections – and these in turn inform important operational and policy decisions.
In the light of these two characteristics of difficulty and importance, we conducted our review to consider whether ONS’s estimates and projections can always bear the weight that is put on them.
We found that ONS is taking a sensible approach, particularly at the national level, drawing on its own expertise and that of external experts. It conducts a very wide range of engagement activities to keep users informed about the statistics. The estimates are highly regarded, but there is a risk that ONS misses the bigger picture of what the population data inform and is not regularly sense checking what it does against local insight. Part of this sense checking involves drawing on the challenges from users in different parts of the country – in effect, for ONS to be open to the insights that come from people who say “those figures don’t reconcile with what we see in our area”. That is not to say that the insight should be taken without question. We are simply urging a creative conversation that regards this sort of feedback as useful intelligence to help sense check and quality assure the ONS estimates.
In short, then, we conclude in this review that ONS needs to build on what it does already and enhance its approach in three ways: improve methods; enhance communication; and embrace challenge.
Ed Humpherson
Director General for Regulation
Our review was initiated in response to concerns raised with us in November 2020 regarding the population projections and mid-year population estimates for Coventry. The concerns were around the perceived inaccuracies of the population estimates on which the household projections and subsequent housing need are based.
The population estimates and projections are important data with implications for many other statistics and which influence decision making by individuals, national and local governments. The projections feed into local planning decisions which can have a long-term commitment and therefore the impact of issues can have far reaching consequences.
It is not within our remit to regulate operational decisions made by government or local authorities, nor to form a judgement on decisions about government policy. Our review focused on the population estimates and projections in the context of the principles in the Code of Practice for Statistics.
What We found
ONS collaborates with a range of experts to determine the methods, data and assumptions which underpin the population estimates and projections. Its approach is generally seen as fit for purpose and is highly regarded internationally. One area of challenge has been migration, where there are limitations in the data available. ONS has sought to address this challenge by introducing some methodological fixes, such as the way students leaving university are identified. However, more needs to be done to investigate the root and scale of the issue associated with students and outward migration.
ONS has a number of methods for quality assuring the statistics, including deep dives, triangulation of data it holds and comparisons against historic data. ONS developed a range of variant projections to cater for the different uses of the data. We recommend that ONS develops case studies of how these variants are being used in practice to support their use more widely, as some users were unclear on which variants would best cater to their needs.
ONS engages regularly with experts, academia, and other users, and participates in relevant events and conferences. We found that while ONS is good at sharing its work outwardly, there is room for improvement in the way it takes on board feedback and handles challenge. We would like to see ONS be more open and responsive to issues when they first arise and view challenge as an opportunity to improve outputs and not a criticism of its approach. We recognise that ONS is balancing competing priorities, but a more open and constructive approach to responding to user feedback would create opportunities for ONS to continually improve the population estimates and projections, and ensure users feel listened to.
We have identified a number of actions for ONS to take in response to our findings. These are provided in more detail later in this report and are summarised below.
Improving methods
- ONS needs to investigate the root and scale of the issue associated with cities with large student populations and communicate its findings publicly, to support the appropriate use of the existing data.
- ONS needs to integrate a more flexible and responsive approach to methodological changes in its design for admin-based population estimates, working with its external partners, so that improvements are more timely.
- ONS should collaborate with others to incorporate local insight and carry out sensitivity analysis to enhance its approach to quality assurance.
Enhancing communication
- ONS should be open with users about its short-term solution to bridge the gap of migration data until the administrative data alternative is fit for purpose and ready to use.
- ONS should provide more specific guidance on interpreting the levels of uncertainty associated with the population estimates and projections, to help support the appropriate use of the statistics.
- ONS should develop case studies of where the variant projections have been used in practice and beneficial to users, to support their use more widely.
Embracing challenge
- ONS should take a more open and constructive approach to responding to user feedback, by improving its complaints procedure and viewing challenge as an opportunity to improve the statistics and outputs.
- ONS should collaborate with experts to frame the statistics for different audiences and scenarios.
Next steps
We expect ONS to reflect on our findings when developing its new admin-based approach to population estimates and projections. ONS should report back to us in July 2021 with its plans for addressing our recommendations. Further check points to discuss progress against plans will be arranged in the second half of the year.
To support the delivery of the recommendations, ONS should focus on determining whether the issues raised here have an impact on other official statistics. We would encourage ONS to engage with the devolved administrations, through its existing working level partnerships, to assess how the issues concerning students and emigration may impact their estimates and projections
Back to top