What we found

Data sources

2.1 ONS has taken an ambitious and open approach to data sources. It is drawing on different sources from across Government and continuing to explore the potential of new sources to improve the accuracy of its estimates. Currently the two major sources in the strategic approach are the RAPID dataset (Registration and Population Interaction Database) sent to ONS annually and the Home Office Border Data. RAPID aims to identify a migrant through their interactions with earnings and benefit systems. The tactical approach draws on datasets including Home Office exit checks data, the Civil Aviation Authority (CAA) data and uses trend and seasonality elements from the historic IPS.

2.2 Over time, ONS plans to integrate data across more input datasets to ensure a wider population coverage and enable more granular reporting. For example, as part of the ABME approach ONS plans to include further Home Office administrative data, HM Revenue and Customs Pay As You Earn Real-time Indicators (PAYE- RTI) (and Self-Assessment), NHS Hospital Episode Statistics, Further Education data (AEDE), Council Tax and others such as Electoral Register and DVLA registrations. Currently, RAPID data is available once a year, it is hoped that in the future it will be available more frequently.

2.3 ONS is aware of, and has set out, the limitations of the data used in its current approaches. For example, the strategic approach will not provide complete coverage of the UK population and some population groups will be hard to estimate. Non-working students is one such group and ONS is investigating ways to best estimate migration in this group. There are also other concerns about the use of administrative data to measure migration, such as the issue that many people can still interact with these administrative systems while living abroad. Also, these administrative data sources all have a different primary purpose which is not directly related to migration and therefore consideration needs to be taken when using and linking these data sources.

2.4 As ONS progresses with its plans, it should make clear which data sources will be feeding in to both the ABME (strategic) and provisional state space model (tactical) approaches, what their strengths and limitations are, and how these additional sources will contribute to improving the migration estimates. For example, some data sources may only be updated annually, and certain sub-groups will therefore only be updated annually. Also, it may be that the provisional estimates are only appropriate to be used for national level estimates, with the final estimates providing the more granular data.

2.5 In general, the user responses to ONS’s approach to using administrative data sources were positive. However, it was noted that these sources are likely to be good at measuring population inflows but less good at capturing those who are leaving the country. Some users did comment that they are waiting until more outputs are available which use the ABMEs before making a judgement.

Methods

2.6 ONS has taken an innovative approach to estimating migration in the absence of the IPS and has been collaborative in its decision making around methods. As mentioned above, ONS currently has two primary ways of measuring international migration, the strategic approach, which produces behaviour-based estimates from administrative sources, and the tactical solution, which uses state space modelling to produce more timely estimates of migration flows.

2.7 The development of the tactical approach was done in collaboration with academics from the University of Southampton and Warwick and came about following the suspension of the IPS in February 2020. A state space model was developed that took the trend and seasonality elements from the historic IPS and additionally applied the continuing time series from other sources. Currently, ONS publishes on a 6-monthly basis with quarterly breakdowns. It is looking to set up data feeds that would allow monthly publication of estimates. State space models are generally considered to deal well with shock events such as the impacts of COVID-19 and EU Exit. However, underpinning assumptions should be reviewed, e.g.: the assumption that EU and non-EU leaving rates are the same – although ONS plans to introduce RAPID data into the model which should address this issue. More information on state space models is included in Annex A.

2.8 However, some aspects of state space models can be challenging to explain to users without touching on quite complex areas. If users feel the model explanation is inaccessible it may undermine confidence in the estimates and users may resist using the estimates produced by the model.

2.9 ONS is continuing to develop its strategic approach to estimating migration, using data from across Government. It has produced two admin-based migration series The first ABME measure is based on the RAPID dataset developed by DWP to provide a view of an individual’s interactions across DWP, HM Revenue and Customs (HMRC) and local authorities (via Housing Benefit) datasets. These interactions include benefits, employment, self-employment, pensions and in-work benefit. RAPID contains a record of everyone who has a National Insurance number (NINo) and for each, the number of weeks of activity (or interactions) within these systems is summarised for each year from tax year. The second ABME measure is based on Home Office Exit Checks data. The method looks at first arrival and last departure within a visa period as an approximation for length of stay in the UK. Work on refining these methods continues including the extension of the use of the RAPID data to include UK nationals and working more closely with the Home Office to include data on EU nationals as it becomes available through their new migration system. ONS is also working on resolving some of the challenges in using RAPID and Home Office Exit Checks data in a complimentary way. For example, understanding the overlap between the two datasets.

2.10 ONS has not yet provided details of its approach to calculating uncertainty associated with its ABME estimates. In order for estimates to be used appropriately, it is important that users understand how the estimate may differ from the “true value” and the degree of confidence that can be placed in the outputs.

2.11 ONS plans to combine the strategic (ABMEs) and tactical (state space model) approaches to provide more timely, higher quality estimates. Greater use of administrative information should reduce the need for some of the assumptions currently present in the modelling and help to improve the accuracy of the estimates. The challenge with the ABME approach is the timeliness of the data, as ONS is waiting to observe whether a person remains in the country for 12 months. To overcome this challenge, ONS is proposing to provide provisional estimates based mainly on the modelled state space method. The final estimate would then be based mainly on the administrative data method. This approach – having provisional and then final estimates – has been used successfully by ONS for other statistics. The highest profile of which is GDP. ONS is intending to discuss with users how to define a migrant, and it could be that in the future, the 12 month definition may become more flexible with shorter durations also included in the publication.

2.12 Additionally, as part of the overall population transformation programme ONS is exploring Demographic Accounts as a method to draw together and achieve fully coherent estimates across population and migration through time. ONS will publish more on this work later in 2022. Under this approach, Bayesian methods are being used to estimate population counts and rates by constructing models for observed data sources and demographic systems. The resulting demographic account is a framework that is built up with layers. This begins with a simple population framework including population stocks, and components of population change due to births, deaths, internal migration and international migration, but can be extended to include more demographic components such as country of birth or household structure.

2.13 Changes to methods will impact on a consistent time series. It is not yet clear how ONS will handle this. For example, in April 2021, ONS published data comparing migrant flows using RAPID and Home Office border data over the period year ending March 2012 to year ending March 2020. The estimates from RAPID showed higher net migration for EU nationals and lower net migration for non-EU nationals compared with Long Term International Migration (LTIM) estimates based on the IPS. Additionally, the 2021 Census results may provide additional information which impacts historical migration estimates.

Quality assurance

2.14 In terms of validating migration stocks and flows, there is no ‘gold standard’. The results from Census 2021 in England, Wales and Northern Ireland will start to become available throughout 2022 and will be crucial to validate the migration stocks for both the tactical and modelled approaches. However, the Scottish Census does not take place until Spring 2022 so data covering the whole of the UK is unlikely to be available until late 2023 at the earliest and we do not know the impacts of the pandemic on this data. ONS also plans to use the Census results to validate migration flows but this will be of more limited use as it is based around a question about where the respondent was one year ago.

2.15 ONS has currently used student accommodation data, estimates from the Labour Force Survey (LFS), Advanced Passenger Information (API) data on arrivals and departures for validation to see if there are regular patterns evident across the sources.

2.16 The Annual Population Survey (APS) is also likely to become a quality assurance source although there are current concerns around the bias and previous reweighting as discussed above. ONS is working on the new Labour Market Survey (LMS) to replace the LFS/ APS which will hopefully be more robust.

User engagement and communication

2.17 ONS engages well with expert users about migration statistics through a variety of forums and groups including the Migration Expert Users Forum, the GSS Migration Steering Group, the Migration Statistics Users Forum and the UK population theme advisory group (UKPTAB). It also engages with users through bilateral meetings on specific issues and topics. The users we spoke with were positive about the level of engagement with ONS and felt that generally their were views were taken on board.

2.18 We did receive comments that ONS does not always clearly acknowledge problems with data in its publications, or it takes a while for these limitations to be acknowledged externally. However, this seems to have been improving. These comments were made specifically with reference to the interim migration estimates from the Annual Population Survey. Following these comments further work was undertaken by ONS to improve notes on limitations in the 17 September 2021 and 25 November 2021 Another area of concern highlighted by users was the reweighting exercise. ONS is undertaking another reweighting exercise to try and address these concerns. It is important that ONS continues to respond to user input to support the quality of estimates, and user confidence in the estimates.

2.19 ONS regularly publishes updates on progress with the Migration Transformation programme. For example, its most recent update was published on 26 November 2021. These updates are also communicated to registered users through email alerts and the Migration and population statistics newsletter. Generally, users felt that they were well informed on plans.

2.20 However, for users who engaged less frequently with the migration transformation programme, these updates can be difficult to navigate, and it is not always clear what the most up-to-date information is. For example, how would a user navigate ONS’s website to find the most up-to-date information on the admin-based migration estimates, modelled estimates or most-up-to-date long-term migration estimate? As a result, some users commented that they are relying on previous year data, while they wait for ONS to develop a new measure.

Linked to the wider population transformation programme

2.21 ONS is transforming the way it produces population, migration and labour market statistics. This programme is complex and we heard how some of the terminology connected to the transformation programme is new and evolving and can be challenging for less expert users – for example, whether at the centre of the system there will be a ‘Bayesian demographic account’ or ‘dynamic population model’ and what these terms mean.

2.22 ONS has recently provided users with a high-level update on its work to transform population estimates, including how the transformation of migration statistics fits within this. We welcome this update, although note that migration outputs were not listed in the planned outputs table. We also note that there are ongoing discussions in some areas, for example plans for the UK population by country of birth and nationality stock estimates from the Annual Population Survey (currently planned for May).

2.23 ONS has also undertaken work internally to bring together the different transformation streams in population, migration and labour market statistics in a coherent way and shared with us an overview of this and a high-level roadmap for the transformation streams work.

Users and uses

2.24 ONS is developing its thinking on how to define a migrant and as part of this is considering its continued use of the UN definition of a long-term migrant (i.e. someone resident in the country for 12 months). It is crucial that feedback from a broad range of users feeds into this development as different users have different uses. For example, some of the users we spoke with were interested in whether a person is here for the short-term (i.e. less than 1 year) or the long-term (i.e. 12 months or more) as most policies apply to those who are here for 12 months or more, therefore supported the continued use of the UN definition. A couple of users did note that if the amount of time a migrant had been here was published as one characteristic of that individual then external analysts could calculate whatever categories were relevant for their research or policy evaluation. Alternatively for some users understanding the legal status of a migrant was helpful rather than how long they have been here. Many users were also interested in the availability of more granular data, however the type of granular data specified varied between users, from lower geographical areas to the type of migrant.

2.25 Some users noted the potential benefits of deriving migration estimates from Home Office visa data, which are more timely and based on likely intentions about length of stay, although it was also noted that this will only be partially complete. ONS is investigating this potential and has recently published some research on the visa data and how it could potentially support the development administrative based migration estimates.

2.26 ONS is proposing to provide provisional estimates based mainly on the tactical approach. The final estimate would then be based mainly on the strategic approach. ONS has used this approach for other statistics, for example GDP. The users we spoke with were generally supportive of this approach and of using administrative data to measure observed migrant behaviour. However, some did note that the modelled provisional estimates and final estimates based on administrative data could potentially result in quite different numbers. Also, it was noted by one user that GDP is more abstract than the number of migrants in the country, which can often be a more emotive issue so the risks may be higher. ONS recognises these risks and that managing expectations through clear communication will be crucial in mitigating them.

Trustworthiness

2.27 ONS is seen as a credible and reliable statistics producer, with highly skilled professional staff. It is effectively prioritising the development and transformation of these high profile and vital migration estimates. It is transparently updating users on plans. However, as noted in recommendations above, more could be done to improve the accessibility of these updates and to explain them in the context of the wider population and migration transformation programme.