In my previous letter, I explained the challenge of lower response rates of the main sample of the Labour Force Survey (LFS), and the additional sample used to build the Annual Population Survey (APS). I committed to respond with our assessment of APS quality, including the feedback provided by key users at a workshop that was held on 2 October 2024.

The high-level assessment following the workshop on 2 October was that:

• The total sample size is still relatively large and at a national level where results remain reasonably robust with a small increase in variability compared  to the pre-COVID period;
• headline data at a regional level is acceptable, but some cross-tabulations (e.g. unemployment by age band) cannot be published as sample sizes are too low;
• the quality of data for local authorities has deteriorated and the majority of users are experiencing issues with producing results at this level from the  APS; and
• The reduction in the achieved sample also impacts on the ability to publish estimates for other smaller segments of the population (for example small age bands or specific demographics).

Our assessment considered evidence from across ONS outputs, including labour market, well-being and population and demography. The evidence included:

• The reduction in the available sample for producing estimates at local levels and small domain levels.
• The increase in confidence intervals of published estimates; and
• The impact on APS estimates of its current weighting approach.

External users also provided views related to their outputs which were in line with the assessment of ONS output.

Overall, our view on the quality of the APS is that while it is robust for national and headline regional estimates, there are concerns with the quality of estimates for  smaller segments of the population, such as local authority geographies. In addition, we recognise the need to review and update the weighting approach used in the APS. Since analysis at the local area is one of the main uses of the APS data, we want to be clear with users about the extent to which different statistics are affected and how and when APS data should be used in analysis. To help give that clarity to users, I would like to request that we suspend the accreditation of ONS outputs using the APS data while we undertake further work on APS quality. Changing the designation of the statistics will help to convey the uncertainty and clearly signal the limitations of the data to users to support their appropriate use. It also demonstrates our commitment to fully understanding, assessing and improving the quality of the APS data.

Our immediate next step is to arrange a follow-up workshop with users in early November. The aims of this workshop will be to:

• Provide clarity on how APS data can currently be used and what the limitations are. The information provided by this discussion will be used to produce a technical note to be published by ONS following the workshop to provide further clarity to the wider set of users of the APS; and
• Assess how quality concerns could best be addressed and any additional actions that would be necessary.

We will update you further after the workshop and the publication of the technical  note. The actions identified for the APS will build on the measures we are already  taking to improve the quality of the LFS.

Alongside this letter, we understand the Welsh Government plan to request that the  accreditation is suspended for Welsh Government APS outputs due to be published  on 9 October.

We will communicate our decision on accreditation to Chief Statisticians at  departments who publish outputs using the APS and encourage them to carry out their own assessment of quality and consider the designation of their APS outputs.

While we work to improve the quality of the APS and therefore the outputs produced  from it, we are committed to keeping both you and our users informed of  developments. In addition to the follow-up workshop with APS users we will also  include information about the APS in our regular updates on the LFS and TLFS.

Yours sincerely,

Michael Keoghan

Related Links:

Ed Humpherson to Michael Keoghan: Suspension of the accredited official statistics status for the estimates ONS produces from the Annual Population Survey

Stephanie Howarth to Ed Humpherson: Request to suspend APS accreditation

Ed Humpherson to Stephanie Howarth: Suspension of accredited official statistics status for Wales estimates produced from the Annual Population Survey