Annex 1: Findings on the Transformed Labour Force Survey

This section of the report provides an update on ONS’s progress against the seven recommendations set out in OSR’s progress report dated July 2023, which were updated in February 2024.

After assessing ONS’s revised plans, we find that our previous recommendations about the TLFS remain relevant.

Actions remaining to meet recommendation (as set out in July 2023 progress update report)

1. In addition to fulfilling its commitment to publish regular updates on the transformation process and continue to widen user engagement, ONS should publish a version of its stakeholder engagement plan.

Evidence

• ONS has widened its user engagement with the introduction of the stakeholder panel and expert data sharing groups.
• ONS has also been publishing updates on the labour market transformation – progress and plans.

OSR judgement

• We consider this recommendation to be partially met.
• Most users we spoke to were aware of the detail about the transformation of the LFS and were pleased that information about the project is being shared in user groups to increase the level of transparency about the TLFS.
• To support trust in the statistics and their acceptability among users, ONS should continue to widen its engagement and publish updated stakeholder engagement plans by April 2025.

2. ONS should publish adequate information on:

• finalised variables and questions (including harmonisation and frequency and coverage of TLFS variables) for the TLFS.
• its analysis of data discontinuities explaining any impact on time series.

Evidence

Further work and analysis on the TLFS design mean that ONS is not yet in a position to finalise elements of the questionnaire. In May 2024, ONS published an initial suite of guidance documents. Analysis of data from the TLFS is ongoing.

OSR judgement

As work to finalise the TLFS is still ongoing, we consider that this recommendation remains relevant.

3. ONS should ensure that the transition from the current Annual Population Survey (APS) to its new equivalent is seamless and continues to support ongoing user need. As part of this, ONS should publish expected timescales on the future availability of datasets, replacing current APS datasets now that the timetable has been extended in the next iteration of the user guides in autumn 2023.

Evidence

ONS has been including information about the APS in its published transformation updates since April 2024 and has been holding monthly workshops for APS stakeholders since October 2024. While users are frustrated that they can no longer obtain estimates at the sub-national level for some variables due to the reduction in sample size (following the reduction in the achieved sample size for the LFS in 2023), they are feeding into ONS’s improvement plans for the APS.

OSR judgement

We consider that this recommendation remains relevant.

ONS should continue to support user needs for the APS data and publish timescales for the future availability of datasets by May 2025.

At the request of ONS and the Welsh Government, we have removed the accreditation from other outputs based on data from the Annual Population Survey (APS).

4. To support public confidence in the transformation process, before the decommission decision is made in the autumn 2023, ONS should:

• explain how it is triangulating the TLFS data with other data sources, such as the census, to sense-check the reliability and accuracy as part of the quality assurance process.
• publish planned activity that will be undertaken and decisions that will be made between October 2023 and March 2024.

Evidence

The further work and analysis on the TLFS design mean that ONS has not yet taken a decision to decommission the LFS. ONS’s update in December 2024 noted that “we will provide an update on timescales in spring 2025, following engagement on the findings of the test activity with our expert groups”.

As work to finalise the TLFS is still ongoing, we consider that this recommendation remains relevant.

OSR judgement

ONS should continue to provide users with regular updates on progress and, as part of its spring 2025 update, include its high-level plan for the next phase of work.

5. ONS should include achieved TLFS response rates in the next iteration of the user guides planned for November 2023 and publish its analysis of mode effects in the autumn 2023.

Evidence

In its update in November 2023, ONS noted that the “average overall response at wave 1 are 39% and quarterly GB person datasets contain roughly three times as many individuals as the LFS”.

OSR judgement

As work to finalise the TLFS is still ongoing, we consider that this recommendation is partially met.

ONS should publish its analysis of mode effects on the TLFS.

6. ONS should ensure that all relevant technical papers describing the methods used in the TLFS are published, accessible and well signposted before the end of December 2023. As part of this, ONS should:

• engage with expert users as part of ONS’s data sharing plans to help quality-assure the methods used on TLFS.
• take account of the results of the current exploratory work on current population weights used in the LFS to help inform the decision about methods used.

Evidence

ONS is engaging with expert and academic users to help quality-assure the methods used. ONS has also published the re-weighted LFS.

In May 2024, ONS published an initial suite of guidance documents, including a description of the initial methodology used to develop the TLFS. However, ONS has yet to publish a description of the latest data processing methodologies chosen or regular outputs from the TLFS.

OSR judgement

We consider that this recommendation is partially met.

ONS should ensure that the next suite of technical papers describing the methods used in the TLFS are published, accessible and well signposted.

7. ONS should publish its plans for the weighting of the LFS until the point of decommission and in the interim publish details about the impact on quality of the current LFS. ONS should also keep users informed about any implications on the development of TLFS weighting methodology.

Evidence

ONS has kept users updated about its plans for reweighting the LFS since its update in April 2024. ONS announced in the update on 3 December and published the reweighted LFS estimates in its December 2024 release Employment in the UK.

ONS published a user guide in May 2024 that included information about the initial methods and weighting used in the TLFS development work.

OSR judgement

We consider that this recommendation is partially met.

Information on the quality of the LFS is being regularly published.

ONS should publish information about the final TLFS weighting methodology.