UK Statistical System UK Statistics Authority Code of Practice
Ed Humpherson to Alex Lambert: Confirmation of National Statistics for the Living Costs and Food Survey

Ed Humpherson to Alex Lambert: Confirmation of National Statistics for the Living Costs and Food Survey

Dated
24 October 2022
Last updated on
26 October 2022
Sent by:
Ed Humpherson
Director General for Regulation, Office for Statistics Regulation
Sent to:
Alex Lambert
Director of Surveys, Office for National Statistics
File

Ed Humpherson to Alex Lambert Confirmation of National Statistics for the Living Costs and Food Survey

Download "Ed Humpherson to Alex Lambert Confirmation of National Statistics for the Living Costs and Food Survey" in PDF format (0.15MB)

Type
PDF
Size
0.15MB

Dear Alex, 

Confirmation of National Statistics for the Living Costs and Food Survey

We have reviewed the actions that your team has taken to address the requirements in Assessment Report number 358: Living Costs and Food (LCF) Survey. On behalf of the Board of the UK Statistics Authority, I am pleased to confirm the continued designation of National Statistics for Family Spending in the UK.  

The LCF is a significant survey measuring household spending in the UK and provides essential information for a wide range of key social and economic measures including price indices. At a time of increasing public interest with a greater focus on living costs, statistics using data from the LCF are important.  

We are really impressed with the way your team has taken swift action to address significant data quality concerns. The team has successfully improved the stability of the LCF processing system through the adoption of Reproducible Analytical Pipelines (RAP) principles. An enhanced quality assurance process and communication channels with users has also led to improved accuracy and timeliness. It is also good to see that many of the planned strategic improvements to the LCF data are now being taken forward as part of ONS’ Household Financial Statistics Transformation (HFST) project. The actions taken so far and your future plans really exemplifies your commitment to enhance and maximise the public value of this survey – which in our view should be celebrated.  

We have included more detail about our judgement in an annex to this letter. I, or my team, would be happy to talk to you or your colleagues through any aspects of this letter or Code compliance more generally.  

I am copying this letter to Adrian Chesson, project executive for the HFST project. 

Yours sincerely,

Ed Humpherson
Director General for Regulation 

Annex A

Requirement / Consideration  Actions taken and commitments made by ONS to meet the requirement  OSR’s evaluation of evidence 
Requirement 1:

Demonstrate a positive direction of travel by making some short-term gains by the end of 2021 as follows:
 
  1. ONS needs to take remedial action to improve the stability of the existing LCF processing system and to develop a new system which meets the needs of users and the staff running the systems.
  2. ONS should enhance its understanding of the value of the statistics by improving its engagement with users, within and outside ONS, to capture a wide range of views and use these to drive its priorities for development.
  3. ONS should reflect on the Government Statistical Service’s User Engagement Strategy for Statistics to help determine the best methods for engaging with users.
  4. ONS should provide a mechanism and relevant access for other teams in ONS who make use of LCF data to be able to contribute to the quality assurance of the data. 
 ONS set up an LCF and RPI (Retail Prices Index) improvement project, which has led to a range of improvements to the existing LCF processing system, including:

  • the implementation of a more-structured quality assurance process and an enhanced data quality assurance module in R and new data aggregation module in Python
  • establishment of the LCF development team (rebranded to Household Financial Survey (HFS) pipeline development) to oversee and maintain the LCF data processing system and statistical processing pipelines of other household financial surveys applying principles of Reproducible Analytical Pipelines (RAP)
  • improvements to business processes
  • producing process maps covering LCF data processing
  • commitment to the full rebuilding of the LCF data processing system module by module, by March 2023
The LCF research team, responsible for the LCF statistical outputs, holds regular curiosity meetings providing users in ONS the opportunity to contribute to the quality assurance of the data. Micro-level data are also provided alongside aggregate data to Economic, Social and Environmental Statistics Group (ESEG) data users.

ONS has set up a Household Financial Statistics Transformation (HFST) project to continue the work. As part of the HFST project, stakeholder engagement and identifying user requirements has been a priority.

ONS has held several detailed workshops with key users across Government, academia and the third sector, as well as expanding its reach to new users via a ‘missing user’ survey. The LCF steering group has been rebranded and expanded to form a new ‘spending experts’ group, this first met in September 2022. Plans are also underway for a public consultation on the vision for the HFST and specific questions around spending data collection in winter 2022/23.

ONS published a progress report outlining its next steps as it continues to improve the LCF processing system, skills of the LCF research team and the continued development of LCF through user engagement.   		ONS has met this requirement through the LCF and RPI improvement project.

The data processing system has been fully reviewed and it is being rebuilt module by module, with the most recent run having successfully completed in parallel with components of the previous processing system.

The implementation of RAP principles to the statistical processing of the LCF has reduced the operational complexity of processing and therefore improved the accuracy and timeliness of the output.

ONS has made good progress on engagement and communications, via the LCF steering group and internal curiosity meetings. The RPI team at ONS has been complimentary about the improved communication channels with the LCF team.

We are encouraged to hear that as part of the HFST project, stakeholder engagement and identifying user requirements has been a priority.   
Requirement 2:

Publish a plan which includes specific actions, deliverables, and a timetable by the end of March 2022, that explains how it will address the following strategic improvements:
 
  1. ONS needs to develop a solution to address user need for more granular breakdowns of data, so that the devolved administrations and other key users can use the statistics in the ways that they need to for the public good.
  2. ONS needs to invest time and resource to pursuing initiatives to improve the quality and robustness of LCF data.
  3. ONS should be open to creative solutions to improve the response rate, such as continuing exploring the use of different short and long form questionnaires/diary, alternative sampling strategies and linking with other data sources, rather than focusing only on increasing the existing sample.
  		The HFST project aims to establish integrated, efficient, inclusive survey operations, to provide higher quality, more-timely and more-granular statistics. It aims to unlock an in-depth understanding of income, wealth, spending and financial resilience across the UK population.

The HFST project has been funded until March 2025 as part of ONS’s Ambitious, Radical, Inclusive Economic Statistics (ARIES) programme. The aim is to exploit opportunities to use alternative data sources and establish integrated survey operations by consolidating three current surveys; LCF; Survey of Living Conditions; and the Wealth and Assets Survey. Many of the strategic improvements for the LCF will be considered as part of this project.

ONS commissioned NatCen to conduct a review to inform future expenditure data collection. The review concluded in March 2022 and covered: 

  • a summary of user requirements for household expenditure data
  • an overview of alternative sources of data, including an assessment of suitability against user needs identified
  • recommendations for a high-level design of expenditure data collection, including highlighting that a move to automation for the collection, coding and processing would have the largest beneficial impact on the survey
  • recommending that moving to digital capture for the diary element of the survey would be the most significant change the survey could make
ONS is taking forward these recommendations as part of the HFST project.

ONS recently published its progress report from March 2022 outlining actions taken and how it intends to address the strategic improvements highlighted in i) and ii).

ONS has also set up a project to test expenditure data collection via digital means. Feasibility of a mobile app, developed by Statistics Netherlands has been assessed in February 2022. The pilot provided helpful insight into the useability of digital tools and ONS are exploring options for the further development of a digital tool as a potential solution. 		This larger sample of the new combined Household Finances Survey (HFS) will enable greater precision in headline estimates, increased stability over time, and more-granular statistics at regional levels and for different population subgroups.

ONS has demonstrated its commitment to improving the LCF through the commissioning of NatCen to conduct an external review and exploring solutions to improve the quality and robustness of the LCF data.

It is also encouraging to see that ONS are exploring the mode of survey with the potential implementation of a digital tool for the diary portion of the LCF. As part of the HFST project, a workstream is fully dedicated to the digital redevelopment of the diary, with ONS engaging with the market to assess options, costs and benefits of various technological solutions. Whilst still in development, this shows that ONS is committed to trialling new solutions to improve response rates

Following the recent publication of ONS’ progress report and action plan, we consider that this requirement has been met. 
Consideration 1:

ONS should consider extending the scope of its project work to include input from some of its key external users, such as those in the devolved administrations, where additional intelligence could be gathered on the use and issues faced by the government in its use of LCF data.  		As part of the LCF external review by NatCen, a detailed user requirement exercise has been carried out to inform the future of its household expenditure data collection. A report has been produced, summarising the requirements of the interviewed LCF data users.    Consideration 1 has been achieved via the NatCen report which has been shared with OSR.   
Consideration 2:

ONS should consider the management of risks throughout the end-to-end production process as part of the LCF projects medium term work and ambitions   		ONS has expanded the scope of the LCF and RPI improvement project to consider wider aspects of the end-to-end process. ONS has established a continuous improvement project aiming at streamlining the LCF receipts processing and data coding and editing processes.

ONS has undertaken a review of the Quality Assurance of Administrative Data (QAAD) for LCF, and plans to publish the results in November 2022.

ONS has initiated the development of more-robust questionnaire change control processes. This evolved to include the harmonisation of the Household Financial Surveys' questionnaire change control processes. So far, this workstream has investigated the current questionnaire change control processes used in each of the Household Finance Surveys, gathered user needs, developed to-be journeys, and designed the proposals for the future governance of the questionnaire change processes. ONS developed a comprehensive Change Control Check list to provide a standard of good practice and user guide.

The next steps for ONS are to finalise the Change Control governance structure, defining the roles and responsibilities; finalise the user requirements and choose the optimal technical solution for automated change control system as part of the HFST Project.   		ONS has made significant progress with this consideration, and the publication of the results of the LCF Quality Assurance of Administrative Data (QAAD) review will add significant assurance for users.  
Consideration 3:

ONS should determine a longer-term solution for the LCF which draws on a broader base of data, international best practice and wider transformation initiatives.  		ONS is continuing its collaboration and exchange of knowledge with the Australian Bureau of Statistics (ABS) through monthly meetings to keep abreast of international developments and best practice in the production of household finance statistics and is re-starting the collaboration with other National Statistical Institutes (NSIs) including Statistics Canada and Statistics Netherlands.

The HFST project is also liaising with a US study providing guidance for modernising and integrating income, consumption and wealth statistics to better inform policy and research. ONS participated in a panel discussion with several NSIs in September 2022.  		ONS has demonstrated its commitment through its engagement with other NSIs to share best practice and learning and therefore met this consideration. 

Related links

Alex Lambert to Ed Humpherson: Confirmation of National Statistics for the Living Costs and Food Survey (25 October 2022)