Summary
Why we carried out this review
A household expenditure survey has been conducted each year in the UK since 1957. Since 2008, this has been in the form of the Living Costs and Food Survey (LCF), which collects information on spending patterns and the cost of living that reflect household budgets. It is the most significant survey on household spending in the UK and provides essential information for key social and economic measures including price indices. LCF data are published annually by ONS in Family Spending in the UK, and are also used in the production of other statistical series such as the Retail Prices Index (RPI).
In recent years, there have been several errors in LCF data which have led to errors in the RPI in both 2019 and 2020 through the use of incorrect expenditure weights. In 2020 for example, if the corrected LCF dataset had been used, it would have led to an upward revision of 0.1 percentage points to the published RPI annual growth rate for six months of 2020. In line with its policy not to make revisions, ONS did not revise the time series for the RPI. The errors in LCF data also affected the Office for National Statistics’ (ONS) Family Spending statistics for years 2017-18 and 2018-19, as they are the main output of LCF data.
In line with the Office for Statistics Regulation (OSR) and ONS strategies, we have carried out an assessment of the LCF against the Code of Practice for Statistics. Our review aims to identify opportunities to improve the quality and public value of LCF data. To inform our review, we spoke to a range of statistics producers and users who make use of LCF data, to understand the impact of LCF on other statistics and data.
What we found
The LCF data processing system is not fit for purpose. The system is unstable, often producing inconsistent results between processing runs of data. The statistics team has endured challenges with the system and resource it is working with to run the LCF, and it has done remarkably well to keep the LCF afloat in spite of these challenges.
We found that the LCF is highly valued by users and is seen as unique in bringing together data on spending habits with information on the households who are doing the spending. Users we spoke to do not see an alternative approach to collecting this information in the short term that does not involve a survey in some form. The production of the LCF is supported by a steering group made up of statistical teams in ONS and other government departments who use LCF data, as well as external think tanks. The LCF team would like to enhance its engagement with internal and external users, across academia, the private sector and other government departments, to ensure developments best meet user needs.
The COVID-19 pandemic has seen the landscape of user need change with increased demand for timely and granular data. The LCF has been underperforming in terms of sample size and response rate in recent years and this was highlighted as the main limitation of the LCF by users we spoke to. Without an increase to the sample size, the devolved administrations are unable to make use of many of the categories available in the LCF data.
The quality assurance arrangements that the LCF team has in place are generally appropriate for the data and go some way to reduce the risk of errors associated with the processing systems. However, the sample size of LCF can make it difficult to determine whether changes between periods are genuine or not. We heard from users of LCF data in ONS that widening access to the LCF data for the purposes of quality assurance could support the LCF statistics team in identifying errors and would allow it to focus on the pre-processing stages.
In 2016, a National Statistics Quality Review (NSQR) was carried out for the LCF. The NSQR recommended several alternatives that could be explored to improve the response rate and thus the achieved sample size of the LCF. The team has taken forward most of the recommendations with no additional headcount being allocated to progress them. The remaining recommendations remain relevant to improving LCF.
The lack of progress in the use of alternative and administrative data sources in the UK has impacted on the quality, accuracy, and international comparability of the LCF data, a perspective which is seen as important to users to gauge the impact of Brexit and the pandemic on UK households.
Requirements and next steps
We have identified several ways the LCF needs to be improved to meet users’ needs and to comply with the highest standards of the Code. Urgently fulfilling the requirements of this assessment is necessary to ensure that the LCF and its outputs continue to be fit for purpose. In order to retain the National Statistics status for Family Spending in the UK, we require ONS to:
a) Demonstrate a positive direction of travel by making some short-term gains by the end of 2021 as follows:
- ONS needs to take remedial action to improve the stability of the existing LCF processing system and to develop a new system which meets the needs of users and the staff running the systems.
- ONS should enhance its understanding of the value of the statistics by improving its engagement with users, within and outside ONS, to capture a wide range of views and use these to drive its priorities for development. ONS should reflect on the Government Statistical Service’s User Engagement Strategy for Statistics to help determine the best methods for engaging with users.
- ONS should provide a mechanism and relevant access for other teams in ONS who make use of LCF data to be able to contribute to the quality assurance of the data.
b) Publish a plan which includes specific actions, deliverables and a timetable by the end of March 2022, that explains how it will address the following strategic improvements:
- ONS needs to develop a solution to address user need for more-granular breakdowns of data, so that the devolved administrations and other key users can use the statistics in the ways that they need to for the public good.
- ONS needs to invest time and resource to pursuing initiatives to improve the quality and robustness of LCF data. ONS should be open to creative solutions to improve the response rate, such as continuing exploring the use of different short and long form questionnaires/diary, alternative sampling strategies and linking with other data sources, rather than focusing only on increasing the existing sample.
We have also highlighted several considerations for ONS to reflect on as part of its LCF and RPI improvement project.
a) ONS should consider extending the scope of its project work to include input from some of its key external users, such as those in the devolved administrations, where additional intelligence could be gathered on the use and issues faced by the government in its use of LCF data.
b) ONS should consider the management of risks throughout the end-to-end production process as part of the LCF projects medium term work and ambitions.
c) ONS should determine a longer-term solution for the LCF which draws on a broader base of data, international best practice and wider transformation initiatives.
We expect ONS to report back to us every quarter, starting from the end of September 2021, demonstrating its progress against these requirements. We will review the National Statistics designation of Family Spending in the UK at each of these points.