Why we carried out this assessment

The Annual Business Survey (ABS) is the largest business survey conducted by ONS in terms of the combined number of respondents and variables it covers. ONS collects data from 62,000 businesses in Great Britain, which is supplemented by data collected by the Northern Ireland Statistics and Research Agency (NISRA) through its Annual Business Inquiry (NIABI) on 11,000 businesses in Northern Ireland. It is the main resource for understanding the detailed structure and performance of businesses across the UK and is a key contributor of business information to the UK National Accounts.

OSR has recently conducted several assessments of ONS’s annual economic outputs: the Annual Purchases Survey, Business Demography Statistics, the Living Costs and Food Survey (LCF) and now the ABS. Across the OSR’s recent assessments, a common theme has emerged to explain the decline in the quality and relevance of ONS business data. A lack of investment, across finance, staff and systems, has meant that ONS has not been able to adequately address data quality and coverage issues, nor develop long-term business data in order to reflect the UK’s evolving industrial mix and the evolution of the digital landscape.

Our review aimed to identify opportunities, and to build on ONS’s initial good ideas to improve the quality and public value of ABS data. To inform our review, we spoke to a range of statistics producers and users who make use of ABS data to understand how the data are currently used, and identify opportunities to improve their public value.

What we found

ONS publishes provisional national data from the ABS on its website in November covering the previous calendar year. Revised national and regional results are then published in May of the following year, around 17 months after the end of the reference period. Whilst ABS data are used extensively in the measurement of long-run economic concepts such as productivity, the significant time delay on the publication of the data means they are not used to the same extent to measure the ongoing impacts of structural and cyclical changes to the UK economy. As a result, ABS data are not meeting users’ needs for timely and detailed data on business performance and users are turning to alternative data sources to gain an understanding of UK business activity. A number of government departments are using business data available on the Fame database, provided by Bureau Van Dijk (BVD), in order to better understand the structure of the UK economy.

Basing annual structural business statistics on alternative data sources (administrative, commercial, or business data already held by other government departments) supplemented with survey data collected more effectively has the potential to improve the quality and granularity of annual business data and reduce the burden on reporting businesses. They could also help to improve the timeliness of the data and render them more useful for analysing business trends and tracking the impact of policy changes. Although moving towards greater use of administrative data sources was part of ONS’s five-year strategy, little progress has been made to date.

The ABS statistical team has recognised the increased demand for more-detailed data and is pulling together ambitious plans, not only to examine the availability of new data sources, but to examine the possibilities for introducing flexibility into parts of the ABS to allow for additional questions to address current business policy issues. However, the ABS production system does not currently allow for the addition of new data sources, nor the linking of other survey data within its infrastructure. This has effectively restricted efforts to improve the breadth and depth of ABS data and its timeliness.

Throughout the COVID-19 pandemic, ONS was successful in meeting the short-term needs of users through the production of real-time information on business activity. It now needs to use this same agility, innovation, and investment to improve its longer-term measures of economic performance.

ONS’s user engagement focuses on the needs of government users. A similar level of engagement needs to be established with users from the private and third sectors, and academia. We welcome that ONS has organised a conference on business data for later in September 2021.

OSR’s Assessment Report of UK Business Demography Statistics noted several areas of uncertainty around the quality of business data held on the Inter Departmental Business Register (IDBR). These included concerns around the incorrect classification of businesses in relation to both industry and structure. The IDBR is used as the sampling frame for the ABS and, as such, some of the same concerns identified in the Business Demography report also apply to ABS data.

Requirements and next steps

We have identified several ways the ABS needs to be improved to meet users’ needs and to comply with the highest standards of the Code.

Requirement 1

ONS needs to urgently prioritise investment in the development of its structural economic statistics, to ensure that the public good is served by them, and in particular to ensure that:

The granularity and timeliness of the statistics are improved, and

The decline in the quality of official economic and business data highlighted in the Bean review of economic statistics does not continue and is reversed.

Requirement 2

ONS should continue, and develop, its current endeavours to establish what existing data – from administrative, private sector or other sources – could be used as the basis for its annual business statistics. These data have the opportunity to provide much more detail, more quickly and cheaply, with lower respondent burden than running a full survey. ONS should then determine what form of supplementary survey needs to be run to collect those data that are not already available from elsewhere (for example sub-national information), and develop ways to harness existing business data (for example use of accountancy software) or collect data more efficiently (for example through electronic questionnaires).

Requirement 3

ONS must develop its understanding of the potential uses and value of ABS data, by engaging better with users both inside and outside of government, to ensure that as far as possible its annual business statistics are providing public value. ONS needs to understand what users require from ABS data and demonstrate how it is going to use feedback to inform a development plan for its structural economic statistics. ONS should reflect on the Government Statistical Service’s User Engagement Strategy for Statistics to assist in selecting the most appropriate methods for engaging with users.

Requirement 4

ONS should provide clearer indications of the fitness for purpose of the ABS statistics and their strengths and limitations with respect to various potential uses. As part of this, ONS needs to understand and communicate to users the influence of IDBR data quality issues on ABS estimates, to ensure that users are well placed to understand the capability and usability of ABS estimates.

Requirement 5

ONS should work more urgently with the UK Data Service to improve the access of researchers to ABS microdata. Improving access to this was also a Requirement (1b) of OSR’s October 2020 Business Demography Assessment Report.

In addressing these requirements, ONS should seek to apply the same innovative and agile approaches it demonstrated in meeting the demand for short-term business data during the pandemic.

We expect ONS to publish a plan by the end of October 2021, which includes specific actions, deliverables and a timetable that explains how it will address the improvements identified in this report. The UK Statistics Authority will take advice from OSR, based on ONS’s progress against this plan to decide whether continued use of the National Statistics designation is merited.