Dear Liz
Annual Business Survey Statistics: Confirmation of accredited official statistics status
We have independently reviewed the actions that your team has taken to address the five requirements outlined in our assessment report that we judged must be met for the Annual Business Survey statistics to maintain their status as accredited official statistics. Accredited official statistics are called National Statistics in the Statistics and Registration Service Act 2007.
I would like to thank your team for its engagement in responding to the requirements set out in our assessment and for your ongoing commitment to improving these statistics. On behalf of the Board of the UK Statistics Authority, I am pleased to confirm that the statistics continue to comply with the Code of Practice for Statistics and should continue to be labelled as accredited official statistics. We have included the detail around our judgement in the annex to this letter.
We are particularly impressed by ONS’s extensive user engagement, which has provided a solid foundation to inform future development plans. We also note the range of other improvements to the existing survey and data collection process, for example, the implementation of electronic data capture. Additionally, the improved accessibility of ONS data platforms has allowed users to access micro-level data and create bespoke cuts and breakdowns of ABS data.
We also note further developments in progress, such as ONS developing data processing systems and improving data dissemination, which should provide users with greater insights and meet their needs for more-timely and more-granular statistics.
Yours sincerely
Ed Humpherson
Annex: Review of actions taken in response to Assessment Report 360: Assessment of Statistics from the Annual Business Survey, produced by ONS
Requirement 1
ONS needs to urgently prioritise investment in the development of its structural economic statistics, to ensure that the public good is served by them, and in particular to ensure that:
a. The granularity and timeliness of the statistics are improved, and
b. The decline in the quality of official economic and business data highlighted in the Bean review of economic statistics does not continue and is reversed.
ONS has published a response on how it plans to meet the requirements set out in our assessment and committed investment in improving ABS (Annual Business Survey) data. The ABS is being redeveloped as part of the Business Transformation programme within the Ambitious, Radical, Inclusive Economic Statistics (ARIES) framework and will continue as part of the Future of Business Surveys and Statistics (FBSS) programme of work. Under these proposals, a new ABS is due for March 2026 (with a 2025 reference period); in the meantime, ONS is working towards improving the timeliness and quality of ABS estimates by taking the following steps:
- In March 2024, electronic questionnaires (EQs) went live, replacing paper-based questionnaires.
- Improved guidance and in-built validation were developed to aid respondent understanding and improve the quality of responses.
- Fully automated disclosure control and improved in-built validation of data responses are in the process of being established.
ONS’s FBSS programme of work, is due to start later in 2024. FBSS is a key step in preventing the decline in the quality of official business data and will review information collected across ONS annual structural surveys with a primary focus on the ABS. The work programme is focused on integrating and coordinating survey designs to improve the business experience and data quality and reduce the number of business surveys. Initial work is proposed to include:
- An assessment of the information collected in the survey, focusing on concepts and user needs, to improve the questionnaire design to minimise respondent burden.
- Research into expanding the coverage of the survey and including new industrial classification codes.
- Research into how ONS can improve engagement with businesses, and how it can work with businesses more effectively to collect the data that ONS needs, for example by establishing a system of account management.
- Moving the ABS to a new platform and developing new data processing systems and results processing systems to replace the current SAS-based system.
Work beyond 2024/25 will depend on the outcomes of the next spending review.
ONS has streamlined the ABS publication process by:
- Considering the optimal time to stop results processing and quality assurance (QA)
- Automating table creation
- Creating code to systemise elements of the QA process
- Automating disclosure control
Optimising the production processes has improved the timeliness of ABS statistics by bringing forward the 2024 national release by six weeks. ONS has committed to making similar time savings for the regional release.
ONS has met user needs for improved granularity by improving accessibility and ensuring a full catalogue of data on platforms, such as the Secure Research Service (SRS), Data Access Platform (DAP) and IDS.
ONS is investigating the possibility of reintroducing ABS standard extracts. The ABS team offers, on request, standard extracts free of charge. At the UK level, the extract covers a wider range of variables, down to five-digit sub-class level of the SIC; the regional extract covers five main variables down to three-digit group level. This service will provide users access to more granular data for over 600 ABS variables.
OSR’s evaluation of the evidence:
Short term, improving accessibility on ONS data platforms has allowed users to access micro-level data and create bespoke cuts and breakdowns of ABS data. Longer term, reintroducing ABS standard extracts will further improve data granularity.
We recognise that ONS’s processing systems are hindering its progress in further improving the quality of the estimates. The new systems will be able to incorporate administrative data sources and should meet users’ needs for more-timely and more-granular statistics with more breakdowns and an improvement in the insights offered.
Requirement 2
ONS should continue, and develop, its current endeavours to establish what existing data – from administrative, private sector or other sources – could be used as the basis for its annual business statistics. These data have the opportunity to provide much more detail, more quickly and cheaply, with lower respondent burden than running a full survey. ONS should then determine what form of supplementary survey needs to be run to collect those data that are not already available from elsewhere (for example sub-national information) and develop ways to harness existing business data (for example use of accountancy software) or collect data more efficiently (for example through electronic questionnaires).
ONS is working with HM Revenue and Customs (HMRC) and other government departments to assess the quality, usability and relevance of data sources that could replace elements of wider survey data collection, such as corporation tax data.
ONS has participated in a cross-government project comparing the ABS capital investment data against other ONS outputs and corporation tax. ONS and HMRC are exploring the best way to communicate the findings of this work, explaining the differences between the data.
ONS has conducted research comparing VAT and ABS data, the findings of which are due to be presented at the Economic Statistics Centre of Excellence conference in May 2024. ONS found missing data and matching reporting units with some sources and discovered that large enterprises often cross several industries and report VAT as a group or as several overlapping subgroups. ONS found VAT returns amalgamated over industries, and difficulties applying suitable deflators (based on industries) could lead to overestimation or errors in the data.
ONS has expanded the use of industry-specific administrative data sources, for example, The Department for Transport’s Sea passenger statistics, and as part of the FBSS, is considering how it can better use alternative data sources to supplement survey data and improve sampling and statistical results processing.
ONS has concluded from its research that the use of administrative data in place of survey data for the ABS in the short term is not sensible. Instead, administrative data are more suitable for benchmarking or QA purposes. ONS uses Companies House data, news articles, academic studies and economics reviews and incorporating other data, such as short-term estimates from monthly GDP, GVA and Monthly Business Survey data, to quality-assure ABS data and better tell the economic story.
OSR’s evaluation of the evidence:
ONS’s expanded use of administrative sources, research and involvement in cross-government projects looking at the use of administrative data demonstrate that ONS is better harnessing existing business data and considering it for future use.
Where survey data are required, electronic data collection allows for greater flexibility and the FBSS aims to continue to improve the questionnaire design and minimise respondent burden. Further to this, improving how ONS works with businesses (for example through account management) should improve how ONS collects data more efficiently.
ONS’s research exploring the use of administrative data is ongoing. However, as development work progresses, we expect ONS to communicate its future plans for surveys and statistics, including plans around the use of administrative data to users more widely. For example, ONS should explain to users the outcomes of any research, such as the trade-offs of using different administrative sources.
Requirement 3
ONS must develop its understanding of the potential uses and value of ABS data, by engaging better with users both inside and outside of government, to ensure that as far as possible its annual business statistics are providing public value. ONS needs to understand what users require from ABS data and demonstrate how it is going to use feedback to inform a development plan for its structural economic statistics. ONS should reflect on the Government Statistical Office for Statistics Regulation Service’s User Engagement Strategy for Statistics to assist in selecting the most appropriate methods for engaging with users.
ONS has engaged with ABS users by:
- Conducting a user engagement exercise in summer 2022
- Organising a programme of stakeholder meetings in autumn 2022
- Holding a Business-user event in April 2023 with roughly 300 attendees
- Sending a newsletter to users of the Secure Research Service (SRS) to ask for feedback
- Contacting user teams in other government departments to get their views on annual surveys
- Working with trade associations, whose industry-level expertise will aid ONS in better understanding specific industries, impacting how ONS quality-assures data, for example, helping to inform GDP balancing decisions and changing validation gates by expected trends.
- Working with National Account stakeholders to better understand any upcoming data and better meet their delivery requirements, for example, through “curiosity sessions” (at the start of results processing) where short-term survey teams (Monthly Business Survey, Retail Sales Index, Construction, GDP) update ONS statistics teams on changes in the data and highlight any underlying “economic stories”. The ABS team is also briefing and offering insight on specific industry issues that are problematic for the ONS Supply and Use team.
ONS user feedback has been collated, documented and included in the scope of its research programme.
ONS is taking a more active role in high-profile and policy-impacting analysis, for example working with HMRC, HM Treasury and The Department for Business and Trade to help them better understand aspects of ABS data to inform budgetary decisions. ONS’s engagement with other government departments is also informing its research into improving the ABS.
OSR’s evaluation of the evidence:
ONS’s engagement with ABS users to improve the coverage and data collected in surveys shows its efforts towards understanding the public value and potential use of its statistics.
Closer relationships with trade associations, for example, should allow ONS to gain a better understanding of users’ needs from a real-world perspective and adapt its processes and outputs accordingly. ONS’s enhanced understanding of ABS users’ needs should lead to improvements in the quality and usefulness of its outputs in the future.
To further improve transparency and reassure users, ONS needs to communicate to users the outcomes of its user engagement with clear plans on how it intends to address users’ needs and demonstrate how it is going to use the feedback it has received to inform its ABS development plans.
Requirement 4
ONS should provide clearer indications of the fitness for purpose of the ABS statistics and their strengths and limitations with respect to various potential uses. As part of this, ONS needs to understand and communicate to users the influence of IDBR data quality issues on ABS estimates, to ensure that users are well placed to understand the capability and usability of ABS estimates.
ONS reviewed, amended and published an updated ABS technical report and Quality Methodology Information (QMI) in January 2024. The text has been updated with more information about sampling and how the Inter-Departmental Business Register (IDBR) is used as a sampling frame. Data quality issues are highlighted in the technical report, for example:
- Types of questionnaires – larger businesses (250+ employees) more often receive long questionnaires and account for over half of ABS turnover estimates.
- New collection requirements can affect the quality of returns – an existing questionnaire may be amended or a new one introduced. Altering an existing questionnaire may impact those who have already received the questionnaire, with the additional questions potentially not applying to them.
- Information on how questionnaires are reviewed – analysis compares response rates, error rates, completion times and the number of queries received.
- The production of standard errors to measure the levels of uncertainty in the data.
ONS has provided reassurance to users regarding IDBR quality (the main sampling frame for the ABS) and reviewed other webpages relating to the ABS; the latest forms are now available on the ONS website (linked in the right-hand side boxes of the webpages – both for users and respondents).
The ABS team is contributing to the development of the new Statistical Business Register by providing feedback on the limitations of the current register and future requirements that will improve the quality of the ABS estimates. The ABS team is updated on any other projects that use the ABS dataset (for example SRS and DAP) and is advising on any constraints or limitations caused by, for example, the use of IDBR as a sampling frame, which will impact the usability of the data for those projects.
ONS has added links from ABS datasets to quality information highlighted in the QMI and technical report. For users who do not read the technical report and the QMI, ONS publishes quality measures, such as standard errors and coefficients of variation, alongside the bulletins for national and regional estimates.
OSR’s evaluation of the evidence:
The updated ABS quality documentation demonstrates a more comprehensive understanding of ABS data quality. It signals to users the strengths and limitations of the data and the influence of the IDBR on data quality.
ONS understands that as the future survey is developed, further work is required to understand and communicate to users the quality and usability of ABS estimates. We look forward to continuing to support ONS with communicating data quality to users, as it develops the ABS.
Requirement 5
ONS should work more urgently with the UK Data Service to improve the access of researchers to ABS microdata. Improving access to this was also a Requirement (1b) of OSR’s October 2020 Business Demography Assessment Report.
The ABS metadata are being migrated to the Integrated Data Service (IDS), which will help inform the next iteration of the SRS. The migration of the ABS is expected to be completed by April 2024.
ONS delivers the ABS dataset to the SRS within one week of its publication. The SRS is responsible for delivering ABS data to the UK Data Service (UKDS). ONS does not have direct control over ABS delivery targets; deliveries from SRS to UKDS are based on user priorities and other commitments. However, discussions between ONS, SRS and UKDS are regular and remain ongoing to address user concerns.
The ABS team plans to review the current data delivery process, which should ensure data delivery to IDS and SRS on the day of publication.
OSR’s evaluation of the evidence:
ABS data access for researchers is now much improved through its availability on the SRS and IDS. We look forward to ONS updating us on its work to reduce the data delivery times to these platforms.