Findings

Data sources and methods

2.1 ONS is highly regarded by users for producing Business Investment statistics as a standalone measure rather than just as a component of Gross Fixed Capital Formation, as is the case with most other national statistical institutes. This approach has been driven by user demand, underscoring ONS’s commitment here to providing statistics that serve the public good. Users acknowledge and appreciate ONS’s leadership in this area. Users also praised ONS for the frequency of BI estimate releases.

2.2 BI estimates are derived from a combination of surveys and administrative data. The response rate of the QCAS, the primary contributor to BI estimates, is currently slightly below pre-pandemic levels and the target range of 80 to 85%, with the most recent response rates from the January to March 2024 revised release being 75.9%.

2.3 The QCAS does not sample small businesses (those with between 0 and 19 employees). Instead, it accounts for them through weighting and benchmarking, using data from the Annual Acquisitions and Disposals of Capital Assets Survey and the Annual Business Survey. However, as observed in the production of Business Enterprise Research and Development (BERD) statistics, excluding small businesses can lead to a sample that is not representative of all businesses, and potentially weakens the statistics’ value. ONS should conduct analysis to understand the impact of not surveying businesses with fewer than 20 employees on the quality of estimates. If this analysis demonstrates a meaningful impact on estimates, ONS should investigate the feasibility of mitigating non-sampling errors. Potential strategies for reducing these errors include sampling small businesses, leveraging administrative data and adopting less-biased weighting systems. Should these methods prove viable, ONS should develop plans for their implementation. ONS should communicate the findings of its impact analysis to users, explaining any quality issues identified and what they mean for use of the statistics, and clearly setting out what action it will take to address any identified issues.

2.4 ONS uses deflators to account for price changes over time. These deflators are essential to understanding how economic data change over time. To produce the volume measures of BI statistics, which account for inflation, as part of this process, ONS uses an R&D price deflator. However, growth in the R&D price deflator has remained unchanged since 2015, resulting in a poor reflection of the fluctuations in inflation in the following years. This outdated deflator can lead to misleading conclusions being drawn from the data. We welcome ONS’s ongoing investigation into the R&D price deflator currently used to produce volume BI estimates to ensure it is the most suitable deflator.

2.5 HMRC and ONS analysts have collaborated to better understand the discrepancies in qualifying plant and machinery data. HMRC has now published findings detailing the main reasons for these differences. Given that users have expressed a strong interest in understanding the discrepancies, we commend ONS for its efforts in addressing these concerns. ONS should ensure that the findings published by HMRC are integrated into the Business Investment Quality and Methodology Information to further enhance clarity and transparency for users.

Revisions to the BI statistics

2.6 Users of BI statistics understand and accept that some level of revision is unavoidable, particularly between provisional and revised quarterly estimates. One user described these revisions as a “workplace hazard”, indicating a general expectation of such adjustments. The nature of business reporting, with some businesses reporting investments later in the reporting cycle, means that preliminary estimates are based on fewer data than revised estimates. To account for this later reporting, ONS applies bias adjustments to provisional estimates and describes this adjustment in its BI statistical bulletins. However, some users have raised concerns about the magnitude of revisions, explaining that revisions to early headline estimates are often material, causing issues for the forecasting they had previously conducted and eroding confidence in the data. For example, in Q4 2022 and Q3 2021, the revised estimates shifted BI figures from positive to negative, altering the data narrative and impacting users’ forecasting models.

2.7 Figure 1 shows the mean revision and mean absolute revision of the provisional and revised estimates, illustrating the extent of these revisions. The size of revisions grew dramatically in 2020 relative to previous years, demonstrating the effect of the COVID-19 pandemic on the quality of preliminary estimates. The size of revisions has decreased since 2020, although revisions in 2023 were still notably above pre pandemic levels. ONS recognises that it has needed to make significant revisions to BI statistics in recent years as a result of the impact on data collection from COVID-19. It took steps to try to minimise these impacts; for instance, in response to reduced survey participation, ONS surveyed imputation methods to address areas of non-response and reviewed their seasonal adjustment.

Figure 1. Difference between provisional and revised estimates (M2–M3).

*2024 Q1 only
Source: ONS

2.8 Users also voiced their frustration to us about revisions to the BI estimates published in the annual National Accounts (the Blue Book). Users explained that growth rates often underwent substantial changes that altered the pre-established economic narrative, leading to confusion and concern about data reliability.

2.9 We recognise the impact of the pandemic on Business Investment Statistics. Substantial revisions were not unique to Business Investment statistics nor to the UK. Other National Statistical Institutes internationally similarly saw revisions increase in their economic statistics during this period. We are reassured that revisions to Business Investment statistics have been reducing since 2020 but also recognise that user trust in the statistics has been impacted as a result of increased revisions. Given ongoing user concerns, we expect ONS to conduct a thorough review of revisions over recent years. This review should aim to identify the reasons behind the revisions and explore potential strategies to minimise them. By reviewing the revisions, ONS should be able to enhance the reliability and credibility of its initial estimates, thus improving user confidence in the data.

2.10 We consider that ONS should enhance its communication efforts to address concerns around revisions to the BI statistics. While ONS already provides explanations for large revisions, more-detailed context and rationale could help mitigate these user concerns. Clearer communication would help users better understand the reasons behind significant changes and ensure they can more accurately interpret the data, preserving the credibility of BI statistics. ONS should communicate the findings of its revisions analysis and any improvements it will take to reduce revisions in future.

Systems, resource and prioritisation

2.11 ONS uses a variety of software systems to produce BI statistics. However, many of these systems are outdated. For example, Common Software (a legacy system) cannot be updated and so increases the risk of errors. The BI team acknowledges the need to modernise its current systems, although it operates within the broader software constraints across ONS. Addressing these issues is a broader challenge across ONS and the BI team has been actively engaging with ONS on the need to modernise. ONS should prioritise modernising its software to help maintain the quality of BI statistics.

2.12 To enhance the efficiency of the production of BI statistics and reduce the risk of processing errors, the BI team should adopt Reproducible Analytical Pipelines (RAP) principles. Implementing RAP would streamline the production process, freeing up resources and improving the overall quality and reliability of BI statistics. ONS is already progressing efforts to modernise data production processes across National Accounts, which will support these goals.

Assured Quality

2.13 We consider that there a need for improved quality assurance processes for BI statistics. One user specifically raised a concern about ONS’s slow response to quality issues flagged by users. This concern was illustrated when the user identified a significant drop in deflators within the BI statistics for a particular year – an anomaly not reflected in their own data. Although the issue was promptly reported to ONS as a processing error, it took more than a year to resolve. While the delay in ONS’s response may be partly due to the National Accounts Revisions Policy, the lengthy delay, along with the lack of transparency regarding the resolution timeline and the actions taken, has heightened doubts about the effectiveness and reliability of the quality assurance processes applied by the BI team. As a result, user confidence in the ONS’s quality assurance practices has been impacted. We note that since this review began in January 2024, the BI team has taken steps to improve its quality assurance processes, including implementing a new tool allowing the team to inspect data quality at low-level breakdowns. Building on this work and to address user concerns, ONS should improve its processes for investigating and rectifying quality issues raised by users, keeping users informed throughout the resolution process.

2.14 OSR has identified the need for improving the provision of quality information alongside estimates. Clear disclaimers should be provided with releases, particularly when there are significant revisions. Additionally, we recommend that the BI team provide a percentage breakdown of input data available at the time of release. Users also stressed the importance of communicating any uncertainty in the data and the quality of the inputs used during the release.

2.15 ONS should improve the quality information provided alongside its granular breakdowns. A good amount of quality information is provided for high-level estimates. However, users felt that the information provided for granular estimates was inadequate (or lacking). Furthermore, users are unclear how granular breakdowns are compiled, highlighting the need for further documentation to aid understanding. There is also a demand for more-granular breakdowns of BI, such as firm size investment data.

2.16 BI statistics are derived from a combination of surveys and administrative data sources. However, the quality information for these sources is outdated; the most recent Quality and Methodology Information document for the QCAS was updated in August 2014. We urge ONS to review the published information and update it as needed. Additionally, access to supporting information for other data sources may be an overarching issue with the ONS website and outside of the control of the BI team. ONS informed us of forthcoming website improvements which could help address some of these problems. Notably, the BI team has already made progress in enhancing quality information by updating the Business Investment QMI in March this year.

User engagement

2.17 Key users of BI statistics reported good active engagement from ONS. ONS consistently engages with key users, maintaining ongoing dialogue to ensure their needs and feedback are addressed.

2.18 However, user feedback indicates that engagement with wider user groups is less comprehensive. Users who analyse data in an advisory capacity have reported feeling inadvertently excluded due to insufficient engagement. This gap highlights the need for ONS to broaden its communication strategies to include a more diverse range of users. The BI team could enhance its user engagement by involving wider users in the quality assurance process. These users’ industry-specific economic insights would not only improve the quality of the BI statistics but also provide the team with a deeper understanding of how users interact with them.

2.19 One approach that users reported having found effective in enhancing engagement and understanding was the use of explanatory webinars such as the ones ONS organised for the BERD transformation work. Similar webinars for BI statistics could improve communication and transparency, particularly by informing users of the reasons for and impacts of large revisions. By adopting such interactive platforms, ONS could ensure that all users, regardless of their level of engagement, are well informed and able to use BI statistics effectively.