Value

ONS Annual Economic Surveys

OSR has recently conducted several assessments of ONS’s annual economic outputs: including the Annual Purchases Survey, Business Demography Statistics, the Living Costs and Food Survey (LCF) and now the ABS. Each of these assessments has highlighted several issues with the data quality and value of these annual surveys. These issues have included significant incoherence between estimates of Intermediate Consumption between the Purchases Survey and the ABS, incorrect standard industrial classification in Business Demography (and, by proxy, the ABS) and data volatility in the LCF. The quality of the data in these annual economic surveys was discussed in Sir Charles Bean’s review of economic statistics which highlighted the poor profile of official business data (paras 4.188-4.190), and the need for improvement either through the use of administrative data or the use of unique identifiers.

Across OSR’s recent assessments, a common theme has emerged to explain the decline in the quality and relevance of ONS business data. A lack of investment, across finance, staff and systems, has meant that ONS has not been able to address data quality and coverage issues, nor develop business data in order to reflect the UK’s evolving industrial mix and the evolution of the digital landscape.

ONS is aware of the need to improve the quality and relevance of these outputs but notes that the investment priority offered to the annual business surveys has not “come above the line”. We recognise that the decision to invest in the development of the ABS is challenging, as ONS will need to find additional resource to add to the already significant ongoing costs of production.

ABS data are being crowded out by alternative sources

In addition to the published data, ONS also supplies bespoke analyses through its special analysis service. However, we found that some government policymakers, industry analysts and think tanks such as Nesta are turning to alternative data sources to gain an understanding of the impact of economic shocks and to track the evolution in industrial structure. Some government departments (including the Department for Business, Energy and Industrial Strategy (BEIS)), use business data available on the FAME database, provided by Bureau Van Dijk (BVD), in order to understand these impacts and developments.

Fame is a live database, which uses financial data recorded at Companies House to describe the economic and financial performance of UK companies across industries and by size band. Fame data have their limitations as they record the performance only of companies, omitting those businesses that are not legally required to file reports at Companies House. The omission of small and self-employed businesses, not required to file annual accounts at Companies House, means that the Fame data universe does not fully represent all businesses. Users therefore need to take these coverage issues into account when using the data.

Crucially however, Fame records a wider range of business characteristics than the ABS, including data on turnover, employment, company structure and ownership and balance sheet information (the latter is not a reporting requirement for medium-sized and smaller companies). The Bank of England used Fame’s greater coverage of business characteristics to examine how COVID-19 affected small UK companies, BEIS used Fame data to estimate the Economic contribution of private companies of significant size and the Intellectual Property Office used the data to examine the Use of Intellectual Property rights across UK industries. With increasing use of the Fame database in policy design and evaluation, it is becoming apparent that ABS data are being crowded out by this alternative dataset.

Investment and creativity

ONS needs to invest in and think creatively about how it can produce better statistics quicker

ONS publishes provisional national data from the ABS on its website in November covering the previous calendar year. Revised national and regional results are then published in May of the following year, around 17 months after the end of the reference period.

This means there is a large time lag between the publication of survey results and the reference period the results refer to. 2020 ABS data will not be available to policymakers until November 2021, based on the current production schedule. OSR recognises ONS’s efforts to improve timeliness by shortening publication of national and regional ABS results, by one and two months respectively, to May each year. We also recognise the important trade-off between timeliness and accuracy, However, whilst these improvements are welcome, the time between data collection and publication remains unhelpful to policymakers who require access to structural business data quickly in order to design economic and business support policies to minimise the impact of market failures such as inefficient trade markets and the dislocation of labour caused by the COVID-19 pandemic. Policy users told us that they would be prepared to forego a degree of data accuracy in favour of more timely data to address these issues.

Whilst ABS data are used extensively in the measurement of long-run economic concepts such as productivity (for example through the EU KLEMS database), they are not used to the same extent to measure the ongoing impacts of structural and cyclical changes to the UK economy.

ONS must increase its use of alternative data sources (administrative, commercial, or business data held by governments) to improve the quality, granularity and public value of ABS data, and reduce the survey’s burden on reporting businesses.

ONS has already begun to consider opportunities for using Fame data. Matching Fame with IDBR data (such as described in this paper) could present ONS with the opportunity to develop better and faster business data to answer the increased demand for detailed industrial and geographical data for measuring, for example, regional GDP, productivity and income and earnings.

The ABS statistical team could use Fame data in conjunction with business data held by government departments to produce a joined-up picture of local, regional and UK supply chains. More-timely data will allow for almost real-time policy evaluation, which would allow the statistical team to build the additional survey modules that are required to address the needs of economic shocks such as Brexit and the COVID-19 pandemic. The statistical team might also consider how HM Revenue and Customs (HMRC) or Companies House administrative data could further improve the timeliness and depth of annual business statistics and then look to complement and broaden this baseline of business data with other commercial data.

The ABS statistical team has recognised the increased demand for more-detailed regional data and the need to build in the capacity to the ABS to be able to measure the impact of structural economic shocks as and when they occur. The statistical team is pulling together ambitious plans, not only to examine the availability of new data sources to address the need for detailed data, but to examine the possibilities for introducing flexibility into parts of the ABS to allow for additional questions to cover current issues, such as, for example, the availability of finance to small and medium sized enterprises during economic recovery. The statistical team acknowledges that there could be lengthy lead times on the development of adaptions to the ABS, and that significant investment would be needed to research how different data streams can be used together, and challenges (such as in matching records) can be overcome. Therefore in response, short-term solutions are being considered, such as working with small groups of companies to gather qualitative information on prevailing issues.

The ABS statistical team has started to review the data required to answer the needs of the UK Government’s levelling up strategy. It has planned qualitative work with businesses to verify the relevance of the data it plans to collect and will review best practice amongst international partners with access to similar administrative (similar tax regimes) data to assist with the apportionment of regional data. The statistical team intends seeking funding for the development of this work under the 2021 Comprehensive Spending Review.

The use of alternative data sources has the potential to enrich the depth, scope, and relevance of annual structural economic data. By answering some of the questions the ABS currently asks, these sources could also offer opportunities to cut down the size of the ABS questionnaire, which is a considerable time and resource burden for statisticians and responding businesses, and enable quicker results. Reducing the length and complexity of the survey should also help to improve the timeliness of the data and render them more valuable.

Other National Statistics Institutes such as the US Census Bureau and the Australian Bureau of Statistics have already demonstrated the capacity of alternative sources to improve the timeliness and scope of their annual business surveys. The following case studies provide examples of ways that ONS could aim to enhance the value of annual economic statistics through linkage with administrative records from the IDBR and external sources.

In considering what data it needs to collect directly from businesses, ONS could further improve the timeliness and detail of the ABS using accountancy software and an electronic questionnaire for survey respondents. These developments may also particularly help reduce the burden on larger businesses. ONS has been considering such developments for a long time, with users indicating that they were aware of proposals to use an electronic questionnaire back in the mid-1990s. The statistics team told us that a proposal had been put forward to its Transformation (Aries) Programme to assess the feasibility of moving the ABS on to an electronic questionnaire. However, the work was not deemed to be a priority and is unlikely to be considered again until 2022.

To implement new arrangements, ONS will need to develop its systems to take on and link new data sources and to introduce the use of accounting software and an electronic questionnaire.

Learning from the pandemic

ONS should apply what it has learnt about collecting data during the pandemic to its long-term economic surveys

During the early stages of the pandemic, ONS used administrative data sources and new survey data to produce very timely indicators of economic performance. Many of the short-term indicators’ contained in ONS’s Economic activity, faster indicators, UK were produced using administrative data; including indicators on businesses’ turnover using HMRC VAT returns. These indicators allowed data to be produced on a timely basis, to help policymakers understand the immediate impacts of the pandemic on the performance of UK business.

These statistics were also complemented by data gathered using new and existing surveys to understand the ongoing impact of the COVID-19.

The Business Impact of COVID-19 Survey (BICS), launched in March 2020, provided users with data on the impact of the pandemic on business operations, including the status of employees following the introduction of the UK Government’s furlough scheme. The economic impact of the pandemic has also been captured via ONS’s Management and Expectations Survey, which includes information on business inputs, outputs, and prices.

The challenge of finding alternative ways to source data which would usually be acquired through surveys has demonstrated how statistical producers could change their business operations and innovate quickly to meet the information demands of users and economic policymakers. ONS was successful in meeting this short-term need and should use these successes as a platform to ensure that this same agility and innovation is introduced to improve its longer-term measures of economic performance. ONS should aspire to improve the timeliness and relevance of its ABS, in a similar way that it developed its online Labour Market Survey.

The survey response rate through the pandemic has been much lower than expected, running at approximately 50%, compared to around 75% in more usual years. The ABS team has been trying to use other more-accessible methods for respondents to provide their data, including the use of an editable pdf file. ONS estimates that approximately 3000 businesses have responded via this method during the pandemic. We commend the ABS team for seeking to ease the burden on its respondents and protect its pressurised response rate using these more-accessible tools.

Technology

Current legacy systems are impeding innovation and improvement to the ABS data

We discussed the possibility of using alternative data sources with ONS, who noted that it has considered using HMRC Corporation Tax data and BVD commercial information to improve the timeliness of the ABS.

The ABS production system, however, does not allow for the addition of new data sources, or the linking of other survey data within its infrastructure. This has negated efforts to improve the breadth and depth of ABS data and its timeliness through the inclusion of new data sources, or data linking.

ONS must invest in the development of its production systems to provide a platform which is sufficiently flexible and agile to allow for the inclusion and linking of alternative data sources. Unfortunately, the development of the ABS system has not been able to attract sufficient investment priority within ONS’s Transformation (Aries) Programme to enable the start of this work.

The constraint on system development, however, should not preclude ONS from establishing links with data suppliers able to provide data which can be used to reduce survey burden and improve data granularity and timeliness. ONS should tell its users its plans for sourcing data that will improve these aspects of the ABS product, such that when the ABS system is developed, ONS will be well placed to improve public value quickly.

Users

ONS needs to speak to a wide range of users and potential users to inform its statistical development

ONS engages with government users through its six-monthly Annual Business Survey Group. The membership of the group comprises representatives from government departments, including BEIS, HMRC, Department for International Trade (DIT), Department for Digital, Culture, Media and Sport (DCMS) and the three devolved nations.

ONS’s user engagement has focused on the needs of government users. It does not carry out the same level of engagement with users from academia, the private and the third sectors. ONS had planned to hold an event in March 2020 titled “Business Statistics and the modern economy” which included a wider range of stakeholders but due to the pandemic, this event was cancelled. It is encouraging to see that ONS is soon to run a UK Business Data User Conference. The conference is being jointly run by ONS and the UK Data Service and features academics from several UK universities focussed on business related research, covering topics such as productivity, investment, research and development, innovation, and business trade. It is also encouraging to note that representatives from the UK Data Service will be leading a session on how researchers can best gain accreditation for their research projects and thereby gain quicker access to ABS microdata, an issue which is discussed in more detail in para 1.61.

Engagement with representatives from the third sector and industry and professional bodies can offer new perspectives on the collection of inclusive and diverse data, information on the nature of business operations and management, and potentially offer the opportunities to link ABS data with industry survey information. It is important that ONS seeks to engage with the third sector as many third sector entities are not defined as “businesses” and as such there is a risk that engagement with and feedback from large components of this sector will be missed.

ONS invites users to provide feedback and to engage in discussion on data issues via its Business and Trade Statistics Community on the StatsUserNet Forum. There has been no recent activity on this forum and ONS should consider what value could be added by it taking a proactive lead in generating discussion about ABS statistics on this discussion channel.

Limited proactive user engagement and inflexible production systems have resulted in an absence of user-driven ABS development. We recognise the impact of the pandemic on the ABS team’s capability to engage and collect the views of a wide range of users. However, it is important to note that users’ interests will have shifted because of Brexit and the pandemic, particularly regarding the way in which businesses operate.

All the users we spoke to provided us with positive feedback on the service they had received from the ABS team. BEIS, DCMS and the Department for Environment Food and Rural Affairs said the ABS team is helpful and able to resolve queries or provide requested data in good time.

Access to the UK Data Service

Some users we spoke to (the former Industrial Strategy Council whose responsibilities now lie within BEIS, DCMS, DIT, Cardiff Business School) rely on the ABS microdata available through the DSSL, rather than just the headline figures in the ABS bulletins and data tables. These users reported to us that obtaining access to the ABS microdata for research purposes is a long and arduous task. One user told us that it had taken over 12 months to gain access.

Gaining ready access to microdata is important to researchers seeking to gain insight into, for example, the impact of Brexit and the pandemic on the location of capital and labour in the economy. The ABS team delivers data to the UK Data Service within a week of publication. However, delays in researchers’ ability to access these data lead to the postponement of these valuable insights, which otherwise could be used readily in the design of economic policy. Difficulties in gaining access potentially displaces important research, such as analysis of policy decisions on the employment dynamics of small and medium-sized enterprises, a key current consideration for the UK economy as it enters its post-pandemic recovery. Similarly, displacement of research examining the impacts of labour market mismatch on UK output and productivity, during this period of significant dislocation of labour, would have profound effects on the targeting of economic policy through economic recovery.

As a matter of urgency, ONS must work with the UK Data Service to significantly improve the speed of access of researchers to ABS microdata via the DSSL.

OSR reported on this issue as part of last year’s assessment of Business Demography Statistics where we noted that: “Improving access to IDBR data for users however is not a straightforward task. Before granting access to the data, ONS first must consider the confidentiality requirements of the Statistics of Trade Act 1947. Access to IDBR data can only be granted in writing to “competent authorities” to obtain “information necessary for the appreciation of economic trends and the provision of a statistical service for industry and for the discharge by government departments of their functions”. The IDBR houses a wealth of untapped data, improved access to which would be of great benefit to the development, introduction, and evaluation of business and economic policy. In the longer term we would support any efforts ONS makes to review the legislation to reflect current access requirements.”

Clarity and Insight

The ABS bulletins are generally clear and easytofollow and supported by helpful graphics. They draw out key messages from the data and offer a degree of insight that is accessible for less-expert users. There is preliminary commentary on various trends including the comparison of indicators for each of the devolved administrations, and industries.Industry contributions to approximate Gross Value Added (aGVA) are shown clearly with graphs and analysis. There are opportunities to develop the commentary in the statistical bulletin, for example, by providing explanations for the differences in aGVA across regions. There are currently no international comparisons in the bulletin, which will be important in the medium term for gauging the impact of Brexit and the COVID-19 pandemic.

ONS publishes a detailed technical report and quality and methods document alongside the ABS statistics, which users identified as being helpful for understanding the context and quality of the data. However, there could be a clearer indication of the fitness for purposes of the statistics and their strengths and limitations with respect to various potential uses. We welcome ONS’s commitment to review this.