Introduction

Overview

1.1 In November 2022, the Office for National Statistics (ONS) requested the temporary suspension of the National Statistics status (now known as accredited official statistics) of sub-regional and sectoral Business Enterprise Research and Development (BERD) estimates, following the discovery of the undercoverage of small and medium-sized enterprises (SMEs) within the BERD data. Following this, the Northern Ireland Statistics and Research Agency (NISRA) made the same request for the Northern Ireland Business Expenditure on Research and Development (NI BERD) statistics. A change in the UK Government tax policy in 2015 gave SMEs increased tax relief for research and development (R&D) activities. This change resulted in a larger uptake of R&D activities by these smaller businesses, which previous ONS and NISRA sampling methods were unable to capture.

1.2 Prompted by stakeholder observations, the ONS worked with HM Revenue and Customs (HMRC) to understand the divergence of these statistics compared with the R&D tax credits data produced by HMRC, leading ONS to significantly improve the sampling framework and methodology used to measure the expenditure of businesses that engage in R&D activity. Such improvements included a larger sample size and a move to electronic questionnaires and data collection.

1.3 The assessment, focusing on ONS’s BERD statistics, uses our Spotlight on Quality framework, which examines four key areas to evaluate the quality of statistics: whether the statistics are produced using suitable data sources; whether appropriate methods are used; transparent quality assurance; and whether the statistics are sufficiently prioritised and resourced proportionately to their use. We also consider the international comparability of the statistics and whether the statistics meet the quality needs of users and are not misleading.

1.4 ONS has implemented a three-phase approach to improve the methodology used to compile BERD statistics:

• phase 1: An interim uplift and adjustment approach was applied to the statistics that ONS published in 2022, covering data for the years 2014 to 2021, bringing BERD estimates more in line with estimates from other data sources that better captured the activity of smaller businesses
• phase 2: The introduction of a larger sample size and improved methodology in producing the lower-level breakdowns of BERD
• phase 3: The refinement of methods and systems, including the delivery and integration into National Accounts in 2025 and the production of a back series of the breakdowns of BERD in late spring 2024

1.5 This assessment took place at the end of phase 2. We consider ONS has made significant improvements to the sampling methodology used to produce the BERD statistics during these development phases. We recognise ONS is working with users of the BERD statistics to implement the final phase for improving the quality of these statistics, which will enable them to be used in the production of estimates of Gross Domestic Product (GDP) and other economic analysis. We will consider reaccrediting these official statistics once ONS completes the final phase of the methodology and has met the requirements of this report.

Requirements

1.6 We have identified two requirements that ONS must fulfil to improve the quality of the statistics to the standards required by the Code of Practice for Statistics.

UK Business Enterprise Research and Development (BERD)

1.7 ONS’s BERD statistics provide estimates of UK businesses’ expenditure on research and development (R&D), the source of funding for this R&D work and the number of people working on R&D in UK businesses.

1.8 BERD statistics are produced following the internationally agreed standards as specified by the Frascati Manual 2015, published by the Organisation for Economic Co-operation and Development (OECD), which defines R&D as “creative and systematic work undertaken in order to increase the stock of knowledge – including the knowledge of humankind, culture and society – and to devise new applications of available knowledge”.

How ONS produces BERD estimates

1.9 BERD statistics are compiled from data received from the BERD survey conducted by ONS, which covers R&D activity in Great Britain (GB). For the statistics covering the year 2022 (published in ONS’s February 2024 BERD publication), approximately 37,000 businesses, sampled by region and published product group (PPG) employment size band, directly from the Inter-Departmental Business Register (IDBR) were surveyed. ONS has reduced its sample to approximately 20,000 businesses for the survey covering the year 2023 following work to consider what overall sample size is adequate to achieve estimates of a good enough standard, while being conscious of the burden placed on businesses to complete the survey. ONS distributes short-form type and long-form type questionnaires to the businesses.

1.10 ONS incorporates estimates for Northern Ireland, produced by NISRA, to produce the UK BERD estimates. NISRA conducts an R&D survey of businesses based in Northern Ireland to compile Northern Ireland Business Expenditure on Research and Development (NI BERD) statistics. The regional estimates for NI BERD published by ONS are slightly higher than the estimates published by NISRA. This is because a small proportion of R&D is performed in Northern Ireland by businesses based in GB and is therefore not included in the Northern Ireland data collection process, excluding these R&D-performing businesses from NISRA’s published estimate. NISRA further explains this in its R&D statistical publication.

Uses of BERD

1.11 BERD statistics are vital for multiple government departments. For example, HM Revenue & Customs (HMRC) uses BERD statistics to conduct impact analysis, and the Department for Science, Innovation and Technology (DSIT) uses BERD statistics to deliver policy advice. The Bank of England (BoE) uses BERD statistics to gain insight into the UK’s innovation and productivity performance, which informs ad hoc analysis and the modelling of policies. BERD statistics are also used by academics and research professionals in writing research publications, which often inform policy debate.

1.12 BERD statistics are also used in the derivation of other statistical outputs, such as Business Investment statistics, which are used to estimate GDP.

1.13 BERD statistics are also supplied to international organisations, such as the Organisation for Economic Co-operation and Development (OECD). UK BERD statistics feed into the core OECD Main Science and Technology Indicators and the Analytical BERD (ANBERD) database, which facilitates international comparisons.