Findings

Data sources and methods

2.1 We consider NISRA to have made significant improvements to the sampling methodology used to produce NI BERD statistics. NISRA now draws its survey sample directly from the IDBR, no longer relying on previous NISRA R&D surveys and other sources, including a filter question on the Northern Ireland Annual Business Inquiry. Drawing directly from the IDBR has enabled a more accurate sampling of businesses, particularly SMEs, that perform R&D in Northern Ireland. This decision also aligns with ONS’s practice, increasing the alignment of NISRA’s methods and the subsequent NI BERD estimates with ONS’s GB BERD data.

2.2 NISRA has used online data collection methods for over a decade. NI BERD is collected through the Integrated Business Survey System, with the majority of businesses responding via the online data collection portal, and the remaining small number of businesses responding with paper forms. We appreciate that a full transition from paper questionnaires to EQs would require a significant investment in infrastructure to handle their distribution, collection and analysis, and we commend NISRA for its aim to transition to 100% online data collection by 2025.

2.3 NISRA publishes a sample questionnaire for its R&D survey, although this was last updated with the NI BERD statistics 2021 release. Users explained that having access to sample questionnaires enables them to gain a better understanding of how BERD data are collected and the quality of these statistics. We encourage NISRA update the sample questionnaire and guidance in line the latest R&D survey as questions are updated.

2.4 NISRA told us that it does not intend to expand the NI BERD survey sample size from 1,500 businesses, as it was felt to be a sufficiently large sample that would produce reliable estimates, reinforced by the move to more-robust sampling methodology. We welcome NISRA’s plan to work with ONS to replicate their sampling process.

2.5 In the February 2024 NI R&D data release, NISRA published NI BERD data breakdowns with time series of various lengths. Some (namely business size, employment and funding) spanned 2018 to 2022, whilst others (namely sector, regional and ownership splits) were limited to 2021 and 2022. However, it is good that NISRA and ONS have begun to discuss the potential of introducing and developing a methodology for a back series of NI BERD estimates.

2.6 Both NISRA and ONS use the GDP deflator to convert their respective current-price BERD statistics into constant-price statistics. Since the GDP deflator also includes non-market output, prices in R&D may not fluctuate consistently with the GDP price deflator. Furthermore, this non-market output played a more significant role during the COVID-19 pandemic period than usual. We encourage NISRA to review, and update where necessary, the deflator used in deriving these constant-price statistics, taking into account the variables that contribute to changes in the price of R&D activity. This will enhance users’ trust in the quality of the constant-price NI BERD statistics, ensuring a representative price movement for the R&D activity is reflected.

Systems, resources and prioritisation

2.7 We praise NISRA’s decision to transfer its imputation methods from SPSS software to R. NISRA explained that this transition has reduced the level of manual intervention required. Consequently, the change has increased the efficiency of production as well as the quality of the NI BERD statistics. We also welcome NISRA’s plans to create a Reproducible Analytical Pipeline (RAP) to further increase the efficiency of NI BERD data processing.

2.8 Throughout the development process, ONS and NISRA have proactively collaborated, holding monthly meetings to share updates on progress and to ensure that respective methodologies are consistent and coherent, the transformation is delivered smoothly, and the resulting estimates are comparable. Users and data suppliers have recognised and praised the efforts made.

Quality assurance

2.9 Users praised NISRA for providing a comprehensive range of granular breakdowns of the NI BERD statistics, including business size, Local Government District (LGD) splits and sectors, enabling them to meet their varied needs.

2.10 The NI BERD statistics are compiled from an annual survey, and therefore annual publication is the most viable option. While this frequency suited most users of these statistics that we spoke to, some indicated that they wanted more frequent (bi-annual) indicators.

2.11 Users also told us that they would like NISRA to explain the drivers of the revisions to enhance their trust in NI BERD data. Whilst NISRA signposted ONS’s articles explaining the transformation of UK BERD statistics, little information explaining the plausibility of the revisions of NI BERD estimates is provided. Users particularly wanted to further understand the revisions made to the uplifted estimates of the proportion of R&D activity carried out by SMEs in Northern Ireland. NISRA should work with users to identify what they require to understand the revisions.

2.12 We welcome NISRA’s plans to investigate the feasibility of developing confidence intervals for NI BERD estimates, to support communicating uncertainty within the statistics to users. The NI BERD team plans to collaborate with the Northern Ireland Economic Trade Statistics and Northern Ireland Annual Business Inquiry teams, to produce a consistent methodological approach and code in R for producing confidence intervals.

2.13 NISRA communicated the quality of the NI BERD statistics clearly and accessibly in the NI R&D Quality Report 2024. The quality report assessed various aspects of the NI BERD statistics, including their relevance, accuracy, coherence and respondent burden, whilst providing contextual definitions to support user understanding. We commend NISRA for providing this information to users in one comprehensive document.

User engagement

2.14 Users praised the accessibility of the NI BERD publications and the supporting explanations, which they felt facilitated their understanding of how NI BERD statistics were compiled and their quality. Users also thought that the direct engagement with the NI BERD team was efficient and effective.

2.15 We welcome the fact that NISRA is currently establishing an expert user group for the NI R&D survey. NISRA plans to use this group to inform users of the ongoing improvements to NI BERD statistics. For recruitment, a notice has been put on the NISRA R&D webpage asking interested parties to get in touch.