Findings

Resources for the ongoing production and development of BESES

Users praised the diligence of the BESES statistics team to us but highlighted that it was their view that the team was overstretched and under-resourced, with data requests for some users taking longer than expected to fulfil. In NISRA’s own user consultation, respondents suggested that BESES would benefit from greater resourcing and staffing levels.

NISRA told us that the time of its BESES statistics team is fully accounted for in meeting the demands of business-as-usual and ad hoc data requests from customers such as Department for the Economy. Accordingly, meeting added future needs, depending on complexity of the predicted development, could (to a degree) be constrained and may require wider available resources to be used flexibly as in the past.

NISRA has improved the efficiency of the data collection of the quarterly business surveys by amalgamating some individual data collection teams into one team. NISRA additionally plans to amalgamate the annual survey data collection teams into one. The intent is to free-up BESES statisticians to focus solely on the production of the statistics, continue to manage quality effectively and further develop the statistics. The BESES statistics team is also working to improve the understanding of the survey across statisticians in the wider teams of economic accounts, EU Exit and NIABI with a view that support could be made available if there is an unexpected absence of a BESES statistician or demand over and above existing requirements.

The BESES publication date is dependent on the NIABI publishing on time as there are several common variables across both datasets and the NIABI publication sets the top-line totals to which BESES is constrained. The BESES statistics team normally has six to eight weeks over the summer months to make any developments. Due to the packed publication schedule and dependence on the NIABI being published on time, there is little scope to make changes outside this small development window. If an unexpected event occurs, then the BESES statistics team risks losing that vital development time. The BESES statistics team, however, despite being small is highly resourceful. For example, it used an opportunity of a delayed publication due to COVID-19 to undertake development work over the winter of 2020-21.

NISRA has resources outside of the immediate team to develop its outputs. For example, NISRA has made capital funding available for projects to improve systems and processes for its statistics through NISRA Tech Lab. The Tech Lab is a dedicated branch designed to harness innovative technologies and improve processing efficiency, through targeted investment and bringing together skilled personnel.

The BESES statistics team has proven that it can meet the information requests of users, even in times of high workload. However, there is clearly little spare capacity in the immediate team to respond to any surge in demand for statistics, or to develop the statistics, and the BESES statistics team acknowledged our concerns around the future development of BESES and (although not in the scope of this assessment) NISRA’s trade statistics more generally, given the limited resources already available to develop the statistics. NISRA also told us it does not currently have the resource (budget or availability of staff) to expand the immediate team. Continuous innovation and improvement to BESES will be an important part of ensuring that they continue to offer the current highly valued insight and good public value beyond the short term. Given the resourcing queries raised by users in response to NISRA’s own consultation, and the ongoing potential for increased demand for data associated with NI’s trading arrangements following the UK’s exit from Europe, it will be important for NISRA to continue to monitor on an ongoing basis the ability, including continuing to flex capacity, to resource production and developments to meet users’ needs and serve the public good.

Improved timeliness and alternative data sources

The time between the reference period and publication of BESES headline results is normally just under 12 months, but due to COVID-19, in 2021 the period was nearer 16 months. Much of that time is when the NIABI fieldwork takes place. BESES headline results are published approximately 6 weeks after the closure of the database, and 3 weeks after the NIABI results (which include BESES constraint totals) are published.

Users told us that the time between the reference period and publication was a key impediment to their usefulness, which was also highlighted in NISRA’s own user consultation. Users from UK government and trade organisations told OSR that to make informed policy decisions, they need more-timely statistics. NISRA told us the main reason for the timing of the publication release compared to the closure of the data collection phase is that the three BESES publications are produced, and quality assured separately. NISRA plans to discuss timeliness with its user group to gain a deeper understanding about those users who would find more-timely results beneficial and for what purpose, along with the impact that not having them earlier is having on their work.

We commend NISRA for developing parts of the production process to reduce the time it takes to produce ad-hoc queries, for example by testing automated statistical disclosure control procedures. NISRA told us that the current statistical disclosure checking process (ensuring no person or organisation is identifiable from the results of an analysis of survey or administrative data, or in the release of microdata) doesn’t impede current publication timetables but does increase response times to queries. Automated disclosure checking has been very successful and NISRA hopes to fully roll-out its use quickly.

In our view, NIABI may not be the most suitable source for providing quicker estimates, but NISRA needs to find a solution that meets users demands for sales and export statistics that can be used for making informed policy decisions. Users told us that they are prepared to accept a trade-off between the accuracy in the estimates for much quicker availability. Taking account of users’ feedback, NISRA needs to consider whether alternative approaches to producing BESES, for example publishing early provisional estimates using an earlier cut of NIABI data, or using alternative data sources, could better meet users’ needs.

We commend NISRA for starting to look at alternative sources for earlier estimates, for example, Eurostat Northern Ireland trade in goods data, which are produced monthly based on HMRC data. NISRA is pursuing direct access to these HMRC data. Another possible solution to improve timeliness could be via the Integrated Business Survey System (IBSS) which is due for an upgrade in 2023. The IBSS is a data capture IT system used by NISRA to administer business survey questionnaires, store and validate all the business data collected. Research work has started to look at what functionality is required and in time is expected to improve the efficiency and timeliness of the data collection process.

Requirement 1: NISRA needs to establish with users what further public value could be added by producing sales and exports statistics more quickly, and to (a) develop plans to meet those needs, and (b) be transparent about where the needs cannot be met.

Improved product-level data

Some users expressed concerns about some of the granular breakdowns of BESES, particularly in relation to information on those businesses exporting services and those trading internationally. NISRA produces data relating to individual countries in the EU, but due to a relatively small proportion of NI businesses trading internationally, NISRA aggregates results outside the EU up across multiple countries to obtain meaningful results, for example North America rather than the countries making up North America. Publishing individual country data may satisfy more users, but we recognise that NISRA is limited by the need to protect confidentiality for certain sectors which may include only a few businesses.

Users told us that they would welcome more-disaggregated BESES with greater levels of sectoral and geographical detail. Examples of the data gaps users expressed to us include data on types of firms and size of firms published more as standard and to be able to compare the different size distributions of different types of firms. Users would like finer geographical breakdown of the markets, for example USA rather than North America.NISRA’s development plan for these statistics published in February 2021 highlights the intention to address these user needs by publishing more-detailed breakdowns and says “Upon completion of this programming project and investigations we will release more granular BESES data as dictated by our disclosure rules”.

Through two-way cooperation between NISRA and HMRC, BESES have helped to improve HMRC’s Regional Trade in Goods statistics. Further cooperation between NISRA and HMRC offers further opportunities to triangulate each other’s data to further improve all NI trade data.

NISRA is keen to respond positively to its users’ needs for better data on NI-GB and NI-Republic of Ireland trade. These users’ needs were echoed in written evidence to the House of Commons Northern Ireland Affairs Committee’s Brexit and the NI Protocol Inquiry highlighting the dramatic effect of both the Trade and Cooperation Agreement and the NI Protocol on both UK imports and exports with the EU and a substantial reorientation of trade between the Republic of Ireland and NI as well as trade between NI and GB[1].

NISRA worked with ONS, the former Department for Exiting the European Union, Department for the Economy and other departments to develop a trade survey of hauliers in 2019 with a view to improve product-level data for trade between NI and GB. However, haulage companies did not have the information needed to supply robust data. New HMRC data being collected by HMRC on trade between NI and GB are likely to be the best source of official product-level data on trade in goods between NI and GB. NISRA has taken active steps to help NI trade statistics users access contemporary product-level trade data and gain a better understanding of trade flows between NI and GB. NISRA has submitted an application to HMRC to access these data with the hope that they will help strengthen its statistics, particularly the product-level data available in the NI Supply-Use Tables.

We support NISRA’s endeavour to share and link for its BESES users to already-existing data to add further insight into NI trade. NISRA has committed, once it has a response from HMRC to its application to access product-level data on NI-GB trade, to advise BESES users quickly on whether they can access product-level NI-GB trade and where and when they can access such data.

[1] https://committees.parliament.uk/writtenevidence/37759/pdf/

Quality of BESES

Users we spoke to as part of this assessment were confident in the quality of BESES and were satisfied that the quality of the statistics was sufficient for their needs. As a sample-based survey, estimates from the NIABI will have some degree of uncertainty and NISRA explains this uncertainty in its external sales statistics. Users told us that they are aware of the limitations of the statistics. ISRA’s user consultation highlighted that most respondents said that NISRA’s advice around strengths and limitations of the statistics was clear, and the statisticians involved were readily available to answer questions relating to the data as well as any limitations or caveats. However, users highlighted that they would appreciate a summary of limitations that can be easily copied and cited.

NISRA has collaborated with the Republic of Ireland’s national statistical institute (Central Statistics Office – CSO) in looking into reasons for and possible mitigating measures about trade asymmetries[1] arising from the trade figures reported by both organisations. This work provided insight into what might be driving significant changes in estimates of trade between NI and the Republic of Ireland. This work has helped to prepare for recent analysis by the CSO into trade between NI and the Republic of Ireland since the implementation of the NI Protocol.

As part of the new rules in the NI Protocol, separately identifiable VAT registrations for NI businesses were introduced from January 2021. This introduction of NI VAT registrations has allowed traders to recognise and report their trade with NI more accurately. Previously it was not always possible to distinguish whether a UK VAT registration related to a trader in GB or NI. Since the start of 2021, CSO has queried with traders when there has been a reportedly large increase in trade between NI and the Republic of Ireland since the start of 2021, in many cases, the increases are explained by changes to supply chains. However, in some cases, traders have reported that they had previously been incorrectly reporting their trading partner as being in GB, when in fact the partner was a trader in NI. CSO has revised its data for a period of two years (2019-2020) to allow more-accurate comparisons of Republic of Ireland’s trade with Northern Ireland before and after EU Exit.

Understanding trade asymmetries plays an important part in helping NISRA to understand the quality of its own statistics. The BESES team is better prepared to answer users’ queries when it understands the differences between BESES and equivalent statistics from the CSO. We commend the work that NISRA has done with the CSO. The Department for the Economy, the Department for Foreign Affairs and CSO are working together to better understand the most recent CSO data. NISRA should continue to work with other organisations such as CSO, ONS (which has its own extensive programme to investigate trade asymmetries) and HMRC as appropriate to further analyse trade asymmetries. Publishing details of its findings can help further reassure users about quality.

[1] Bilateral trade asymmetries occur when the reported exports from country A to country B do not match the reported imports to country B from country A.

Presentation of results

As a result of NISRA’s ongoing engagement, users reflected that the public value of BESES has increased over the last few years. BESES have been essential for explaining trade flows between NI and GB both in the run up to and post EU Exit. Users told us that the NISRA team is helpful, provides added analysis within the constraints of the data and that it clearly communicates any issues with the data. NISRA has acted on user feedback by producing documents to explain the differences between BESES and HMRC RTS methodologies and produced PowerPoint presentations at points on the timeline towards EU Exit. NISRA receives high volumes of queries via email and phone and responds to them often promptly. Additionally, NISRA has also provided training sessions about BESES to colleagues in the Department for the Economy. However, some users told us that although the BESES statistics team is responsive and there are good working relationships, they would always welcome new and innovative ways to engage them such as the use of webinars or user fora after publication so they can ask general questions about the statistics.

The majority of BESES results are now presented in both HTML and PDF formats. NISRA also supplies alternative data presentations of BESES, for example demonstration tables in Excel which are not an open format and reference tables in ODS, an open format that improves the usability, accessibility, and machine readability of statistical spreadsheets. However, the Open Document Spreadsheet files are not easily accessible from the statistics landing page; NISRA should update the landing page with a textbox to highlight to users the different formats available and improve user access to the statistics.

NISRA publishes several background and methodology documents alongside the statistics on its website, including a Quality Report, a Summary of Usage document, a Development Plan and a Background Notes These documents give a detailed and helpful account of the methods used to produce the data and highlight any limitations but there is a lot of repeated information making navigation confusing for users. For example, background information to BESES and user lists are provided in a separate background information document, in the summary of usage document and again in the development plan. The “uses” of BESES are covered in both the summary of usage and development plan.

Requirement 2: To aid use and re-use of BESES data, NISRA should (a) update the statistics landing page with clear information to users on how to navigate BESES and an explanation of the different formats available (b) improve the presentation of quality information.

Understanding the effects of inflation on BESES can be important for users because it gives them a more-accurate reflection of the real value of trade, the competitiveness of businesses in international markets and could lead to more-informed policy decisions around investments. We recognise that producing real-terms estimates depends on the availability of appropriate deflators and at present trade-in-services deflators are not published. However, we consider that as part of its ongoing engagement with users, NISRA should establish the extent of demand for such a series and use that knowledge in its prioritisation.