Albert Einstein was reputed to have said that curiosity has its own reason for existing and the important thing is not to stop questioning. This blog is a case study example of the value of indulging curiosity to help strengthen the quality of valuable official statistics- and in this blog we look at ONS’s Employees in the UK statistics from Business Register and Employment Survey (BRES) survey data. Professor Sir Charlie Bean in his 2016 Independent Review of UK economic statistics, reflected that to better understand the modern economy and people’s lived experiences of the economy, ONS statisticians should be more curious about their statistics and what stories they are telling. He set out three inter-linked ingredients that are needed to help meet this objective of building a ‘curious’ ONS that is more responsive to changes in the economic environment and better meets evolving user needs.
Chart 1 Three inter-linked ingredients of building a more curious ONS: Strengthen QA, improve appreciation of the statistics in use, raising staff knowledge.
Source: Independent Review of UK economic statistics, Sir Charles Bean
In early summer of 2018, Cambridge Ahead (CA), a membership organisation involving people and organisations dedicated to the successful growth of the Greater Cambridgeshire region, contacted both OSR (the Office for Statistics Regulation) and ONS. CA drew our attention to their concerns about a lack of alignment between employment growth estimates for the region from its own database compared to those from ONS’s BRES-sourced statistics as exemplified in Chart 2 below: Source: Analysis provided by CBR based on latest data
Chart 2 showing Average employment growth over six year 2012-2018 comparing BRES and CA estimates.
After engaging with ONS and CA we felt that engagement between the two organisations provided an opportunity for ONS to pursue a potentially new way of testing the quality of the BRES estimates. The purpose of this blog is not to go into the detail about the reasons why the estimates don’t align, we are more interested in how the curiosity could lead to improving the insight and quality of these employment statistics.
We saw that ONS and CA, exchanged details of their methods and details about their data. Both appreciated the strengths of the other – for example BRES offers labour market data at low levels of geography and CA’s analysis introduces a greater understanding of the employment market in this region.
ONS has described BRES statistics as providing a comprehensive picture of jobs in the UK but recognise that these statistics, like most derived from sample surveys, display some limitations and may not immediately pick up new businesses. ONS intends through its transformation programme to source employment numbers from PAYE income tax data. This offers the potential to give ONS close to real-time employment information from every business with a PAYE scheme, including newer businesses. This would provide a wealth of new information that ONS can use to provide even more detail and more timely figures for statistics users.
While this is exciting news and very much to be welcomed, developments like these can take time to implement. We feel there are more immediate opportunities to strengthen the insight that these statistics offer to users as well as quality assurance of the source data.
CA believe that they have a comparative advantage compared with ONS in capturing smaller companies, which don’t get picked up on the IDBR (Inter-Departmental Business Register – the sampling frame for the ONS data) due to such businesses not being registered for VAT or PAYE purposes. In an area with a lot of small start-up companies, this could be a significant cause of differences. The sample allocation for BRES was last reviewed 10 years ago. Sample re-allocation is important because of changes in the economy over time; the existing allocation may not reflect the current structure of the labour market. In the OSR’s assessment of ONS’s Labour Markets Statistics we have required ONS to review and update the sample allocation (our Requirement 7a). We expect ONS to have acted upon our requirement by March 2021 with a formal update on a quarterly basis. In the meantime, ONS should publish an action plan which sets out its proposals for addressing the Requirements.
What has been instructive about ONS opening its data to scrutiny and challenge is that the engagement progressed beyond an attempt merely to reconcile differences in estimates. The challenge posed by the alternative estimates has been the impetus to check out the reasons why and question whether the methods and current checks used to produce BRES estimates are adequate. The engagement has been successful in that it has been a key input to some of the requirements that the OSR has made in its assessment of ONS’s labour market statistics leading to better employment statistics not just for one region but for all regions throughout the UK.
ONS is keen to continue to engage with CA to help better understand the labour market in Cambridge and Peterborough. This curiosity work between ONS and CA underscores the importance of covering all three of Sir Charlie Bean’s ingredients of curiosity – improving the appreciation of the statistics in use, raising staff’s knowledge and strengthening the quality assurance. This was always an exercise in taking advantage of additional data to see whether official statistics could be improved. In this respect curiosity has proved that it has a very important reason to exist.