Dear Sir Ian
CORONAVIRUS, THE UK ECONOMY AND SOCIETY, FASTER INDICATORS
I am writing to endorse the new experimental faster indicators that ONS has published providing information about the impacts of the Coronavirus (COVID-19) pandemic on the UK Economy and Society. This is ground-breaking and remarkable work from everyone involved in producing these timely and valuable statistics under challenging circumstances.
My team has conducted a rapid regulatory review of these statistics, which are constructed from rapid response surveys, novel data sources and experimental methods, using the existing ‘Faster indicators of economic activity: UK’ release as a vehicle to publish those series developed for understanding the impacts of COVID-19. We have reviewed the extent to which they have been produced in accordance with the Code of Practice’s Trustworthiness, Quality and Value pillars, while taking account of the pressures ONS has faced to deliver timely statistics about a rapidly evolving national emergency. A summary of our findings, including recommendations, is set out below. More detailed feedback has been provided to the ONS statistics team responsible for producing the statistics.
Value
- We welcome the rapid development of the new statistics published alongside some of the existing faster indicators in a re-purposed weekly statistical release. The appetite for faster information on UK economic activity has never been higher. Judging by the response of the media and commentators to the first release, the statistics will prove invaluable to decision-makers. Additionally, further public good is likely to be delivered from their use by a wide range of organisations and the general public in measuring some of the economic and societal impacts of the pandemic in close-to real-time.
- While extensive user consultation about the questionnaire topics was not possible due to time constraints, ONS worked with policy and analytical leads in government departments to help identify priority topics for the of the new Business Impact of Coronavirus (COVID-19) Survey (BICS). Within ONS, the statistics team has involved the ONS COVID-19 Outputs Group, ONS’s National Accounts Steering Group and the COVID Rapid Response team about their needs from the survey. Additionally ONS staff in publishing, digital and content were asked for guidance in meeting the perceived needs of the typical ‘inquiring citizen’ user for this analysis – defined as those who are often motivated to check out statistics because they want to find out more about a topical issue they have seen on the news, such as changes to indices like RPI or CPI. We welcome your commitment to continue to take account of wider user feedback about the published outputs as time progresses.
- There is frequent dialogue between the ONS’s COVID-19 Outputs Group and the statistics team about introducing further faster indicators to the release over the coming weeks. These include the use of new sources of data on, for example, changes to households’ spending during the pandemic and the development of some geographic and industrial sector breakdowns of BICS survey findings. As the quantity of data in the release is likely to expand; statistical coherence is particularly important to avoid large amounts of information obscuring some of the key messages from the statistics. We encourage you to reflect on the evolving user need and available data to adapt the approach for how to present the various indicators, for example whether a dashboard would suit.
- We welcome the very helpful metadata that ONS has published about the faster indicators since their inception, particularly about how much weight users should place on the statistics as lead indicators of other economic output indicators e.g. Gross Domestic Product (GDP). ONS have given some suitable warnings in this release about inappropriate use for example of the price indicators on High Dependency Products. It will be helpful to enhance these caveats for ONS to include references to articles that have previously explained the role of the faster indicators and how to interpret them in relation to the other short-term economic indicators.
Quality
- As with many surveys there will be biases and sampling variability in the resulting statistics. We welcome that ONS has alerted users to the fact that results from the BICS survey reflect the characteristics of those who responded and not necessarily the wider business population. In due course it may be possible to expand and say more on the possible distortive effects of the non-response biases, and about the population of the businesses surveyed and the kinds of smaller businesses such as many self-employed businesses which are not surveyed.
- Given the circumstances and the salience of the topics covered, there will be additional scrutiny of these statistics compared to the previous faster indicators of economic activity: UK release. We encourage ONS to ensure that the methodological information provided about the sample design, response and any potential biases for the BICS survey, choices of products included in the High Dependency Products listing are sufficiently accessible and detailed to support this enhanced degree of public scrutiny.
- Due to the rapid development of the new indicators there is less reliance on statistical process control and a higher degree of expert judgement in reviewing and challenging the data. This is particularly the case for price data for High Dependency Products and the BICS data on business impacts. The quality of the shipping indicators has been checked against subsequent imports data and shows a reasonably high level of correlation. However, there is seasonal variation in the shipping data that cannot yet be adjusted for until a longer tie series of data has been established. ONS has set up a ‘hub and spoke’ model where experts in the different ONS statistics teams (the ‘spokes’) can advise those at those producing the statistics (the ‘hub’) about the quality of the data. As the data collection process matures ONS might seek to develop its quality assurance to incorporate some further checks on the data.
Trustworthiness
- We commend the level of resources that have been reprioritised quickly to enable people to work on this. It has required a considerable effort from lots of different teams. ONS should ensure that resourcing requirements are reviewed regularly as the publication develops and ensure that staff wellbeing is maintained while delivering under pressure.
We look forward to seeing these statistics develop as further outputs are published. As set out in the guidance on changes to statistical outputs you can include a statement in your release such as “These statistics have been produced quickly in response to developing world events. The Office for Statistics Regulation, on behalf of the UK Statistics Authority, has reviewed them against several key aspects of the Code of Practice for Statistics and regards them as consistent with the Code’s pillars of Trustworthiness, Quality and Value.”
I am copying this to Jonathan Athow, Deputy National Statistician for Economic Statistics.
Your sincerely
Ed Humpherson
Director General for Regulation