Dear Sir Ian

CORONAVIRUS (COVID-19) INFECTION SURVEY

I am writing to endorse the approach taken by ONS and partners in developing new statistics on COVID-19 infection. The statistics, published first from a pilot study in England estimate how many people have the COVID-19 infection, and how many are likely to have had the infection, even if they did not realise it at the time. This information is important for scientists and decision-makers trying to model the impact of COVID-19, because it makes models better able to predict the effect of school closures, social distancing, and other measures aimed at reducing the spread of the virus.

My team has conducted a rapid regulatory review of these statistics. We have reviewed the extent to which they have been produced in accordance with the Code of Practice’s Trustworthiness, Quality and Value pillars, while taking account of the pressures that ONS has faced to deliver timely statistics about a rapidly evolving national emergency. I really appreciate the time that your colleagues have spent talking through the development of the survey with my team. A summary of our findings, including recommendations, is set out below.

Value

• ONS’s agility in launching and resourcing this new survey has been remarkable. The rapid development of these statistics is essential in helping better understand the spread of the disease. The data will help scientists and the government work out how to manage the pandemic better, in order to save lives and protect the country’s health and care services from being overwhelmed.
• The commitment shown by the team to develop, improve and continue to evolve the presentation of the statistics has already been commendable. The statistics are being presented alongside an already crowded, and sometimes confusing, landscape of other data. ONS will need to consider carefully how it communicates clearly and unambiguously to all types of user what the basis for these statistics is and how it differs from others they might have seen. ONS’s plans to publish blogs should help to provide further insights.
• Continued collaboration with experts and producers of related statistics to ensure a clear and comprehensive narrative is vital. In particular, there is increasing focus on, and interest in, the reproduction number – R. We understand that ONS is still considering the role of the data in its own estimation of R, and in contributing to others’ research and modelling. Given the high profile of R, ONS should ensure that the role of these data in estimating R is clear.

• We welcome ONS’s engagement with colleagues from the devolved administrations about extending the survey UK-wide. Doing so will in turn add value to the separate understanding and decision-making in each of the UK’s nations. To ensure clarity of messaging, ONS will need to clearly outline expectations over when and how the statistics will expand into new areas.

Quality

• Early data reveal a tremendous response to ONS’s invitations to participate in the survey. Given the scale of ambition for the eventual sample size – 130,000 households and 300,000 survey respondents – the commitment to continued funding of the study is welcome. We are encouraged by early signs of the response to the survey, and ONS’s commitment to ensure a good sample size. Maintaining good response rates is essential to presenting robust results both for the population as a whole, and breakdowns.
• The clear presentation of statistical uncertainty is a helpful reminder to users that these data are based on a survey, and an essential element in ensuring the appropriate use of the data. Many of the concepts being measured are technical ­– ONS should consider how best to describe the statistics and technical terms for a range of users. For example, use of terms like ‘false positive’ and ‘false negative’ may not be clear to all. ONS will also need to clearly explain how accurate the tests being used are, the reliability of the methods for administering the swab tests and how much weight can be placed on the findings.

• The initiation of a range of data quality checks, developed at speed across all the organisations involved, provides reassurance about quality. We welcome that these checks have continued to be refined as the survey has developed, and the commitment to further review them to provide additional reassurances around quality. Your proposals to publish more-detailed technical and quality information will provide further reassurance to users about quality. We would encourage ONS to share any lessons learnt among the various teams that are developing new statistics, using these to enhance the recent guidance on urgent quality assurance of data as appropriate.

Trustworthiness

• The rapid reprioritisation of resources to enable people to work on this survey has required a considerable effort across ONS. Recognising the pressure staff at all levels may be under, and considering the ambitions for future iterations of the survey, balancing resourcing requirements with staff well-being will be vital.
• I am also happy to confirm the exemption from the Code of Practice for Statistics’ standard publication time of 9.30am to permit a later release time for today’s statistics of 2pm. This will enable the statistics to be as timely as possible under the circumstances, and I welcome your commitment to release the statistics at 9.30am from next week.

We look forward to seeing these statistics develop as further statistics are published. Based on the findings of our review you can include a statement in your release such as “These statistics have been produced quickly in response to developing world events. The Office for Statistics Regulation, on behalf of the UK Statistics Authority, has reviewed them against several key aspects of the Code of Practice for Statistics and regards them as consistent with the Code’s pillars of Trustworthiness, Quality and Value.”

I am copying this to Iain Bell, Deputy National Statistician and Director General for Population and Public Policy; and Pete Benton, Director of Population and Public Policy Operations.

 

Yours sincerely

Ed Humpherson

Director General for Regulation