Dear Sarah

ONS COVID-19 Infection Survey ad hoc analysis

Thank you for raising your concerns about the Office for National Statistics’ (ONS) ad hoc analysis of the number of school workers, key workers and other professions in England who had COVID-19, which was included in the 6 November Coronavirus (COVID-19) Infection Survey bulletin. It is important that data are shared in a trustworthy way and that government can draw on reliable data to make important decisions.

We have considered the concerns you raised and respond to them below. We also want to make you aware that we are undertaking a review of the COVID-19 Infection Survey as a follow up to the Rapid Review of the survey we completed in May 2020. We will consider some of the points you raised in more detail through this review, which we expect to publish in early 2021.

Presentation of statistics in the Office for National Statistics bulletin

You highlighted concerns about the ad hoc analysis published by ONS as part of the COVID-19 Infection Survey results on 6 November. We do not think that ONS intentionally presented the analysis in a misleading way. However, there are some changes which could have been made to the analysis and the accompanying text to support those reading the bulletin in understanding the analysis.

ONS has published a statement in response to some of the concerns raised about the analysis, which addresses some of the issues you raised. We have also considered the concerns:

End date of the analysis: ONS explained that the ad hoc analysis only covered the time period up to 16 October, not 31 October like other analysis in the bulletin, because the occupation data requires further processing and is only available around two weeks after the initial headline results. This meant that outputs could not be produced as quickly for this analysis as other parts of the publication. It would have been helpful if ONS had given a rationale for the discrepancy when the data were first published.

Evidence of a change: We agree that ONS could have done more to explain the uncertainty around the estimates for education staff categories, and the implications of this. It could have been clearer that no evidence of a difference is not the same as evidence of no difference. It should also have explained the impact of small numbers of cases in some categories on the ability to determine any differences. ONS has now updated the analysis to include “all teachers” in one category.

Use of data by the Chief Medical Officer (CMO) prior to publication

My team has also looked into the timeline for the sharing of information between ONS and the UK Government and SAGE. In our view the CMO was not quoting the ad hoc analysis published by ONS on 6 November when he addressed the Science and Technology Committee on 3 November. ONS shared the results of the analysis with SAGE on 4 November, which was then referenced in the report that was made public on 13 November. We have been told that the evidence which informed the CMO’s response on 3 November included other published ONS data, data on COVID-related deaths by occupation. His response was consistent with the Statement from the UK Chief Medical Officers on schools and childcare reopening made in August 2020, though we would have liked to see clearer references to the evidence quoted in this statement.

The Coronavirus (COVID-19) Infection Survey is a critical source of information on the pandemic, so it is important that the statistics meets the highest standards of the Code of Practice for Statistics. We will investigate further the issues you have raised with us as part of our in-depth review of the trustworthiness, quality and value of the Infection Survey statistics.

Yours sincerely

 

Ed Humpherson

Director General for Regulation