Dear Victoria
Compliance review of statistics in development from the Winter Coronavirus (COVID-19) Infection Study, England and Scotland
Thank you very much for inviting us to independently review the trustworthiness, quality and value of the statistics in development from the Winter Coronavirus (COVID-19) Infection Study (Winter CIS). The Winter CIS was jointly developed by the Office for National Statistics (ONS) and the UK Health Security Agency (UKHSA), covering households in England and Scotland. On 21 December 2023, using raw survey data from the Winter CIS, UKHSA first published a series of reports titled Estimates of Epidemiological Characteristics.
Information about COVID-19 (that can be caused by the coronavirus SARS-Cov-2) continues to be of high interest for expert users, such as public health and NHS officials, health researchers and the media, who have used your Winter CIS estimates over the winter for a number of reasons:
- to act as an early warning for any potential outbreaks of COVID-19 for those planning and delivering the NHS and other services preparing for winter stressors.
- to observe the prevalence of coronavirus variants in the community.
- to supplement and triangulate existing coronavirus community surveillance data, such as acute respiratory infection incidents.
- to raise public awareness of COVID-19 infection levels in England and Scotland.
Involving users to assist with continuing development
I welcome the innovation and agility shown by your team to develop the analyses throughout the study period. This has been demonstrated on two fronts: through transparent joint and collaborative working, with continuous learning, allowing your team to iterate your working processes; and by trialling a different methodology while encouraging expert user feedback.
We note from a recent Blog by ONS that statistics to capture information about respiratory viruses will continue to evolve and we consider that retaining the label ‘official statistics under development’ seems appropriate. To inform future iterations of infection studies more generally, it is good practice to continue to engage with users of the Winter CIS to help develop the methods, data and metadata transparently.
Coherence across the UK
Both ONS and yourselves have been very clear to link to other statistics about the coronavirus and COVID-19. We welcome this approach to provide coherence with existing coronavirus data from the rest of the UK.
Users of the Winter CIS have identified the need for comparable statistics across the UK, particularly to understand the effects of coronavirus on the lives of individuals. During any future planning for a study of this nature, we consider that time should be taken by statistics producers to fully consider the possibilities for the development of coherent UK-wide statistics.
Ensuring transparency of methods
It is good that you have published a Quality and Methodology Information report which explains the uncertainty, strengths and limitations of the estimates that you have published. To produce these modelled estimates, you used Multi-level Regression with Poststratification (MRP) methodology. The statistics team told us it has submitted the methodology and data for academic peer review, which is expected to be published and so made available for public scrutiny, which is good practice for official statistics in development.
Insight and forward plans
Over the course of the study, as well as descriptive analyses about the latest estimates, you have published an Explainer article and followed up with news releases so that summary information is made more accessible for the less expert user. Once you have finalised publication dates for future research or analyses using Winter CIS data, such as the modelled Infection Hospitalisation Risk for Scotland, as well as using informal ways to let people know, it would be good practice to also pre-announce these as ad hoc official statistics.
I would like to thank your team for its positive engagement with us during this review. Please do not hesitate to get in touch if you would like to discuss any aspects of this letter.
Yours sincerely
Mark Pont