Dear Ed,

COVID-19 Infection Survey Statistics

Further to the Office for Statistical Regulation (OSR) review of the Office for National Statistics (ONS) COVID-19 Infection Survey (CIS) in March 2021, I am pleased to provide you with an update on changes we have made to further develop CIS statistics. A summary of the changes made to meet the recommendations highlighted in your review and our next steps are outlined below.

Value

In your review, you highlighted the need for us to continue to openly engage with CIS users, and we have strengthened our approach in a few ways. Firstly, we have increased our resource and capability to engage with users in a wider range of media. This includes social media and in-depth discussions with interested stakeholders, including members of the public, who wish to present challenge and ask further questions to understand our approach.

In October 2021, the CIS User Group was launched which is comprised of key survey users from other government departments, modelling groups, and the devolved administrations. Group members were invited as a result of us proactively seeking those who CIS findings would be relevant to, who had prior engagement with us regarding CIS data, or who were known to use CIS data for decision-making purposes. The monthly meetings give users the opportunity to influence CIS plans and developments and provides ONS with specific feedback on user requirements and priorities. For example, user group members recently indicated that a higher antibody threshold of 800ng/ml would be useful to them and advised us how to communicate the outputs to best suit user need. As a result, we are now in the process of introducing this higher threshold into our antibody analyses.

Your recommendations about improving accessibility of our statistics are well aligned with the UK Statistics Authority strategy ‘Statistics for the Public Good’, which we aim to embody in all work produced. Since your review, we have significantly developed our popular COVID-19 Latest Insights Tool, which provides a summary of all pandemic-related data designed with the public in mind. A variety of features have been added to the tool, including the addition of comparisons of our data to external COVID-19 data. In July 2021, the Coronavirus Round-Up, another successful product summarising ONS publications, was incorporated into the Latest Insights page. This resulted in an increase of web traffic to Latest Insights and simplified the user experience, providing a single destination for pandemic data.

Since the review, we have jointly won the 2021 Harding Prize for Useful and Trustworthy Communication. CIS was awarded the prize due to accurate and impartial communication on changing infection rates during the pandemic, which shaped decision-making and became a trusted and reliable source of information for the public.

In 2021, Latest Insights had the highest number of page views on the ONS website for most months of the year. User research has revealed a substantially higher satisfaction score than other ONS products, with the general public providing positive feedback, especially for the simple narratives accompanying the data. To further improve accessibility, we have increased the prominence of

Latest Insights and other CIS outputs on the ONS website and regularly promote outputs on social media. Future plans for our insights tool are outlined later in this letter.

Trustworthiness and Quality

In accordance with your recommendation to improve the published information concerning CIS methodology, further revision has taken place to our communication of methodology with users. The COVID-19 Infection Survey: methods and further information article has enhanced transparency. It includes detailed information regarding the methods used to collect the data, process it, and calculate the statistics. This article continues to be updated, with the latest update in February 2022. Additionally, we now publish technical articles that communicate more complex analyses and explain the methodology. These articles give users insight into the depth and breadth of topics that can be explored using CIS data, beyond our regular outputs, and are aimed at users who wish to understand the technical detail.

As per your recommendation to publish information regarding the quality management approach of CIS, the COVID-19 Infection Survey: methods and further information article includes information regarding quality assurance and how improvements are made. In line with your recommendation to strengthen the documentation of the data processing pipeline to aid user understanding, a process map has been added to the article to provide an overview of the flow of data through the pipeline.

In your review, you advised that we continue to be transparent regarding the methods used for data collection and the representativeness of the survey. In addition to the aforementioned methods and further information article, a Coronavirus (COVID-19) Infection Survey QMI was published to provide users more insight into the CIS sample, recruitment, response rates, and weighting approaches along with information regarding the survey quality, methodology, strengths, and limitations.

Your review also highlighted the need for users to understand how CIS outputs compare to other COVID-19 data sources. As previously mentioned, the COVID-19 Latest Insights Tool now contains a comparisons page to allows users to explore trends across data sources. This has involved collaboration with the UK Health Security Agency (UKHSA) and other stakeholders. For example, NHS Test and Trace data from the GOV.UK coronavirus dashboard is compared with CIS infection estimates. The tool offers explanations as to why differences may exist across the data sources.

Next Steps

To further enhance user understanding of how CIS data compares to other COVID-19 sources, Latest Insights is currently under review. We have appointed a new team leader who will focus on the strategic direction of Latest Insights and how it should be adapted as the pandemic landscape continues to shift. Several objectives have been outlined, including to provide users with a more customisable, interactive experience, to incorporate more external data sources, and to investigate how the tool could enhance user understanding of each data source in the wider context to provide a more coherent story. Regular monitoring of web metrics and user research will help us understand changing user need and we will retain the flexibility to adapt accordingly.

In your review, you highlighted the need for us to publish the governance structure of CIS. The CIS programme governance is currently being revised and we will publish this information once it has been finalised.

With regards to improving engagement with users, CIS is aware of the need for further public engagement. The CIS teach-ins on complex methods are being expanded over the coming months, to provide users with more opportunities to understand, discuss, and challenge CIS methodology. We are also exploring how we can reach other types of users with this information. We are considering launching an external-facing outputs consultation to ensure the analysis we produce is relevant for the inquiring citizen.

It was highlighted in your review that there was no information in the public domain regarding the long-term plans for CIS. Plans for the next phase of the survey have been published in a blog post by Esther Sutherland, Head of Strategic Development for CIS. This blog can be accessed through the ONS National Statistical website and outlines our priorities and ambitions for the survey. We have also commenced discussions with UKHSA about the future of the CIS and will share these publicly as appropriate.

Thank you again for your insightful review, we continue to welcome feedback on the COVID-19 Infection Survey. If you have any questions, please don’t hesitate to get in touch.

Yours sincerely,

Ruth Studley
Deputy Director for Infection Survey Analysis, COVID-19 Infection Survey, Office for National Statistics

Esther Sutherland
Head of Strategic Development, COVID-19 Infection Survey, Office for National Statistics