In our latest guest blog Helen Barugh, Head of Statistics Policy and User Engagement, discusses transforming the statistics produced by the UK Health Security Agency…
What is the Health Security Agency?
The UK Health Security Agency (UKHSA) is responsible for protecting every member of every community from the impact of infectious diseases, chemical, biological, radiological and nuclear incidents and other health threats. We are an executive agency of the Department for Health and Social Care.
We collect a wide range of surveillance data about diseases ranging from influenza and COVID-19 to E.coli to measles. We publish statistics related to planning, preventing and responding to external health threats. You can find our statistics here.
UKHSA was born in October 2021 during the COVID-19 pandemic. The pandemic had a significant impact on the organisation, including statistical production, and the repercussions of that are still being felt.
Producing and reviewing official statistics
I joined UKHSA in August 2022, one of four recruits to a new division supporting the statistics head of profession. Our division, which has a mix of statisticians at different grades, aims to transform UKHSA statistical production and dissemination. We have all produced statistics in other government departments, and we use that expertise to provide advice, guidance and practical support to all aspects of statistics production and dissemination. Our division also includes two content designers who actually publish UKHSA’s official statistics.
One of the most important parts of our work is a programme of reviews looking in-depth at each UKHSA official statistics publication. This is a big programme of work, encompassing around 35 statistical series covering a range of topics and including weekly reports right through to annual reports. The reviews aim to:
- bring consistency to our statistics production and outputs.
- improve efficiency and quality assurance through the adoption of reproducible analytical pipelines (RAP) in line with our RAP implementation plan.
- improve compliance with the Code of Practice for Statistics.
- embed user engagement as a regular and standard activity.
We are part of the way through this programme, having reviewed around 20 series and with another 15 to go. We expect to finish the reviews by late summer 2024.
How do we review our statistics?
Our reviews have three main phases.
- Desk-based research includes assessing products against the Code of Practice for Statistics, assessing publications for accessibility and clarity, reviewing desk notes and analysing google analytics to draw out insights about users.
- Discovery work with the team explores the journey from data acquisition through to publication, understanding the processes used and the quality assurance in place. Sometimes we shadow a production cycle to really understand how the process works and how it can be improved. We also discuss user engagement with the team to investigate what they know about their users and how they assess any changing or emerging needs.
- Once we have all the information we need, we write a report to summarise our findings and agree recommendations for improvement with the production team.
How do we make changes in practice?
We work with the production team, providing practical support, training and guidance as they implement the recommendations. We are aiming for incremental improvement, and the review provides a baseline against which we can measure success.
The reviews have given a terrific insight into the good practice within our organisation, as well as the challenges of producing some of our statistical products and the legacy processes that now need updating. Despite all the challenges of producing statistics during the pandemic, UKHSA statistics teams have been putting out very detailed and thorough statistics, in some cases on a weekly basis and with very short turnaround times between receiving data and publishing. Google analytics indicate that in general the readership is high, and user engagement so far has shown that products are highly valued and appreciated by users working in health protection and healthcare settings.
Areas for improvement are often similar across different statistical series. For example, production methods are not as reproducible as they could be. There are opportunities to introduce reproducible analytical pipelines (RAP) and build in automated quality assurance that will improve the efficiency and accuracy of production. We’ve also found that most outputs are aimed at a technical and clinical audience which limits their impact for the general public or for meeting the public good. As an organisation, we need to do more to understand the wider uses of our statistical products and adjust their presentation accordingly.
What difference are the reviews making?
One of the key benefits of the reviews has been building relationships between our division and UKHSA statistics teams. Our work only really has impact where teams get onboard and are enthused to make positive changes. So I’m delighted about the impact our work is having, with statistics teams working hard to improve their products. For example, charts are being redesigned to conform with best practice, RAP is being implemented in some publications, our first quality and methodology information report has been published and most publications are now in HTML. It all feels very positive!
So, what next?
We have more reviews to do to make sure we have an accurate picture of all our official statistics, and lots of opportunities to support production teams to make improvements. We’re planning more user engagement and also participating in the cross-government user consultation on health and social care statistics which will give us some valuable user feedback to help shape our statistics in the future.
We need to decide whether any publications designated on GOV.UK as ‘research and analysis’ should really be designated as official statistics. And as we bring about improvement right across the UKHSA statistics publications, we will be aiming for more products to be accredited by OSR to provide that stamp of approval that we’ve done the right things and are now meeting the highest standards of trustworthiness, quality and value.
If you’re interested in talking to us about our programme of reviews please do get in touch: UKHSA_HOPSTATS@ukhsa.gov.uk. We’re happy to share more about what we’ve learnt as well as the materials we’ve developed to support the review process.