“Statistics may be used to provoke and to challenge the status quo; but Ofcom is entitled to insist – in the public interest – that they should not be misused so as to mislead”. Not our words but the words of a recent High Court Judgement following an intervention by Ofcom to challenge the presentation on GB News of vaccine statistics produced by the UK Health Security Agency (UKHSA).
When concerns are raised to OSR about the communication or presentation of statistics, we are often asked to comment on whether we judge something to be misleading. Misleadingness, how to define it and what this means in our context as a statistics regulator, is something that we routinely come back to.
Being able to intervene and publicly challenge the misuse of statistics is a crucial part of meeting our statutory objective of ‘promoting and safeguarding the production and publication of official statistics that serve the public good’ under the Statistics and Registration Service Act 2007. When statistics are misused, it damages public confidence in data and those communicating the messages.
As this High Court Judgement shows, we are not alone in tackling misuse of data and misleading communication. The High Court supported Ofcom’s right to intervene and emphasised that the misleading presentation of statistics cannot be dismissed as just a matter of opinion.
“The purpose of both the caveat on effectiveness and the contextual statement was to sound a warning against the simple and undifferentiating comparison of groups. Yet, an undifferentiating comparison was undertaken on the Show”. – High Court Judgment
This sets a valuable formal legal precedent and echoes many of OSR’s messages around the importance of pre-empting statistics being used in a way that has the potential to mislead. We often set recommendations to producers in our regulatory work that state the importance of highlighting information on quality, caveats and context alongside the statistics to support appropriate use. But it’s just as important to do these things to prevent misuse of the data – which UKHSA had done in this case.
“We present data on COVID-19 cases, hospitalisations and deaths by vaccination status. This raw data should not be used to estimate vaccine effectiveness as the data does not take into account inherent biases present such as differences in risk, behaviour and testing in the vaccinated and unvaccinated populations.” COVID-19 vaccine surveillance report, UKHSA
The Ofcom case demonstrates why our advice and the Code of Practice for Statistics should not be thought of as a tick box exercise. Preventing misuse of data is an essential part of protecting the value and impartiality of statistics which in turn serve the public good. When communicating quality, and uncertainty in particular, it can be tempting for statistics producers to fall back on generic wording such as ‘users should exercise caution’ but as my recent blog highlights, these statements don’t go far enough in supporting use and interpretation of the statistics.
It is much harder to react quickly to and debunk misuse once it has happened, especially in an age of social media, so effort should be made to prevent it happening in the first place. We will continue to work on identifying best practice around how to prevent misuse and to support producers of statistics to communicate statistics through different means, so that they can be understood by audiences with varying levels of understanding.
A good rule of thumb is to consider how a reasonable person would interpret the statement being made and ensure that this is not likely to be misleading in the absence of additional information.