You may have noticed that the last 18 months or so have been rather unusual. In fact it’s getting difficult to think about what things were like before masks, distancing and the universal smell of alcohol gel. And there’s another change to which we have become accustomed – the daily parade of statistics, the use of graphs on the news, and the huge presence of scientific and statistical discussion, both in the media and among ordinary people who are not even statisticians!
The scale and ambition of the health data being made available would have been unthinkable just two years ago, as would be the complexity and sophistication of the analyses being conducted. But the Office for Statistics Regulation’s ‘Lessons Learned’ report argues that we should not be complacent: we need to press harder for more trustworthy, better quality, and higher value statistics.
There are a few recommendations that stand out for me. First, Lesson 9 focusses on improved communication. Back in May 2020 I stuck my neck out on the Andrew Marr show and criticised the press briefings as being a form of ‘number theatre’, with lots of big and apparently impressive numbers being thrown around without regard for either accuracy or context. This attracted attention (and 1.7m views on Twitter). But although some dodgy graphs continued to appear, the presentation of statistics improved. Crucial to communication, however, is Lesson 1 on transparency – it is essential that the statistics underlying policy decisions, which affect us all, are available for scrutiny and are not cherrypicked to avoid those that might rock some political boat. This requires both constant vigilance, and appropriate clout for professional analysts.
Lesson 7 deals with collaboration, reflecting the extraordinary progress that has been made both in collaboration across governments and with academic partners, all of whom have shown themselves (against archetype) to be capable of agile and bold innovations. The Covid Infection Survey, in particular, has demonstrated both the need and the power of sophisticated statistical modelling applied to survey data. Although of course I would say that, wouldn’t I, as I happen to be chair of their advisory board, which has enabled me to see first-hand what a proper engagement between the ONS and universities can achieve.
Finally, Lesson 3 addresses the idea that data about policy interventions should not just enable us to know what is happening – essentially ‘process’ measures of activity – but help us to evaluate the impact of that policy. This is challenging; Test and Trace has come in for particular criticism in this regard. For statisticians, it is natural to think that data can help us assess the effect of actions, with the randomised clinical trial as a ‘gold-standard’, but with an increasing range of other techniques available for non-experimental data. Again there is a need to get this up the agenda by empowering professionals.
An over-arching theme is the need for the whole statistical system to be truly independent of political influence from any direction. While this is enshrined in legislation, a continued effort will need to be made to make sure that work with data lives up to the standards expressed in the Code of Practice for Statistics, in terms of trustworthiness, quality and value. The pandemic has shown how much can be achieved with the right will and appropriate resources, and OSR’s ‘Lessons Learned’ point the way forward.
David Spiegelhalter is a Non-Executive Director of the UK Statistics Authority, which oversees the work of the Office for Statistical Regulation.