Ground truth

Do statistics provide a sort of ground truth, anchoring our perceptions of the world?

Last year I was invited to participate in a conference on digital innovation in mental health, run by the excellent Dr Becky Inkster. The theme was “ground truth”, that is, the perceptions and experiences of each individual and how they feel about their lives. In the end, I wasn’t able to participate. But the idea of ground truth has sat with me, and I’ve wanted to explore further whether there is any mileage in thinking about statistics as one source of ground truth.

What I mean by this is that statistics provide a lens for us to understand our world, our environment: how good schools are, how expensive houses are, how dangerous society is, how many cases of a pandemic there are in our neighbourhood.

We usually tend to think about these things as products of conscious, active user engagement – in other words, we envisage that someone wants to know about the quality of a school so they obtain the statistical information that can inform them. This type of use is active, visible and incredibly important. Indeed a common theme of OSR reports and reviews is the need for Government bodies to think more about and engage with these types of uses and users.

But I’ve started to wonder whether there is a less visible type of use. Everyone builds a narrative of how the world works based on their personal experiences and information they encounter every day, some of which will come from intermediaries like journalists and social media. This information contributes to building a picture in our minds about ‘how things are’, like assembling a jigsaw puzzle. Hearing a reference to some statistics on the news may lodge something in a person’s mind – not because they’ve actively studied the issue, but because they’ve picked it up in passing.

This type of use of statistics would be very difficult to research. What role might be played by “motivated reasoning” – in other words, how it works if someone hears or reads something that they agree with, as opposed to what they don’t agree with? And how could researchers access these confirmation biases around information that is casually as opposed to purposively picked up?

We have published a literature review on the public good of statistics. We wanted to see what research had taken place on this concept of the public good of statistics – a common phrase that features in the Statistics and Registration Service Act that set up the UK Statistics Authority. My colleague Mary Cowan has summarised the findings in her review, but in short, we found some good empirical work on what people say about their confidence in and use of statistics; some attempts at quantification of value, particularly for economic statistics; and some broader approaches based on a social capital notion of public good.

But all of these approaches tend to rely on a conscious use of statistics – a user who is aware of statistics and actively uses them in choice-making. We found much less on the contextual use of statistics, where a broad awareness of a topic (how is the economy doing) is informed by a range of observations, including statistics picked up in passing through intermediaries, like broadcast news.

It’s possible that there is nothing in this – that the idea of a ground truth role for statistics is misplaced. Or it may be the case that it’s difficult or perhaps impossible to research effectively, or least not possible without extensive participation observation studies.

But we are interested in it. If you know of any research that opens up this area of passive use of statistics, and how it influences perception, let us know!

How can official statistics better serve the public good?

How good are government statistics? In a recent seminar we asked members of the Government Statistical Service for three words they would use to describe Government Statistics. Among the top words we got back were ‘trustworthy’, ‘quality’ and ‘informative’. It was striking how closely these aligned to the three pillars of our Code of Practice for Statistics – Trustworthiness, Quality and Value – and encouraging to us, as the regulator of official statistics, to hear our message echoing with others 

Official statistics play a central role in answering society’s most important questions. The most salient questions currently facing society concern the COVID-19 pandemic, its impacts and societal responses to it. Data and analysis have been crucial in informing government and individual’s decisions and supporting public understanding.  

But the uses of official statistics extend far beyond the pandemic into peoples’ everyday lives: whether you are making decisions as a head teacher, or choosing your child’s school, developing policy on social housing, or trying to decide whether and where you should buy a house, have an interest in your local library remaining open, or are considering the country’s major economic decisions, you may well be using official statistics. This is why it’s so important that the UK’s statistical system responds to society’s information needs with insightful statistics.  

In a world of increasingly abundant data, expectations are higher. Individuals have become accustomed to information on many aspects of society in near real time with increasingly detailed breakdowns. Official statistics need to respond to these demands for information. Our work as a regulator of official statistics puts us in a unique position to reflect on the UK government statistical system and, in July, we set out our view on the current state of government statistics 

At their best, statistics and data produced by government are insightful, coherent, and timely. They are of high policy-relevance and public interest. During the COVID-19 pandemic, we’re seeing the kind of statistical system that we’ve always wanted to encourage – responsive, agile and focusing on users. However, the statistical system does not consistently perform at this level across all its work. In our report we address eight key areas where improvements could be made across the system. 

  1. Statistical leadership 
  2. Voluntary Application of the Code, beyond official statistics 
  3. Quality assurance of administrative data 
  4. Communicating uncertainty 
  5. Adopting new tools, methods and data sources 
  6. Telling fuller stories with data 
  7. Providing authoritative insight 
  8. User engagement 

In each area, as well as talking about what we would like to see, we highlight examples of statistical producers already doing things well, which others can learn from and build on.  

Our 5-year Strategic Business Plan sets out our vision and priorities for 2020-2025, and how we, as OSR, will contribute to fostering the Authority’s ambitions for the UK statistics system. In all our work, we will continue to champion the work producers do, celebrate the things they do well, and encourage them to continue to improve the statistics they produce so that, together, we can ensure that official statistics better serve the public good.  

Please get it touch if you’d like to discuss the report further.