How do people use official statistics to make decisions?

Sofi Nickson, Head of Research at OSR, shares why OSR is interested in the role statistics play in decision making by members of the public along with what we know about this so far, and invites others to share evidence they have on the topic.

When I first heard about the Office for Statistics Regulation (OSR), I assumed it simply checked whether statistics are accurate or not. It wasn’t until last year, when I happened upon a job advert for OSR that I looked into what it really does. It turned out that I was somewhat off the mark in my assumption – OSR’s vision is not as it happens limited to ‘accurate statistics’, but it is the far more inspiring ‘statistics that serve the public good’. For the past few years, colleagues across OSR have deepened their understanding of what this may look like through their regulatory work and supporting research programme, which I am now lucky enough to lead. Part of the research programme is understanding the role official statistics can play in decision making by members of the public and, in this blog post, I explain why we are interested in this and what we know so far, then I invite you to share your thoughts on the topic.

Statistics serving the public good

The Statistics and Registration Service Act states that serving the public good includes assisting in development and evaluation of public policy, and informing the public about social and economic matters. ‘The public’ here could be anyone outside of government, in fact a report from workshops on whether scientists understand the public states that ‘there’s a thousand publics out there that one could address, any of whom has to be understood… in order to know how to deal with them, how to work with them, engage them, try to benefit them’. We have begun understanding how some publics play a role in statistics serving the public good, such as non-governmental organisations that may use statistics to provide services, businesses who can use them to adapt their practices and better meet needs, and we have evidence from analysing applications to access public data suggesting that academics see themselves holding a role in providing an evidence base for decision making. Even the media plays a part, with ESCoE research finding that journalists help translate statistics for public consumption.

The point here is that for statistics to serve the public good, they must be a tool both for government and for those beyond. When we look at statistics use outside of government, we have heard from a wide range of civil society organisations through our regulatory activities about their uses of statistics on behalf of the public. This is a way for statistics to serve the wider public indirectly, without individuals using statistics themselves. However, we are currently missing an important piece of the puzzle – what use looks like for individual citizens outside of such organisations. This family or individual-level use of statistics is far less visible, but may be no less valuable in serving the public good. In OSR we want to shine light upon these hidden uses of statistics by exploring how individual citizens use statistics in their professional and personal lives, and what they value in statistics.

The more we can understand, the better we can ensure our regulatory decisions and recommendations support members of the public statistics users. This topic is vast, so to narrow it down we are starting with how individual citizens use official statistics to make decisions, and we are focussing on three areas:

1. Whether members of the public find value in using statistics to make decisions (including whether they do use statistics at all)
2. Whether members of the public feel equipped to use statistics to make decisions in the way they are currently presented
3. How statistics inform decisions (including how much influence they have alongside other factors).

Do members of the public find value in using statistics to make decisions (and do they use statistics at all?)

A quick search about using statistics to make decisions gives lots of potential examples. For example, using statistics on school performance to inform where you want to educate your child, or using statistics on crime to decide where to live. But we currently lack evidence about whether and how these potential uses play out in real life – do people actually use statistics like this, or do we just think they could. Even more specifically, do people use official statistics, or are their information needs being met by other sources.

In the Public Confidence in Official Statistics 2021 survey, just over half of respondents (52%) agreed to some degree that statistics helped them to make a decision about their lives. However, we don’t know from this what sort of statistics or decisions respondents were thinking about. We also don’t know what might support the other 48% of respondents get value from using statistics as well, or even whether this 48% want to use statistics at all. It may be that individuals are already satisfied with organisations in civil society using statistics on their behalf.

Do members of the public feel equipped to use statistics to make decisions in the way they are currently presented?

From our commissioned research into statistical literacy we saw great variability amongst the general public in skills linked to statistical literacy, and have concluded that responding to this is all about the communication. We have evidence on how to communicate statistics to non-specialists, for example recommendations from a programme of work by ESCoE which explores communicating economic statistics to the public. Despite strong recommendations, there is more that could be done to improve the communication of statistics to non-expert users, which is why in our business plan for 2023/24 we commit to championing the effective communication of statistics to support society’s key information needs. We don’t profess to know everything in this area though and are always interested in learning more.

How do statistics inform decisions (and how much influence do they have among other factors)?

We have uncovered an abundance of literature about human decision making, including how heuristics and biases sit alongside ‘rational’ evidence-based choices. From this we recognise that it is unlikely anyone bases their decisions on statistics alone, but we still don’t know how influential official statistics are and where they sit alongside other evidence. Are they seen as compelling and trustworthy? What factors influence this?