Sofi – Office for Statistics Regulation

Our Head of Research discusses how the findings from the 2023 Public Confidence in Official Statistics Survey are relevant to OSR.

At the Office for Statistics Regulation (OSR), we have a vision that statistics serve the public good. This vision guides our regulatory activities, which means we must have a deep understanding of how statistics can serve the public good, including what actions bring us closer to or further away from our goal.

This understanding is supported by evidence such as UK-wide surveys which gather public views about official statistics. For England, Scotland and Wales, this information is collected by the Public Confidence in Official Statistics (PCOS) survey, which is conducted independently on behalf of the UK Statistics Authority. For Northern Ireland, the Northern Ireland Statistics and Research Agency (NISRA) use the Northern Ireland Continuous Household Survey to explore Public Awareness and Trust in Official Statistics (PATOS). The findings from PCOS 2023, which were published this week (14 May 2024), and PATOS 2022 provide valuable insights into attitudes towards official statistics.

In this blog post, we highlight some of the findings from PCOS 2023 and PATOS 2022, and discuss how they align to our activities in OSR. This demonstrates how we see our work serving the public good.

While we are including data from both PCOS and PATOS in this blog post, the two are not directly comparable. For example, the surveys cover different years, and PCOS removes people from their analysis who said they ‘don’t know’ or didn’t provide an answer to questions, while PATOS includes ‘don’t know’ responses. Because of the difference in approach to which responses are included or not, when we say ‘respondents’ in this blog post for PCOS data we mean people who were able to express a response other than ‘don’t know’, whereas when referring to PATOS data ‘respondents’ includes people who indicated that they ‘don’t know’.

Building confidence through trustworthiness

As the Head of Research at a statistics regulator, you may need to forgive me for being specific and precise about technical statements in this blog post. However, I want to emphasise that these surveys report what respondents say, and in the case of PCOS, people who responded that they ‘don’t know’ or who didn’t respond are not included in reported percentages of respondents. The PCOS survey largely asks questions about the UK Statistics Authority’s production branch, the Office for National Statistics (ONS). As a NISRA-produced survey, PATOS focusses on NISRA statistics.

In both PCOS 2023 and PATOS 2022, the majority of survey respondents reported that they trust these statistics producers, with 87% reporting that they tended to trust ONS or they trusted it a great deal, and 85% reporting the same for NISRA.

At OSR, we assert that trust in official statistics is important if they are to serve the public good – people will not use evidence that they feel is unreliable . However, we also recognise that trust cannot be forced; it must be earnt. Our Code of Practice for Statistics (the Code) includes a pillar called ‘Trustworthiness’, which embodies the idea that producers of statistics must consistently demonstrate that they deserve trust. This pillar encourages honesty, independence, reliability and competence. By holding statistics producers to the standard for trustworthiness set out by the Code, OSR supports trust within our statistical system, which we assert helps statistics to serve the public good.

Addressing concerns with intelligent transparency

Equally important to knowing people’s reasons for trusting official statistics is understanding their reasons for not trusting them. In PCOS 2023, respondents who indicated that they did not trust ONS statistics were asked to select a reason for this. Respondents most commonly attributed their lack of trust to statistics being misrepresented by politicians (49%), although many also expressed that they believe the statistics alone do not tell the whole story (45%) and that the government has a vested interest in or manipulates the results (41%).

To address these reasons for not trusting statistics, we assert that there is a need for producer bodies to follow OSR’s principles of Intelligent Transparency:

Equality of access: Data used by the government in the public domain should be available to all in an accessible and timely way
Enhancing understanding: Citations for sources and appropriate explanations of context should be provided alongside the information
Independent decision making and leadership: Decisions about the publication of statistics and data (such as the content and timing) should be free from political influence

These three principles tie directly to the three most commonly reported reasons for not trusting ONS statistics:

Equality of access allows people to see the statistics themselves without having to rely on how they are represented by public figures
Enhancing understanding can help users to understand exactly what parts of the story statistics cover (and what parts lie beyond their scope)
Independent decision making and leadership may help alleviate concerns around government manipulation of official statistics

Advocating for, and supporting the implementation of, Intelligent Transparency is therefore another route OSR takes to building trust in official statistics. Intelligent Transparency sits alongside our casework function, where we investigate potential issues with official statistics such as how they are used in public debate. Here, we use our voice to stand up for statistics, reporting publicly where we have concerns and highlighting good practice. We anticipate that our work protecting statistics in public debate will allow more people to be confident in the use of official statistics, and therefore allow these statistics to better serve the public good.

Sofi Nickson, Head of Research at OSR, shares why OSR is interested in the role statistics play in decision making by members of the public along with what we know about this so far, and invites others to share evidence they have on the topic.

When I first heard about the Office for Statistics Regulation (OSR), I assumed it simply checked whether statistics are accurate or not. It wasn’t until last year, when I happened upon a job advert for OSR that I looked into what it really does. It turned out that I was somewhat off the mark in my assumption – OSR’s vision is not as it happens limited to ‘accurate statistics’, but it is the far more inspiring ‘statistics that serve the public good’. For the past few years, colleagues across OSR have deepened their understanding of what this may look like through their regulatory work and supporting research programme, which I am now lucky enough to lead. Part of the research programme is understanding the role official statistics can play in decision making by members of the public and, in this blog post, I explain why we are interested in this and what we know so far, then I invite you to share your thoughts on the topic.

Statistics serving the public good

The Statistics and Registration Service Act states that serving the public good includes assisting in development and evaluation of public policy, and informing the public about social and economic matters. ‘The public’ here could be anyone outside of government, in fact a report from workshops on whether scientists understand the public states that ‘there’s a thousand publics out there that one could address, any of whom has to be understood… in order to know how to deal with them, how to work with them, engage them, try to benefit them’. We have begun understanding how some publics play a role in statistics serving the public good, such as non-governmental organisations that may use statistics to provide services, businesses who can use them to adapt their practices and better meet needs, and we have evidence from analysing applications to access public data suggesting that academics see themselves holding a role in providing an evidence base for decision making. Even the media plays a part, with ESCoE research finding that journalists help translate statistics for public consumption.

The point here is that for statistics to serve the public good, they must be a tool both for government and for those beyond. When we look at statistics use outside of government, we have heard from a wide range of civil society organisations through our regulatory activities about their uses of statistics on behalf of the public. This is a way for statistics to serve the wider public indirectly, without individuals using statistics themselves. However, we are currently missing an important piece of the puzzle – what use looks like for individual citizens outside of such organisations. This family or individual-level use of statistics is far less visible, but may be no less valuable in serving the public good. In OSR we want to shine light upon these hidden uses of statistics by exploring how individual citizens use statistics in their professional and personal lives, and what they value in statistics.

The more we can understand, the better we can ensure our regulatory decisions and recommendations support members of the public statistics users. This topic is vast, so to narrow it down we are starting with how individual citizens use official statistics to make decisions, and we are focussing on three areas:

Whether members of the public find value in using statistics to make decisions (including whether they do use statistics at all)
Whether members of the public feel equipped to use statistics to make decisions in the way they are currently presented
How statistics inform decisions (including how much influence they have alongside other factors).

Do members of the public find value in using statistics to make decisions (and do they use statistics at all?)

A quick search about using statistics to make decisions gives lots of potential examples. For example, using statistics on school performance to inform where you want to educate your child, or using statistics on crime to decide where to live. But we currently lack evidence about whether and how these potential uses play out in real life – do people actually use statistics like this, or do we just think they could. Even more specifically, do people use official statistics, or are their information needs being met by other sources.

In the Public Confidence in Official Statistics 2021 survey, just over half of respondents (52%) agreed to some degree that statistics helped them to make a decision about their lives. However, we don’t know from this what sort of statistics or decisions respondents were thinking about. We also don’t know what might support the other 48% of respondents get value from using statistics as well, or even whether this 48% want to use statistics at all. It may be that individuals are already satisfied with organisations in civil society using statistics on their behalf.

Do members of the public feel equipped to use statistics to make decisions in the way they are currently presented?

From our commissioned research into statistical literacy we saw great variability amongst the general public in skills linked to statistical literacy, and have concluded that responding to this is all about the communication. We have evidence on how to communicate statistics to non-specialists, for example recommendations from a programme of work by ESCoE which explores communicating economic statistics to the public. Despite strong recommendations, there is more that could be done to improve the communication of statistics to non-expert users, which is why in our business plan for 2023/24 we commit to championing the effective communication of statistics to support society’s key information needs. We don’t profess to know everything in this area though and are always interested in learning more.

How do statistics inform decisions (and how much influence do they have among other factors)?

We have uncovered an abundance of literature about human decision making, including how heuristics and biases sit alongside ‘rational’ evidence-based choices. From this we recognise that it is unlikely anyone bases their decisions on statistics alone, but we still don’t know how influential official statistics are and where they sit alongside other evidence. Are they seen as compelling and trustworthy? What factors influence this?

Can you help?

If you have read this far it will be clear that we have a lot of questions about how statistics can serve the public good. In OSR, asking questions is in our nature – as a regulator our judgements and decisions are informed by the evidence we have, so we are always seeking to learn more. If you know of any research, examples, or information that you think could inform our understanding of the role official statistics play in how members of the public make decisions then we would love to hear from you – please get in touch with us at research.function@statistics.gov.uk

Earned, not given: Public confidence in statistics and how this informs OSR’s work