Trustworthiness

The Code of Practice for Statistics sets out that Trustworthiness is about confidence in the people and organisations that produce statistics and data. It is a product of the people, systems and processes within organisations that enable and support the production of statistics and data. Trustworthiness comes from the organisation that produces statistics and data being well led, well managed and open, and the people who work there being impartial and skilled in what they do.

Key Findings

• Intelligent transparency
• Transparency of development plans
• Challenge of inappropriate use of statistics
• Resources

Over the last year we in OSR have continued to champion transparency and equal access to data and statistics. We have published a blog, statement on transparency and the role of Heads of Profession and regulatory guidance. The former chair of the UK Statistics Authority attended the Civil Service board to talk about the need to ensure equality of access to data. We currently have a page on our website dedicated to transparency.

In order for statistics to serve the public good they need to be available in the first place, but this in itself isn’t enough and this is why we talk about intelligent transparency. At its heart intelligent transparency is about supporting a transparent and accessible approach to communicating not just statistics, but also other types of data and research too like management information or evaluations. There are three core principles which combine to support intelligent transparency: equality of access, enhancing understanding and analytical leadership. Intelligent transparency is at the core of many of the practices outlined in the Code of Practice for Statistics.

In last year’s report we stated that producers should make transparency the default across all statistics and data. Over the last year we have seen good examples of improvements to the transparency and equality of access to statistics. Statistics producers have set up specific pages for the publication of statistics and data used in the public domain, such as the Scottish Government Coronavirus additional data in information page. Blogs have been published to aid understanding of the statistics that are of high public interest, such as the Welsh Government Chief Statistician’s blog. Departments have made an effort to make granular data available to help understanding of topical questions such as different Russian entities, individuals and ships subject to sanctions published by HM Treasury.

However, we have continued to have to intervene on occasions where unpublished statistics are used in the public domain or not sufficiently well explained.

A challenge we have observed is balancing the need to make data available with the need to be clear about limitations which impact the interpretation and appropriate use of statistics. In late 2021, analyses published by the UK Health Security Agency (UKHSA) of COVID-19 case rates by vaccination status were misinterpreted or misused – for example, the figures were used to support anti-vaccine misinformation in podcasts and online. In November 2021, we welcomed the changes which UKHSA made to the presentation of its analysis, but noted that there was further room for improvement to minimise the potential for misuse. UKHSA continued to develop its publication to support appropriate use and published a blog explaining how analytical choices and underlying biases impacted interpretation of the statistics. This example highlighted the need for producers to make it easy for people to find and understand objective information and help to minimise the spread of misinformation. This can be achieved if producers consider any risks associated with the publication of statistics, explain any caveats prominently and are agile in responding to public debates. We continue to work with producers, including UKHSA, to understand the lessons learned from examples such as this during the pandemic.

A further challenge that we have heard is keeping up with the range of ways data are being published. Data are being published in a range of places on government websites, with a range of labels including management information, transparency data as well as statistics. This can make the data difficult for users to find. There are also varying levels of information accompanying the data to ensure users can draw robust conclusions from them.

We will continue to champion and encourage improvements to making data and statistic easier to find, access and understand for users.

What we would like to see

• Intelligent transparency being the default for all statistics and data
• Statisticians feeling confident and supported to ensure statistics are equally accessible to all
• Any limitations of the statistics and data are clearly and prominently explained to users to aid understanding and prevent misinterpretation.
• Data are published where a user can easily find them

The transparency of development plans for statistics continues to be an issue. Some statistics producers are not making their future plans for their statistics publicly available. Of those plans that are published, some are out of date or lack detail about the developments, making it difficult for users to know what future value to expect from the statistics of interest. These plans should also be transparent about what is not currently possible.

ONS Population statistics are undergoing transformation with a move to greater use of administrative data and development of higher frequency estimates, for example more timely model-based estimates, which will draw on various survey, administrative, and possibly commercial data. In January 2021 ONS published a blog setting out the range of population statistics that it planned to publish in 2022 and it continued to provide updates through its population statistics and sources guide. These types of outputs enable users to better understand upcoming changes, and what new statistics to expect and when.

What we would like to see

• Development plans published as standard to enable all users to understand upcoming changes to statistics and help drive priorities for development.

Over the last year, statisticians have been willing to step in and respond to ensure the statistics can be used appropriately. For example, in January 2022, when deaths data were used to claim that the number of deaths from covid was a lot lower than official statistics showed, ONS statisticians rapidly put out a blog explaining the deaths statistics and how correct conclusions can be drawn from them. In last year’s report we argued that statisticians should have greater freedom to engage openly about data and statistics and their limitations, both within and outside government. There continues to be to be an increased presence of statisticians on social media in a professional capacity, helping make them more accessible and their responses quicker. ONS have also launched a podcast to enable statisticians to discuss further the statistics that they produce. This comes against a back drop of increased discussion and challenge of statistics, particularly on social media. It is important that the voices of government statisticians are heard in these debates.

This year has seen the successful roll out of the Data Masterclass for Senior Leaders created by the Data Science Team at 10 Downing Street. It is designed to improve data and analytical literacy in senior public-sector leaders and help them create and champion a data culture in their organisation. It has seen over 3,000 learners enrol on the course. These sorts of initiatives may not only help increase the awareness of the value of data in organisations; they may also help them understand the factors that support appropriate use of statistics by public sector organisations.

What we would like to see

• Producers being proactive in considering how their statistics might be misquoted or misused and including relevant information alongside the statistics to mitigate this risk.
• Departments empowering statisticians to communicate with users, challenge the inappropriate use of statistics and support understanding of statistics.

Continued pressure on the system, coupled with issues with recruitment and retention, are leading to strain on analysts and statisticians and may inhibit innovation and collaboration in the future.

This report highlights the enormous breadth of work being delivered by the statistics system. In speaking to statistics producers, an issue that was flagged multiple times with us is that of resources. The demands on the statistics system have increased, but at the same time there have been difficulties recruiting and retaining staff. The pandemic provided opportunities for statisticians to expand and demonstrate their skills in delivering new analysis at pace. Many of these statisticians got promotions or moved into other roles or sectors. A competitive labour market has made it more difficult to recruit into the Government Statistical Service and organisations are reporting high vacancy rates. Recent announcements from the Government about reducing the number of civil servants are likely to increase the pressure on resources within the statistical system through potentially not being able to replace statisticians when they leave.

We are hearing of Heads of Profession having to make decisions about which statistical releases to continue to publish, or publishing data-only releases to meet demand. These decisions will need to be taken in collaboration with users to identify where changes in user need can facilitate the cessation, or reducing the frequency or detail of outputs. In addition, the use of technology, such as that set out in our Reproducible Analytical Pipeline review, may support the more efficient delivery of some sets of the statistics. In last year’s report we set out the need for producers to consider how best to use better infrastructure, processes and systems to improve the efficiency and sustainability of its processes and that need continues this year.

What we would like to see

• Departments ensuring that Heads of Profession have the skilled and resourced teams that they need to deliver statistics in line with the Code of Practice for Statistics.
• Conversations with a wide range of users to identify where statistics can be ceased or reduced in frequency or detail to save resource if appropriate.
• Use of better infrastructure, processes and systems to ensure the efficient and sustainable delivery of statistics.