OSR publishes its review: The public good of statistics; what we know so far

Today we have published our review ‘The public good of statistics; what we know so far’. This review has been published as part of our Research Programme launched in 2019, to better understand the public good of statistics.

Mary Cowan, who is a Research Specialist working on the programme, outlined how the review will help us to better understand how statistics serve the public good by giving us better clarity on what is known, and what is not known about this subject. She said:

“This review contributes towards our overall aim, but this is just a starting point for the research programme. The review considers how the public good is defined. This phrase is sometimes used interchangeably with other similar phrases, ‘public interest’ for example, but it is not well understood what the ‘public good’ really means for the public themselves.”

As well as discussing this definition, the review covers four approaches to how the public good of statistics can be measured and understood.

Ed Humpherson, Director General for Regulation at OSR, commented on the importance of the review and said:

“This literature review is an important first step in establishing an evidence base for the public good of statistics. This will help to guide future work in our Research Programme, as we try to develop our understanding of the public good of statistics”.

At OSR, we want to make sure that statistics are relevant, useful, and clearly communicated to the public. The first two parts of our mission are achieved through our regulatory work, and the third part will be achieved through the Research Programme. You can read more in our Research Programme Overview.

 

Related Links

Are statistics serving the Public Good? Our Research Programme is going to help us find out

OSR launches its review of poverty statistics

Following on from our blog published earlier this year on the trouble with measuring poverty, we are beginning our review on income-based poverty statistics in the UK. We’ve been working in this space for a while, from regulatory reviews of benefit and deprivation statistics to casework on the use of poverty statistics in the public domain. It has become clear to us that a review of poverty statistics is needed to enhance understanding and decision making in this area.

We will consider the extent to which the statistical system is providing coherent and comprehensive information required to support decision making on poverty. Engaging with users, producers and decision makers is a huge part of the review. Find out how to get involved.

Ed Humpherson, DG Regulation said:

“We are very excited to be starting our review of poverty statistics. This work really is crucial so that we can make recommendations to improve the statistics, to inform policy, which will ultimately make a difference to people’s lives. This is something we always keep in mind in our work and in the recommendations that we make. We are really keen to hear from a range of stakeholders on this work, so if you’re interested in taking part, please contact us.”

The Office for Statistics Regulation (OSR) Publishes its Reviews of UK Benefit Statistics

OSR has today published a package of reviews concerning UK benefit statistics. This includes a full assessment of Department for Work and Pensions’ (DWP) benefit statistics for Great Britain and a compliance check of the Department for Communities’ (DfC) benefits statistics summary for Northern Ireland.

There has been heightened interest in benefit statistics during the COVID-19 pandemic and it is great to see DWP and DfC demonstrate their ability to respond and adapt to changing demands by providing real-time insights into the take up of Universal Credit.

Our findings point to a lot of untapped potential in DWP’s benefit statistics. We found a strong need for DWP to engage more proactively with users to understand their needs and ensure the statistics on legacy benefits keep pace with statistics on Universal Credit. Our report suggests a range of improvements that we consider necessary to ensure that these statistics can continue to be badged as National Statistics.

We are pleased to see the close collaboration DWP and HMRC have taken to develop a joint annual publication on the numbers of individuals claiming common combinations of benefits that DWP and HMRC are responsible for. As part of our review, we have written to HMRC to welcome this collaboration and encourage it to ensure that this development can be completed quickly to enhance the public value of benefit statistics.

Our compliance check of the benefits statistics summary for Northern Ireland confirmed that these statistics should continue to be designated as National Statistics. We found several positives in the way that DfC produces and presents these statistics and we welcome DfC’s enthusiasm in exploring the use of Reproducible Analytical Pipelines in the production of its statistics. Our letter to DfC also sets out several ways in which the statistics could be further developed to enhance the public value of the statistics.

Are statistics serving the Public Good? Our Research Programme is going to help us find out

Since the launch of our research programme, we have been finding out exactly what problems currently exist around understanding the public good of statistics, such as:

  • difficulties defining and measuring the public good
  • knowing little about how the public actually use statistics in their daily lives
  • not knowing the best way to communicate statistics to the public

Mary Cowan, who is a Research Specialist working on the Research Programme, said:

“It was important for us to identify the problems and gaps that exist so that we could plan the next stages of our work. Now that we have this knowledge, we can act to overcome these problems. We have developed research questions that will help us define projects which will get us closer to achieving our vision of statistics serving the public good.

We know that we won’t achieve this alone, and we know we are not alone in wanting statistics to serve the public good, so that’s why we’ll be working with stakeholders who share our enthusiasm for this topic. The knowledge we gain through this programme will also help statistics producers to identify changes which are needed to ensure their work serves the public good. This could lead to official statistics being seen as more trustworthy, more valuable, and a public asset, which could also increase public support for their data being used in statistics production.”

At OSR, we want to make sure that statistics are relevant, useful, and clearly communicated to the public. The first two parts of our mission are achieved through our regulatory work, and the third part will be achieved through this Research Programme. You can read more about this programme in our Research Programme Overview.

First Award Winners Announced for the Voluntary Application of the Code Award 2020

Today, we are proud to announce the winners of the first ever Voluntary Application of the Code Award. This new and prestigious Office for Statistics Regulation (OSR) award is given in partnership with the Royal Statistical Society (RSS) and our media partner Civil Service World.

This rewards excellence in the voluntary application of the ‘pillars’ of Trustworthiness, Quality and Value (TQV) that stand at the heart of the Code of Practice for Statistics. ‘Voluntary application’ was introduced in 2018 by OSR alongside the publication of its refreshed Code of Practice for Statistics. This is where an organisation commits to TQV in how it produces, and releases information not classified as official statistics. Voluntary Application is available for organisations outside Government, and also for Government organisations publishing other types of information like research and open data.

The volume, standard and variety of entries for the award was amazing, making the judging extremely difficult. However, after much deliberation we selected two organisations as the first joint winners and a highly commended runner-up.

These winning organisations are:

  • The Ministry of Housing, Communities & Local Government (MHCLG): For their Building Safety Programme Monthly Data Release that started in December 2017, following the Grenfell Tower tragedy. This provides the public, stakeholders and Parliament with the latest data on high-rise residential buildings with Aluminium Composite Material (ACM) cladding – reassuring the public on progress and holding HMG and building owners to account in make buildings safe (full case study here).
  • The Scottish Fiscal Commission (SFC): Since its creation three years ago, SFC have adhered to the Organisation for Economic Co-operation and Development (OECD) Principles for International Financial Institutions, so were already following the majority of the Code principles. But in March 2018, they published a Statement setting out ongoing commitment to voluntarily apply the Code. This has helped them develop a reputation – not just in Scotland but also at UK level and internationally – for providing independent and credible forecasts that enhance public confidence (full case study here).

Highly commended is:

  • Universities and Colleges Admissions Service (UCAS): For voluntarily application since September 2019, to innovate and improve the accessibility of all outputs. While they have always provided timely and transparent reporting on the HE admissions cycle in the United Kingdom, the pillars of the Code provide a structured and verifiable means of demonstrating commitment to improving UCAS’s data. Over 2018 and 2019, they nominated a Head of Profession for Statistics and a working group to consult on the key changes necessary in order to voluntarily adopt the Code. The work they have completed strengthens their reputation amongst key stakeholders and the public as producers of high quality, useful analytical outputs (full case study here).

Baroness Onora O’Neill of Bengarve, who’s ‘Trustworthiness’ work inspired this award said:

“This new award has proved an exciting opportunity for those pioneering organisations that have voluntarily applied the Code to their statistics to gain additional recognition for their public commitment to high quality and trustworthiness. It has never been more important to recognise the fantastic work that has been undertaken by a wide range of organisations, in applying the principles of the Code and enhance the public value for their users.”

Ed Humpherson, Director General for Regulation and lead judge commented:

“We have had a tremendous quality and variety of entries for this new award, and without doubt have identified two credible and worthy winners. This shows how flexible the Code of Practice is to support public confidence in statistics and data in the public domain.”

Stian Westlake, Chief Executive of the Royal Statistical Society, said:

“The RSS is delighted to have partnered with the Office for Statistics Regulation on this new award. It is essential that statistics are credible to ensure public trust, so this award plays an important role in highlighting and commending those organisations that are voluntarily holding their statistics to a high benchmark. We look forward to a wealth of great entries again next year, demonstrating the merits of the Voluntary Application process.”

The Voluntary Application Award will be presented again in 2021, and information on how to apply will be available early in the New Year. Keep up to date with all of our news on Twitter @StatsRegulation or subscribe to our newsletter.

COVID-19 surveillance and registered deaths data review

Information available on COVID-19 cases and deaths has been developed rapidly in a constantly shifting environment. The work being done by analysts to get this information into the public domain is commendable. There will always be a desire for improvements to the timeliness and completeness of data, but this should not undermine the huge efforts being made by individuals and organisations to deliver timely data to support decision making and inform the public.

Our vision is statistics that serve the public good. We aim to support producers of statistics and data to achieve this while championing the needs of the public. We have undertaken a short review of the data releases on COVID-19 cases and deaths – at a UK level and for each country within the UK – to help understanding of the available sources and to highlight strengths and our view on areas for improvement. This document outlines the findings from our review, that is necessarily only a snapshot of what are very fast-moving developments.

In reviewing the various statistical outputs, we have been guided by the three pillars of the Code of Practice for Statistics: Trustworthiness, Quality and Value. Trustworthiness refers the governance that surrounds the production of statistics; Quality refers to the characteristics of the data; and Value considers the extent to which the statistics answer users’ questions.

Summary of findings

There have been many developments to the data and supporting information available on COVID-19. Analysts have made huge efforts to deliver the information and have shown a willingness to address concerns and make rapid improvements.

There is great value in having timely data, such as the daily surveillance data covering the UK that is published less than 24 hours after the data reporting period. It provides an important leading indicator of the trend in COVID-19 testing, cases and deaths, which is essential to inform operational decisions being made at pace. However, the speed at which these data are made available means there has been a trade off with completeness, and the limitations of the UK data have not been fully explained.

The nature and extent of the uncertainty around the UK estimates of deaths associated with COVID-19 has not so far been made clear. However, we are aware of efforts being made to improve the clarity and transparency of the material that accompanies the daily briefing, including drawing on support from the Government Statistical Service (GSS).

In contrast, the weekly death statistics published for England and Wales, Scotland and Northern Ireland provide a more complete measure of the number of deaths associated with COVID-19, but these statistics are released with a greater time lag.

ONS’s publication of its forward workplans in this area is a helpful development for stakeholders and it is important that other nations provide detail about their plans to keep users of the statistics informed. We understand that the GSS is considering the accessibility of all the information on COVID-19 to allow users to navigate all outputs from a central hub, such as the GSS health and care statistics landscape.

Areas for further development

  1. It is important to maintain public confidence and trustworthiness of statistics that are used to inform public debate. The nature and extent of the uncertainty around the UK estimates of deaths associated with COVID-19 should be clarified.
  2. All statistics producers should show they are actively considering the diverse and changing user need for COVID-19 statistics, by publishing detailed plans for improvements, for example, information about the occupancy of intensive care units or beds, or on person characteristics, such as ethnicity.
  3. The GSS should consider the accessibility of the information and allow users to navigate all COVID-19 related outputs from a central hub, such as the GSS landscape.

Statement from the Office for Statistics Regulation – COVID-19 Update

We welcome the work of analysts across a range of organisations in providing the public with information related to the COVID-19 pandemic. The pace at which these organisations have set up new data collection and dissemination processes has been unprecedented and enabled timely updates on the number of COVID-19 cases and deaths. We warmly congratulate all those who are contributing to this effort.

Estimates for the number of cases and deaths for the whole UK are being published and each of the four nations within the UK should continue to collaborate to enable UK reporting as statistics are developed further.

In our discussions with producers of statistics, we have seen a commitment to continuously improve the information provided to the public. Following these discussions, there have been improvements to provide greater clarity, including:

  • clarification of what the daily figures published by Department of Health and Social Care (DHSC) do and do not include (those who have died in hospitals and who have tested positive for COVID-19);
  • improvements to the supporting information on the Public Health England (PHE) dashboard, including clearer explanation of the sources and coverage of these figures;
  • explanation in the Office for National Statistics (ONS) weekly deaths statistics of how COVID-19 related deaths impact on the figures;
  • clarification of how ONS and DHSC figures relate to each other through the joint statement published by DHSC and ONS;
  • cessation of publication of a “patients recovered” figure by Public Health England because of data quality limitations.

Clear explanations of what the data mean for Northern Ireland and the commitment to regular publication times in Scotland, Wales and Northern Ireland also support user confidence.

It is important that statistics producers continue to enhance the information available to the public. We have been assured that statistics producers are working to make developments, including:

  • further breakdowns of the data and more information about hospital admissions;
  • more information on the capacity of and demands on the health system, building on the analysis of NHS 111 calls published by NHS Digital;
  • further explanation of how the figures from the UK’s four nations compare to one another;
  • and, in the medium term, greater information about the demographic characteristics of people who are confirmed as having or having had COVID-19.

COVID-19 and the regulation of statistics

Today, The Director General for Statistics Regulation has made the following statement:

“This is a very difficult time for everyone as the UK adjusts to rapid changes in society and the economy. The priority is to protect the health and safety of individuals – including through changing working patterns and practices – and to support the information needs of society.

“Organisations that produce official statistics are rightly showing flexibility and adapting what they collect and publish to respond to this new environment. The Office for Statistics Regulation fully supports this flexibility and the responsiveness shown by producers of statistics.

“We have prepared a package of measures including guidance on factors that producers should consider when making changes to data collection, statistics production and release. We have also set out an approach to rapid regulatory reviews, potentially including National Statistics status, for any new outputs published by producers which inform the public about the coronavirus and its economic and social impacts. We will support sensible changes to existing production practices.”

Celebrating one year since the refresh of the Code of Practice for Statistics

This time last year, the UK Statistics Authority launched the refreshed Code of Practice for Statistics. 

The refreshed Code has supported and developed thinking on statistics as a public asset based on three pillars – Trustworthiness, Quality and Value.

This has enabled users and producers to have a set of principles that underpin independent statistics production and presentation, ensuring confidence in this essential public asset. This is important because statistics frame public debate. They inform public understanding of what’s going on in the world. They help provide a basis for people to make decisions – whether as policy makers, citizens, professionals or businesses. And they create a common ground for debate – about what’s working, what isn’t working and what needs to change.

 

Speaking on the first anniversary:

Sir David Norgrove, Chair of the UK Statistics Authority said: “It’s essential that people can have confidence in the statistics produced by Government. The Code of Practice for Statistics aims to provide the framework to ensure that statistics are trustworthy, good quality, and valuable. Statistics will serve the public good if producers follow the principles and practices set out in the Code.”

Ed Humpherson, Director General for Regulation said: “In a world of abundant data people want to sift the useful information from the unreliable. The principles of trustworthiness, quality and value really support this aim – and this is why the Code has been endorsed widely inside and outside Government.”

 

Guest blogging on the first anniversary:

Sir David Spiegelhalter, Former RSS President said: “A personal highlight of my last year as RSS President was the new Code of Practice for Statistics … Its emphasis on the trustworthiness of statistics gets to the core of what we are about as a community.” You can read Sir David’s blog here.

John Pullinger, National Statistician said: “The real power of the code is as an antidote to the narratives about loss of trust in institutions in general and in the use of data in particular.” Read John’s blog here.

Steve Ellerd-Elliott, Head of Profession at the Department for Work and Pensions said: “A new aspect of the Code that we have embraced is voluntary application to extend trustworthiness, quality and value to statistics that we routinely publish statistics on benefits, employment programmes and other important subjects that aren’t Official or National statistics.” You can read Steve’s blog here.

 

Note for Editors

The refreshed Code is available in three formats: as a booklet, an online PDF and a newly developed interactive code with case study examples – please visit here.

The Code of Practice for Statistics is an important tool in protecting official statistics by setting the standards which must be met to ensure the public can have confidence in them.

The Code standards are used:

  • by statisticians, and other analysts when producing and publishing official statistics;
  • by the Office for Statistics Regulation when determining whether official statistics can be called National Statistics;
  • by the Authority when publicly challenging the misuse of statistics;
  • to support judgements about the publication of wider numerical information; and
  • to enable statistics that are not currently National or Official statistics to voluntarily apply the principles and support users.

If you have an enquiry or require further information about the refreshed Code of Practice please email the regulation team.

The Social Metrics Commission Voluntarily Adopts the Code of Practice for Statistics

The Office for Statistics Regulation is pleased to announce that the Social Metrics Commission has become one of the latest organisations to publish a statement of voluntary compliance with the UK Statistics Authority’s Code of Practice for Statistics.

This demonstrates the Commission’s full commitment to applying the pillars of Trustworthiness, Quality and Value which underpin the Code. The Commission is one of the first non-Government bodies to voluntarily comply with the pillars of Authority’s Code of Practice.

Ed Humpherson, the Director General for Regulation at the Office for Statistics Regulation said: “I am delighted that the Social Metrics Commission has chosen to apply the pillars of the Code of Practice for Statistics to prepare their new poverty measure. They set a strong example of how to be transparent, reassure users and ensure high standards.”

Philippa Stroud, Chair of the Social Metrics Commission said: “Our approach to measuring poverty is underpinned by the need to build consensus and trust in their work and the analysis they produce. The Commission hopes to play a central role in improving the measurement of poverty in the UK, and consequently, drive better outcomes for people and families who are struggling to make ends meet.”

To see the latest list of organisations to voluntarily adopt the Code or find out more voluntary application please visit the online Code or contact regulation@statistics.gov.uk.