Office for Statistics Regulation (OSR) has been exploring how National Statistics designation is understood.
We currently describe ‘National Statistics’ as the most important official statistics that have been demonstrated, through assessment, to meet the very highest standards of trustworthiness, quality and value, set out in the Code of Practice for Statistics. Only they carry the unique logo of the National Statistics tick mark, having been designated by OSR, as the regulatory arm of the UK Statistics Authority.
Back to basics – refreshing the designation
OSR has completed its review of the National Statistics designation to consider what is needed from the designation now. The designation was first introduced in the UK statistics system in 2000, so for over two decades it has been the mark of showing the highest statistical standards. Only official statistics that are assessed as fully compliant with the Code are designated as National Statistics. It has also protected official statistics against interference.
But the world has changed over that time – it is now data driven, with new sources and ways of sharing information. At the start of the review, the big question was what is needed from the designation: by producers of statistics; by users of statistics; and by us, the regulators of statistics.
Our review has involved an iterative series of conversations with producers of official statistics and some key stakeholders. We’ve developed a range of ideas about ways of evolving the designation and how to support the application of the Code of Practice for Statistics – these are summarised in our ideas paper.
After a final series of discussions with statistics producers and stakeholders, we put a recommendation, to refresh the designation, to the UK Statistics Authority Board, which has approved the move. We are now planning an implementation project to finalise the changes. We will be conducting user testing before we launch the refreshed designation. We are keen to have volunteers to join in the testing – if you are interested in being involved, please let us know by emailing firstname.lastname@example.org.
What we found out
We currently describe National Statistics as being the official statistics that are assessed as fully compliant with the Code, that is, they meet the highest standards of trustworthiness, quality, and value. During our conversations, we found some complications stemming from this way of understanding and applying the National Statistics (NS) designation.
There is a degree of mystique about the NS designation. The label is unclear and ambiguous, and potentially sensitive. It has led to an unhelpful distinction between National Statistics and other official statistics. Some intermediaries (such as the media, who present some official statistics to their own audiences) told us that they have interpreted the NS badge as indicating which statistics they should use – potentially missing out on using many hundreds of useful sets of statistics. It is worth remembering that most of the highly important statistics in the pandemic have not been designated as National Statistics.
The designation has become an artificial boundary between the designated and other official statistics. Producers told us that they have embedded the Code of Practice in the way their analysts produce official statistics – it has become their professional standard. They do not make a distinction in how they apply the Code – they do not typically look to achieve a higher level of Code compliance for National Statistics. Instead, they seek to comply with the Code in all their statistical outputs.
Our emphasis on full compliance has missed highlighting this compliance by official statistics. Our approach also goes beyond the requirements of the Statistics and Registration Service Act 2007. The Act calls on the Authority to conduct an assessment to determine whether the Code has been complied with. Our focus on continued full compliance by National Statistics, however, has set an unrealistic goal. Instead, we now recommend getting back to the basic requirement of the Act.
How to refresh the designation
We are reframing the designation to focus on its characteristic as an independent review that confirms the statistics comply with the Code by showing trustworthiness, quality, and value.
Independent review was one of the benefits of the designation that producers described, providing a different, external perspective on the statistics. It also meets one of the concerns of users, to ensure producers are held to account. They felt it was not enough for producers to promise to comply with the Code but that the commitment should be verified.
In focusing on independent review, we are moving away from using the label of ‘National Statistics’ in published statistics, although it remains the legal name for designated statistics whose Code compliance has been confirmed by the Authority. In our designation refresh project, we will look at how we can help explain and make clear which official statistics have been verified as showing trustworthiness, quality, and value. It may be the case that a symbol will be helpful, but we will ensure that we avoid recreating the artificial boundary.
Alongside this reframing of the designation, we will refresh our assessment process. We will apply the learning we gained during the early days of the pandemic, when we employed rapid reviews to quickly evaluate new statistics, and ensure that assessment is flexible and proportionate – fit to the meet the challenge of providing independent review to a wider range of official statistics.
In our designation review we suggested some new ways of supporting producer Code culture and processes. We are now aiming to develop these ideas further, working with producers. These ideas all have a strong emphasis on self-evaluation and are approaches that can help individual analysts, teams, as well as organisations, be guided by TQV.
- The Code ‘ABC’ approach promotes the benefits of self-evaluation, peer review, and independent review, to provide three levels of verification giving reassurance to users and insight to producers on compliance.
- The Code Maturity Model encourages the proactive application of the Code by organisations, to see how practice matches with four levels of achievement: beginning, progressing, accomplishing, and exemplifying, against the three pillars of the Code. It can help statistics leaders to target their developments, encourage learning and sharing across statistics teams, and promote strategic approaches in organisations.
- The Quality Grade tool can help analysts to summarise the key quality issues with their statistics to inform users and guide their use.
These new approaches and the reframing of the designation provide an opportunity for producers to learn from the best of their current good practice, effectively respond to the challenges of complex data and changing contexts, as well as provide greater reassurance to users about the statistics and better support their use.
And, in this data-rich world with statistics based on new and multiple sources, and varied forms of dissemination, knowledge about the provenance and quality of the data and statistics have become even more important for users. Accountability of producers is an essential requirement for public confidence.
Our reframing of the designation and refreshed assessment provide the means to assure and grow public confidence in all official statistics:
- Users will know that producers are accountable to a regulator
- Users will see that the statistics are:
- Not politically motivated
- Independently reviewed
- Users will:
- Feel they are listened to by responsive producers
- Be confident that statistics reflect their world
- Judge that statistics are good enough for their needs
Outline of the designation refresh project
The project will run through autumn 2022 and is split into three work streams:
Work stream 1
Focuses on how to explain the designation so that producers and stakeholders clearly understand it. This development includes other key terms and concepts such as ‘experimental statistics’ and ways to highlight that producers are accountable to the statistics regulator. The development will involve user testing of all the solutions identified with the assistance of communications experts and producers, including the presentation and dissemination champions.
Work stream 2
Focuses on supporting producer culture and processes and the three tools or approaches proposed to assist with the application of the Code. These developments are being done in partnership with producers, including the quality champions.
Work stream 3
Focuses on refreshing the assessment process and will be done by the regulatory team in OSR considering how to confirm compliance in short form assessments and in-depth reviews such as focusing on method and quality. It will also establish the means for ongoing engagement with statistics users, to ensure their experiences inform our decisions.
Please do let us know if you are interested in being involved by emailing email@example.com.