End of Code consultation – reflections on our road trip

It has been great to get out and about to speak about the draft Code of Practice for Statistics over the past few months as we ran over 30 sessions. Our team has covered over 4000 miles all around the UK since the start of the consultation in July, visiting many departments and agencies that produce official statistics. We’ve also met a wide range of stakeholders – some working in other areas of analysis and data, plus others outside of government but with a keen interest in statistics.

The consultation closes today – 5 October – and now the work begins to reflect on all the feedback we have received. It has been terrific to see the genuine enthusiasm for the Code and its importance in setting standards. We strongly feel the responsibility in ensuring that the Code clearly sets out these standards, supported by guidance to show what it looks like to comply with the Code.

The road show seminars reminded us of the need for the Code to reach everyone in producer organisations – to ensure that everyone understands their individual responsibilities. This becomes essential in dealing with pre-release access, and other aspects of orderly release such as releasing at the standard time of 9.30am. Other aspects of sharing data and statistics are of big concern to analysts – from providing low quality data under Freedom of Information legislation to questions about sharing data with colleagues working on modelling prior to publication. The use of worked examples on our website, alongside the Code, will go a long way in helping to support understanding of how the Code applies in different situations.

We have also spoken about our wider ambition for the Code. We see its principles as having universal application with all kinds of data, being a useful guide particularly when publishing data that are important in helping to inform users or supporting them when making decisions. The audience at the Code seminars suggested some challenges that we will face as a regulator in managing voluntary compliance but also identified some very real opportunities.

Overall we found strong support for our proposed framework of ‘Trustworthiness, Quality and Value’ from those involved in producing official statistics. But we also heard from some who found the concepts to have a degree of overlap – highlighting that good quality and relevant statistics lead to trust. Under the proposed framework, it is essential that those producing statistics demonstrate why they deserve public confidence and why their statistics can be trusted.

“…it is essential that those producing statistics demonstrate why they deserve public confidence and why their statistics can be trusted.”

We are grateful to everyone who attended one of our sessions for being so willing to hear about the Code and to participate in discussions. We hope to be able to continue the conversations once we have published the refreshed Code, to support all those involved in publishing official statistics in understanding what it means for them in their role, as well as promoting its wider application.

Our next steps are to review the consultation responses and all of the feedback we have received throughout the consultation period. We will be publishing a response that summarises the feedback, including how we plan to incorporate it in the final version of the second edition of the Code of Practice. Do keep an eye out for future blog posts to stay updated on the latest developments.

Five whys: the Code of Practice

A useful rule of thumb from the world of Lean process improvement is to never ask ‘why’ just once. If you want to improve something, ask ‘why’ several times. That way, you can get to the real drivers of a problem.

For example, imagine that work has stopped in your factory. You see it’s gone dark. So the first why is obvious: Work has stopped because no one can see what they’re doing. But then you ask why again. You’re told it’s because the light bulbs failed. Obvious solution – get more light bulbs.

But why did that happen? Because the bulbs weren’t checked regularly.

Why? Because it was no-one’s job.

The Lean solution: don’t just buy light bulbs. That just addresses the immediate issue. Instead, give someone responsibility to achieve the outcome of uninterrupted provision of light.

How is this relevant to the Code? Well, it gets to the heart of what we want to achieve.

Let’s apply the why approach to producing statistics.

Your colleague tells you that you have to announce the publication of your statistics in advance.
Why?
Because the Code of Practice for Statistics tells us to.
Why?
Because we should release statistics in an orderly way.
Why?
Because we want to show we have integrity.
Why?
Because we want to show demonstrate that we are trustworthy and have trusted people, systems and processes.
Why?
Because it demonstrates that the public can have confidence in our statistics.

We set out to create a Code that is more than a collection of detailed rules. We wanted the Code to be principles-led – more flexible, more supportive of judgement, more able to be applied to a wide variety of scenarios.

Now, principles can sometimes get a bad press. Too vague. Too open to interpretation. Perhaps too flexible.

We want our Code to have the good features of principles, without the downsides. We’ve tried to do this by including (updated) versions of the detailed practices from the original Code – but structured under core aspirational principles. These in turn are grouped into three pillars: trusted processes and people (trustworthiness); quality data (quality); and valued uses (value).

The beauty of this approach is that it explains why specific things matter; what they achieve. It fits with the notion that the key behaviour for people producing analysis and statistics is curiosity – about the data, about the questions the analysis is trying to answer, about the aspect of the world the statistics are illuminating. We’ve also included a new principle of innovation – it is all about the desire for improvement, the restless asking of questions like ‘why’ and ‘how’.

The ‘whys’ start bottom-up, starting with an individual practice and working up to the pillars. The Code can work the other way – as a series of ‘how’ questions, starting top-down. You want to secure public confidence. How? Through the three pillars of trustworthiness, quality and value. How do you deliver those – by thinking hard about how to align with the principles:

Principle	Pillar
Confidence in statistics is dependent on the integrity of those producing statistics; the behaviours and actions of producers should reflect public interest and this should be apparent to users	Trustworthiness: trusted people, systems and processes
Statistics production should be underpinned by strong leadership, effective and transparent planning, and clear lines of responsibility and accountability for observance of the Code
Sound professional and technical skills are needed to ensure good statistical judgement
The privacy of individuals and business information must be protected in the production and release of statistics and data, ensuring legal obligations are met
Using and understanding the most appropriate data sources is the foundation of producing robust statistics	Quality: robust data, method and statistics
Transparent judgements about statistical definitions and methods, together with judgements about strengths and limitations, are essential in supporting confidence in the quality of the statistics
Producers should demonstrate how they assure themselves that the statistics are robust and reliable
Statistics should be consistent and comparable, while remaining relevant to society
Statistics must be equally available to all and not released partially to selected audiences	Value: statistics that serve the public good
Statistics should help to clearly answer society’s important questions
Producers should understand and promote the variety of uses and potential uses of statistics
Statistics need to continue to evolve to remain relevant in a changing world
Statistics should be produced from data which has been compiled in an efficient way

And if you feel like you need more specific guidance as to what these principles mean in practice – under each principle there are in fact a series of practices, literally what do in practice. And to help people use the Code, we’re planning to create a guide to the Code which explains the purpose behind each element.

Of course, you can jump off this thought process at any point. If you are looking to comply with a specific practice and it makes sense – then you don’t need to ask a succession of why questions. Similarly, if you are pretty confident in how you secure public confidence, you don’t need to drill down into the detailed practices.

So. We don’t really think of the Code as new at the detailed level. In fact, most of the existing practices are incorporated in the new Code.

Instead, we’ve adopted the philosophy of Lean, based on the why (and how) questions. Armed with this, we’ve sought to breathe new life into what was a list of detailed practices. We’ve given meaning and purpose to these practices – so that they can continue to underpin statistics and data as a public asset.

Breakfast in the Boardroom

Reflections on the Code and the wider analytical community

Kimberly Cullen, Statistics Assessor

It was the bacon roll that did it. Ed, our DG, had just asked if I wanted to attend a breakfast meeting with several Directors of Analysis from across government to discuss the refreshed Code of Practice for Statistics. I hesitated – was thinking about my deadlines when I heard one of the most beautiful sentences in the English language: There will be bacon rolls.

Want me involved? Bring. Food. It really is that simple.

So bacon rolls aside (they were divine), how was the meeting? Fortunately there was a positive response to the Code changes (always a good start to a meeting). And, crucially, there was agreement that the revised document presents an opportunity to raise awareness across analytical communities. This was great news – but what is this opportunity and how do we at OSR frame it and then use it?

We all know there’s not just official statistics that are produced in government departments but administrative data, management information, research, financial data, secondary analysis, and modelling to name a few types of numerical information. Often these are not published nor end up as official statistics but nonetheless play a significant role in policy and decision making particularly in Ministerial government departments. I often hear throughout the Code seminars of the urgent and unyielding demands to provide information which are not official statistics and yet required to inform policies and decisions. Those working in this environment ask us repeatedly, how does the Code assist in this ambiguous space? How do those in the non-official stats analytical community use the principles and practices? I wish a bacon roll could turn their frowns upside down but sadly food is not the answer in this respect (I know, I can’t believe it either. . . ).

The draft Code contains helpful principles for wider analysis but the feedback suggested that we need to be clearer that the Code relates to the publication of such material (whether secondary analysis, modelling, etc.). And that what is published should be equally accessible to all. Both excellent points and it occurred to me that I had always just assumed – because I work with the Code daily – that everyone understood that. Right. We need to work on our messaging.

“What is published should be equally accessible to all.”

Use of the Code does not mean an obligation to publish all analysis and data in departments – only what ends up as official statistics. The message to the analytical community is that the Code principles of trustworthiness, quality and value (TQV) can apply to all numerical information. The Code has the potential to enhance public confidence in wider numerical information. TQV underpins official statistics but are universal properties to which all of us that work with information and data aspire towards. The Code can be a helpful guide to those collecting, analysing, and disseminating any type of numerical information. We need to get that message out.

Based on comments from the breakfast, we also realised the need to highlight the synergy between related guidance, such as standards applied in other professional groups (e.g. actuaries) and those using the Aqua Book for producing quality analysis. The Code is a tool to support numerical analysis and our principles do not supersede the expected standards in other related professions and industries.

Many thanks to the senior leaders that came to the meeting and provided such excellent feedback on the Code. As a result, we now recognise there is a spectrum of regulatory and advocacy activities that we are working within. We are regulators of the Code when assessing official statistics and we advocate the Code with the rest of the analytical community. How far down the advocacy route we go depends upon the audience and type of data but that requires further development which we have started.

We are keen to show the wider benefits of the Code principles and practices and support all those in the analytical community through a two-pronged approach of regulation and advocacy. It’s a rather exciting time in the office. Watch this space. . .

Richard Laux: Benefits from voluntary compliance

Richard Laux, Deputy Head of Regulation

In our consultation launch document, we floated the idea that a wide range of organisations might wish to comply with the Code of Practice for Statistics on a purely voluntary basis. We are keen to hear others’ views on this approach; in this blog I want to explain some aspects of our emerging thinking, to help stimulate debate.

Who might ‘voluntarily comply’?

By law the Code applies to a range of organisations, almost all from the public sector, which are ‘producers of official statistics’. In effect these organisations must ‘statutorily comply’ with the Code.

But many other organisations in the third sector, in academia, in local government, and in the commercial sector produce numerical information which are valuable to society – that is, the information has the potential to enhance debate and support informed decision-making. For example, local authorities publish performance information such as how much household waste is recycled and the percentage of council tax they collect. Companies publish information like their charitable donations and their carbon footprint. And universities which publish information about the employability of their students, their success in widening participation access, and their funding and spending. All of this information has public value: they provide insight, generate understanding and help people answer key questions.

What does ‘voluntary compliance’ entail?

We don’t think that a one-size-fits-all approach to voluntary compliance is the best way forward, and we would naturally support innovative approaches. But at the same time, we think it is likely that there will be some common approaches across organisations that are interested. For example, it seems sensible for an organisation to review its own policies and procedures against the principles of the Code, and to document its findings, including any parts of the Code that it does not fully comply with.

The organisation should focus on describing how it secures the three pillars of the Code – Trustworthiness, Quality and Value (TQV) – which are universal, and draw on the principles in the Code to guide its judgement. Where it is unsure how to demonstrate its approach to TQV, it might make sense to look at the detailed practices as a prompt or guide – although we believe that voluntary compliance will work best when targeted at the level of principles rather than practices. And it seems sensible for an organisation that considers that it is voluntarily compliant to say so on its website; if it considers that it is partly compliant, it seems sensible to explain its thinking together with any plans it has to enhance the extent of its voluntary compliance. We are happy to provide advice about the Code, to help organisations understand their own compliance.

Organisations in the commercial sector might not immediately recognise the relevance of the concept of public value to their work. But we would encourage them to think broadly – for example, in many cases the information they publish helps to make markets work efficiently, inform shareholders, and helps customers make decisions. These are all ways of providing public value.

What are the benefits?

From the perspective of the organisation, voluntarily compliance offers the opportunity to:

compare existing practices against the high standards required to enhance public confidence in its data and statistics;
draw attention to its high standards of statistical activity;
demonstrate an aspirational commitment to trustworthiness, quality and public value.

The impact of doing so is likely to vary between organisations – but is likely to include better numerical information, greater confidence in the use of that information, and competitive advantage. In some areas, we know that official statistics draw upon the numerical information produced by these organisations, and so documented voluntary compliance provides a degree of assurance about the trustworthiness of the organisation and the quality of the source.

What will the OSR’s role be?

We are happy to provide advice and guidance about the Code and, where possible, attend workshops to help the organisation; we might also be able to put organisations in similar sectors in touch with each other, to develop networks. And we will consider whether there are ways of presenting the Code that make it more accessible to organisations with an interest in voluntary compliance. Although it would not be possible to designate organisations’ statistical products as National Statistics, there may be scope for us to look at the quality of the statistics, if we were invited to do so.

Any questions?

If you have any questions about voluntary compliance, please contact me on regulation@statistics.gov.uk

More generally, the values of OSR include being: externally engaged and connected; enquiring and open-minded; and inclusive and listening to others – we want to listen and learn. So, if this blog stimulates any other thoughts, please let me know.