Voluntary Adoption

How we won our Award for Statistical Excellence

As the deadline for applications for our Statistical Excellence award nears, the joint-winners of the 2020 award, the Scottish Fiscal Commission, have written a guest blog about how they won and what the award means to them. Apply for the 2021 Award for Statistical Excellence in Trustworthiness, Quality and Value by 30 April 2021.

What is the Scottish Fiscal Commission?

We are Scotland’s independent fiscal institution (IFI). The Commission was created to serve the needs of increased fiscal devolution in Scotland. We are a young organisation, established in 2017. Our official economic and fiscal forecasts are used by the Scottish Government to formulate the Budget, the Scottish Parliament to scrutinise the Budget, and stakeholders and the media to inform public debate. We are not considered an official statistics producer, but we made an active choice to apply the Code of Practice for Statistics wherever possible.

The Code and the award

Since our creation, we followed the majority of the Code of Practice for Statistics principles. But in March 2018, we published a statement setting out our ongoing commitment to voluntarily apply the Code.

Last year, following encouragement from former team members, Mairi Spowage and Laura Pollitt, we decided to apply for the award. It was the first year the category had been created so we had no template to go on. Although the Commission is not an official source of statistics, we’re committed to upholding the pillars of the code – trustworthiness, quality and value – so it was just a question of evidencing how we do this.

We outlined our processes which include routinely pre-announcing our publications; helping data providers to plan their work by publishing an annual Statement of Data Needs, ensuring we accompany our publications with underlying data so others can use them and by communicating our work as widely as possible via our website, social media and events.

We also outlined our success in developing a reputation for delivering independent and credible forecasts, for which we were recognised by the OECD in an independent review published in 2019.

We were delighted to receive the award. We know that our stakeholders and ultimately the public depend on us to uphold the highest standards of statistical practice as a matter of course, so we were grateful to the Royal Statistical Society and the Office for Statistics Regulation for the opportunity to have our work endorsed in this way.

Any civil service organisation voluntarily applying the Code of Practice for Statistics to their work can apply for this year’s award. You can see the full list of organisations who have committed to apply the Code and more information about applying the Code on the Code of Practice website.

2021 Award

Apply for the 2021 Award for Statistical Excellence in Trustworthiness, Quality and Value by 30 April 2021.

The people behind the Office for Statistics Regulation in 2020

This year I’ve written 9 blogs, ranging from an exploration of data gaps to a celebration of the armchair epidemiologists. I was thinking of making it to double figures, setting out my reflections across a tumultuous year. And describing my pride in what the Office for Statistics Regulation team has delivered. But, as so often in OSR, the team is way ahead of me. They’ve pulled together their own year-end reflections into a short summary. Their pride in their work, and their commitment to the public good of statistics, really say far more than anything I could write; it’s just a much better summary.

So here it is (merry Christmas)

Ed Humpherson

Donna Livesey – Business Manager

2020 has been a hard year for everyone, with many very personally affected by the pandemic. Moving from a bustling office environment to living and working home alone had the potential to make for a pretty lonely existence, but I’ve been very lucky.

This year has only confirmed what a special group of people I work with in OSR. Everyone has been working very hard but we have taken time to support each other, to continue to work collaboratively to find creative solutions to new challenges, and to generously share our lives, be it our families or our menagerie of pets, all be it virtually.

I am so proud to work with a team that have such a passion for ensuring the public get the statistics and data they need to make sense of the world around them, while showing empathy for the pressures producers of statistics are under at this time.

We all know that the public will continue to look to us beyond the pandemic, as the independent regulator, to ensure statistics honestly and transparently answer the important questions about the longer term impacts on all aspects of our lives, and our childrens’ lives. I know we are all ready for that challenge, as we are all ready for that day when we can all get together in person.

Caroline Jones – Statistics Regulator, Health and Social Care Lead

2020 started off under lockdown, with the nation gripped by the COVID-19 pandemic and avidly perusing the daily number of deaths, number of tests, volume of hospitalisations and number of vaccines. This level of anxiety has pushed more people into contacting OSR to ask for better statistics, and it has been a privilege to work at the vanguard of the improvement to the statistics.

To manage the workload, the Health domain met daily with Mary (Deputy Director for Regulation) and Katy, who manages our casework, so we could coordinate the volume of health related casework we were getting in. We felt it important to deal sympathetically with statistic producers, who have been under immense pressure this year, to ensure they changed their outputs to ensure they were producing the best statistics possible. It’s been rewarding to be part of that improvement and change, but we still have a lot of work to do in 2021 to continue to advocate for better social and community care statistics.

Leah Skinner – Digital Communications Officer

As a communications professional who loves words, I very often stop and wonder how I ended up working in an environment with so many numbers. But if 2020 has taught me anything, it’s that the communication of those numbers, in a way that the public can understand, is crucial to make sure that the public have trust in statistics.

This has made me reflect on my own work, and I am more determined than ever to make our work, complex as it can be, as accessible and as understandable to our audiences as possible. For me, the highlight of this year has been watching our audience grow as we have improved our Twitter outputs and launched our own website. I really enjoy seeing people who have never reached out to us before contacting us to work with us, whether it be to do with Voluntary Application of the Code, or to highlight casework.

As truly awful as 2020 has been, it is clear now that the public are far more aware of how statistics affect our everyday lives, and this empowers us to ask more questions about the quality and trustworthiness of data and hold organisations to account when the data isn’t good enough.

Mark Pont – Assessment Programme Lead

For me, through the challenges of 2020, it’s been great to see the OSR team show itself as a supportive regulator. Of course we’ve made some strong interventions where these have been needed to champion the public good of statistics and data. But much of our influence comes through the support and challenge we offer to statistics producers.

We published some of our findings in the form of rapid regulatory review letters. However, much of our support and challenge was behind the scenes, which is just as valuable.

During the early days of the pandemic we had uncountable chats with teams across the statistical system as they wrestled with how to generate the important insights that many of us needed. All this in the absence of the usual long-standing data sources and while protecting often restricted and vulnerable workforces who were adapting to new ways of working. It was fantastic to walk through those exciting developments with statistical producers, seeing first-hand the rapid exploitation of new data sources.

2021 will still be challenging for many of us. Hopefully many aspects of life will start to return to something closer to what we were used to. But I think the statistical system, including us as regulators, will start 2021 from a much higher base than 2020 and I look forward to seeing many more exciting developments in the world of official statistics.

Emily Carless – Statistics Regulator, Children, Education and Skills Lead

2020 has been a challenging year for producers and users of children, education and skills statistics which has had a life changing impact on the people who the statistics are about.  We started the year polishing the report of our review of post-16 education and skills statistics and are finishing it polishing the report of our review of the approach to developing the statistical models designed for awarding grades.  These statistical models had a profound impact on young people’s lives and on public confidence in statistics and statistical models.

As in other domains, statistics have needed to be developed quickly to meet the need for data on the impact of the pandemic on children and the education system, and to inform decisions such as those around re-opening schools. The demand for statistics in this area continues to grow to ensure that the impact of the pandemic on this generation can be fully understood.

Producing quality, trustworthy and valuable information? Why wouldn’t you?

Mairi Spowage, Deputy Chief Executive at the Scottish Fiscal Commission, describes using the Code of Practice for Statistics in a forecasting organisation.

First, a bit of background to the Commission. The Scottish Fiscal Commission is the independent fiscal institution (IFI) for Scotland, set up to produce independent forecasts of the Scottish economy, income tax, VAT, devolved taxes and devolved social security. We have a different remit and way of operating from the Office of Budget Responsibility (OBR), the IFI for the UK. We are not considered an official statistics producer.

I moved to the Commission in 2016, to help set it up, establish the modelling teams, and recruit the analytical staff. The requirements of these roles meant we required a mixture of economists and statisticians.

I used the Code of Practice for Official Statistics 1.0 extensively in previous roles as a statistician in the Scottish Government. Previously, I was responsible for the production of the National Accounts and public sector finance projections for the Scottish Government. These are high profile statistics, and the Code was an important tool in both ensuring and demonstrating independence of production. It gave us a helpful framework for interacting with users and Ministers.

“I became a cheerleader …”

Why was I interested in bringing this experience into the Commission? Firstly, it is worth saying that as an IFI, we try to adhere to the OECD Principles for these bodies. I was struck by how well many of these principles chime with the Code – at their heart they encourage transparency, accessibility, independence of production and user engagement.

I became a cheerleader for the principles of the Code within the organisation. The first objective was to get buy-in from members of the Commission and our Chief Executive. Given the synergy of the Code with the IFI principles, they were wholeheartedly behind using this framework to shape our work and demonstrate independence, transparency, high quality analysis and user engagement.

I ran sessions on Code 1.0 with new staff as they joined, emphasising how it helps to ensure orderly release and transparency of production. At this point we hadn’t made a public statement about the Code, but the rigour of statistical production and the principles of openness were being brought into all the teams as they were formed. It is helpful to highlight practical examples which show the value of working within this framework.

“This has become a valuable way of working…”

The new (draft at this point) Code 2.0, with its aim to apply to the wider community of analysts producing numerical information, made it easier for us to make a public statement about what had become an important part of our way of working. Ensuring trust in us as an organisation, demonstrating the quality of our methods, and making sure that our outputs have the maximum value to our users is central to our approach. This has become a valuable way of working for both professions in the Commission, not just for the statisticians.

We released our first publications in September 2017 and organised user events to ensure we were meeting user needs. We implemented an extensive, robust and documented quality assurance process as part of our forecast production in December 2017.

Since then we have both implemented and publicised other elements of our work to demonstrate our efforts to stick to the spirit of the Code. These are simple measures, but are practical things that organisations can do to be more accessible, open and transparent.

Advanced notice

We now publish a monthly forthcoming publications email and news story, to highlight the month of publication up to a year ahead and the exact date at least 6 weeks in advance. It allows us to be in touch with users regularly, especially important in between forecast rounds, when we are likely to be publishing more specialist working papers.

It also allows us to demonstrate and publicise our outputs, so users know when to expect our publications. By being transparent about when we will publish outputs, and sticking to publication dates, we also demonstrate our independence.

Have policies in place

We have developed a corrections and revisions policy which allowed us to publicise transparently how the Commission will deal with any analytical errors or revisions in our work. I know that this can be uncomfortable for some organisations: obviously, the best scenario is one in which the corrections policy never needs to be used. However, anyone who has worked on large analytical documents will know that, no matter how rigorous quality assurance procedures are, errors will sometimes happen. Setting out how these will be dealt with in a proportionate manner is part of building user confidence in an organisation which is a mature, confident and professional provider of analysis.

Another simple policy is that from the first statutory publication in September 2017, we have assured that every table and chart in our document is made available in spreadsheet form at the time of publication. This may seem like a small point, but it is amazing how often this is not done and leaves users struggling to reuse information. This is a really easy way to increase public value and ensure snapshots of published information are preserved in an organisation.

“We are keen to meet user need for information and analysis …”

At the heart of our efforts to comply with the real spirit of the Code is user engagement, and in particular responsiveness to user feedback. An example of this is the additional paper we produced on income tax in March 2018 that focused on how we had estimated the taxpayer behavioural response in December 2017. There was huge interest in our estimate of the tax raised from the Government’s announced income tax policy and a clear desire for more information. Our additional paper also included some analysis that, due to user feedback, we will now include in any analysis of future policy changes.

We held an event where we presented this analysis to interested users. We are keen to meet user need for information and analysis where it can shed light on areas within our remit.

“…demonstrate our commitment to the Code…”

We brought together all of our policies which demonstrate our commitment to the Code in one document which we published in March 2018. We plan to add to this over the months and years to come as we find new ways to be transparent, responsive and accessible. Because our staff come from a mixture of backgrounds, including those with experience of producing official statistics, we also aim to promote a collaborative environment where different analytical professions continue to learn from each other.

As I’ve gone on about how we value user feedback, I’d better say that you should let us know if you have any ideas about how the Commission can improve and expand this, whether in the document, or in practice.

So, drop us an email at info@fiscalcommission.scot if you want to get in touch.

Voluntarily Applying the Code

For me there was something exciting, and a bit nerve-wracking, about waiting to see the results of the Code consultation, specifically about ‘voluntary compliance’. It’s a bit like waiting for exam results or the outcome of an interview.

But the consultation responses were great – positive, constructive, and in places quite challenging. Respondents identified the potential advantages of voluntarily adopting the Code – as a means to ensure and improve quality and to enhance transparency and trust in their statistics. While the core target audience for the Code are organisations producing official statistics, there was widespread agreement that the three pillars of the Code – Trustworthiness, Quality, and Value – and the principles are transferrable to other organisations.

At the same time it was noted that some of the detailed practices of the Code (such as about the role of statistical Heads of Profession) are civil service constructs. I conclude from this that organisations interested in voluntary adoption and in applying the Code should, as the consultation document suggested, focus on the pillars and principles, and refer to the practices primarily to help interpret the principles in their own contexts.

But two sets of concerns were identified:

To what extent might organisations outside the public sector – which might have different drivers, such as profit making or lobbying – wish to voluntarily adopt the Code? I see this as a question for those organisations – we are, after all, talking about something voluntary. If an organisation sees advantage in aligning its work with the Code, then we would encourage them to do so.
What would be the Authority’s role in checking that an organisation voluntarily applying the Code is in fact compliant, to avoid the risk that statistics (and the Code) are brought into disrepute? I’m very clear that the Authority’s regulatory role is defined (in law) quite tightly – we are not in a position to ‘police’ voluntary application in the way we actively monitor the extent of compliance of official statistics, through the Assessment function. But there is an opportunity for the Authority to provide guidance on the Code, to support those interested in its voluntary adoption. For example, we’re planning to share some case studies online early next year.

I noted above that one of the main advantages of applying the Code principles is about enhancing trust by being transparent. Transparency requires organisations to make information available. So, the Authority considers that any organisation wishing to say that it voluntarily applies the Code should publish a statement alongside the statistics, setting out the extent to which it complies (and, where appropriate, areas of non-compliance).

Others can review the statement, form their own judgements, and potentially offer challenge. That might be uncomfortable for the organisations in question – but it’s key to continuous improvement. And the very process of opening one’s working practices up to external scrutiny and being seen to respond to feedback is at the heart of building trustworthiness.

There was also some comment in the consultation around the application of the Code by official statistics producer organisations to data and analysis that are not official statistics; for example, when publishing statistical research, management information or forecasts. Just to be clear, for these types of outputs we are also advocating the voluntary adoption of the Code: where an organisation chooses to adopt and apply the Code principles and makes a public statement about how it does so.

Voluntary adoption of the Code: where an organisation chooses to adopt and apply the Code principles and makes a public statement about how it does so.

We are working on a guide to this voluntary adoption and application of the Code which we will make available through our website. It will be interactive and supported by examples that illustrate how organisations are applying the Code.