Producing quality, trustworthy and valuable information? Why wouldn’t you?

Mairi Spowage, Deputy Chief Executive at the Scottish Fiscal Commission, describes using the Code of Practice for Statistics in a forecasting organisation.

First, a bit of background to the Commission. The Scottish Fiscal Commission is the independent fiscal institution (IFI) for Scotland, set up to produce independent forecasts of the Scottish economy, income tax, VAT, devolved taxes and devolved social security. We have a different remit and way of operating from the Office of Budget Responsibility (OBR), the IFI for the UK. We are not considered an official statistics producer.

I moved to the Commission in 2016, to help set it up, establish the modelling teams, and recruit the analytical staff. The requirements of these roles meant we required a mixture of economists and statisticians.

I used the Code of Practice for Official Statistics 1.0 extensively in previous roles as a statistician in the Scottish Government. Previously, I was responsible for the production of the National Accounts and public sector finance projections for the Scottish Government. These are high profile statistics, and the Code was an important tool in both ensuring and demonstrating independence of production. It gave us a helpful framework for interacting with users and Ministers.

 

“I became a cheerleader …” 

Why was I interested in bringing this experience into the Commission? Firstly, it is worth saying that as an IFI, we try to adhere to the OECD Principles for these bodies. I was struck by how well many of these principles chime with the Code – at their heart they encourage transparency, accessibility, independence of production and user engagement.

I became a cheerleader for the principles of the Code within the organisation. The first objective was to get buy-in from members of the Commission and our Chief Executive. Given the synergy of the Code with the IFI principles, they were wholeheartedly behind using this framework to shape our work and demonstrate independence, transparency, high quality analysis and user engagement.

I ran sessions on Code 1.0 with new staff as they joined, emphasising how it helps to ensure orderly release and transparency of production. At this point we hadn’t made a public statement about the Code, but the rigour of statistical production and the principles of openness were being brought into all the teams as they were formed. It is helpful to highlight practical examples which show the value of working within this framework.

“This has become a valuable way of working…”

The new (draft at this point) Code 2.0, with its aim to apply to the wider community of analysts producing numerical information, made it easier for us to make a public statement about what had become an important part of our way of working. Ensuring trust in us as an organisation, demonstrating the quality of our methods, and making sure that our outputs have the maximum value to our users is central to our approach. This has become a valuable way of working for both professions in the Commission, not just for the statisticians.

We released our first publications in September 2017 and organised user events to ensure we were meeting user needs. We implemented an extensive, robust and documented quality assurance process as part of our forecast production in December 2017.

Since then we have both implemented and publicised other elements of our work to demonstrate our efforts to stick to the spirit of the Code. These are simple measures, but are practical things that organisations can do to be more accessible, open and transparent.

Advanced notice

We now publish a monthly forthcoming publications email and news story, to highlight the month of publication up to a year ahead and the exact date at least 6 weeks in advance. It allows us to be in touch with users regularly, especially important in between forecast rounds, when we are likely to be publishing more specialist working papers.

It also allows us to demonstrate and publicise our outputs, so users know when to expect our publications. By being transparent about when we will publish outputs, and sticking to publication dates, we also demonstrate our independence.

Have policies in place

We have developed a corrections and revisions policy which allowed us to publicise transparently how the Commission will deal with any analytical errors or revisions in our work. I know that this can be uncomfortable for some organisations: obviously, the best scenario is one in which the corrections policy never needs to be used. However, anyone who has worked on large analytical documents will know that, no matter how rigorous quality assurance procedures are, errors will sometimes happen. Setting out how these will be dealt with in a proportionate manner is part of building user confidence in an organisation which is a mature, confident and professional provider of analysis.

Another simple policy is that from the first statutory publication in September 2017, we have assured that every table and chart in our document is made available in spreadsheet form at the time of publication. This may seem like a small point, but it is amazing how often this is not done and leaves users struggling to reuse information. This is a really easy way to increase public value and ensure snapshots of published information are preserved in an organisation.

“We are keen to meet user need for information and analysis …”

At the heart of our efforts to comply with the real spirit of the Code is user engagement, and in particular responsiveness to user feedback. An example of this is the additional paper we produced on income tax in March 2018 that focused on how we had estimated the taxpayer behavioural response in December 2017. There was huge interest in our estimate of the tax raised from the Government’s announced income tax policy and a clear desire for more information. Our additional paper also included some analysis that, due to user feedback, we will now include in any analysis of future policy changes.

We held an event where we presented this analysis to interested users. We are keen to meet user need for information and analysis where it can shed light on areas within our remit.

“…demonstrate our commitment to the Code…”

We brought together all of our policies which demonstrate our commitment to the Code in one document which we published in March 2018.  We plan to add to this over the months and years to come as we find new ways to be transparent, responsive and accessible. Because our staff come from a mixture of backgrounds, including those with experience of producing official statistics, we also aim to promote a collaborative environment where different analytical professions continue to learn from each other.

As I’ve gone on about how we value user feedback, I’d better say that you should let us know if you have any ideas about how the Commission can improve and expand this, whether in the document, or in practice.

So, drop us an email at info@fiscalcommission.scot if you want to get in touch.

Voluntarily Applying the Code

For me there was something exciting, and a bit nerve-wracking, about waiting to see the results of the Code consultation, specifically about ‘voluntary compliance’. It’s a bit like waiting for exam results or the outcome of an interview.

But the consultation responses were great – positive, constructive, and in places quite challenging. Respondents identified the potential advantages of voluntarily adopting the Code – as a means to ensure and improve quality and to enhance transparency and trust in their statistics. While the core target audience for the Code are organisations producing official statistics, there was widespread agreement that the three pillars of the Code – Trustworthiness, Quality, and Value – and the principles are transferrable to other organisations.

At the same time it was noted that some of the detailed practices of the Code (such as about the role of statistical Heads of Profession) are civil service constructs. I conclude from this that organisations interested in voluntary adoption and in applying the Code should, as the consultation document suggested, focus on the pillars and principles, and refer to the practices primarily to help interpret the principles in their own contexts.

But two sets of concerns were identified:

 

  • To what extent might organisations outside the public sector – which might have different drivers, such as profit making or lobbying – wish to voluntarily adopt the Code? I see this as a question for those organisations – we are, after all, talking about something voluntary. If an organisation sees advantage in aligning its work with the Code, then we would encourage them to do so.
  • What would be the Authority’s role in checking that an organisation voluntarily applying the Code is in fact compliant, to avoid the risk that statistics (and the Code) are brought into disrepute? I’m very clear that the Authority’s regulatory role is defined (in law) quite tightly – we are not in a position to ‘police’ voluntary application in the way we actively monitor the extent of compliance of official statistics, through the Assessment function. But there is an opportunity for the Authority to provide guidance on the Code, to support those interested in its voluntary adoption. For example, we’re planning to share some case studies online early next year.

I noted above that one of the main advantages of applying the Code principles is about enhancing trust by being transparent. Transparency requires organisations to make information available. So, the Authority considers that any organisation wishing to say that it voluntarily applies the Code should publish a statement alongside the statistics, setting out the extent to which it complies (and, where appropriate, areas of non-compliance).

Others can review the statement, form their own judgements, and potentially offer challenge. That might be uncomfortable for the organisations in question – but it’s key to continuous improvement. And the very process of opening one’s working practices up to external scrutiny and being seen to respond to feedback is at the heart of building trustworthiness.

There was also some comment in the consultation around the application of the Code by official statistics producer organisations to data and analysis that are not official statistics; for example, when publishing statistical research, management information or forecasts. Just to be clear, for these types of outputs we are also advocating the voluntary adoption of the Code: where an organisation chooses to adopt and apply the Code principles and makes a public statement about how it does so.

Voluntary adoption of the Code: where an organisation chooses to adopt and apply the Code principles and makes a public statement about how it does so.

We are working on a guide to this voluntary adoption and application of the Code which we will make available through our website. It will be interactive and supported by examples that illustrate how organisations are applying the Code.