Summary
The Office for Statistics Regulation (OSR) provides independent regulation of all official statistics produced in the UK. Statistics are an essential public asset: we aim to enhance public confidence in statistics produced by government by setting the standards they must meet in the Code of Practice for Statistics through the pillars of Trustworthiness, Quality, and Value (TQV).
This document provides guidance on how the principles in the Code of Practice can help in designing, developing and using models to improve their Trustworthiness, Quality and Value (TQV). Misuse and a lack of transparency of models can undermine public confidence, especially in the statistics they produce and decisions they inform. This guidance has been created to cover both traditional statistical techniques, (e.g linear regressions), and newer techniques (e.g machine learning), when they are used to create outputs that inform decision making and/or public policy. In this guidance, tick box statements have been included so that you can apply the principles in your work.
Part I explores the planning of a model. It provides steps to ensure your model meets the pillars of Trustworthiness, Quality and Value before you begin development. The main factors which should guide your decision to use a model are the purpose of the work, the user need and the social context surrounding the work. By thinking through these three things, you can demonstrate the appropriateness of your chosen technique.
This section discusses the following questions:
- What is the question you are trying to answer?
- What is the user need?
- What ethical and legal issues do you need to consider?
- Are the roles and responsibilities clear?
- Does your team have the right skills?
- Is resource sufficiently prioritised?
Part II focuses on the steps you should take to best develop and use your model to serve the public good. Part of this is ensuring that users of the model, both directly and indirectly, are at the heart of any decisions made around model usage.
This section discusses the following questions:
- Is there data of suitable quality?
- What is the right type of model?
- Are there opportunities for collaboration?
- Is the model clear and accessible?
- How will model quality and performance be measured?
All of the questions above explore important principles under the Trustworthiness, Quality and Value pillars. Thinking through these questions will enable you to demonstrate these pillars in your work as well as champion the themes of transparency, coherence and collaboration in a practical sense.
This is the final version of this guidance. It supersedes version one, which was published in October 2021. This new guidance reflects feedback we received from external stakeholders on version one, as well as OSR’s recent work on Securing public confidence in algorithms and our QCOVID case study. We would, however, still welcome feedback so please get in touch with regulation@statistics.gov.uk if you would like to share your thoughts and use cases.