6. Draw on analytical standards and expert functions
Trustworthiness (T) Quality (Q)
Look out for these letters throughout this section for indicators of the ‘Think TQV’ approach
It is important that there are clear standards (T) for analysts in government and that analysts have the access to the guidance that they need. Accessing the right guidance requires drawing on advice, expertise and resources, both within departments and beyond. Having clear guidance helps to ensure that analytical approaches, standards and techniques follow professional best practice (Q), wherever possible, and comply with established legal and ethical standards (T). Clear standards also help to ensure efficiency and analytical coherence by drawing on established sources, definitions and approaches to communicating strengths, limitations and uncertainty (Q), rather than reinventing the wheel.
There are varying approaches to governance and the structure of analytical staff in the UK, with each of the Devolved Administrations sometimes taking their own approaches. OSR heard about good work being carried out by different parts of government in this area, including the Government Analysis Function, the Government Statistical Service and the ONS Data Science Campus. Crucially, there are opportunities to build links between different professions around shared values, standards and priorities for analysis, and by drawing on UK expert functions.
Our conversations brought up two important aspects when using the analytical system. This includes the need to:
- Follow and promote professional standards; and
- Draw on UK expert functions.
In this section, rather than offer case studies, we have shared information on several key standards and expert groups that can be drawn on. This is not an exhaustive list of the standards and groups that exist across government, but rather ones that have been highlighted in our work.
Follow and promote professional standards
A variety of established professional standards (T) set out expectations for undertaking methodologically sound, robust and assured (Q) analysis across government.
The Analysis Functional Standard sets expectations for the planning and undertaking of analysis across government to support well-informed decision making; to deliver better outcomes, and improve the lives of citizens. It provides direction and guidance for permanent secretaries, directors general and chief executive officers of arm’s-length bodies to ensure appropriate governance of analysis, as well users and producers of government analysis including non-analysts and external consultants to ensure quality and consistency of analysis across government organisations.
The Code of Practice for Statistics from the Office for Statistics Regulation (OSR), sets the standards that producers of official statistics should commit to, to support public confidence in how statistics are produced and used. But the three pillars of the Code – Trustworthiness, Quality and Value – support all types of evidence.
Committing to OSR’s TQV framework on a voluntary basis can be done by any producer of data, statistics and analysis, whether inside government or beyond. It helps teams produce analytical outputs that are of high quality, useful for supporting decisions and well respected, supporting public confidence for published analytical outputs. The three pillars focus on:
- Trustworthiness: the organisation’s commitments to professionalism, sound governance and separation of statistics and data from policy
- Quality: appropriate data sources and methods, together with a commitment to quality assurance
- Value: clarity and insight as to what the data and statistics can, and cannot, be used for.
Many organisations already voluntarily apply the TQV framework. Among them:
The UK Health Security Agency (UKHSA) voluntarily applied TQV to its Coronavirus (COVID-19) in the UK Dashboard – the official UK government website for data on COVID-19. It did so to show its commitment to transparency and desire to increase user trust in the dashboard, to help it as a producer to demonstrate what it was doing well, and to identify areas for improvement. Applying TQV allowed UKHSA to feel confident that its work was easy to access, remained relevant, and supported understanding of important issues in relation to COVID-19.
The Office for Local Government (Oflog) voluntarily applies TQV to its Local Authority Data Explorer. While the data explorer is not designated as official statistics, Oflog has made a public commitment to uphold TQV, which helps it emphasise that transparency and an ambition to produce high-quality analytical outputs that inform decision making and the public, underpin the explorer’s production and release.
The Scottish Fiscal Commission (SFC) sees its voluntary application of TQV as important in building public confidence in its outputs. The SFC uses voluntary application to help demonstrate why its outputs can be trusted. Examples of what the SFC does as part of this commitment include preannouncing publications, which means stakeholders know when to expect outputs, and having a transparent policy to handling corrections and revisions. The SFC also publishes all its data in accessible spreadsheets so they can be easily re-used, and has an active external engagement strategy encompassing the public, the Scottish Government, the Scottish Parliament, and other key decision makers.
The Aqua Book provides guidance on producing quality analysis for government. It was produced following the ‘Review of quality assurance of government analytical models’, by a cross-departmental working group on analytical quality assurance.
The Magenta Book provides guidance on what to consider when designing an evaluation. It looks at the types of evaluation (process, impact and value-for-money) and the main evaluation approaches (theory-based and experimental), as well as setting out the main stages of developing and executing an evaluation.
The Green Book provides guidance on how to appraise and evaluate policies, projects and programmes. It also provides guidance on the design and use of monitoring and evaluation before, during and after implementation. Green Book guidance applies to all proposals that concern public spending, taxation, changes to regulations, and changes to the use of existing public assets and resources.
Draw on UK expert functions
The UK has a range of analytical expert and advice functions that support the suitable (Q) and responsible (T) production of analysis across government. They are open to providing support to anyone in government undertaking analytical work both in terms of collaborative projects and bespoke analytical advice.
The Centre for Applied Data Ethics was established by the UK Statistics Authority in 2021 to provide practical support and thought leadership in the application of data ethics by the research and statistical community. The Centre aims to provide a world leading resource that addresses the current and emerging needs of user communities, collaborating with partners in the UK and internationally to develop user-friendly, practical guidance, training and advice in the effective use of data for the public good.
The Centre for Applied Data Ethics Independent Advisory Committee (CADEAC) has been established to advise the UK Statistics Authority on the strategic direction, outputs and impact of the Centre. The committee is formed of experts from government and academia. CDEAC have produced an easy-to-use ethics self-assessment framework for researchers to review the ethics of their projects throughout the research cycle. The framework provides a timely means for researchers to identify potential ethical issues within their research proposals and an accurate and consistent estimation of “ethical risks” across research projects.
Led by the 10 Downing Street Data Science team, Evidence House aims to radically upskill civil servants in data science, software development and AI while delivering innovative solutions to crowdsourced problems. It does this by:
- Bringing skilled Civil Servants together, putting them in teams and have them compete to generate new solutions to priority problems through multi-day hackathons.
- Taking the best ideas from the hackathons and developing them further, by deploying members of the community.
- Supplementing this hands-on experience with learning and development opportunities for members, from introductory courses in programming to one off seminars and specialist workshops.
Since being established last year, the programme has amassed 850 members to whom it has delivered 12,500 hours of hands on in person technical upskilling, while through its hackathons numerous projects have been developed further, with some in trial across government currently.
The ONS Data Science Campus was established in 2017 as a Centre of Excellence with the purpose of applying data science, and building data skills, for the public good across the UK and internationally. The campus seeks to capitalise on a new generation of tools and technologies to exploit the growth and availability of these new data sources and innovative methods to provide rich informed measurement and analyses on the economy, the global environment and wider society.
The goals of the ONS Data Science Campus are to:
- investigate the use of new data sources, including administrative data and big data for public good;
- help build data science capability for the benefit of the UK.
The campus works with experts from various UK and international organisations including academic institutions, commercial and third sector organisations and other National Statistical Institutes. It collaborates on a wide variety of initiatives including, but not limited to, data science, including, capability building research and data sharing.
The Government Data Quality Framework produced by the Government Data Quality Hub supports a strong data quality culture, through principles and practices to assess, communicate and improve data quality. It sets out five principles which act as a guide to help create a strong data quality culture. The principles are:
- Commit to data quality – Create a sense of accountability for data quality across your team or organisation, and make a commitment to the ongoing assessment, improvement and reporting of data quality.
- Know your users and their needs – By researching and understanding user needs we can ensure data are fit for purpose and prioritise our efforts on the data which are most important.
- Assess quality throughout the data lifecycle – Data quality should be managed across the data lifecycle, paying close attention to quality measures and assurance at each stage.
- Communicate data quality clearly and effectively – It is important to clearly communicate the quality of your data so that users can decide if it meets their needs
- Anticipate changes affecting data quality – Not all future problems can be predicted. Where possible, anticipate and prevent future data quality issues through good communication, effective management of change and addressing quality issues at source.
The principles underpin best practice approaches to data quality and explain procedures and attitudes to help facilitate this.
The Regulatory Policy Committee (RPC) is the independent regulatory scrutiny body for the UK Government. It is an is an advisory non-departmental public body, sponsored by the Department for Business and Trade.
As a centre of excellence, the RPC provides expert advice on the quality of evidence and analysis used to inform government regulatory proposals. This independent advice and scrutiny helps ensure that ministerial policy decisions are based on accurate evidence, and helps to produce better regulation. Through effective early engagement with departments, the RPC plays a key role in improving the evidence used to support ministerial decision making.
The publication of RPC’s advice is also of value for parliamentary scrutiny and assists stakeholders in understanding the impact of regulatory proposals.