1. Foster an evidence-driven culture

think-trustworthiness-quality-value

 

Trustworthiness (T)     Quality (Q)     Value (V)

Look out for these letters throughout this section for indicators of the ‘Think TQV’ approach

Fostering an evidence-driven culture supports effective policy-making and sound operational decisions that can, in turn, improve the lives of citizens. An evidence-based approach helps people to make well-informed decisions about policies, programmes and projects by putting the best available evidence from research at the heart of policy development and implementation. Publishing more of the analytical evidence that government produces, in a transparent and accessible way, supports accountability, evaluation and improved outcomes. It can also build public confidence in an organisation’s commitment to evidence-based decision-making and the appropriate use of data more generally.

Fostering an evidence-driven culture is only possible through first recognising the value (Value) that sound analytical evidence offers, for example, to determine how to operationally work smarter rather than harder; to understand which policy interventions work and those which do not; or to support society’s needs for information. Being able to draw conclusions from analytical evidence with confidence requires the evidence to be based on suitable data and methods (Quality) and be produced and used in professional and orderly manner (Trustworthiness).

Our conversations brought to light four important aspects of fostering an evidence-driven culture. These include the need to:

  1. Facilitate evidence-based policy and decisions;
  2. Have non-analysts create demand for analytical evidence;
  3. Have visible analytical leaders at the highest levels;
  4. Champion outstanding analytical work.

Facilitate evidence-based policy and decisions

Extract from Declaration on Government reform:

“We will put data at the heart of our decision-making, learning explicitly from the approach we have taken in responding to COVID-19.”

Advocating for evidence-based policy and decisions, and leading by example in doing so, helps others to see their benefits, as well as how they can effectively use evidence in their own contexts. Analysts and non-analysts both have important roles and responsibilities (T) in supporting a strong, evidence-driven culture and making evidence publicly available to support society’s information needs and public accountability (V), wherever possible.

The Scientific Advisory Group for Emergencies (SAGE) provides scientific and technical advice to support government decision makers during emergencies. SAGE was most recently convened in January 2020 to provide expert advice on a novel coronavirus, now known as SARS-CoV-2. SAGE is made up of experts from a broad range of disciplines from both inside and outside government, including academics, chief scientific advisers, and members of scientific advisory groups across departments. It advised the government on a wide range of science topics related to the pandemic response, ensuring that authoritative analytical evidence was available to the government, to inform new policies and interventions in a timely manner.

The Government Chief Scientific Adviser made a commitment to publish all scientific advice provided by SAGE. This evidence is in the form of scientific papers and the minutes of SAGE meetings, which means that the public are also informed about the evidence that SAGE’s advice to government is based on.

The vision for the Office for Local Government (Oflog) is to provide authoritative and accessible data and analysis about the performance of local government, to support its improvement. Oflog aims to become a centre of expertise in the use of data in managing local government and to increase understanding – among citizens, civil society, central government and local government itself – about data on the performance of local authorities.

Oflog’s Local Authority Data Explorer allows users to compare key metrics relating to local government performance for authorities with similar responsibilities and attributes, drawing on the Nearest Neighbours Model developed by the Chartered Institute of Public Finance and Accountancy (CIPFA). To support confidence among users and the wider public, Oflog voluntarily applies OSR’s Trustworthiness, Quality and Value (TQV) framework for the Local Authority Data Explorer, and states that the transparency of high-quality analytical outputs to inform decision making and the public, underpins its production and release.

Extract from Declaration on Government reform:

“We must do better at evaluating the success of our programmes – not just in terms of meeting budgets but meeting the needs of those we serve. An Evaluation Task Force to act as an inhouse scrutineer not just of value for money in programmes but effectiveness against published ambitions.”

The Evaluation Task Force is a joint Cabinet Office-HM Treasury unit providing specialist support to ensure that robust evidence on the effectiveness of policies and programmes sits at the heart of government spending decisions. Its goal is to increase the effectiveness and efficiency of Government decision-making, and to improve confidence that the decisions made by policymakers, and the money that Government spends, is improving people’s lives.

The ETF delivers a range of activities to tackle barriers to robust evaluation in government and to foster a culture of evaluation. By 2025, the ETF aims to: ensure that all new major programmes in government have robust evaluation in place; all departments publish findings on policy evaluations in a timely and transparent way; stop the replication of programmes that have been shown not to work and learn best practice from those that do; and make the UK Government a world leader in evidence-based policy making.

Since its establishment in April 2021, the ETF has advised on evaluation methods for 305 government programmes worth a total value of £139 billion, it’s overseeing the £15m Evaluation Accelerator Fund, the £37.5m Labour Market Evaluation and Pilots Fund and the development of an Evaluation Registry to centrally store all government evaluations with aims to increase cross-government data sharing and increase government transparency and accountability. It is also running the train the trainer Evaluation Academy and publishing an updated 5 year What Works Strategy aiming to improve the way government uses What Works evidence to inform decisions about public services.

The Department for Transport (DfT) has been identified as one of the strongest departments in Government in terms of its analysis function over the last few years (based on feedback from the Senior Civil Service functional standards survey along with assessment against the analytical functional standard). Its strong evidence driven culture is supported by the strength of expertise across the analytical community and its structure, with most analysts working in teams embedded in policy directorates, plus a central team to maintain standards and deliver cross-cutting strategic work. This structure has supported close working between policy and analysts, and DfT are building on this, empowering policy customers to be more engaged and intelligent customers and users of analysis through structured learning and development for non-analysts to increase their awareness of analytical approaches and capabilities to use data.

DfT’s Permanent Secretary played an important role through their vocal passion for, and commitment to, policy and performance evaluation. This helped to embed a strong evaluation culture, driving expectation for evidence use through all levels of the department, so that analytical evidence for evaluation became seen as a core workstream, rather than as a separate entity.

Non-analysts create demand for analytical evidence

A strong evidence-driven culture needs non-analysts at all levels to actively seek out analysis to inform new policy areas and key operational decisions (V) as part of a standard, professional way of working (T). Very senior leaders have an important role here: people we spoke to told us that ministers and permanent secretaries can set the tone regarding the department’s approach to analysis. If for every new initiative they ask: “how is it going to be measured?” (Q) or “what are our plans for publishing our findings?” (T) it becomes the norm to integrate data into all policy and decision-making. In this way, they can make a significant difference in driving demand for analysis in decision-making and set the right tone for evidence use throughout their departments.

In his 2020 Ditchley lecture, the Rt Hon Michael Gove stated: “it is imperative that we learn the hugely valuable lessons that lie buried in our data”. He committed to ensure government has the skills to make better use of data. He also stated that “Government needs to evaluate data more rigorously” and that “Government must also ask itself if its people have the skills necessary for the challenges that I have set out”.

Then, in his 2021 launch of the Declaration on Government Reform speech, Michael Gove outlined some benefits and responsibilities associated with using government data: “Government data, effectively used, thus becomes a liberator of individuals. It enables us to arrive at conclusions drawn from evidence about what works to improve peoples’ lives. It lets us compare interventions between areas. And it gives students and their families the ability to hold government to account. Handling data requires government, of course, to protect privacy and safeguard individuals’ rights. But data can be anonymised, ensuring that government departments observe the right protocols and are transparent in their working.”

The Director General at the Department for Business & Trade (DBT) has been a champion for analysis, monitoring and evaluation in the department. By bringing their experience of working in other areas of government with a strong analytical focus, they have worked to ensure that evidence is now part of the rhythm of what DBT ministers see.

Key factors in achieving this have been: getting ministers used to using analytical evidence, demystifying it, and; showing the power of evidence, by drawing on departmental ambassadors of impactful analytical projects to communicate specific examples of evidence in practice. While there may be some variation in the extent to which a given Minister or Permanent Secretary may naturally recognise the value of analytical evidence, taking such an approach to highlight its power and benefits in specific cases, has been instrumental in building support and demand and informing policy development.

Visible analytical leaders at the highest levels

A strong evidence-driven culture also ensures that analysts have a seat at the right tables and have channels (T) to advocate their professional advice within government and to the public. Representation of analysis at senior levels and in key, influential, decision-making conversations helps to embed expectations and demand for evidence and build a culture of evidence-based decision-making. Analysts should also be visible and have public channels to communicate clear, impartial insights (V) both within government and to the public.

Sir Patrick Vallance as Chief Scientific Officer and Professor Sir Ian Diamond as National Statistician, were visible champions for analytical evidence during the COVID-19 pandemic. Their public profile helped to normalise the role of analysis in government decision making, especially through their attendance at the televised Downing Street briefings with the Prime Minister and other senior Ministers.

This provided powerful visibility of senior government analysts standing alongside senior government ministers. It was clear through the format of briefings that their role was to provide trusted, professional, objective analytical advice based on the best available evidence. This was in turn used to inform both the public and the decisions taken by Government based on that advice.

The Chief Statistician’s blog is a regular platform for the Chief Statistician of Wales to speak out on statistical matters. The Chief Statistician uses this channel to champion the work of Welsh statisticians, provide clarity on planned developments, and provide guidance on the correct interpretation of a range of Welsh government statistics. This open communication approach keeps users and the wider public informed on the latest statistical developments.

In January 2022, the Chief Statistician set out the potential impact that the policy change to no longer require people to take a PCR test after initially testing positive for COVID-19 could have on published COVID-19 case data. They explained the changes and provided authoritative advice on interpreting data. This included, for example, informing the public that while the published data on COVID-19 cases may appear to show a fall in cases in the short-term, this may not actually be the case. This provided a clear, authoritative, public, analytical voice on a matter of keen public interest.

Champion outstanding analytical work

Everyone in government has a role in advocating for, promoting and championing strong analytical insights that can help answer key public and policy questions and improve the lives of citizens (V). As well as sharing the benefits of these insights as knowledge across government and publishing them to inform wider society, demonstrating the value of analytical work supports the case for resourcing future analytical projects (T).

The Analysis Function Analysis in Government Awards (AiG Awards) recognise and celebrate excellent analytical work across government, by members of the Analysis Function, who include colleagues who work within government departments, government agencies and arm’s length bodies. There are six separate award categories:

  1. Collaboration
  2. Communication
  3. Impact
  4. Inclusion
  5. Innovative Methods
  6. Professor Sir Ian Diamond Rising Star Award

Contributions by colleagues outside of the Analysis Function are recognised through the Collaboration Award, which celebrates a person or team who has collaborated between teams, departments, other professions, external organisations or researchers to produce an impactful piece of analysis or analytical project.

Back to top
Download PDF version (368.96 KB)