Systemic Review Outline: approaches to user engagement in the Defra Group

Users of Defra Group statistics, please help us by completing our short survey. It takes approximately 5 minutes to complete and closes on 21st February. We really appreciate your feedback.

Background

To ensure that statistics are valuable and provide insights, statistics producers must understand how the statistics are used and what questions they need to answer. This can be achieved by communicating and engaging with users.

What we are doing and why

We are exploring approaches to user engagement by the Department of Environment, Food and Rural Affairs (Defra) and its Arm’s Length Bodies – collectively called the Defra Group.

The review has two main objectives:

  • To understand and document the range of approaches to user engagement currently adopted across the Defra Group statistics portfolio.
  • To identify key features of effective and impactful user engagement for the different types of audience (current and potential).

The Defra Group has a broad and diverse portfolio of official statistics and National Statistics, covering the environment, food and farming, fishing and rural communities. The public profile, public interest and impact of these statistics is varied, and therefore is likely to require different approaches to engaging with users. For this reason, the Defra Group is an ideal candidate for the review, which may provide broader learning about user engagement for the statistical community.

To make the scope of the project manageable, we are focusing on a small subset of 10 outputs which reflects the diversity of the portfolio and which covers the core Defra Department and the Arm’s Length Bodies.

How we are doing it

The review uses three sources of evidence.

Source 1: workshop with Defra Group statistics teams

We will discuss users and uses, approaches to user engagement, challenges to user engagement and support and training for user engagement.

Source 2: user survey

The survey will gather views from users of Defra Group statistics on how the Defra Group communicates and engages with them.

Source 3: measures of web use

Web analytics data provided by the Defra Group will tell us about public interest in the statistics.

Contact

For more information about the project, or to share feedback, please contact Job de Roij.

Systemic review outline: Statistical Leadership

Background

Leadership is an essential element of the Code of Practice. We have a principle dedicated to ‘Independent decision making and leadership’ and leadership is identified in the Code as essential for enabling statistical innovation, collaboration and coherence across departments. In practice, statistical leaders need strong systems to support them in their challenging roles and statisticians also need effective leaders to make progress and support them in their development.

What we plan to do and why

In the context of a new National Statistician, an evolving analytical landscape, and the recent Public Administration and Constitutional Affairs Committee review of UKSA governance, our statistical leadership systemic review will identify the primary characteristics of strong statistical leadership in government. We will identify a range of factors that enable effective statistical leadership: where it is most important; the consequences that can occur when it is missing; and the various structural aspects that support or inhibit it.

The overall aim of the review is to help ensure statisticians are valued and have a voice and influence in decision making. To this end, we want to support statistical Heads of Profession (HoPs) to perform their roles to best effect, making sure they have what they need to be effective and identify any barriers to effective leadership with a view to helping to overcome them. We will consider how best to use our position to influence existing structures in government to help to ensure statistical leaders have a voice and influence in decision making. There are also likely to be useful lessons around how we in OSR, and others in UKSA might better support current and future statistical leaders.

How we will do it

We will carry out research and engage with a range of statistical/analytical leaders and senior decision makers to explore the different departmental structures that support effective statistical leadership and where improvements can be made. We will examine statisticians’ roles and profiles, the skills they need to be effective, what the pipeline for future statistical leaders looks like, and how future leaders’ careers are being developed.

We will look for opportunities to facilitate knowledge sharing to support cross-organisational learning around the ways that current and future statistical leaders can be better supported, use our influence to address barriers to effective leadership where needed, and build advocates for the importance of strong statistical leadership as we go. We will also look to develop additional guidance and case study material to support the leadership elements highlighted in the Code of Practice.

Taking part

We would love to have your involvement in this work, so if you are interested in contributing to our statistical leadership review or would like to receive an alert as more information becomes available, please get in touch.

Contact for more information:

Email: Oliver Fox-Tatum (Lead), Tel: 01633 455 848

Email: Helen Miller-Bakewell, Tel: 0208 039 0314 or

Email: Oliver Anand, Tel: 0208 390 322

Systemic Review Outline: Mental Health Statistics

Background

Many people in the UK will experience a mental health problem each year, and most of us know at least one person who has been affected by a mental health condition at some point in their life. Every week there are reports in the media covering many aspects of mental health. Health policies attempting to address these many and varied issues have been introduced in all four countries across the UK in recent years. This review aims to look across the spectrum of mental health statistics and explore whether the statistical system is providing the information required to support individuals and policy makers.

What we plan to do and why

This project is intended to explore the extent to which statistics on mental health are able to answer the key questions that surround this area in society today. We want to understand users’ needs in relation to mental health statistics, and examine whether the statistics that are currently available meet these needs.

We want to see producers in all parts of the UK improve the trustworthiness, quality and value of their statistics on mental health so that they achieve parity with statistics about physical health, NHS performance and outputs. This is detailed in our work programme for 2019/20, which we published recently following engagement with both producers and users of statistics.

How we will do it

As each of the UK’s countries has separate policies on mental health, the review will take a phased approach, looking at the statistics produced in each country in turn. We have begun with England as it is the largest in terms of the volume and complexity of mental health statistics that are published.

The scope of mental health statistics is broad. As part of the initial scoping phase for each country we will determine the specific focus of the review. Our scoping work for England is now complete, and we are initially focusing on statistics on people’s experiences of mental health. This could include the incidence and prevalence of mental health conditions, individuals’ experiences of their mental health conditions, people’s access to services and journeys between different providers of services, and the outcomes of interventions.

Stage 1: system mapping

We have built up a picture of mental health statistics in England to help inform our understanding of what statistics relating to people’s experiences of mental health exist. To help with this we held meetings with some of the key producers of mental health statistics in England.

Stage 2: stakeholder engagement

Following on from stage 1, we are consulting with a range of statistics users who have an interest in mental health statistics in England. This work will be completed by early 2020.

Stage 3: collate findings and explore options for improvements

We will review the evidence gathered for the England part of the review and present it to key stakeholders. For areas where we feel improvements are needed, we will work collectively with relevant parties to identify solutions. We aim to publish our initial findings from this first part of the review by summer 2020.

Taking part

If you use mental health statistics or would like to be able to use them either personally or professionally, then we would like to hear from you. Some examples of questions that you might be interested in are:

  • Are the mental health statistics that you need available?
  • Are the statistics you need easy to find?
  • Are the statisticians responsive to your needs?
  • Do I have the information I need to better plan mental health services at a local level?
  • Is young people’s mental health getting worse?

You may have many other interesting and varied feedback so please get in touch with us at the regulation team. Our initial focus is on speaking to people who want to use statistics about the outcomes of mental health care in England.

The value of statistics on policing to public debate

Background

Policing is changing. Technology presents new challenges as well as opportunities for smarter working. The mix of crimes that police deal with now has changed, so that despite previous reductions in numbers of crimes recorded the time spent on dealing with crime has not. There are many non-crime related demands on police time and the police deal increasingly with complex problems that can call for multi-agency responses.

What we plan to do and why

Public debate around what the police do, or what they should do, and the resources they need to do these things do not necessarily reflect actual demands on the police. We think there is a need for public debate to be better informed. Our initial investigations suggest that current statistics and data on policing don’t tell the narrative of how policing has changed or reflect to the public what the police do and the resources they use to do it.

In this review, we will consider how well statistics and data support the public’s understanding of policing and serve the public debate about the police and policing in the UK, and what contribution statistics and data can make to improving public debate. We have four lines of investigation:

  • Policing structures and activities and demands placed on the police
  • Recent public debate about the police and policing
  • The statistics that describe aspects of the police and policing
  • Obstacles that prevent statistics better informing public debate

How we will do it

The review has four separate strands:

Stage 1: Intelligence gathering – We will look at policing statistics in the UK to identify what statistics exist in England, Wales, Scotland and Northern Ireland and where there are gaps. Separately, we will review published reports and attend events to develop our understanding of what the police are asked to do and how they do it.

Stage 2: Media analysis – We will analyse recent media coverage to identify the public debate around policing and to understand the extent to which current statistics are used to inform that debate.

Stage 3: Stakeholder Engagement –We will interview police leaders and others with insight into policing and demands placed on the police. Along with talking to statistics producers and decision makers in government, we will also listen to this group’s views about police data and statistics. The third group we want to speak to are journalists and other intermediaries in the public debate.

Stage 4: Share findings and highlight areas for improvement – We will publish updates during the review on the emerging findings from our four lines of investigation and an overview of our findings at the end.  If there are areas where the statistics can be improved or where there are gaps in statistics serving the public debate we will identify what is needed to bring about the necessary change and publish these as next steps.

What we want to happen as a result

This review is our first look at how statistics and data in the UK support the public’s understanding of policing. We will use it to determine the changes we think are needed to statistics and data, and how best to influence or bring about that change. Our aim from the review and any follow-up work is to increase the value of statistics on policing to improve public debate on this topic. Ultimately, we want to see statistics and data that better support the public’s need for information about policing, increase transparency and improve accountability of elected representatives.

Systemic Review Outline: National Accounts Classification processes, decisions and communications

Background

The Office for National Statistics (ONS), in determining how public spending should be classified for statistical purposes, can have wide impacts on key economic statistics as well as consequential impacts on the resourcing of government-sponsored projects over many years. It is essential that there is transparent independence around the decision-making process and good communication of the reasons underpinning those decisions. Increasing demands are being made on the classifications decision-makers, at the same time as questions are being asked about the Post-Brexit classification arrangements and how independence will be maintained. This vital area of statistical work, coordinated and managed by ONS, has not been reviewed in terms of meeting Code of Practice for Statistics principles, particularly those relating to its resourcing and transparency.

What we plan to do

We will examine the processes and governance of classification decision-making around what is inside and outside of the public sector for the purposes of public sector finances. We will focus on the value classifications advice from ONS adds to decision-making about funding public projects, the roles of the various committees, how classification work plans are decided and who they’re decided by. We will also examine the robustness of the governance structures in place that facilitate the smooth implementation of new and innovative funding models. These elements are essential for safeguarding independence in the classification process and as well as ensuring inter-generational commitments are made explicit for the purposes of holding decision-makers to account.

How we will do it

Stage 1. Background Research

We will review the guidance currently published online which relates to the classification process. We will review selected historic classification cases to better understand how the process works, particularly when applied to high profile or controversial classification decisions. We will review media coverage of decisions more comprehensively (in part to identify stakeholders).

Stage 2. Gather primary evidence

(a) from talking to decision-makers inside the classifications process

We will meet with people closely involved in classifications decisions, for example those who sit on key committees and statisticians in the Economic Statistics Classifications Team at ONS. Positive engagement with these internal experts will help to inform us and allow us to be aware of any changes or updates to the process. It will also give us the opportunity to ask any questions we may have at this stage.

(b) from talking to external stakeholders

Talking to decision-makers inside the classifications process should give us a comprehensive understanding of the current classifications approach. Once this is achieved we can begin meeting with external stakeholders; primarily academics, journalists and members of departments and devolved administrations. These meetings will be particularly important as they’ll give us various outside perspectives regarding the classifications process. We also plan to meet with EUROSTAT officials, this will allow us to gain a greater understanding of the level of involvement EUROSTAT has in in the classification process and how the UK classification process compares internationally.

Stage 3. Collate and review findings

We will analyse and review our findings, checking for consistency with Code of Practice for Statistics principles around independence, sufficient resourcing and transparency. We will determine whether to make any recommendations to ONS around

  • advice and decisions arising from the classifications process and how lessons are learned from decisions that cause controversy
  • how stakeholders are made aware of and react to the processes, advice and decisions
  • classifications approach post-Brexit.

Stage 4. Dissemination

We might host a meeting to discuss and help refine our emerging findings or put them into a report for publication. Any wider follow up activities would be determined on the basis of what we found.

What we want to happen as a result?

All aspects of the classifications process will be code compliant, ensuring policy makers, the wider statistical community and the general public can be fully confident in its transparency, independence and governance.

Timings to be announced

If you are interested in contributing to our work or would like to receive an alert as more information becomes available, please get in touch.

Contact for more information: email Guy Manning or telephone 020 7592 7800

Public value of statistics on public finances in a devolved UK

Background

The financial environment for devolved nations is changing with devolved new tax and borrowing powers bringing about increasing autonomy with a commensurate need to strengthen accountability.  This devolution of powers has created a situation where there are likely to be multiple sources of tax revenue and government spending statistics. Already there is a range of publications available from different government departments, which may be confusing to some users. There is a clear need for coherence in these existing statistics alongside the new statistics which will be introduced.

At the UK level we have in the Whole of Government Accounts (WGA), widely regarded as a world-leading development in public sector financial reporting. WGA has provided a step change in the ability of the government to understand and manage its financial position. The improved transparency provided by the WGA has helped the UK Parliament to scrutinise the effects of government policy better, aiding the work of the Public Accounts Committee and other parliamentary committees in holding the UK government to account.

At the devolved nations levels users do not have access to similar insights into the public finances that the WGA provides for the UK. We believe that devolved administrations need to be aware of, and manage, the net financial position. This is the difference between the assets they use to deliver vital public services such as healthcare and education, and the liabilities that purchasing or building assets create. The net difference between assets and liabilities (the net financial position) is a useful indicator of the overall financial position. Such a measure alongside informed commentary allows the key issues and risks arising from public sector assets and liabilities to be understood and managed. An investment statement is now a vital tool for holding modern fiscally responsible governments to account

What we plan to do and why

We will conduct extensive desk research looking at:

  • The existing published devolved public finance statistics
  • Existing data on the results of the Block Grant Adjustments to the Barnett formula and the consequences on devolved funding
  • Comment in the public domain about the strengths and limitations of current statistics and data on devolved government funding
  • Modern investment statements from advanced small nations such as New Zealand providing commentary accessible to the citizen in the street

We will engage with statisticians in both HM Treasury and in the nations and regions where there are significant devolved budgets and revenue raising capabilities such as London. Our purpose will be:

  • to establish the plans that administrations have made to address people’s key questions and provide them with the insights about national and regional funding that will help them.
  • Clarify administrations’ plans to help users easily find the statistics that they are looking for, and how they will be guided to go to the statistics that are most likely to answer their questions.
  • Pursue with the HM Treasury their intentions for the regular publication of statistics about the workings of the Barnett formula in a single, coherent and consistent publication and the possible voluntary application of the Code of Practice for Statistics to these statistics as recommended by a House of Lords Select Committee

We will explore the appetite for Statements issued by the respective Devolved Administrations from time to time describing and stating the current value of the State’s assets and liabilities, as well as changes since the last statement and foreseeable changes in the medium term.

We will investigate with stakeholders what statistics might exist to produce the first national Investment statements and what statistics might need to be developed over time to meet people’s expectations. The kinds of stakeholders we wish to engage with include:

  • Official independent forecasters such as the Scottish Fiscal Commission
  • policy-makers in Devolved and Central Governments
  • long-range infrastructure planners
  • Economic consultants, researchers and academics
  • Audit bodies both nationally and in devolved nations
  • National investment bodies
  • Regional and Devolved Nation based think tanks
  • Media
  • Business Bodies
  • Parliamentarians and their research and library bodies
  • Citizens with an interest in inter-generational equity and fiscal responsibility

We want to do this because transparent government accounts are a vital mechanism through which Parliaments, Assemblies and people can hold the government to account for the money it spends and the services it provides. The details on how devolved budgets are determined are already described as too opaque, which undermines accountability.

Despite the need for greater insight, historically the Treasury and the devolved nations published limited official statistics about devolved public spending. This is not to say that data has not been put into the public domain, however there is sometimes a fine line between increased transparency and ‘data dumping’. The latter can be inimical to transparency and good government. It is the task of government statisticians to produce  official statistics which are intelligible, readily accessible, with objective and impartial commentary.

It is difficult to establish comparable levels of spending in England for devolved functions as they are different in each part of the United Kingdom. Clear, thorough and readily accessible data on public spending across the United Kingdom are not being provided.

What we want to happen as a result?

We intend to enhance the coherence and transparency of devolved statistics on public finances with a view to improving the statistics to help people make better decisions.

Further, we seek to improve the transparency and accountability over the investment taxpayers have made in the balance sheets of the devolved nations.

Timing

  • Before the end of 2018 we will write to Chief Statisticians in the relevant jurisdictions. Our letters will set out the broad results of our review of coherence and transparency of their statistics on Government funding. We will also make any appropriate recommendations for improving the statistics to help people make better decisions.
  • By the end of March 2019, we will publish our findings on ways to improve the transparency and accountability over the investment taxpayers have made in the future wellbeing of the people of the devolved nations.

If you are interested in contributing to our work or would like to receive an alert as more information becomes available, please get in touch.

Contact for more information: email Iain Russell or telephone 01329 447704

 

Related Links:

Systemic Review: The Public Value of Devolved Public Finance Statistics (May 2019)

“An investment in knowledge pays the best interest” – Steps towards transparent fiscal statistics (May 2019)

Systemic Review Outline: Adult Social Care

Outputs

2020:

2019:

We closed the Northern Ireland strand of the review as summarised in a letter published in March 2019.

 

Background to the project

During 2018-19 the Office for Statistics Regulation (OSR) carried out a systemic review of adult social care statistics.

The review explored the adult social care statistics landscape across the UK to better understand the strengths and weaknesses, and share the learning across the four nations, recognising that policy variations have led to differences in how statistics are produced.

We wanted to further understand users’ needs for statistics about adult social care and discover and share best practice in the compilation, harmonisation, data linking potential, publication and statistical commentary of these statistics.

 

What we want to happen as a result of this systemic review

Adult social care statistics across the UK are coherent, harmonised, insightful and accessible and answer society’s important questions thus ensuring that decision-makers, policy officials and the public have the information they require.

 

Systemic Review Outline: Innovation in Children, Education, and Skills Statistics

Background
The Office for Statistics Regulation views statistics as an essential public asset; innovation in
statistics is vital to secure the public value of statistics and can add real value for users.
Systemic reviews help us explore issues of public value and identify developments that the
statistics system may need to make.

What we plan to do and why
This review will focus on the Children, Education, and Skills (CES) area which includes statistics
on various topics – for example teachers and lecturers, learners, and those not in education,
employment or training – and covers all stages of education from early years to university and
beyond.
Through this review, we will explore innovations and improvements being made to CES
statistics and in doing so champion and celebrate this important work. In addition we want to be
able to understand user needs when it comes to CES statistics and share this with producers to
help inform priorities in their development plans for the future.

How we will do this
The CES statistics area is wide and covers many different statistics outputs produced across
the UK. To get a broad understanding of the range of innovations being undertaken we will ask
producers to tell us about their work through a questionnaire. Based on the outcomes of this we
will review the scope of the review and possibly focus on activity in a particular area of the
statistics.
We then plan to delve more deeply to understand the benefits realised from improvements and
any barriers faced by producers. At this point we will also speak with researchers, academics,
policy, and other user groups to hear what they think have been valuable innovations in CES
statistics and what they would like to see in the future.

Timings
Producer Questionnaire, September/October 2017
Undertake producer and user interviews, October – December 2017
Publish a short report containing the review findings, Spring 2018.

Contact for more information on the CES systemic review:
Marie McGhee marie.mcghee@statistics.gov.uk
Louisa McCutcheon louisa.mccutcheon@statistics.gov.uk

Systemic Review Outline: the Public Value of Justice Statistics

Background

The Office for Statistics Regulation is keen to ensure statistics that relate to similar themes are readily accessible to users, able to answer the questions society needs to answer, and that there is a coherent narrative across sources. We see statistics as a public asset and want to maximise their value to users. Systemic reviews help us explore issues of public value and identify improvements that the statistics system may need to make.

What we plan to do and why                                                                

We published our review of the Public value of crime and justice statistics in the UK in April 2016. This work focused on crime and criminal justice. It identified a need for statistics that join-up across the system, yet the range of government bodies and other organisations  involved in processing data on crimes and delivering justice services, particularly in England and Wales, means that this is not always the case. We now want to continue our investigation of the public value of justice statistics by extending it to cover civil and family justice, offender management and outcomes, and public attitudes to the justice systems in the UK. Our principal aims will be to uncover what questions are being asked of justice statistics and what questions the current statistics can’t answer, and to check for any fundamental issues affecting justice statistics.

How we will do it

Stage 1: system mapping – we will build-up a picture of justice statistics in the UK to help inform our understanding of what statistics exist within and across each legal jurisdiction (England and Wales, Scotland, Northern Ireland).

Stage 2: stakeholder engagement – building on stage 1, we will consult with a range of statistics producers and users who have an interest in justice. We will gather views about whether the statistics that are currently available answer key questions and what gaps exist. We will also want to find out about emerging areas for which evidence might be needed in future and how well the systems in place can respond to such demands. If you want to speak to us about this work please get in touch: regulation@statistics.gov.uk

Stage 3: collate findings and explore options for improvements – we will review the evidence gathered and present it to key stakeholders. The findings from this review will also be considered alongside the findings from our review of crime and criminal justice statistics to look for common themes. For areas where we feel improvements are needed, we will work collectively with relevant parties to identify solutions.

Stage 4: follow-up reviews – we are aware that some aspects of crime and justice are still to be covered in our reviews (for example, policing), while this review might identify other areas requiring further attention. We will publish our plans for any further reviews of justice statistics once this current project is complete.

What we want to happen as a result

Justice statistics in the UK provide accurate and appropriate information with sufficient insight for those who use them to make decisions, and the system operates effectively and can anticipate the needs of the future.


Related Links

Public value of justice statistics

 

Systemic Review Outline: data linkage

Review published on 11 September 2018:

Joining Up Data for Better Statistics

 

Background

The Office for Statistics Regulation is keen to ensure that statistics are able to answer important questions for society. We see statistics as a public asset and want to maximise their value to users. Systemic reviews help us explore issues of public value and identify improvements that the statistics system may need to make.

What we plan to do and why

We now want to investigate the UK statistical system’s ability to provide greater insight to users via linked data (that is, data from more than one source which can be brought together securely to help answer questions). Stakeholders have already told us about various barriers to using linked data effectively, for example problems negotiating access and, if approved, long delays before data are actually available. The 2017 Digital Economy Act contains a number of provisions designed to significantly improve this situation. This review will look at whether we have the right structures in place to ensure that the benefits of the DEA are fully realised, this will include looking at how departments identify the most important and relevant questions to ask using linked data. We will also be exploring what this new landscape of linked data means for the Code of Practice for Statistics and how we might incorporate questions about using linked data into our assessments.

How we will do it

Stage 1: priority setting – we will review the evidence we have already collected from stakeholders and use that to prioritise a set of specific areas on which to focus our attention. Stakeholders will be consulted about our review aims at this stage. If you would like to help us shape this work, please get in touch.

Stage 2: stakeholder engagement and system mapping – we will consult with a range of statistics producers and users who have experience of working with linked data, or who have an interest in using linked data to answer policy questions, to explore what kinds of changes would be needed to help government and other users identify important questions and use linked data effectively. We will also map out the current data linkage landscape, and what it is likely to look like after the Digital Economy Act, to help us focus the work around the perspective of users. This mapping will cover data linkage systems in government departments across the UK, the health services, universities and the private sector (where they have an involvement in linkage projects that support public policy making).

Stage 3: collate findings and explore options for improvements – we will review the evidence gathered and present it to key stakeholders. For areas where we feel improvements are needed, we will work collectively with relevant parties to identify solutions.

Stage 4: follow-up reviews – we anticipate that we will continue our monitoring of data linkage issues beyond the timeframe for this initial phase of the work in order to assess the impact of the DEA – and where necessary to continue adapting our regulatory approach – in real time. This initial phase of the work might also identify new areas requiring attention.

What we want to happen as a result

Data linkage is widely used to answer society’s important questions in a timely manner.

Timing

Stage 1: priority setting – September 2017

Stage 2: stakeholder engagement and system mapping – September to November 2017

Stage 3: collate findings and explore options for improvements – December 2017 to March 2018

Stage 4: follow-up reviews  – Summer 2018 onwards

 

If you want to speak to us about this work please email the regulation team