Ed Humpherson to Ken Roy: Confirmation of NS designation for Defra statistics on air quality and emissions of air pollutants

Dear Ken 

CONFIRMATION OF NATIONAL STATISTICS DESIGNATION FOR DEFRA STATISTICS ON AIR QUALITY AND EMISSIONS OF AIR POLLUTANTS 

We have reviewed the actions that your team has taken to address the requirements in Assessment Report number 344Statistics on Air Quality and Emissions of Air Pollutants 

 On behalf of the Board of the UK Statistics Authority, I am pleased to confirm the continued designation of National Statistics for the air quality and emissions of air pollutants statistics 

 These statistics are valuable because they support understanding of air pollution – major environmental threat to human health in the UK. They inform the development of policies that aim to improve air quality by reducing emissions of air pollutants. The statistics provide a good overview of trends over time in the concentration of air pollutants and the relative contribution of different sources of emissions of air pollutants.  

 The team responsible for producing the statistics has responded very positively to the Requirements of our report. The team has made changes that enhance the coherence, accessibility and clarity and insight of the statistics and data for a wide range of users. We are particularly pleased with the more joined-up presentation of the two sets of statistics, and with the progress made on developing a local air quality data warehousewhich will maximise access to, and the value of, these data for specialist users. In some areas, the team went beyond the Requirements and implemented additional changes, such as converting the statistics bulletins to HTML format. We welcome the team’s commitment to continuous improvement. 

 We have included more detail about our judgement in an annex to this letter. I, or my team, would be happy to talk you or your colleagues through any aspects of this letter or Code compliance more generally.  

 National Statistics status means that official statistics meet the highest standards of trustworthiness, quality and value and is something to be celebrated. We welcome that you already publish a statement alongside the statistics which reflects the National Statistics status, and we encourage you to update this on an ongoing basis as you continue to develop the statistics. 

 I am copying this letter to Philip Taylor, the responsible statistician.  

 Yours sincerely  

 

Ed Humpherson 

Director General for Regulation 

 

Related Links

Statistics on air quality and emissions of air pollutants

Ed Humpherson to Ken Roy: User engagement in the Defra Group

Dear Ken

User engagement in the Defra Group

Today we published the findings of our review of user engagement in the Defra Group[1].

The aims of the review were to develop a better understanding of the range of approaches to user engagement currently adopted within the Defra Group, and to identify the key features of effective and impactful user engagement.

We identified many examples of good practice and are encouraged by the extent to which teams are engaging with users. We found that many teams promote their statistics and tell users about changes to the statistics, that they have a good understanding of the types of users and uses and that they engage regularly with well-established networks of known users in a variety of ways. We also welcome that the Defra Group is taking a joined-up, strategic approach to user engagement to gain insight into users and uses and to encourage and foster user engagement within the department.

We also identified several ways in which user engagement could be improved or enhanced. Some teams could be more proactive in their engagement by seeking the views of users and other stakeholders and exploring a range of ways of involving them in the production and development of the statistics. Teams could enhance transparency and demonstrate their knowledge of the types of users and uses, and the extent of their engagement, by publishing information about users, uses and user engagement activities. As a whole, the Defra Group could be more strategic and impactful by encouraging and supporting teams in their user engagement and could continue to build its capability in areas like social media and web analytics to enhance communication with users and develop understanding of use.

We recognise that the last few years have been particularly challenging for the Defra Group in terms of resourcing, due to preparation for EU exit, and more recently, COVID-19. It is right that your short-term focus is on maintaining official statistics outputs and prioritising analysis that supports understanding of the effects of COVID-19 on food, farming and the environment. This means the coming year will not be a normal time for statistics production and there may limited scope or capacity for enhancing user engagement. While the recommendations in this report should be seen as a reference point for more normal times, some recommendations are achievable in the short term, and we encourage you and your teams to consult this report on an ongoing basis.

I would like to thank all individuals in the Defra Group who gave up their time to contribute to this review for their open and positive engagement with my team. I look forward to future engagement with you and your statistics teams as you enhance your user engagement.

Yours sincerely

Ed Humpherson

Director General for Regulation

[1] By the Defra group we mean the Core Department and Executive Agencies, Forestry Commission and those Defra Arm’s Length bodies that are designated as producers of official statistics: Environment Agency, Joint Nature Conservation Committee, Marine Management Organisation and Natural England.

 

Related links:

User engagement in the Defra Group

Compliance Check: Monitor of Engagement with the Natural Environment (MENE) Statistics

Dear Ken

COMPLIANCE CHECK OF THE MONITOR OF ENGAGEMENT WITH THE NATURAL ENVIRONMENT (MENE) STATISTICS

Thank you for your letter outlining Natural England’s plans for its new People and Nature Survey, the replacement for the Monitor of Engagement with the Natural Environment (MENE) Survey. You intend to publish the first statistics from the new survey as experimental statistics, with a view to putting them forward for assessment shortly after publication.

We have undertaken a compliance check to look at the transition to the new survey. Given the proposed changes, we have focused on the quality of the MENE data and statistics. However, through our conversations with the MENE team we also learned about the value and trustworthiness of the statistics.

We summarise our main findings below, and highlight some things you might like to consider as part of developing the new statistics.

New method

While the new survey has similar objectives to MENE, the move to an online panel method for data collection is a major departure. As far as we know, they will be the first official statistics based on data collected through an online panel. This presents an opportunity to innovate but also some potential challenges to the quality and trustworthiness of the statistics. We agree that the People and Nature Survey statistics should be fully assessed against the Code of Practice for Statistics, to ensure meet the highest standards of trustworthiness, quality and value.

Quality

The team demonstrated a good awareness of the limitations of the MENE survey, and told us how it has enhanced the quality of the data in the last few years. We heard that the team has invested a lot of resource in working through the 10-year MENE dataset to ensure it is clean, consistent and comparable; that it has strengthened its quality assurance (QA) processes; and that it has enhanced resilience in the team by spreading expertise across a wider range of staff and learning from past experiences.

We welcome that QA, and reference to the Code of Practice for Statistics, were built into the contract for the People and Nature Survey, and are pleased that the team is considering building a more collaborate approach to QA for the new survey. We encourage the team to apply the lessons learned from cleaning and maintaining the MENE dataset to develop a good understanding of, and document, the quality of the People and Nature Survey data.

Value

MENE is the leading source of information on people’s engagement with the natural environment, and is used widely inside and outside government. Users in Defra told us that MENE, and the new People and Nature Survey, are key data sources for evaluating policies in the 25-Year Environment Plan. The team has established successful partnerships with several universities, which has led to detailed analyses and insight into the association between people’s contact with nature and health and wellbeing. We heard how engagement with external users and stakeholders has helped colleagues in Natural England and Defra recognise the value of the MENE data and statistics, and we welcome that it has led to a better long-term partnership between Natural England and Defra. It will be important to communicate clearly the value of the new People and Nature Survey and we encourage the team to continue its wide engagement to promote and develop the new statistics.

MENE review

Last year, Natural England commissioned an external review of MENE to better understand the value of the MENE data and statistics to users and to scope out options for piloting different approaches for the People & Nature Survey. The outcome of the review fed into the requirements for the new survey, but the team also made immediate changes to how it reported the MENE results following stakeholder feedback. For example, the team released a GIS local authority dashboard, allowing users to access and integrate local-level data, and developed an interactive, visual story map summarising and reflecting on 10 years of MENE data. We welcome that the team took the opportunity to develop these additional outputs to enhance the value of the MENE statistics. We would welcome similar levels of innovation in the People and Nature Survey outputs.

The draft report of the MENE review has been ready since May 2019 but is still awaiting external peer review. To enhance transparency around the development of the new survey, and to inform users of the lessons learned from MENE, we encourage the team to publish the report as soon as possible.

Our Agriculture, Energy, Environment Domain Lead, Job de Roij, will continue to engage with the statistics team throughout the transition to the new survey.

I am copying this letter to Rose O’Neill and Simon Doxford, the responsible analysts at Natural England.

Yours sincerely

Mark Pont
Assessment Programme Lead

 

Related Links:

Ken Roy to Job de Roij (November 2019)

Monitor of Engagement with the Natural Environment (MENE) Statistics

Dear Job,

Monitor of Engagement with the Natural Environment (MENE) – National Statistic Compliance Check

Thank you for meeting with the Natural England team in relation to the Compliance Check of MENE. I understand that you discussed the MENE Review and transition to a new People and Nature Survey.

The MENE survey ended in February 2019 and results have been published on Gov.UK. Following user engagement, the team are developing a new People and Nature survey that will build on and replace MENE. The People & Nature Survey for England will, like MENE, aim to collect data on people’s experiences and attitudes towards the natural world. It is designed to meet the needs of a range of MENE users, new users and provide data for the Government’s 25 Year Environment Plan Indicator Framework. The People & Nature Survey will be a new dataset, with some new questions alongside some from MENE, and will use an online panel method to collect data.

Data collection for the new People & Nature Survey will begin in April 2020, with the first annual release in autumn 2021. Given the significant changes to survey design and method, we propose publishing the first statistical release as Experimental Statistics. We intend to ultimately achieve National Statistics status for the new survey, and aim to undergo assessment shortly after the first statistics are published. The experimental phase will allow us to better understand and mitigate for any impacts in relation to trust, quality and value in relation to the transition.

Do get in touch if you require any further information.

Yours

Ken

Head of Profession for Statistics (Defra group)
Natural England

 

Related Links:

Mark Pont to Ken Roy (December 2019)

Assessment of statistics on air quality and emissions of air pollutants

Dear Ken

ASSESSMENT OF STATISTICS ON AIR QUALITY AND EMISSIONS OF AIR POLLUTANTS

We have today published our assessment report covering these statistics. I am grateful for the positive contribution of your staff and for their engagement throughout the assessment process.

Air pollution is one of the most high-profile and important environmental issues. These air quality and emissions data and statistics are therefore vital to supporting public understanding of the problem and informing public debate, and for the development and evaluation of policies that aim to reduce emissions of air pollutants and improve air quality. Both sets of data and statistics have a wide range of uses and users, and users told us how valuable they are.

Our report identifies many areas of good practice, particularly around quality. These include: the use of the most appropriate data sources with rigorous and robust quality assurance procedures; a close working relationship between Defra and Ricardo, and their effective engagement with data providers, other contractors, stakeholders, and topic experts, to ensure high data quality; and an ambition to further improve the emissions estimates.

Our report identifies some important ways in which the value of these data and statistics should be enhanced. The two sets of statistics form a package and are closely linked, but the statistics bulletins do not help users understand this relationship. To provide a coherent view of these connected topics and generate greater insight for users, both sets of statistics should be integrated as far as possible. As part of achieving this, the air quality and emissions statistics team needs to engage more directly and proactively with users of the statistics to develop a better understanding of the needs of different types of users. We also consider that users’ needs for data would be better served through better access to more granular, disaggregated air quality data and existing local air quality data.

We judge that these statistics can continue to be designated as National Statistics once you have met all six Requirements of this report. This will demonstrate that your statistics meet the highest standards of trustworthiness, quality and public value and comply with the Code of Practice for Statistics. We have agreed that you will report back to us by May 2020 outlining how you have addressed the Requirements. Aspects of some Requirements will take longer to implement, such as the user engagement work and the improvements to the UK-AIR website; for these, we expect you to send us a detailed update on progress towards meeting the Requirements by May 2020.

Please feel free to discuss any aspect of this with me or my team at any time.

Yours sincerely

Ed Humpherson
Director General for Regulation

 

Related Links:

Assessment report 344: Statistics on air quality and emissions of air pollutants