Dear all,

Local Authority Collected Waste Management statistics

We have reviewed compliance with the Code of Practice for Statistics of the four sets of waste statistics published by the Department for Environment, Food and Rural Affairs (Defra), the Scottish Environment Protection Agency (SEPA), the Welsh Government and the Northern Ireland Department of Agriculture, Environment and Rural Affairs (DAERA):

• Local Authority Collected Waste Management Statistics for England (Defra)
• Household Waste Statistics for Scotland (SEPA)
• Local Authority Municipal Waste Management Statistics for Wales (Welsh Government)
• Northern Ireland Local Authority Collected Municipal Waste Management Statistics (DAERA)

We reviewed the trustworthiness, quality and value of the statistics, including the coherence of the data source, methods and quality assurance (QA) arrangements, and the presentation of the statistics.

I am pleased to confirm that the statistics for England, Wales and Northern Ireland can continue to be designated as National Statistics. The statistics for Scotland are not currently designated as National Statistics; they are official statistics. We carried out a more comprehensive review of the Scottish statistics and gathered feedback from a small number of users to support their continued development towards National Statistics status.

The waste statistics are one of the key sets of environment statistics. They provide high-quality information about the volume of waste generated and recycled at the local authority level. As waste and recycling are primarily local issues, this level of granularity is essential. The statistics and data are also important at the national level, as they are used to monitor progress against waste and recycling targets. Issues relating to resources and waste, in particular, plastic pollution, continue to be a focus of public interest and debate on the environment and these waste statistics contribute to public understanding of these issues.

Our key findings and recommendations across all sets of statistics are presented below. Detailed findings by country are presented in Annex A. We encourage you, where relevant, to reflect on the findings for other countries, to learn from their approaches and practices.

We identified several shared strengths:

• All local authorities submit waste data through the WasteDataFlow (WDF) Data are entered and processed in a standardised way, generating a robust data series on household waste for each country, although countries define and categorise waste slightly differently. Most countries have introduced a more flexible question format which allows local authorities to report more accurate and complete information on waste treatment and disposal.
• All countries maintain strong, constructive relationships with data suppliers (local authorities). For instance, Defra and SEPA hold regular WDF user group meetings, which provide a forum for discussing data quality issues and developments and gathering feedback on the statistics from local authorities. It is good that you each took into consideration the resource pressures that local authorities have faced during the COVID-19 pandemic, supporting them where necessary (for example, by extending data submission deadlines), and that you have are adapting your QA checks to ensure that data quality remains high. We welcome this ongoing, proactive engagement with data suppliers, who are also key users of the statistics.
• In general, the four countries collaborate closely when collecting waste data and producing waste statistics. For instance, all countries are represented on the WDF Operational Group, which meets annually to discuss methodology and processes as well as ad hoc WDF and local authority statistics issues. We also heard how statisticians worked together to develop the harmonised UK household waste measure (‘waste from households’) and are currently collaborating on developing a new waste tracking system (see below).
• The ‘waste from households’ measure, which is used by UK and Devolved Governments to monitor and report compliance with the EU’s Waste Framework Directive recycling targets, allows users to compare recycling rates between UK countries. Defra presents a summary of this measure in a separate bulletin, UK Statistics on Waste, but some countries also comment on trends in the ‘waste from households’ measure in their bulletins and explain how it differs from country-specific household waste measures.
• Defra, with the support of governments and regulators in Scotland, Wales and Northern Ireland, is leading the development of a new, innovative electronic waste tracking system. It aims to create a single point of reference for a waste transaction that can be used by all four countries. The new system will likely replace existing waste data collections, including WDF, and is expected to fill known information gaps (such as on what happens to waste when it moves from production to recovery or disposal, and flows between recycling facilities) and improve the coherence of waste definitions across the UK.

We also have some general recommendations and suggestions for improvements across all sets of statistics:

• The credibility of recycling is dependent on final overseas destinations. There is still confusion among the public about where the UK’s recycling goes, with a significant quantity of some types of material being shipped abroad for recycling. To support public understanding of recycling, we recommend that waste statisticians in all four countries work together to produce an accompanying “explainer” on how waste is defined as having being recycled and how it counts towards the headline recycling rate. We think it would be helpful if it also summarises the main definitional differences in recycling rates between countries and explains why figures based on the harmonised measure do not always match up across different outputs, for instance, due to data revisions.
• To enhance transparency around user engagement, and encourage further engagement from users, we recommend publishing a summary of existing and planned user engagement activities. It should explain how you are listening to users and acting on their views to develop the statistics. You may like to consult our review of user engagement in the Defra Group to inform your thinking in this area. We encourage you to share learning and insight from user engagement with waste statisticians in the other countries.
• COVID has impacted many local authorities’ waste collections and waste data submissions. This is expected to have knock-on effects on the statistics, especially the volume of waste generated and recycled. It is important that you understand the effects of COVID on data quality and trends in the statistics and explain these to users.
• The new accessibility regulations for public sector bodies came into force on 23 September 2020. You should ensure that your outputs, including the statistics bulletins, and statistics landing pages meet these requirements and reflect on any additional steps you could take to make your statistics more accessible to users, in line with the Code of Practice.

Our Agriculture, Energy and Environment team will continue to engage with you in the coming months to follow up on the highlighted areas for improvement. We thank you and your teams for your positive engagement throughout the review process.

I am copying this letter to Alex Clothier, Katherine Merrett and Andrew Woodend (Defra), Peter Ferrett (SEPA), Stuart Neil (Welsh Government), and Siobhan Carey (Northern Ireland Statistics and Research Agency).

Yours sincerely

Mark Pont

Assessment Programme Lead

 

Annex A: Key findings and recommendations by country

Local Authority Collected Waste Management Statistics for England (Defra)

Strengths

• The team has a good understanding of the main users and uses of the statistics and interacts with users and other stakeholders in a range of ways. It has a strong working relationship with Defra policy colleagues, as evidenced by close collaboration during the response to the pandemic. It engages with industry bodies, waste management consultancies and local authorities through a resources and waste data group, which meets regularly to discuss use of waste data for infrastructure planning. The team also engages with organisations such as the Waste and Resources Action Programme (WRAP) on specific data issues. We welcome that different types of users are involved in the development of the data and statistics.
• The bulletin is well-structured and engaging, with impartial commentary that explains the main differences and trends in the statistics. Relevant new content continues to be added to the bulletin, for example, on final waste destinations. The visualisations are effective and aid interpretation of the statistics. The maps are particularly informative, illustrating the variation in recycling rate, and how it has changed since the previous year, across England.
• The methodology summary published alongside the statistics contains a good overview of the WDF system and highlights its main limitations. It explains the nature of changes in methods and their impact on the statistics. Definitions of the three reported measures of household waste are clear and highlighted prominently throughout the bulletin, supporting understanding for non-expert users. We are pleased that the team has been using our Quality Assurance of Administrative Data (QAAD) framework to review QA arrangements and welcome the level of detail published about the QA process.

Areas for improvement

• To gain further insight into user needs and uses of the statistics, the team could be more proactive in its user engagement. For instance, it could attempt to identify and engage with potential users of the statistics, including academics.
• To ensure that the information on the statistics landing pages is current and helpful, we recommend refreshing all landing pages. The pages would benefit from more information on what the waste data are and why they are collected. For instance, to highlight their wider relevance and importance, we suggest adding an overview of or links to the key waste policies in England, including the Resources and Waste Strategy and the 25 Year Environment Plan. The Defra air quality and emissions statistics landing page provides a good example of how to do this.
• To enable users to easily explore the rich dataset and facilitate re-use of the data, the team may like to consider developing an interactive dashboard or data tool like those DAERA or SEPA have developed.
• The UK Statistics on Waste bulletin highlights the extent to which England is meeting its waste and recycling targets. We think it would be helpful if the local authority collected waste management bulletin also commented on this aspect of the statistics.
• The waste data collection process in England is more complex than in other UK countries due to the large number of local authorities and the involvement of a contractor (Jacobs). To help users understand the flow of data through the system, the team could add a process map that illustrates the different stages and QA arrangements.
• We encourage the team to explore the feasibility of developing a metric like SEPA’s carbon metric, which measures the whole-life carbon impact of waste. Such a measure would add insight on waste’s contribution to climate change in England.

Household waste statistics for Scotland (Scottish Environment Protection Agency)

Strengths

• The team engages effectively with users inside and outside government and employs a range of approaches to understand use and listen to users. It works closely with policy teams in SEPA and analysts in the Scottish Government and Zero Waste Scotland. It also engages regularly with academics and industry bodies through Scotland Waste Data Strategy activities. In 2019, the team conducted a user survey to better understand the users of the waste statistics, their views on the presentation and content of the statistics bulletins, and to identify gaps in their needs. We encourage the team to continue this proactive engagement to ensure that the statistics are relevant and insightful for all types of users.
• The Waste Data Strategy hub on Scotland’s Environment web, maintained by the statistics team, is an excellent repository of waste-related information in Scotland. It brings together news and updates, documents from user events, guidance, case studies and data. The two interactive data tools on the website aid interpretation of the statistics by allowing users to produce customisable charts and data sets.
• The users we spoke to told us they valued the statistics bulletins and datasets as the provide fixed points of reference which reduce the risk of misinterpretation. The bulletins give a good overview of the short- and medium-term trends in the statistics. The commentary is impartial and reasons for changes are discussed. Information about waste policies, regulations and targets in the quality report and on the SEPA website is clear and helpful.
• The carbon metric, which measures the whole-life carbon impact of waste, is innovative, insightful and well-established, giving an indication of waste’s contribution to climate change in Scotland. We look forward to seeing the continued refinement of this valuable metric.
• The QA process for the data is rigorous and continually improving. The users we spoke to recognise the effort the team invests in maintaining data quality. The team recently conducted a survey to better understand the QA principles, standards and checks undertaken by local authorities, which provided useful insight. It is also good that the team developed an automated data validation tool for local authorities to improve and standardise the level of QA.

Areas for improvement

• It important to be transparent about the outcomes of individual user engagement activities. We think it would be helpful to reinforce user engagement by publishing a summary of responses to the 2019 user survey, explaining how the team intends to respond to feedback and improve the statistics.
• The accessibility of the waste statistics landing pages and other waste pages on the SEPA website needs to be enhanced. These pages are not as user-friendly as they could be for non-expert users and information across pages is often inconsistent or out-of-date. For example, the publication calendar on the household waste data page confuses the ‘date when published’ with the reporting period and the links on the waste statistics regulations page are old or broken. All pages would benefit from a content review and refresh. Also, the users we spoke to told us the team may assume a higher level of user knowledge than is realistic. The team should consider the needs of different types of users when producing the bulletins, data tables and quality report to ensure that the information is accessible to non-expert users.
• To make users aware of the coherence and comparability of the Scottish statistics with those produced by the other UK countries, we think it would be helpful to report and comment on trends in waste and recycling calculated using the harmonised UK ‘waste from households’ measure. The insight and relevance of the household waste bulletin could be enhanced by commenting on the extent to which the Scottish Government is meeting its waste and recycling targets.
• The most recent edition of the quality report (covering 2017 data) was published in July 2019, a year and a half after the end of the reporting period. To ensure that quality and methods information is timely and supports interpretation of the statistics at the time they are published, it should be published alongside the statistics. The quality report should also explain how the carbon metric is calculated and describe the strengths and limitations of the approach used.
• Published information about QA arrangements is limited. We recommend that the team apply our Quality Assurance of Administrative Data (QAAD) framework and publish a summary of its findings to assure users of the comprehensive QA process. It should map the flow of data through the system to help users understand the quality at all stages of the production process. We encourage the team to publish a summary of findings from the QA survey of local authorities as part of this documentation, to highlight the variation in approaches and data quality.
• The list of recipients with pre-release access (PRA) should be published on the SEPA website and be reviewed regularly.

Local Authority Municipal Waste Management Statistics for Wales (Welsh Government)

Strengths

• The statistics landing page is user-friendly and explains why waste data are collected and where they come from. All data are published on the StatsWales website, which allows users to produce and download customised data and tables and charts. Metadata, summarising the main limitations and caveats, are published alongside the data tables and charts to help users interpret the statistics.
• The annual bulletin is well-structured and captures the main trends in the statistics. It highlights the extent to which the Welsh Government is meeting its waste and recycling targets and contains links to related Welsh statistics and waste statistics from the rest of the UK. The information boxes spread throughout the bulletin define the key terms and measures and explain potential inconsistencies in the data.
• Changes in data quality, such as the improvement in data accuracy which led to a recent revision in the recycling rate, are explained in the quality report. The quality report also highlights potential sources of bias in the data due to, for example, the splitting out of household and non-household waste (which some local authorities collect together).

Areas for improvement

• Apart from a 2017 user consultation on changes to outputs, we found little evidence of proactive engagement with users, particularly those outside government. To ensure that the statistics are relevant and insightful for all types of users, the team should aim to establish an ongoing dialogue with a range of users and involve them in the development of the statistics.
• The insight of the bulletin commentary could be enhanced by discussing reasons for changes over time. The bulletin, quality report and statistics landing page would benefit from more-detailed information or links on waste policy in Wales and the rest of the UK, to help users contextualise the statistics.
• We encourage the team to explore the feasibility of developing a metric like SEPA’s carbon metric, which measures the whole-life carbon impact of waste. Such a measure would add insight on waste’s contribution to climate change in Wales.
• Three organisations are responsible for the management of waste data and engagement with data suppliers in Wales – the Welsh Government, Natural Resources Wales, and the Waste and Resources Action Programme (WRAP). The roles and responsibilities of each organisation, and how they work together to deliver high quality data, should be explained,
• To support user understanding of the data source (WDF) and methods, we think it would be helpful to explain how the statistics are calculated from the data submitted by local authorities. This should cover the question format and how it has changed over time to enable collection of more granular information on end destinations of waste.
• The quality report contains a basic description of QA arrangements, but the level of detail is not proportionate to the complexity of the data. For example, it does not cover the checks and validation carried out by local authorities. To reassure users about data quality, the team should produce more-thorough documentation that maps the flow of data through the system. It may like to consult DAERA’s Administrative Data Source Quality Report for an example of this.

Northern Ireland Local Authority Collected Municipal Waste Management Statistics (DAERA)

Strengths

• We welcome that the team has applied the learning from our compliance check of the Northern Ireland June Agricultural Census statistics to other DAERA statistics, including the waste management statistics, to enhance their quality and value. Recent improvements have focused on making the statistics and data more accessible and reusable for a wide range of users through the development of new outputs (such as infographics, an interactive dashboard and a time series dataset), and improving and publishing more-detailed information about QA arrangements (see below). It demonstrates a commitment to continuous improvement.
• We are pleased to see recent proactive engagement with external users of the statistics, for example, through a workshop in early 2020. This provided valuable information on how the statistics are used and feedback on the presentation of the statistics, which is being used to drive improvements. The team has also promoted the statistics via the DAERA statistics user group newsletter, created to inform users during the pandemic. We encourage the team to continue building its network of external users and involve them in the development of the statistics.
• The annual bulletin is informative and engaging, providing a coherent overview of waste and recycling in Northern Ireland. It presents estimates of the recycling rate using both the Northern Ireland household waste definition and the harmonised UK measure (‘Waste from Households’) and helpfully compares the recycling rates between UK countries. It contains clear descriptions of relevant policies and detailed and impartial commentary on progress against waste and recycling targets.
• The published quality information is comprehensive. In addition to the clear descriptions of data sources and methods in the annual bulletin, an Administrative Data Source Quality Report is published which discusses in depth the WDF system and the quality assurance arrangements at all stages of the production process, including the checks and validation carried out by local authorities, the Northern Ireland Environment Agency data control team and the statistics team.

Areas for improvement

• To enhance the usefulness of the Administrative Data Source Quality Report, a process map illustrating the flow of data through the system could be added. To help users contextualise the data, key quality and methods information, including limitations and caveats, could be added to the interactive dashboard and datasets.
• We encourage the team to explore the feasibility of developing a metric like SEPA’s carbon metric, which measures the whole-life carbon impact of waste. Such a measure would add insight on waste’s contribution to climate change in Northern Ireland.