UK Statistical System UK Statistics Authority Code of Practice
Compliance check

Ed Humpherson to Ian Lonsdale: Assessment of Statistics from the People and Nature Survey England

Dear Ian

Assessment of statistics from the People and Nature Survey for England

Following our assessment of statistics from the People and Nature Survey for England produced by Natural England, we set out four requirements that we judged that you needed to meet in order for these statistics to become accredited official statistics. Accredited official statistics are called National Statistics in the Statistics and Registration Service Act 2007.

We have independently reviewed the actions that your team has taken to address these requirements. On behalf of the Board of the UK Statistics Authority, I am pleased to confirm that they comply with the standards of trustworthiness, quality and value in the Code of Practice for Statistics and should be labelled accredited official statistics.

We have been impressed with your team’s engagement and willingness to address these requirements.

I am copying this letter to Lindsey Clothier, Deputy Head of Profession for Statistics at DEFRA, and to the team at Natural England – Tom Marshall, Caitlin Clark, and Sergio Milheiras.

Yours sincerely

Ed Humpherson
Director General for Regulation

 

Related Links:

Annex: Review of actions taken in response to Assessment Report 374: Statistics from the People and Nature Survey for England (PaNS), produced by Natural England

Ed Humpherson to Ian Lonsdale: Assessment of statistics from the People and Nature Survey for England

Assessment of compliance with the Code of Practice for Statistics: Statistics from the People and Nature Survey for England

Mark Pont to Sian Rasdale: Official Development Assistance Statistics

Dear Sian

Official Development Assistance Statistics

We recently completed our compliance check of FCDO’s Official Development Assistance (ODA) Statistics against the Code of Practice for Statistics. These are important statistics that measure aid flows by UK official agencies or their executive agencies (including but not limited to FCDO) to developing countries and multilateral organisations in line with the international definition of ODA. They include total UK spend on ODA, and as a proportion of Gross National Income (GNI) in the form of the ODA:GNI ratio. These statistics have been of significant public interest not least since 2020, when the UK government announced a temporary reduction in its target spend for ODA from 0.7 per cent of GNI to 0.5 per cent of GNI effective from 2021. More recently the statistics were referred to in the media with reporting of the FCDO’s ODA programme data for its international programme for the next two years.

Our review found that the statistics demonstrate widespread compliance with the Code of Practice and should continue to be designated as National Statistics. It is clear both from the published outputs and from talking to your team how much hard work has gone into the development of these statistics and how much ambition there is to further improve them.

We found a range of positive features, as well as some areas for improvement that we consider would enhance the trustworthiness, quality and value of the statistics:

  • We commend your team for its responsiveness to user feedback by developing a dashboard, which aims to provide users with much greater insight. The dashboard when finalised, will enable users to interact with the data will bring greater transparency of UK aid data.
  • It is also good that your team considered the public value that Gross Public Expenditure statistics provided, in response to user feedback which found these statistics no longer remained relevant or useful, discontinued their production.
  • The merger of the former Foreign and Commonwealth Office and the Department for International Development resulted in the need to bring together two separate reporting accounting systems and different methods used to estimate ODA-eligible administrative costs. After reviewing these methods, against the OECD Development Assistance Committee (DAC) directives, FCDO has introduced coherence in the reporting accounting systems and an interim method to calculate the ODA-eligible proportion of FCDO administration costs. We commend FCDO on the transparency of its interim methodology.
  • In 2018 OECD DAC introduced changes to the methodology used to calculate ODA, which affected the treatment of loans. To help the public understanding and address any misconceptions of what ODA includes, FCDO published clear technical notes to explain how ODA is calculated. However, to further improve the transparency, FCDO might publish additional information explaining to users where the OECD guidance can be applied directly and where it needs to be interpreted in the UK’s local context.
  • Using a new centralised financial information database, FCDO has recently updated information on data sources, showing the wide range of other government department administrative sources used to compile the ODA statistics. The new database will also bring further opportunities to improve data collection processes. To reassure users about the quality of these sources and improved data collection processes, FCDO should consider updating its Quality Assurance of Administrative Data (QAAD) using OSR’s QAAD toolkit to support this process.

Looking beyond Statistics on International Development:

  • In October 2022 the Independent Commission for Aid Impact (ICAI) raised four recommendations in its Transparency in UK Aid review, which included:
    1. Setting and applying standards for transparency to all aid portfolios (including arm’s-length bodies)
    2. Committing to achieving a standard of ‘very good’ in the Aid Transparency Index by 2024
    3. Resuming the publication of forward aid spending plans, cross departmental development results and country aid priorities
    4. Working with other donors to support greater use of International Aid Transparency Initiative (IATI) data.
  • In December 2022 FCDO published a response to the review, where it has:
    1. Rolled out of a single finance and HR IT system throughout the department, which has allowed all ODA programmes in the FCDO to be brought onto a single platform. This has allowed for a more unified transparency process, with systems to assess, approve and collate ODA programme data.
    2. Committed to achieving a standard of ‘very good’ in the Aid Transparency Index by 2024 and publish forward looking ODA allocations and is carrying out improvements to publications of country development strategies.
    3. Committed to publishing annual reports and accounts information that support accountability and describe programme allocations clearly.
    4. Committed to supporting greater use of IATI data across all recipient countries of ODA.
  • To further improve transparency, it would be helpful if FCDO published its plans on how it will address the publication of arm’s-length bodies information, along with a record of any significant decisions to continue, discontinue, adapt, or provide statistics through other means. These decisions should be supported and driven by user feedback.

We would like to thank you and your team for your positive engagement during this review. In order to continue complying with the Code, we ask that you report back to us with your plans for addressing our recommendations above by December 2023. Our Economy, Business and Trade domain will continue to engage with you and your team on progress in the coming months. Please do not hesitate to get in touch if you would like to discuss any aspects of this letter further.

I am copying this letter to Jane Casey, Head of ODA delivery, eligibility and reporting.

Yours sincerely

 

Mark Pont

Ed Humpherson to Scott Heald: Assessment of Accident and Emergency (A&E) Activity Statistics in Scotland

Dear Scott

Assessment of Accident and Emergency (A&E) Activity Statistics in Scotland

We have completed our assessment of the weekly A&E activity statistics produced by Public Health Scotland (PHS). We judge that these statistics can be designated as National Statistics once we have confirmed that the requirements set out in our report have been met. We have also carried out a compliance check of the monthly waiting times statistics and I am pleased to confirm that these statistics should continue to be designated as National Statistics. The weekly and monthly statistics are now available together on a new platform reporting statistics on A&E departments across Scotland. This platform is a positive and innovative development and welcomed by many users in terms of improved accessibility and functionality.

Waiting times continue to be of high public interest across the UK and I am grateful for the positive engagement from you and colleagues in the A&E statistics team throughout the assessment process. It was particularly encouraging how the team adopted a proactive approach and implemented many of our emerging recommendations during the development of the new platform. Of key importance is your ongoing work to ensure that users are aware of the differences in hospital site inclusions between the weekly and monthly statistics to support appropriate use of the statistics – for example, only the monthly statistics include all units and departments relevant to the Scottish Government target that 95% of people attending A&E should be seen, admitted, discharged or transferred within four hours of arrival. You publish some of the disaggregated datasets only monthly. This should be kept under review in case there is further user demand for that level of granularity on a more frequent basis.

We identified three requirements that would strengthen both sets of statistics: extending user engagement to a wider range of users, assisting users to make appropriate cross-UK comparisons, and consider ways to communicate uncertainty to further aid user interpretation of the statistics. Implementing these requirements will ensure that the statistics continue to meet the highest standards of the Code of Practice for Statistics. As discussed, we expect you to report back to us by September 2023 on how you have met these requirements.

I am copying this letter to Emma McNair, Information Consultant at Public Health Scotland.

Yours sincerely

Ed Humpherson
Director General for Regulation

 

Related Links:

Assessment of Accident and Emergency (A&E) Activity Statistics in Scotland

Scott Heald to Ed Humpherson: Assessment of Accident and Emergency (A&E) Activity Statistics in Scotland

Mark Pont to Steve Ellerd-Elliott: Benefit Sanctions statistics

Dear Steve

Benefit Sanctions statistics

We recently completed our compliance check of your Benefit Sanctions statistics against the Code of Practice for Statistics. Even though these statistics are still in development, they are important official statistics valued by users across government and those with an interest in the Benefits system. We have made several recommendations to support the continued development of these statistics.

We considered the Trustworthiness, Quality and Value of these statistics in relation to the Code and have appreciated the positive and constructive way that the team has engaged with us during our review.

Our review found many examples of good practice. We would particularly like to recognise the complexities around the development of these statistics and the re-instating of the currently suspended measures of legacy benefits sanctions, and the need to balance the pace of progress with making sure that the statistics are suitably robust.

The background information and guidance for the statistics is comprehensive and provides transparency around the issues with the data and methods and that those issues are being addressed. There are appropriate health warnings throughout the bulletin about making comparisons across benefits due to the different data sources used and methodological approaches. Making the data available for wider re-use through Stat-Xplore enables users to carry out their own further analysis and enhances the usefulness of the data.

From our conversations with your team, it is clear that a great deal of work is going on to re-instate the currently suspended measures and develop the published series further to add more value and meet user needs. This is while dealing with the limitations associated with the data being collected for operational reasons and not specifically for statistical production – for example, the data being held across several different systems and difficulties with accessing the data in a useable format. It was good to hear that regular conversations are happening between the statistics team and policy colleagues within DWP in particular to understand the context behind the year-on-year increases in reasons for sanctions decisions – an area flagged by users who would like to understand more around these reasons. The statistics team also told us that there has been a positive change in the way ethnicity data are collected during benefit applications which should enable ethnicity breakdowns to be published in future – which would address a data gap we have also seen identified by users.

However, it is difficult for users to see and appreciate this work happening as this has not been communicated to them. Sharing details of both activity to date and planned work with users would help them to understand the amount of effort that is going into making the Benefit Sanctions statistics as useful and useable as possible. When sharing your plans, you should also provide details of how you plan to prioritise work going forward so it is clear to users when developments may occur, which will also help manage their expectations.

As the plans to reinstate the suspended measures progress and with other valuable developments in the pipeline, we fully support the team’s plans to engage more widely with users beyond the regular user forums and stakeholder engagement that are already in place, and which are providing helpful channels for known users to provide feedback. Our user engagement guidance provides a useful framework to help plan these activities and this wider engagement can help inform any future decisions regarding these statistics that will help to ensure that they remain useful, relevant and continue to support understanding of this important area. The Government Analysis Function is also developing a User Support and Engagement Resource (USER) hub which contains helpful guidance on engaging with users.

In our review we identified a gap in the available quality assurance information, and we are pleased to hear from the team that it is following our Quality Assurance of Administrative Data (QAAD) guidance in order to provide both the team and users the necessary assurance about quality and to help users understand how the Benefit Sanctions data are collected and processed. The team has told us that it will publish this as soon as it is ready.

Our Labour Market and Welfare domain lead will continue to engage with you and your team on progress in the coming months. Please do not hesitate to get in touch if you would like to discuss any aspects of this letter further.

Yours sincerely

Mark Pont

Assessment Programme Lead

Mark Pont to Lucy Vickers: Adult Substance Misuse Treatment Statistics for England

We recently completed our compliance check of the Department for Health and Social Care’s (DHSC) Adult Substance Misuse Treatment statistics against the Code of Practice for Statistics. The statistics are published in a provisional monthly release supplemented with an annual report the following year. The statistics were designated as National Statistics in 2015 and provide valuable insight into adults receiving treatment for substance misuse and the characteristics of different substance users. I am pleased to confirm that these statistics should continue to be designated as National Statistics. 

We carried out this review because the responsibility for the production and publication of the statistics was transferred from Public Health England (PHE) to the Office for Health Improvement and Disparities (OHID) within DHSC in October 2021. 

Our compliance check found many positive features that demonstrate the trustworthiness, quality and value of the statistics. For example: 

  • These statistics are important for policy makers and service providers, at the local level and for England as a whole. The value of the statistics is enhanced by the commentary and context provided throughout the annual report – for example, a section is included in the statistics about the policy context, there is discussion of the impact of the coronavirus pandemic on the statistics and trends in characteristics since 2005/06 are presented. Furthermore, the statistics are published in a variety of formats to suit different users’ needs such as charts, data tables and the interactive ViewIt tool. 
  • The value of the statistics is also enhanced by the level of granularity of the different regions. The provisional monthly release provides a breakdown of trends at a local authority level, allowing users to understand their local area. 
  • It is great to hear that the team has a good understanding of users of its provisional monthly data. These users include treatment providers, clinicians, local authorities and the general public. It is also good to hear that the team consults with users every few years on planned changes to the underlying dataset. Engagement is an essential part of maintaining the relevance and usefulness of the statistics for users. 

We identified some areas that would further enhance the trustworthiness, quality, and value of these statistics, some of which the team was already considering: 

  • It is good to hear of the team’s plans to update the Quality and Methodology report published on the National Drug Treatment Monitoring System this year. The report is exceptionally detailed and provides good assurance to users regarding the quality of statistics. However, the report was last updated in 2020 so it is not clear whether there have been changes to quality or methodology that users should be aware of. We consider it would also help users’ understanding if you were to add a short summary of the essential aspects of the statistics’ quality and methods, such as any limitations or what is and is not included in the statistics, within the annual report. This would help support appropriate use of the statistics without having to access a different website.  
  • While the team has a good understanding of users of its monthly statistics, we support its plans to expand user engagement activities relating to the annual report. Gaining a better understanding of what users require from the annual publication over and above the monthly statistics and what users want regarding the presentation of the statistics would be helpful as you continue to develop them. 
  • Finally, it is good that the team has committed to make some small changes that will enhance the demonstration of the trustworthiness of the statistics. This includes adding contact details to the annual release, adding information about roles of individuals to the pre-release access list and providing greater clarity regarding the move from PHE to OHID.   

I would like to thank your team for their positive engagement with us during this review. Please do not hesitate to get in touch if you would like to discuss any aspects of this letter.  

Yours sincerely 

Mark Pont 

Assessment Programme Lead  

Mark Pont to Sarah Crofts: Compliance review of ONS’s Health Index for England

Dear Sarah 

Compliance review of ONS’s Health Index for England

We recently completed our compliance check of the Health Index for England against the  Code of Practice for Statistics. The Health Index is an important new development that provides an overall picture of health both for England as a whole and for regions and local authorities. The index is comprised of 56 indicators across a broad range of aspects of health within the three main domains of Healthy People, Healthy Lives and Healthy Places. This allows users to understand how health and its components has been declining or improving over time, and how it varies from area to area. The most recent release, published in November 2022, provides important insights into health in the earlier stages of the coronavirus (COVID-19) pandemic at national, regional and local authority level. 

Overall, our review found that the statistics demonstrate compliance with many areas of the Code of Practice. It is clear both from the published outputs and from talking to your team how much thought and hard work has gone into the development of the Index to date. We found a range of positive features, as well as some areas for improvement that we consider would enhance the trustworthiness, quality and value of the statistics. Addressing these areas will be beneficial as you work towards obtaining National Statistics designation for the Health Index. 

  • Both the main release and the local authority tool provide a good insight into how aspects of health have changed (or not) since the Health Index started in 2015. The explanations of the findings are clear, as well as possible reasons for some of the findings being provided. The improvements to the release for 2020 data reflect a lot of work from your team. 
  • It is good that you have collaborated with such a wide range of experts in the form of the Expert Advisory Group (EAG) in constructing the Health Index. We are also pleased that you have worked with The Alan Turing Institute to quality assure the Health Index methods, as well as having plans in place to develop the methodology further. This demonstrates a clear commitment both to ensuring that the methodology is sound, and to ongoing refinement of the methodology as you continue to develop the Index. We note that planned future developments include a health projections model and enabling local authorities to produce their own health indices. Your ongoing commitment to further developing the scope and utility of the Health Index is likely to benefit a wide range of users. 
  • It is evident from our conversations with your team how much consideration it has put into the various methodological aspects of the Index and how these might be taken forward, yet it is sometimes hard for users to see or appreciate this. Sharing details of both activities to date and planned work would help users to understand the amount of effort that is going into making the Index as useful and useable as possible. When sharing your plans, you should also provide details of how you plan to prioritise work going forward so it is clear to users when developments may occur. 
  • You should take into account uncertainty when presenting the statistics to help avoid over or under-interpreting. For example, you could explain changes more carefully, as there are some occasions where small differences are presented as though they are statistically significant, and some users might interpret these as being more meaningful than they really are. Your team told us that it has done a lot of work to try and understand how best to understand uncertainty in relation to the Index, and that it plans to continue to do so. It is fundamental to be clear to users about sources of uncertainty to help them in interpreting the findings appropriately. It would be helpful if you could bring together your work so far and publish, separately, some information on this and your plans for your users.  
  • Your team told us about its user engagement activities, and particularly the work that it has carried out at the local level to understand who uses the Index, what for, and what decisions it might inform. A notable example that the team told us about is specific NHS Trusts using the Health Index framework to develop a version at a lower geography to better understand and target health issues in their authorities. Expanding this excellent work to understand more about users and uses of the Index at national level and in the third sector will help to add further value. We welcome that your team is starting to develop its plans in this area.  
  • Although you have improved the timeliness of the most recent release compared to previous ones, it is still relatively out of date. It is good that you are looking to further improve the timeliness of the Health Index, potentially with an early provisional release, as well as exploring the trade-offs between timeliness and accuracy. Seeking users’ views on this possible provisional release as part of the planned user engagement work outlined above, would help you understand whether a more timely but possibly less accurate early release will be of benefit to them and make the Health Index more useful overall.  
  • Your team told us that it follows Reproducible Analytical Pipeline (RAP) principles in its work and gave strong examples of this. There are some areas where future developments could still be possible, such as automation of your quality assurance procedures or investigation of ways to semi-automate the factor analysis. We encourage you to keep this on your work programme and to continue to investigate where further improvements might be possible. 

 I would like to thank your team for their positive engagement on this review. Our Health and Social care domain will continue to engage with you and your team on progress in the coming months. Please do not hesitate to get in touch if you would like to discuss any aspects of this letter further.   

 Yours sincerely 

Mark Pont 

Assessment Programme Lead 

Mark Pont to John Marais: Statistics on modern slavery in the UK

Dear John 

Statistics on modern slavery in the UK

We recently completed our compliance check of your Modern slavery in the UK and Child victims of modern slavery in the UK compendium articles against the Code of Practice for Statistics. We carried out a complementary compliance check of Home Office’s National Referral Mechanism (NRM) statistics, as the articles identify the NRM statistics as the best available source of the number of potential modern slavery victims in the UK.  

Modern slavery is a term for all forms of human trafficking, slavery, servitude and forced labour. Measuring the prevalence of modern slavery is challenging because there is no single source or method that can accurately quantify the number of victims. Our review focused primarily on the value and coherence of the articles, looking at how they bring together and explain statistics and data from a range of sources.  

Both articles are standalone releases, and there are currently no plans to update them. Given the importance and public profile of modern slavery, and the insight generated by the articles, we encourage you to put in place plans to update them, particularly the modern slavery in the UK article. We identified a number of strengths as well as a couple of ways in which the value and quality of the articles could be enhanced.  

Both articles draw on a wide range of data sources to present a detailed picture of the scale and nature of modern slavery. They bring together official statistics from UK Government departments and the Devolved Administrations, with data which are not official statistics, from the Crown Prosecution Service, charities and support services. The statistical commentary is clear and accessible, with explanations of key trends in the data and reasons for changes over time. The modern slavery in the UK article contains a wealth of background information and context, including an excellent flow chart of the NRM process, through which potential victims are referred.   

We found a range of other positive features that demonstrate the value and quality of the statistics: 

  • The collaborative approach used to produce the articles is excellent. Your team worked with stakeholders across academia, government and the third sector to understand information needs, source data, and review data quality. The guest blog post by Professor Sir Bernard Silverman, published on the same day as the modern slavery in the UK article, is a good endorsement of the collaboration and the value of the articles. 
  • The articles are some of the few statistics outputs that bring together crime and justice data from England and Wales, Scotland and Northern Ireland. The driver for creating a UK view is the UN’s Sustainable Development Goals (SDG) indicators, several of which measure the UK’s progress towards combatting modern slavery. For some sources, data from UK countries cannot be combined due to differences in definitions and recording processes – for example, for modern slavery offences recorded by the police. Nonetheless, the articles provide the most comprehensive and coherent picture of modern slavery in the UK.  
  • The articles demonstrate a good awareness of the sensitive nature of modern slavery. Both articles contain a ‘Finding help’ section, with contact details of victim support services. The child victims article intersperses statistical commentary with quotes from victims. This is a powerful way of bringing the statistics to life and highlighting the impact of modern slavery on victims.   
  • Both articles are clear about the challenges of measuring modern slavery; in particular, the lack of a definitive source of data or suitable method to quantify the number of modern slavery victims in the UK. The articles are transparent about the nature of all data sources and their limitations, including the lack of comparability of data sources due to differences in coverage or data collection methods.  
  • The framework used to evaluate and categorise relevant data sources – each data source has been badged as a ‘legal’, ‘support’ or ‘awareness’ indicator – is sensible and helps users understand the complexity of the modern slavery data landscape. It is good that this framework was developed and tested with stakeholders.  
  • The accompanying article on Sourcing data on child victims of modern slavery gives a clear overview of how suitable data sources were identified and reviewed for the SDG indicators relating to modern slavery and children in the UK. 

We also identified two ways in which the quality and value of the statistics could be enhanced: 

  • To aid user understanding of the quality of all data sources, information should be added on the strengths and limitations of data which are not official statistics, including data from victim support services, helplines, and applications, and how data from these sources were reviewed and quality assured. We encourage the team to continue exploring other data and data sources that could add insight on modern slavery. 
  • Signposting between the two articles could be improved, particularly as the child victims’ article does not contain the same level of detail as the modern slavery in the UK article. We recognise that the child victims’ article was reduced in size to make it more accessible and user-friendly.   

Thank you to your team for its positive engagement during this review. We look forward to continuing to engage with your team. Please get in touch if you would like to discuss any aspects of this letter further or if we can offer further assistance as these statistics continue to develop. 

I am copying this letter to Meghan Elkin and Peter Jones, heads of the Centre for Crime and Justice at ONS; Tom Bucke, Head of Unit for Drugs, Exploitation and Abuse Analysis at Home Office; and Jon Simmons, Head of Profession for Statistics at Home Office. 

Yours sincerely 

Mark Pont 

Assessment Programme Lead 

 

Related Links:

Mark Pont to Jon Simmons: National Referral Mechanism and Duty to Notify statistics

Mark Pont to Jon Simmons: National Referral Mechanism and Duty to Notify statistics

Dear Jon 

National Referral Mechanism and Duty to Notify statistics 

We recently completed our compliance check of your National Referral Mechanism (NRM) and Duty to Notify statistics against the Code of Practice for Statistics. We carried out a complementary compliance check of the Office for National Statistics’ (ONS) Modern slavery in the UK and Child victims of modern slavery in the UK compendium articles, to help us understand the modern slavery data landscape and how the National Referral Mechanism (NRM) statistics fit into this landscape. 

Modern slavery is a term for all forms of human trafficking, slavery, servitude and forced labour. Measuring the prevalence of modern slavery is challenging because there is no single source or method that can accurately quantify the number of victims. The NRM currently provides the best available estimate of the number of potential victims in the UK. It is therefore a highly valuable data source on modern slavery. Potential victims that come to the attention of authorised ‘first responder’ organisations are referred to the NRM. NRM figures are known to be an undercount because many cases of modern slavery remain hidden or unreported.   

We found a range of positive features that demonstrate the value and quality of the statistics: 

  • Since taking over production of the NRM statistics from the National Crime Agency in 2019, Home Office has improved the presentation and accessibility of the statistics, for example, by adding commentary to the statistical bulletin and releasing the data tables in a reusable format. Information has been added on different aspects of the NRM process, such as the number of ‘conclusive grounds’ decisions, which adds insight on the proportion of referrals deemed by Home Office to be genuine cases of modern slavery. A user told us that they value the data on the Duty to Notify process, which was added to the bulletin in 2021. It is good that the statistics team has used internal and external peer review to drive improvements.  
  • The users we spoke to, and the letter we received from academic and sector users of modern slavery data, highlighted the importance of, and need for, more data on key areas of modern slavery, including immigration outcomes, criminal justice outcomes and support access. We therefore welcome the work your team is doing to link NRM data with immigration data. Data linkage is the only way to fill certain information gaps on modern slavery and immigration, such as the number of modern slavery victims that have applied for asylum, and the number of people coming to the UK via different routes. To maximise the value of the NRM data and generate further insight, we encourage your team to be ambitious in its data linkage plans, by exploring opportunities to link NRM data with a range of other data sources.  
  • The statistical bulletin is transparent about methods changes, such as the 2019 change to the recording of exploitation types. A caveat about the lack of comparability with previous data gives helpful advice to users. Making the caveat more prominent would further support appropriate use of the statistics.  
  • It is good that anonymised disaggregated NRM data are freely available to academic researchers through the UK Data Service. Home Office has also given some non-governmental organisations and members of the public access to the anonymised data. 

We also identified several ways in which the value and quality of the statistics could be enhanced: 

  • The insight of the statistical bulletin could be improved by adding a clearer narrative of victims’ journey through the NRM process. To help users visualise and better understand the NRM process, we encourage you to add a flow chart like the one in ONS’s Modern slavery in the UK article. While it is good that the bulletin discusses possible drivers of changes in the number NRM referrals over time, it should acknowledge that recent increases in the number of referrals may reflect changes in awareness and identification of potential victims by ‘first responder’ organisations, as argued in the ONS article. It should also explain why the NRM and Duty to Notify figures are likely to be undercounts, and that the true number of modern slavery victims may be higher.  
  • The statistics are not granular enough to meet some users’ needs. The users we spoke to said that they would welcome further breakdowns of the NRM referral data in the bulletin and data tables, including breakdowns by exploitation sub-type, age of victims, and location of exploitation. They also told us they would like to see outcome decisions broken down by UK and non-UK nationals. Expanding the datasets released via the UK Data Service would enhance their value too. We recommend that you engage directly with users to understand their needs regarding the release and presentation of the statistics. 
  • We heard about the challenges that academic users have faced in terms of permissions and access required for research involving the NRM. Difficulties with accessing modern slavery data were also highlighted by the former Independent Anti-Slavery Commissioner in her Annual Report 2021-2022. Improving data access for research would support better public understanding of modern slavery and allow greater scrutiny of the response to modern slavery. 
  • The bulletin contains limited information about the quality of the statistics. To assure users about the quality of data and data standards, and aid interpretation of the statistics, the quality and methods information should be significantly expanded. The information should cover the NRM live management information system, how data are recorded, the system’s strengths and limitations, and sources of bias in the data. It should discuss data accuracy and reliability, comparability and coherence, and other aspects of data quality. Given the variety of authorised ‘first responder’ organisations referring potential victims and sharing data, it should also explain how data are quality assured at all stages of the data collection and statistics production process. Our Quality Assurance of Administrative Data (QAAD) framework might be helpful for reviewing and reporting quality. 
  • The bulletin states that the statistics may differ from those in previous or future bulletins, due to updates to case files on the live management information system. The scale, nature and impact of revisions should be explained, to assure users about the consistency and comparability of the statistics.  
  • Modern slavery is a relatively data-rich area, as evidenced by ONS’s modern slavery in the UK compendium article. The NRM bulletin should signpost other data sources on modern slavery to help users easily find related statistics.  

Lastly, it is important that the integrity and credibility of the NRM statistics are not undermined. In a recent letter to Professor Jennifer Rubin, we highlighted the need for clarity and transparency around the evidence used to support public statements about modern slavery. We welcome the commitment in Professor Rubin’s response to ensuring that public statements are sourced from published statistics or other reliable evidence. 

Thank you to your team for its positive engagement during this review. We look forward to continuing to engage with your team. Please get in touch if you would like to discuss any aspects of this letter further or if we can offer further assistance as these statistics continue to develop. 

I am copying this letter to Tom Bucke, Head of Unit for Drugs, Exploitation and Abuse Analysis at Home Office; John Marais, Deputy Director Crime, Income and Wealth Division at ONS; and Meghan Elkin and Peter Jones, heads of the Centre for Crime and Justice at ONS. 

Yours sincerely 

Mark Pont 

Assessment Programme Lead 

 

Related Links:

Mark Pont to John Marais: Statistics on modern slavery in the UK (March 2023)

Maya Esslemont and Anna Powell-Smith to Ed Humpherson: Modern slavery data (October 2022)

Ed Humpherson to Jennifer Rubin: use of National Referral Mechanism statistics (December 2022)

Jennifer Rubin to Ed Humpherson: use of National Referral Mechanism statistics (January 2023)

Mark Pont to Stephanie Howarth: School Workforce Annual Census statistics

Dear Steph

School Workforce Annual Census statistics

We recently completed our Compliance Check against the Code of Practice for Statistics of the School Workforce Annual Census (SWAC) statistics. Our review primarily focused on the Value pillar and identified several strengths of the SWAC statistics as well as a number of ways in which we consider the quality and value of the statistics could be further enhanced:

  • The SWAC statistics provide valuable information on teachers and support staff in local authority maintained schools in Wales and are the only comprehensive source of information on teachers’ pay, meeting a vital policy need. We understand that you are planning on exploring the possibility of developing a teacher pay data series that is coherent with one or more of the other UK nations. We consider this to be a positive step that will enable users to build a cross-UK picture in this policy area and we look forward to the outcomes of this work.
  • The SWAC data are available in the Secure Anonymised Information Linkage (SAIL) Databank. This means that they can be linked to other data sources to provide valuable insights into the school workforce staff in Wales, greatly increasing the value of the statistics. One example of such work was the linking of the SWAC data to the shielded patient list (SPL) in 2020 which enabled analysis of the number of teachers and teaching assistants on the SPL by local authority as well as by school phase and medium. This analysis contributed to a strong evidence base for policy decisions relating to the reopening of schools during the COVID-19 pandemic. The Administrative Data Research (ADR) Wales Planned Programme of Work for 2022-2026 outlines several planned research areas using the SWAC data including: the role of Welsh language skills and usage in the teaching workforce, pay and progression differences in the teaching workforce, and the impact of school leadership on pupil outcomes. This work by ADR Wales will help inform policy, operational decisions and the legislative programme.
  • The team has a good understanding of the key users of the SWAC data and has engaged with them throughout the development of the statistics. We understand that you are currently working on extending your engagement beyond your key users. This will enable you to gather a wider range of views on the statistics which will help to maximise public value by ensuring that the statistics continue to meet the needs of users and potential users.
  • There are other similar data in this space, for example, statistics on the school workforce published by the Education Workforce Council (EWC). We encourage you to continue to investigate and explain any differences between the SWAC statistics and alternative data sources. We consider that a close working relationship with such organisations would benefit both the statistics and the users.
  • We welcome the use of technology and automated processes that are already in use in the collection and production of the SWAC statistics. We recommend that you consider incorporating principles of the Government Statistical Service’s Reproducible Analytical Pipelines (RAP) which will help you to ensure that your processes are of high quality, transparent and efficient.

I would like to thank you and your colleagues for their positive engagement on this review and hope the points outlined above are beneficial in the further development of the SWAC statistics. Please do not hesitate to get in touch if you would like to discuss any aspects of this letter further.

I am copying this letter to Gareth Thomas, Head of School Workforce Statistics, Welsh Government.

Yours sincerely

Mark Pont
Assessment Programme Lead

Mark Pont to Emma Hickman: Housing, private rental, and housing purchase affordability statistics

Dear Emma

Housing, private rental, and housing purchase affordability statistics

We recently completed our compliance check of your housing affordability, private rental affordability and housing purchase affordability statistics against the Code of Practice for Statistics. These are important official statistics that present affordability ratios based on housing costs and incomes at various geographies. They are important to a range of users to monitor housing trends and inform policy and other decisions. Such uses include the design of schemes to support first-time buyers onto the property ladder; to inform local authority house building targets; and commercial decisions that require an understanding of the demand for different types of housing. The statistics should also provide valuable insights to users in the context of the current cost of living crisis. This letter presents our findings, including feedback to consider alongside the planned development of these statistics.

Our review found a range of positive features that demonstrate the trustworthiness, quality, and value of the statistics:

  • The headline housing affordability statistics are well-established and used for local planning and housing policy. The statistics are produced to high standards and are well-presented, accessible and supported by a range of insightful visualisations. The new private rental and housing purchase affordability outputs provide further valuable insights into different aspects of housing affordability. The team should be commended for its efforts in using household income data to overcome known limitations of the headline series.
  • The team has established an effective quality management approach. This approach includes, for example, collaboration with analysts in the devolved administrations to obtain their expertise which provides additional reassurance around data quality ahead of the statistics publication. The approach is documented extensively and accessibly in the Housing affordability and Additional measures of affordability Quality and Methodology Information (QMI) reports.
  • Extensive engagement with producer teams across the UK to gain buy in and expertise while developing new outputs. The engagement includes that with the Department for Levelling Up, Housing and Communities and the devolved administrations, facilitated though the Government Statistical Service (GSS) cross-government housing and planning working group. The engagement has drawn on cross-government work on the coherence of income data sources to determine the most appropriate sources for developing private rental and housing purchase outputs. This has ensured that consistent income data sources have been used to measure housing affordability in the different UK countries and regions, wherever possible.
  • The team has a range of plans to further enhance the quality and value of the statistics:
  • to enhance value by developing new mortgage affordability estimates from administrative data based on the proportion of people’s income spent on mortgage repayments, and estimates housing affordability for small geographical areas. This will enable a greater understanding of the extent of mortgage affordability than has been available to date, and on variations in affordability at lower levels of geography.
  • to expand the coverage and the data sources used for the forthcoming private rental affordability statistics to include estimates for Wales and Northern Ireland for the first time this year, and potentially for Scotland in 2023. This will help to better inform relevant policies relating to private rented sector provision in these countries.
  • to automate statistical production processes to enhance quality and efficiency, starting with private rental affordability. This will speed up the production process by three months, improving the statistics’ relevance and value, and enable policy and decision makers to understand the latest trends sooner. We published areview into the use of Reproducible Analytical Pipelines (RAP) principles and overcoming barriers, which may be useful to the team as it considers a more automated approach.

We also identified areas that may enhance the trustworthiness, quality, and value of these statistics:

  • With recent increases in mortgage interest rates there is growing interest in mortgage affordability. The team has identified a potential mortgage data source produced by the Financial Conduct Authority (FCA), but has not yet been able to gain access to the data. We support the team’s ambitions to access these data which may enable the production of robust measures of mortgage affordability. These statistics will be of significant value to policymakers and others looking to understand the extent of recent mortgage interest rates changes on the affordability of mortgages for homeowners across the UK.
  • References and links between the three affordability outputs should be clear enough to enable users to be aware of the related estimates and navigate easily between them. There is also potential to provide more insight as to how these statistics fit into ONS’s work on the cost of living, to provide a coherent overall picture. This will enable users to understand the relevance of these statistics in this broader context.
  • Summary information about the team’s planned developments for these statistics is included in the GSS cross-government Housing and planning statistics work programme and the QMI reports. Users can provide feedback via a generic email link. However, more-proactive engagement with users to seek their feedback on the most recent developments would help to maximise the public value and usefulness of these statistics. The team told us that it is considering writing a blog on its forthcoming development work.
  • We note that not all future planned releases are on the ONS release calendar, for example, planned outputs on mortgage affordability or housing affordability for small areas. The team should ensure that these releases are added to support trustworthiness; to provide users with as much advance notice as possible; and to promote the potential value of the new statistics for answering users’ key questions.
  • The descriptions of limitations of the statistics in the QMI documents are comprehensive and helpful. A range of uses for the statistics is provided in the QMI reports. However, the bulletins could be clearer on the exact and potential uses for each series, so that users know when to use one measure over another. While it’s good to highlight the main caveats in the bulletins, we note that sometimes the volume of caveats presented might be seen to play down the usefulness of the statistics. We encourage you to consider how to best balance openness about limitations with clarity around the statistics value for answering the key questions.
  • There is potential to enhance the value of the published quality information in relation to the administrative data used, for example by reviewing data suppliers’ assurance processes for the sources provided in relation to the team’s use of these data, and communicating this to users. This will demonstrate transparency and further enable users’ appropriate interpretation of statistical quality. You may find our Quality Assurance of Administrative Data (QAAD) framework helpful.

I would like to thank your team for their positive engagement on this review. My team will continue to engage with your team as it develops these statistics. Please do not hesitate to get in touch if you would like to discuss any aspects of this letter further. I am copying this letter to Rachel Skentelbery, Deputy Head of Profession for Statistics at ONS, and to Tony Wilkins, Tim Pateman and Nick Richardson, the responsible statisticians.

Yours sincerely

Mark Pont
Assessment Programme Lead