Dear John
Knife-Enabled Crime Statistics for England and Wales
As you are aware, we recently completed our compliance check of the Office for National Statistics (ONS) and Home Office’s knife-enabled crime statistics for England and Wales against the Code of Practice for Statistics. These are published in ONS’s Crime in England and Wales statistical bulletin. Our review focused on the new methodology developed by Home Office analysts to improve the way that knife crime is categorised.
Knife and sharp instrument offences (known as knife-enabled crime) are a high priority policy area for the UK Government, with several elements of the Government’s Beating crime plan being linked to efforts to reduce levels of knife crime in England and Wales. It is important that data on knife-enabled crime collected by police forces are of high quality and accurately reflect trends in this type of crime.
To provide additional context to the offence-based police recorded crime series, police add details to crime records, such as whether a crime involved a weapon or was a hate crime. These details are often manually added by a police officer or police staff by way of a flag (and are known as flagged collections). Concerns were raised by police forces with the Home Office about variable quality of the collections, including knife crime, due to the inconsistent application of the flags.
To improve the data quality of flagged collections, Home Office set up the National Data Quality Improvement Service (NDQIS). Knife crime is the first collection on which NDQIS was tested. NDQIS uses a computer-assisted classification tool to review crime records held by the police. It scans data fields, including free text fields, and examines them using a simple ruleset and bespoke data dictionary to determine whether an offence involved a knife or sharp instrument.
Home Office statisticians worked closely with police forces to develop, refine and implement the tool. The ruleset goes through a quality assurance process with rigorous testing to understand the impact changes have on the number of knife-enabled crimes. For instance, several police forces carried out a peer review on the classification. The quality of the data received from police forces are assured by the Home Office as detailed in the user guide.
The tool has standardised how knife-enabled crime is defined and recorded across police forces in England and Wales (so far, the tool has been rolled out to 37 out of 43 forces) and has led to knife-enabled crimes being more accurately categorised. This has improved the quality of both the input data and the output statistics. The statistical bulletin highlights that the aim is for this tool to be rolled out to all forces by early 2022. Until then, each set of quarterly figures is not directly comparable with those previously published, because data collection practices are not consistent. However, the back series data are adjusted for each quarterly publication, which allows users to make historical comparisons within that release.
We welcome that the team at Home Office is aiming to apply similar ruleset-based methods for other flagged collections such as domestic abuse, child sexual abuse and hate crime. This is a really good example of taking improvements and maximising their impact by applying them more widely. We look forward to hearing how the team progresses with rolling out the NDQIS tool and improving the data quality for these flagged collections.
We identified several ways in which we consider that the transparency around the new methodology and future developments should be enhanced:
- The NDQIS tool was developed by a third party, which makes it difficult for the public to fully scrutinise the model. The team at Home Office told us that NDQIS will be brought in-house within Home Office, which we welcome, as it should enable fuller public scrutiny. To demonstrate transparency and alert users and other stakeholders about future changes, we encourage you to publish and regularly update an NDQIS development plan. This will be important as NDQIS is rolled out to other flagged collections like domestic abuse that are of high public interest.
- The ONS methodology article gives a simple overview for general users on how the method works, but it lacks detail, which more technically experienced users may need to fully understand how the tool was created and how it works. It is important that users have enough information about methods to enable them to evaluate the appropriateness of the statistics in relation to their intended use. We encourage you to review your current publishing arrangements to ensure that this can happen.
- The methodology article only briefly mentions limitations and uncertainty, for example, when explaining the level of certainty that a crime involves a knife or sharp instrument. To help users interpret the statistics, a summary of the main limitations and the steps that were taken to minimise their impact should be added to the methodology article.
Thank you to your team and the team at Home Office for their positive engagement during this review. We look forward to continuing to engage with you and the Home Office, and we hope our findings inform the ongoing development of the knife-enabled crime statistics. Please get in touch if you would like to discuss any aspects of this letter further or if we can offer further assistance as these statistics continue to develop.
I’m copying this letter to Kevin Smith, Lead Statistician on Violent Crime at Home Office and John Flatley, Programme Director Crime Statistics Production and Analysis at Home Office.
Yours sincerely
Mark Pont
Assessment Programme Lead