Dear Iain

Telephone-operated Crime Survey for England and Wales

Thank you for inviting my team to review your newly instated Telephone-operated Crime Survey for England and Wales (TCSEW). Statistics from the Office for National Statistics (ONS) Crime Survey are vital for understanding the prevalence, type and impact of crime experienced by individuals, households and society. I am writing to endorse the approach taken to adapting the Crime Survey and to congratulate your team for their work to ensure that it continues to run.

Until March 2020, the Crime Survey for England and Wales (CSEW) was a face-to-face survey. Since the advent of the COVID-19 pandemic, your team has worked hard to establish a telephone survey to replace the face-to-face survey. This telephone survey launched in May 2020. Given the changes in survey methodology, I support your decision to publish crime statistics based on data from the TCSEW as experimental statistics.

Our rapid review has focused on the extent to which the survey is being run in line with the expectations set out in the Code of Practice for Statistics. Our findings are based on discussions with your team, as well as publicly available information and draft templates for the August crime statistics publication (which did not contain any data) that your team was able to share with us.

We appreciate the openness with which your team has engaged with us and its clear desire to continue to provide and improve information about crime in England and Wales during the pandemic. Our review identifies strengths of the telephone survey and areas we consider priorities for improvement in crime statistics over coming months.

 

Trustworthiness

• ONS has been proactive and open in public communications about changes to the Crime Survey. Changes to the survey were first announced alongside crime statistics published in April, then more information about the telephone survey and the statistics it would provide were published in July in a progress update to the published plan for improving for crime statistics. The crime statistics team also published a blog yesterday (25 August), which explains, in a very accessible way, what to expect from today’s release of crime statistics based on the TCSEW. We consider that these communications will have helped users of the survey to understand the changes taking place and the strengths and limitations of the new statistics. We would encourage the team to continue to take opportunities to share and invite comment on its work, and we are pleased to hear that the team is planning an event with the Royal Statistical Society to talk about the survey.
• More broadly, we consider the published plan for improving crime statistics to be very good – it is thorough and kept up to date. It provides users with helpful insights into developments being made to the statistics and about ONS’s progress. To make it easier for users of the statistics to know how they can be involved to help prioritise statistical plans, we would encourage your team to explain in the plan how they engage with users and how users can get involved. We are pleased that your team agreed to make the plan more visible, by referring and linking to it directly in the August publication of the statistics.

Quality

• The TCSEW is based on re-contacting respondents interviewed in the previous two years on the CSEW. We were reassured to hear that your team has drawn on survey methodology experts within ONS to help it design the new survey. In addition, the team has taken into account expert advice from Kantar Public, the external organisation contracted to run the Crime Survey.
• Because of the limitations of the telephone survey sample, the survey method will only deliver estimates of appropriate quality for a period of up to nine months. Therefore, it is good that the team is already thinking about how it can continue a telephone survey after this period, which ends in January 2021, if a return to face-to-face interviews has not been possible. The team is working closely with other areas of ONS that are testing alternative telephone sampling methods and we have confidence that these steps will ensure the TCSEW is able to continue as long as it is needed. In the longer term, your team plans to develop a programme of research to consider the optimal survey delivery mode for the Crime Survey (face-to-face, telephone or online).
• We are really pleased to hear that response rates to the TCSEW so far have been very good, and higher than expected. This means the team is confident that estimates from the survey will enable users to continue to understand the prevalence of crime in England and Wales. Your team has committed to monitoring the response rate over time to ensure this continues to be the case.
• Having seen a template of the August publication, your team has provided brief, but clear information on the main changes to the survey and on the limitations of the TCSEW. This includes that data cannot be compared to previous estimates from the face-to-face CSEW, and that there are some data gaps. We encourage ONS to ensure that technical information about the survey sample design, response rates, weighting and any potential biases is also sufficiently accessible and detailed to support public scrutiny. We understand the team plans to publish this information alongside the next release of the statistics, in October.

 Value

• The statistics released today are outside of the normal quarterly crime statistics release schedule. We welcome this publication from your team, which introduces the TCSEW ahead of the next planned publication in October, and which means statistics about crime in England and Wales during lockdown are available in the public domain as early as possible.
• Your team and today’s statistical publication are open about the fact that the content of the TCSEW has been reduced compared to the face-to-face survey: this has been to achieve an optimal length for telephone interviews, and because some questions on sensitive topics are inappropriate for telephone interviewing. While extensive user consultation about the questionnaire was not possible due to time constraints, we are pleased to hear that ONS worked with key users of crime statistics across government to help identify priority topics for the survey and maintained these where possible.
• This consideration of user needs is evident in the inclusion of a short COVID-19 related module of questions. Currently this module includes questions relating to perceptions of crime, the police, and anti-social behaviour during the pandemic, but it is a flexible module, which will be reviewed each month. This will ensure questions can remain relevant and better support understanding of the ongoing impact of COVID-19.
• There are some important gaps in data and statistics from the TCSEW. Of particular note is the unavailability of statistics about sexual assault, partner abuse, abuse during childhood and the preferred measure of domestic abuse and domestic violence. It is good to hear that the team is already planning to publish an article focusing on domestic abuse during the pandemic, and that the team is discussing with a range of stakeholders how it can best integrate data into this article from other organisations.
• Another key data gap compared to the CSEW is data for 10 to 17 year-olds, which will no longer be available because it is not feasible to conduct telephone interviews with this age group at this time. The team has included proxy questions for 10 to 15 year-olds as part of the COVID-19 question module. These questions will collect information on online activities in children during the pandemic, including time spent online and any negative experiences while online (whether they have been threatened, verbally abused, or had rumours spread about them). While it is good to have these data, it will be important for the team to gather feedback from users, so that it can continue to refine its understanding of user needs and priorities for filling evidence gaps. These findings should help shape developments to the statistics.
• Finally, we welcome ONS’s engagement with colleagues from the devolved administrations, particularly Scottish Government, which is now taking a similar approach to measuring crime in Scotland, having had to pause its own face-to-face survey (the Scottish Crime and Justice Survey) in March. We would encourage ONS to share any user feedback or lessons learnt from the first publication of results from the TCSEW among the teams that are developing similar new statistics.

As set out in our rapid review guidance you can include a statement in your methodology note such as “These statistics have been produced quickly in response to developing world events. The Office for Statistics Regulation, on behalf of the UK Statistics Authority, has reviewed them against several key aspects of the Code of Practice for Statistics and regards them as consistent with the Code’s pillars of Trustworthiness, Quality and Value.”

We will continue to engage with your team and look forward to seeing these statistics continue to develop.

I am copying this letter to Liz McKeown, Director of Public Policy Analysis in ONS, John Marais, Deputy Director of Crime, Income & Wealth Division in ONS, and Daniel Shaw, Head of Profession for Statistics at Home Office.

Yours sincerely

 

Ed Humpherson

Director General for Regulation