Dear Roger,
Rapid Review of Statistics from the Scottish Victimisation Telephone Survey
I am writing to endorse the approach taken in producing statistics from the Scottish Victimisation Telephone Survey (SVTS) and to congratulate your team for their work to ensure that statistics on victimisation continue.
Statistics from the Scottish Government’s Scottish Crime and Justice Survey (SCJS) are vital for understanding the extent, prevalence, type and impact of crime experienced by individuals, households and society. Since the advent of the COVID-19 pandemic, your team has worked hard to establish a telephone survey as an additional collection to the face-to-face survey, which was suspended in March 2020 and therefore not able to report on crime during the pandemic. This telephone survey launched in September 2020 and data were collected over a six-week period. Given the changes in survey methodology, I support your decision to publish crime statistics based on data from the SVTS as experimental statistics.
My team has conducted a rapid regulatory review of these statistics. We have reviewed the extent to which they have been produced in accordance with the Code of Practice for Statistics Trustworthiness, Quality and Value pillars. Our findings are based on discussions with your team, as well as publicly available information and draft templates for the February 2021 SVTS publication (which did not contain any data) that your team was able to share with us.
We appreciate the openness with which your team has engaged with us and its clear desire to continue to provide and improve information about crime in Scotland during the pandemic. Our review identifies strengths of the telephone survey and areas we consider priorities for improvement should the SVTS continue after its initial publication.
Value
- We welcome this publication, which provides a snapshot of victimisation in Scotland during the COVID-19 pandemic. It is positive that the SCJS data collection for 2019/20 was not greatly affected by the pandemic and the latest statistics are expected to be published in March 2021. We recommend the team to publish its plans for the long-term future of the SCJS once the procurement process for the new SCJS is complete.
- Your team and today’s statistical publication are open about the fact that the content of the SVTS has been reduced compared to the face-to-face survey; this has been done to achieve an optimal length for telephone interviews, and because some questions on sensitive topics are inappropriate for telephone interviewing. It is good that the team was able to engage with users on the changes to the questions and have used the feedback to prioritise which gaps to focus on filling, should the STVS continue for longer than expected. For example, users were keen to have questions about computer fraud and it was good to hear that the team will fill this gap by doing a deep dive, analysing the volume of cybercrime in Police Recorded Crime (PRC) this year. We encourage the team to use other National Statistics data sets in this way to fill the gaps in the STVS and provide a coherent picture of crime and victimisation.
- We understand that, for ethical and safeguarding reasons, which we support, important gaps include a lack of data about; drug use, stalking and harassment, partner abuse and sexual victimisation (the sensitive topics which were inappropriate to ask respondents about over the phone). We recommend that consideration is given to using other sources such as PRC will help to fill these gaps until the team is able to develop a long-term plan.
- It was good to hear about the team’s engagement with colleagues from ONS and the Northern Ireland executive; this type of collaborative learning and knowledge sharing is important. We encourage the team to share any user feedback or lessons learnt from the publication of results from the STVS with the teams that are developing similar new statistics.
Quality
- The SVTS is based on re-contacting respondents interviewed in the previous two years of the SCJS. We were reassured to hear that your team has drawn on survey methodology experts within ONS and has taken into account expert advice from ScotCen Social Research and Ipsos MORI Scotland, the external organisations contracted to run the SCJS.
- We note that the survey sample (2,654) is smaller than the SCJS (5,500) and as a result the team will not be able to carry out analysis of victims experiences due to the lack of detailed breakdowns in the data. We appreciate therefore that the statistics will focus on a comparison between pre- and post-COVID crime volume, providing a snapshot of crime victimisation in Scotland during the pandemic. The team has also been able to report on multiple and repeat victimisation, providing granularity with demographic breakdowns along with area characteristics.
- We were pleased to hear, despite the smaller sample size, the statistics will be able to provide comparisons to the Telephone Crime Survey for England and Wales (TCSEW), on the perceptions and prevalence of crime questions. This will ensure the continued comparability and coherence of the crime surveys in Great Britain.
- Your team has provided clear information on the main changes to the survey and on the limitations of the SVTS. This includes which data can and cannot be compared to those from the ONS survey.Your team shared the template of the February publication and the draft detailed technical report which will be published alongside the statistics; we were pleased that this provides greater detail on the sample design, weighting and potential biases. Your team has also improved the technical report during the review, illustrating commitment to enhancing users’ ability to scrutinise the changes more fully.
Trustworthiness
- The team has been open and transparent about changes to the Crime Survey with users. Changes to the survey were first announced in March 2020, including the specifics on the suspension of face-to-face data collection and the effects on the SCJS. The team also held discussions on survey design ideas with key stakeholders, contractors, and Office for National Statistics (ONS) and Northern Ireland Research and Statistics Agency (NISRA) analysts, including a workshop revisiting the 2004 Scottish Crime and Victimisation Survey calibration exercise: a comparison of survey methodologies. Throughout this period the team has issued ScotStat communications and published a helpful Q&A document on the suspension and data collection.
As set out in our rapid review guidance you can include a statement in your methodology note such as “These statistics have been produced quickly in response to developing world events. The Office for Statistics Regulation, on behalf of the UK Statistics Authority, has reviewed them against several key aspects of the Code of Practice for Statistics and regards them as consistent with the Code’s pillars of Trustworthiness, Quality and Value.”
Thank you for team’s positive engagement throughout this review, we look forward to seeing how the crime survey develops post COVID-19.
I am copying this letter to Amy Wilson, Head of Justices Analytical Services; David Smith, Head of Safer Communities Analytical unit and Anna Saunders, Crime Survey Lead.
Yours sincerely
Ed Humpherson
Director General for Regulation