Mark Pont to Ole Black: Statistics on Internet Access and Use

Dear Ole

Thank you to your team for engaging with us as we discussed the findings of our recent review of ONS’s internet-related statistics, published in two streams as: Internet access – households and individuals, Great Britain and Internet Users, UK.

We recognise the importance of having high quality statistics about internet access and use which serve the public good. We welcome the methodological improvements made relating to the Internet Access statistics with the transformation from face-to-face to a mixed-mode design. It is good that this generated a sufficiently large returned sample that the statistics for 2020 were robust enough not to be affected by COVID-19.

Our review found that ONS’s internet statistics do not cover key topics such as the speed and equality of broadband access, and provision of fibre broadband. In response to our review and the challenges presented by the COVID-19 pandemic, we welcome your commitment to pause and review your stakeholder requirements for statistics on the digital economy, including access and use of the internet. This provides a great opportunity to work collaboratively with others, and look to harness the power of alternative data sources to deliver greater insights. We look forward to ONS engaging and working with stakeholders to develop these statistics.

I have copied this letter to Phil Davies in your team.

Yours sincerely

 

Mark Pont

Assessment Programme Lead

Ed Humpherson to Sir Ian Diamond: Assessment of Business Demography Statistics

Dear Sir Ian

Assessment of Business Demography Statistics

We have completed our assessment of ONS’s business demography statistics. I am grateful for the positive contribution from you and your team throughout the assessment.

Our report suggests that a range of improvements is needed for these statistics to fully meet National Statistics standards. We will formally review whether the statistics merit their continued designation as National Statistics at two key points.

First, upon publication of the next set of annual business demography statistics, we expect ONS to have made some short-term gains. Second, ONS must publish a detailed plan, explaining how it will address the improvements identified in this report, including plans for reviewing the funding of the Statistical Business Register.

Business demography statistics are crucial for understanding UK business dynamics, especially as the economy emerges from the COVID-19 pandemic. I am impressed with ONS’s innovative work to develop, in partnership with Companies House, a weekly indicator of business births and deaths to track the impact of the COVID-19 pandemic. The subsequent publication of experimental quarterly business demography statistics has further demonstrated ONS’s ability to make a sizeable shift in its data offering. In making the improvements needed, I hope that ONS can build on its work over the last few months.

ONS has an amazing, yet largely untapped, resource in the Inter-Departmental Business Register (IDBR) that with development could help business demography statistics achieve the status of leading economic indicators, as they have in the USA and France. Part of untapping the value of IDBR data is making them available, subject to appropriate safeguards around confidentiality. We note that the legislation under which much IDBR data are collected and held – the Statistics of Trade Act – is over 70 years old. We would support any efforts that ONS were to make to review this legislation.

I note that organisational responsibility for the IDBR is separate from the rest of economic statistics. This has resulted in the IDBR not receiving the corporate priority that it clearly needs, nor playing as integral a role in economic statistics as it should. In responding to the requirements in our assessment, I encourage ONS to review these responsibilities and ensure that the IDBR can sit at the heart of economic statistics production.

Yours sincerely

 

Ed Humpherson

Director General for Regulation

 

Related Links

Assessment Report: UK Business Demography Statistics

Ed Humpherson to Iain Bell: Compliance check of ONS Overseas Travel and Tourism statistics

Dear Iain

Compliance check of ONS Overseas Travel and Tourism statistics

I am writing about the compliance check of ONS Overseas Travel and Tourism statistics which we carried out last year. Our review focused on two issues: the sample size of the survey used to calculate the statistics and how the tourism team engaged with users. We set out several requirements that ONS was to meet by July 2020 to ensure compliance with the Code of Practice for Statistics. These included improvements to user engagement and publishing better information about quality and methods.

I would like to commend the hard work your team has carried out since our review. Taking into account the impact of COVID-19 on the International Passenger Survey, which provides the data for these statistics, we consider that ONS has demonstrated great progress in meeting our requirements. Developing new ways to engage with users has resulted in ONS reaching and understanding a broader group of users – for example hosting user events and seminars, carrying out an online survey and distributing a regular newsletter. Implementing an improved methodology to deal with the known underestimates for visitors from some countries means that users have more accurate information. Users are also now better supported in understanding the statistics and their limitations thanks to your published guidance on interpreting confidence intervals. I welcome your commitment to ensure the longevity and sustainability of these improvements.

I would also like to welcome your review of tourism statistics which you plan to carry out over the next six months and support your plans to explore additional sources to further enhance the scope and value of these statistics. I encourage you to ensure that the approach to, and findings of, the review are open and transparent to users. Given that COVID-19 led to suspension of International Passenger Survey operations in mid-March, I welcome your team’s efforts to publish tourism statistics for the affected period. This approach meets user needs as far as possible, while acknowledging the impact on quality through temporary suspension of National Statistics status.

We appreciate the openness and enthusiasm with which your team has engaged with us over the last year and look forward to receiving further updates on the outcome of your review.

Yours sincerely

 

Ed Humpherson

Director General for Regulation

Ed Humpherson to Iain Bell: Assessment of 2021 Censuses in the UK – ONS Response to Preliminary Findings

Dear Iain

Assessment of 2021 Censuses in the UK – Office for National Statistics (ONS) Response to Preliminary Findings

We have reviewed the actions that your team has taken to address the findings in Assessment Report 345: 2021 Censuses in the UK – Preliminary Findings. These findings aim to build on the existing work being undertaken by Census offices, providing further direction and focus on pre-existing plans.

The Office for Statistics Regulation’s (OSR’s) advice on the National Statistics designation will be based on the trustworthiness, quality and value secured by Census offices. That advice will take account of the actions taken to address findings or requirements from OSR as part of this assessment process. While many of the Phase 1 findings can only be fully actioned or addressed over longer timescales, it is my view that ONS has responded positively to the assessment findings so far. ONS has been open to challenge from the assessment team and I am encouraged that improvements have been made in a number of areas. The Census is one of the most important sources of official statistics and I recognise the ongoing commitment of ONS to deliver high quality data and statistics; recognising the challenges and changes in ways of working that have had to be managed during the COVID-19 pandemic.

Since we published our assessment report in October 2019, my team has engaged with various members of your staff working on the Census to understand the improvements you have made since then. I welcome the openness and flexibility that has been shown to the assessment team. In evaluating how ONS has responded to our findings we have taken account of published materials from ONS including the evidence report Response to actionable findings from Phase 1 of the National Statistics Accreditation, alongside information gathered through meetings and workshops held with your teams. I consider that some actions taken by ONS are of note.

  • My team held a number of workshops with staff who have been working to develop methods, on areas such as item imputation and census coverage adjustment. We consider that ONS is taking a rigorous approach to its methods development. We are very pleased to see that methods papers, which have been reviewed by your external Methodological Assurance Review panel, are now in the public domain. This is a significant improvement to aid transparency and share knowledge and expertise with the wider community of expert users and stakeholders.
  • Some findings from our assessment report involved making improvements to recognise the needs of UK Census data users. ONS has taken a lead role in addressing these findings and has established a UK Census data working group – with representation from across the Census offices; we recognise the commitment and energy of ONS staff working in this area.
  • The questions to be asked in the Census in 2021 have been finalised. However, we are aware that some stakeholders have outstanding concerns with the guidance on the Census sex question. ONS has continued to engage with users and stakeholders about this and we recognise the efforts being made to provide answers or explanations on areas of concern. ONS should seek to address outstanding concerns raised by users within its further question testing and research on the guidance on the sex question. ONS should share the outcomes of this research in a transparent and open way.

We consider that further action is still needed in some areas identified for improvement in our assessment report – this is partly due to changing circumstances such as the impacts on programme delivery given the pandemic. For example, while ONS has plans to make improvements to its website, to date there has been little change made to make it more accessible. Key documents and materials are not easy to find and navigation on the webpages is not intuitive. ONS should implement its plans over the coming months to improve the accessibility of research and other Census information on its website.

In the assessment report the need to be clear to users on the impacts of country-specific decision making on UK Census data and statistics. Due to the decision by Scottish Ministers to delay the Census in Scotland, it is even more important ONS work with the other Census offices to explain the impacts of this to UK Census data users.

From speaking with your teams, it is clear ONS recognises the importance of communicating to England and Wales Census data users about delivering its Census in 2021 and on its scheduled plans for Census outputs. Users should be kept up to date and be provided with further detail as more information on the timing and content of statistical outputs becomes available.

We have included more detail about our judgement in an annex to this letter (See PDF download). I, or my team, would be happy to talk you or your colleagues through any aspects of this letter or Code compliance more generally.

I am copying this letter to Sir Ian Diamond, National Statistician and Stephanie Howarth, Interim Chief Statistician, Welsh Government.

Yours sincerely

Ed Humpherson
Director General for Regulation

Ed Humpherson to Iain Bell: Telephone-operated crime survey for England and Wales

Dear Iain

Telephone-operated Crime Survey for England and Wales

Thank you for inviting my team to review your newly instated Telephone-operated Crime Survey for England and Wales (TCSEW). Statistics from the Office for National Statistics (ONS) Crime Survey are vital for understanding the prevalence, type and impact of crime experienced by individuals, households and society. I am writing to endorse the approach taken to adapting the Crime Survey and to congratulate your team for their work to ensure that it continues to run.

Until March 2020, the Crime Survey for England and Wales (CSEW) was a face-to-face survey. Since the advent of the COVID-19 pandemic, your team has worked hard to establish a telephone survey to replace the face-to-face survey. This telephone survey launched in May 2020. Given the changes in survey methodology, I support your decision to publish crime statistics based on data from the TCSEW as experimental statistics.

Our rapid review has focused on the extent to which the survey is being run in line with the expectations set out in the Code of Practice for Statistics. Our findings are based on discussions with your team, as well as publicly available information and draft templates for the August crime statistics publication (which did not contain any data) that your team was able to share with us.

We appreciate the openness with which your team has engaged with us and its clear desire to continue to provide and improve information about crime in England and Wales during the pandemic. Our review identifies strengths of the telephone survey and areas we consider priorities for improvement in crime statistics over coming months.

 

Trustworthiness

  • ONS has been proactive and open in public communications about changes to the Crime Survey. Changes to the survey were first announced alongside crime statistics published in April, then more information about the telephone survey and the statistics it would provide were published in July in a progress update to the published plan for improving for crime statistics. The crime statistics team also published a blog yesterday (25 August), which explains, in a very accessible way, what to expect from today’s release of crime statistics based on the TCSEW. We consider that these communications will have helped users of the survey to understand the changes taking place and the strengths and limitations of the new statistics. We would encourage the team to continue to take opportunities to share and invite comment on its work, and we are pleased to hear that the team is planning an event with the Royal Statistical Society to talk about the survey.
  • More broadly, we consider the published plan for improving crime statistics to be very good – it is thorough and kept up to date. It provides users with helpful insights into developments being made to the statistics and about ONS’s progress. To make it easier for users of the statistics to know how they can be involved to help prioritise statistical plans, we would encourage your team to explain in the plan how they engage with users and how users can get involved. We are pleased that your team agreed to make the plan more visible, by referring and linking to it directly in the August publication of the statistics.

Quality

  • The TCSEW is based on re-contacting respondents interviewed in the previous two years on the CSEW. We were reassured to hear that your team has drawn on survey methodology experts within ONS to help it design the new survey. In addition, the team has taken into account expert advice from Kantar Public, the external organisation contracted to run the Crime Survey.
  • Because of the limitations of the telephone survey sample, the survey method will only deliver estimates of appropriate quality for a period of up to nine months. Therefore, it is good that the team is already thinking about how it can continue a telephone survey after this period, which ends in January 2021, if a return to face-to-face interviews has not been possible. The team is working closely with other areas of ONS that are testing alternative telephone sampling methods and we have confidence that these steps will ensure the TCSEW is able to continue as long as it is needed. In the longer term, your team plans to develop a programme of research to consider the optimal survey delivery mode for the Crime Survey (face-to-face, telephone or online).
  • We are really pleased to hear that response rates to the TCSEW so far have been very good, and higher than expected. This means the team is confident that estimates from the survey will enable users to continue to understand the prevalence of crime in England and Wales. Your team has committed to monitoring the response rate over time to ensure this continues to be the case.
  • Having seen a template of the August publication, your team has provided brief, but clear information on the main changes to the survey and on the limitations of the TCSEW. This includes that data cannot be compared to previous estimates from the face-to-face CSEW, and that there are some data gaps. We encourage ONS to ensure that technical information about the survey sample design, response rates, weighting and any potential biases is also sufficiently accessible and detailed to support public scrutiny. We understand the team plans to publish this information alongside the next release of the statistics, in October.

 Value

  • The statistics released today are outside of the normal quarterly crime statistics release schedule. We welcome this publication from your team, which introduces the TCSEW ahead of the next planned publication in October, and which means statistics about crime in England and Wales during lockdown are available in the public domain as early as possible.
  • Your team and today’s statistical publication are open about the fact that the content of the TCSEW has been reduced compared to the face-to-face survey: this has been to achieve an optimal length for telephone interviews, and because some questions on sensitive topics are inappropriate for telephone interviewing. While extensive user consultation about the questionnaire was not possible due to time constraints, we are pleased to hear that ONS worked with key users of crime statistics across government to help identify priority topics for the survey and maintained these where possible.
  • This consideration of user needs is evident in the inclusion of a short COVID-19 related module of questions. Currently this module includes questions relating to perceptions of crime, the police, and anti-social behaviour during the pandemic, but it is a flexible module, which will be reviewed each month. This will ensure questions can remain relevant and better support understanding of the ongoing impact of COVID-19.
  • There are some important gaps in data and statistics from the TCSEW. Of particular note is the unavailability of statistics about sexual assault, partner abuse, abuse during childhood and the preferred measure of domestic abuse and domestic violence. It is good to hear that the team is already planning to publish an article focusing on domestic abuse during the pandemic, and that the team is discussing with a range of stakeholders how it can best integrate data into this article from other organisations.
  • Another key data gap compared to the CSEW is data for 10 to 17 year-olds, which will no longer be available because it is not feasible to conduct telephone interviews with this age group at this time. The team has included proxy questions for 10 to 15 year-olds as part of the COVID-19 question module. These questions will collect information on online activities in children during the pandemic, including time spent online and any negative experiences while online (whether they have been threatened, verbally abused, or had rumours spread about them). While it is good to have these data, it will be important for the team to gather feedback from users, so that it can continue to refine its understanding of user needs and priorities for filling evidence gaps. These findings should help shape developments to the statistics.
  • Finally, we welcome ONS’s engagement with colleagues from the devolved administrations, particularly Scottish Government, which is now taking a similar approach to measuring crime in Scotland, having had to pause its own face-to-face survey (the Scottish Crime and Justice Survey) in March. We would encourage ONS to share any user feedback or lessons learnt from the first publication of results from the TCSEW among the teams that are developing similar new statistics.

As set out in our rapid review guidance you can include a statement in your methodology note such as “These statistics have been produced quickly in response to developing world events. The Office for Statistics Regulation, on behalf of the UK Statistics Authority, has reviewed them against several key aspects of the Code of Practice for Statistics and regards them as consistent with the Code’s pillars of Trustworthiness, Quality and Value.”

We will continue to engage with your team and look forward to seeing these statistics continue to develop.

I am copying this letter to Liz McKeown, Director of Public Policy Analysis in ONS, John Marais, Deputy Director of Crime, Income & Wealth Division in ONS, and Daniel Shaw, Head of Profession for Statistics at Home Office.

Yours sincerely

 

Ed Humpherson

Director General for Regulation

 

Iain Bell to Ed Humpherson: Covid-19 infection survey publication time

Dear Ed,

We continue to review many aspects of statistical production and publication with regards to informing measures to respond COVID-19.

The latest results of the COVID-19 Infection Survey were published at 11am today, brought forward from the originally scheduled 12pm publication time. This was discussed and agreed between ONS and the OSR before the publication. The decision was made as part of understanding emerging operational requirements for the latest evidence on COVID-19 infection rates and incidence.

We anticipate that forthcoming weekly publications of results from the COVID-19 Infection Survey will be at 12pm.

Yours sincerely

 

Iain Bell

Deputy National Statistician and Director General, Population and Public Policy

Office for National Statistics

 

Related Links

Ed Humpherson response to Iain Bell: Covid-19 infection survey publication time

Ed Humpherson response to Peter Benton: Temporary suspension of National Statistics designation for overseas travel and tourism statistics

Dear Pete 

 TEMPORARY SUSPENSION OF NATIONAL STATISTICS DESIGNATION: OVERSEAS TRAVEL AND TOURISM STATISTICS 

Thank you for your letter of 22 July, regarding statistics on overseas travel and tourismGiven the suspension of the International Passenger Survey in mid-March which provides the underlying data for these statistics, I support your decision to temporarily remove the National Statistics designation. 

I welcome your plans to publish data for the first quarter of 2020 later this weekwhich will be highly valued by users. I encourage you to ensure that the impact of the missing March data on the quality and interpretation of the statistics is communicated clearly. am pleased to hear that you have been engaging with users about the impact of COVID-19 on these statistics and encourage you to continue this over the coming months as you plan for the safe return of the survey. 

We will continue to engage with your statisticians, via our Travel, Transport and Tourism team, and look forward to considering the restoration of National Statistics status once data collection has resumed and you are assured the statistics can meet the standards expected by the Code of Practice for Statistics. 

 Yours sincerely 

 

Ed Humpherson 

Director General for Regulation 

 

Related Links

Peter Benton to Ed Humpherson: Temporary suspension of National Statistics designation for overseas travel and tourism statistics

Peter Benton to Ed Humpherson: Temporary suspension of National Statistics designation for overseas travel and tourism statistics

Dear Ed,

I am writing to ask you to consider a temporary suspension of the National Statistics status for the international travel and tourism estimates based on the International Passenger Survey (IPS).

The IPS stopped interviewing on 16th March 2020 due to safety concerns arising from the Covid-19 situation. It has been possible to produce estimates both for March and for the first quarter of 2020, but we are unable to confirm that the estimates fully meet the required quality standards demanded of National Statistics.

This is partly because the volume of data is greatly reduced for March. Also, we cannot be certain that the data obtained for March are representative: that the characteristics of passengers interviewed in the first half of the month represent those travelling in the second half of the month. Traveller numbers were decreasing at this time but had not yet fallen to the extremely low numbers seen from April onwards.

We have investigated the impact that the lack of data for the last two weeks of March will have on the Q1 2020 and March 2020 estimates. The data is weighted to the total traffic numbers provided at a monthly and quarterly level by the Civil Aviation Authority (CAA) and the Department for Transport (DfT). Therefore, we are weighting an incomplete dataset to the complete traffic figures which are not suitable for weighting at a weekly level. Travel started to be significantly affected towards the end of March as COVID-19 restrictions started to come into force. Hence, our investigation suggests that the IPS estimates for March and Q1 2020 have a degree of uncertainty that makes it preferable to suspend National Statistics status for this period. We have nonetheless made every effort to ensure that the estimates are of the highest possible quality in the circumstances.

Given the constraints on data collection experienced by the IPS, I am requesting temporary de-designation of travel and tourism statistics based on the survey on the basis set out by the published COVID-19 guidance on the suspension of National Statistics status.

Subject to your agreement, we propose to remove National Statistics labelling for March and Quarter 1 of 2020 outputs in the first instance. The survey’s output is suspended for the period from April 2020 until interviewing resumes. It may be advisable to continue the suspension of National Statistics status for a further limited period after interviewing resumes,depending on our assessment of the quality of data obtained.

Yours sincerely,

 

Peter Benton

Director of Population and Public Policy Operations, Office for National Statistics

 

Related Links

Ed Humpherson response to Peter Benton: Temporary suspension of National Statistics designation for overseas travel and tourism statistics

Ed Humpherson to Iain Bell: Coronavirus (Covid-19) Infection Survey statistics

Dear Iain

CORONAVIRUS (COVID-19) INFECTION SURVEY STATISTICS

Following conversations with your team, I am happy to confirm the exemption from the Code of Practice for Statistics’ standard publication time of 9.30am to permit a later release time of noon each Friday for statistics from the COVID-19 infection survey.

This new time allows additional time for preparing and assuring the quality of the release and messaging which is important as the complexity of the analysis grows. I welcome the team’s commitment to keep the release time under review as the survey continues to evolve.

Yours sincerely

 

Ed Humpherson

Director General for Regulation