Dear Glyn

Monthly Indicators from the National Survey for Wales

I am writing to endorse the approach you have taken to develop the National Survey Wales given the current COVID-19 situation. The primary purpose of the survey is to provide views of adults in Wales on a wide range of issues affecting them, which is key to monitor changes and make decisions. Your team has responded effectively to the COVID-19 pandemic in adapting the survey to produce monthly results using a different data collection method. I would like to congratulate everyone involved for their work to produce these valuable statistics in challenging circumstances and confirm that these statistics can be labelled as National Statistics as part of the National Survey suite of National Statistics outputs.

My team has conducted a rapid regulatory review of the published information. We have reviewed the extent to which they have been produced in accordance with the Code of Practice’s Trustworthiness, Quality and Value pillars, while taking account of the pressures you and your teams have faced to deliver timely statistics about an important topic. A summary of our findings is set out below and more detailed feedback has been provided to your team.

Value

• The primary purpose of the statistics is to provide views of adults in Wales on a wide range of issues affecting them. The statistics allow changes to be monitored and meet policy needs.
• Your team worked quickly to implement a different data collection method (from face-to-face to telephone) to produce these statistics.
• The survey questionnaire has been adapted to include new questions on coronavirus across the different topic areas: shopping habits; mortgage payments; getting GP appointments; employment; hours worked; and childcare impacts.
• You informed users of the change to the statistics through publishing an update on your website; users would benefit from this update forming part of the National Survey landing page to ensure it has a high level of visibility and the change is well understood.
• We welcome the ongoing dialogue the survey team has had with the sponsored bodies that co-fund the survey (Sport Wales, Arts Council of Wales and Natural Resources Wales) and policy teams within Welsh Government to ensure that the survey results could feed into making important decisions. Your plans to extend this engagement will help develop the survey over the coming months, and ensure appropriate insights are delivered to a wider range of users.
• The initial statistical report is clear, informative and easy for users to understand; we look forward to it evolving over the coming months as topics change and user needs are more fully understood.

Quality

• The new questions have been developed more quickly than usual. Welsh Government experts in questionnaire design and testing are leading the development of the survey questions to overcome the fact that questions can’t be tested as extensively as would usually happen. Referring to other research – such as the COVID-related questions on the Office for National Statistics’ (ONS) opinions survey – is also helpful to build in quality. The further cognitive testing that you have planned will in time offer further reassurances to users about quality.
• The survey contractor (ONS) has carried out a set of quality checks on the data. Alongside your own checks, this means they are produced to a level of quality that meets users’ needs.
• It is exceptional that the survey has achieved a high response rate in its first month. We consider that your plan to reduce the contacted sample for June and July is sensible, and it is good that you have committed to monitor the response rate over time in order to adjust the sampling arrangements as needed during the constantly changing situation.
• The quality report is thorough about methods, including explanations of strengths and limitations, confidence intervals and accuracy of the data that aid interpretation of the statistics.

Trustworthiness

• The survey data are collected in line with Government Statistical Service guidanceon data collection during the COVID-19 crisis.

We would like to thank the team for engaging with us during this review and we look forward to seeing these statistics develop as more data are collected on the impact of COVID-19.

Yours sincerely

Ed Humpherson

Director General for Regulation