Dear Lucy,

Statistics from the Adult Oral Health Survey in England

Thank you for inviting OSR to carry out an independent review of the suite of statistics from the Adult Oral Health Survey (AOHS) in England, following the transfer of publishing responsibility from NHS Digital to the Office for Health Improvement and Disparities (OHID), within the Department for Health and Social Care (DHSC).

We have reviewed the published data, the statistics report and the supporting documentation, which included the technical report. We also spoke to a range of interested stakeholders. We found that OHID has established processes to demonstrate that it is a trustworthy producer of statistics and we found a range of positive features that demonstrate the quality and value of the statistics. Based on our review, we are content that the statistics reports and metadata from the AOHS comply with the standards of trustworthiness, quality and value in the Code of Practice for Statistics and can remain accredited official statistics. Accredited official statistics are called National Statistics in the Statistics and Registration Service Act 2007.

The AOHS was designed as a continuation of the long-running Adult Dental Health Surveys, carried out since 1968. Whilst the survey was commissioned in 2019, it could not be rolled out as expected because the COVID-19 pandemic forced a swift change in the design of the survey. Accordingly, the AOHS was carried out as a web and paper survey in 2021, with no oral examination element, and the first suite of outputs was published in December 2022. The statistics team has been transparent about the transfer and publications clearly highlight that the responsibility for publishing the statistics has transferred from NHS Digital to OHID. It is helpful for users that links are provided to previous adult dental survey statistics published by NHS Digital.

Both the statistics report and the technical report provide information on the changes to the survey methodology since the 2009 output, and the reasons why, and include appropriate caveats about trend analyses. Uncertainty around the survey estimates is indicated in some of the data in the statistics report, as it provides whisker charts at the geographic level of England, with bars to indicate the 95% confidence intervals. The commitment to publish confidence intervals on all the charts in future reports will help users to understand the uncertainty around other estimates too. Users we spoke to felt that the technical report clearly explained the methodology, the achieved sample size and was transparent about the limitations of the results. Users also told us that they would like to see the actual survey questionnaire to assist with their research, which we understand will be published alongside the next release on 23 November.

We spoke to a range of users, for example dental care commissioners, public health officials, academics, lobby groups and dental professionals. These users felt that this initial output was an important report that looked at the demographics of dental patients and how access to dental care had changed during the pandemic. These users also emphasised the unique nature of the survey and, therefore, placed a lot of value on the statistics from it. They used the data for analysis to inform adult oral public health policy, to commission and target health promotion activities, to perform academic research and teaching and for lobbying activities.

We also identified some ways in which the quality and value of the statistics could be enhanced, summarised in the bullets below:

• Whilst the technical report does make it clear that the change in survey mode has reduced comparability with previous surveys in the series, the statistics report itself does not. OHID should consider performing trend analysis where possible and indicate more clearly how the statistics can and cannot be compared over time.
• Quality assurance processes are not fully explained in the published Whilst the technical report highlights the quality assurance processes performed on the data by the survey contractor (National Centre for Social Research), it does not outline how quality is carried through to the publication of the data and the statistics reports. We understand that this information will be published alongside the next release on 23 November.
• Extensive user engagement was carried out during the development of the survey. However, engagement appears to have fallen away slightly and a few of the users we spoke to felt somewhat disengaged with current plans and activities around the survey. The statistics team will explore promoting the releases using social media, ensuring that users are notified about the planned publication dates and informed how to access the statistics reports and data.
• Users we spoke to are concerned that, whilst the topic of access to dental care during COVID-19 is important, it might quickly lose relevance. There has been a large gap since the 2009 survey, which included an oral examination element. We understand that the statistics team will engage with users to seek feedback on how to add more value to the survey data for them. For example, by linking survey and administrative datasets or by including information at more granular geographic levels. Once they are agreed, we consider that OHID’s plans for the next set of activities and statistics on adult oral health should be published.

Thank you to your team for its positive engagement during this review and we look forward to continuing to engage with you and your team. I am copying this letter to Jess Goshawk-Dumbrell, Head of Central Statistics at DHSC and Kate Jones, Consultant in Dental Public Health at OHID.

Yours sincerely

Mark Pont