Dear Chris
Assessment of National Study of Health and Wellbeing in England – Adult Psychiatric Morbidity Survey
Thank you for the correspondence outlining the actions that have been taken to address the requirements in Assessment Report number 326 on National Study of Health and Wellbeing in England – Adult Psychiatric Morbidity Survey. I am also grateful for the continued open and constructive dialogue about the related Autism Spectrum Disorders statistics. I have considered the two sets of statistics in turn below.
National Study of Health and Wellbeing in England – Adult Psychiatric Morbidity Survey
On behalf of the Board of the Statistics Authority, I have reviewed these on the advice of the Assessment team and I am pleased to confirm the continued designation of National Study of Health and Wellbeing in England – Adult Psychiatric Morbidity Survey as National Statistics. The statistical team in NHS Digital and its contracted partners, NatCen, have shown a positive attitude towards the assessment process, viewing it as an opportunity to improve the statistics. I particularly welcome the changes that NHS Digital will introduce when it publishes the 2014 statistics on 29 September 2016 to make the statistics more accessible – for example: by presenting topic chapters separately so that users can pick the topics of interest to them and share them more widely; and by making the statistics available for re-use in a non-proprietary format. Other welcome additions are the new-style summary that provides some overall insight about mental health trends and treatments, and the helpful information about the quality of the statistics, including any uncertainty around the estimates.
I think that the improved overall transparency of the methods and strengths and limitations of the statistics should provide for even greater user confidence. I was also pleased to hear that you are taking steps to engage users early in the design of future surveys of adult mental health, and that you will be issuing your engagement plans alongside the published statistics.
National Statistics status means that official statistics meet the highest standards of trustworthiness, quality and value and is something to be celebrated. We invite you to include a statement alongside the statistics which reflects the National Statistics status. You may wish to base your statement on the boilerplate text included in the assessment report.
As you know, because these statistics are published only every seven years, the Assessment team’s advice has in large part been made on the basis of restricted access to draft NHS Digital documents and commitments made by your team. When the final statistics are published on 29 September my team will review the materials to ensure that they are consistent with the draft evidence. If, in the unlikely event that any of these are inconsistent with the Code of Practice for Official Statistics, I will contact you at the earliest opportunity.
Autism Spectrum Disorders Statistics
We have considered carefully the issues around the autism spectrum disorder statistics. We applaud you for commissioning and overseeing groundbreaking research to fill a gap in the understanding of the prevalence of autism in the adult household population. We note the difficulty in trying to measure a phenomenon within a household survey context that is formally determined by clinical diagnosis, especially when the prevalence rate is low.
Consequently we commend you for the way that you have engaged experts who have been innovative in developing and adapting tools that could be used to collect data on autism prevalence as part of a household survey, and for the professional way that they have gone about this work. We also praise their diligence in documenting the ways that they have developed these tools within peer-reviewed academic journals.
Following our discussions back in May, we have considered whether the status of these estimates as Experimental Statistics is merited. First, we note that while the overall estimate is the best available, and that you have clearly set out confidence intervals as an indication of the uncertainty in the estimates arising because of sampling, this estimate is based on a small number of positive cases identified within the sample, and has a relatively wide confidence interval. Second, we judge that the methods used to produce the estimates within this context would benefit from a wider degree of assurance from within the scientific and clinical community to offer the highest assurance of quality, and a clearer articulation of the impact of any non-sampling errors on these estimates. I consider that the estimates are of public value as a general indicator of prevalence of autism within the adult household population, and therefore a strong foundation to be able to discuss their development further.
I consider that they are perhaps less useful for decision making where a more precise estimate would be needed. I conclude therefore that at this stage, APMS statistics on autism should be regarded as Experimental Statistics, and not be designated as National Statistics.
The Code of Practice for Official Statistics describes Experimental Statistics as those that “are published in order to involve users and stakeholders in their development and as a means to build in quality at an early stage.” This labelling reflects the statistics’ status as having real potential value and gives you a strong platform to engage with a wider community of scientific and clinical experts with a direct interest in autism to consider the further development of these statistics. As part of that development, it may be helpful to consider what existing administrative data from within clinical settings could be used either in
the production or triangulation of statistics on the prevalence of autism.
Please feel free to discuss any aspect of this with us at any time. I am copying this letter to John Pullinger, the National Statistician, to Noel Gordon, the Chair of NHS Digital, and to Claire Thompson, Statistical Governance Manager at NHS Digital.
Yours sincerely
Ed Humpherson