Dear Tony

UK Armed Forces mental health statistics

As you are aware, we recently completed our compliance check of the Ministry of Defence’s (MOD) UK Armed Forces mental health annual statistics against the Code of Practice for Statistics.

Mental health is receiving increasing attention from the public, media and government as an important public health issue. There is significant interest in the mental health of serving personnel in the UK Armed Forces, given the potentially challenging conditions in which they operate, for example, when deployed to places likes Iraq or Afghanistan. It is important to understand trends in the prevalence of mental health disorders, such as anxiety, depression and Post-Traumatic Stress Disorder (PTSD), among serving personnel, as well as the care they receive and the services they use.

We found a range of positive features that demonstrates the value and quality of the statistics:

• The statistical bulletin provides a detailed, clear, and insightful account of mental health in the UK Armed Forces. Where possible, rates of mental health disorders in the UK general population are presented alongside the rates for serving personnel. This puts the findings into context and helps users understand how representative the Armed Forces are relative to the wider population. The bulletin also comments on differences between groups of serving personnel, for example, the different branches of the Armed Forces. The flow chart at the start provides a helpful overview of mental health care and services.
• Your team engages with relevant groups outside of MOD, including mental health academics, charities, the NHS and the Office for Veterans’ Affairs. This ensures that a diversity of user views feeds into the production and development of the mental health statistics.
• In general, quality and methods information is comprehensive. The background quality report contains a good summary of the data quality and limitations of the various administrative data sources; changes to data collection methods and their potential impact on the statistics are well-explained; and the main quality assurance steps for all data sources are described.
• Use of a patient identifier enables your team to link health data with deployment records. This data linkage adds insight on the effects of deployment to Iraq or Afghanistan on mental health, for example, whether it is associated with certain mental health disorders.
• To compare cases between personnel which have and have not been deployed, rate ratios (RR) are calculated. We welcome the introductory video your team created about RR, which makes this information more accessible to a non-expert audience. We are pleased that your team has received positive feedback from users on the video.

We also identified some ways in which value and quality could be enhanced:

• There is a mental health sub-group of the England Health Statistics Steering Group (EHSSG). Joining this group would enable your team to link up with wider work across government on mental health statistics and to stay abreast of new developments on this topic, which may lead to enhancements to the UK Armed Forces mental health statistics.
• To help users understand the value of these statistics and the decisions they influence, a description could be added to the bulletin about why it is important to measure and report mental health in the armed forces, who uses the statistics, and what the statistics are used for.
• These statistics cover serving personnel only. The team should consider adding a short summary of (or links to) available data and research on the mental health of veterans, to improve the coherence of information on this topic for users. Also, to helps user easily find related MOD health statistics, links could be added to other statistics such as the results from the Armed Forces Continuous Attitude Survey (Afcas), which asks serving personnel about factors influencing intentions to leave, including mental health provision.
• For certain administrative data sources, the information about data quality could be expanded. For instance, the background quality report highlights the varying quality and completeness of data from the Joint Personnel Administration (JPA) system and mentions improvements to JPA data quality. To support user understanding of these data sources, it would be good if you explained what these data quality issues and improvements are and how they affect the statistics.

This is the last in a series of compliance checks of MOD statistics. Across the series, we identified a few common themes where we think improvements could be made to the whole of MOD’s statistics portfolio, to enhance the value and trustworthiness of the statistics.

• User engagement is largely focused on stakeholders within MOD, such as policy teams. We found some examples of engagement with wider relevant groups, but on the whole teams should widen their engagement by reaching out to users beyond MOD to enable a wider range of improvements to statistics. They need to proactively engage with external users to develop a better understanding of the broader uses and impacts of the statistics, and to capture the views of these users on the trustworthiness, quality and value of the statistics. This will ensure that the statistics remain relevant, accessible and insightful to a wide range of users. Being transparent about who you have engaged with and how you have engaged with them may help to facilitate further engagement. The Government Statistical Service’s user engagement strategy for statistics is expected to support development and capability in this area, and we look forward to seeing how MOD implements the strategy.
• Pre-release access (PRA) arrangements could be improved. For some sets of statistics, such as the continuous attitude surveys, the list of eligible recipients is long. The number of recipients should continue to be regularly reviewed and where possible, kept to a minimum, to ensure that the statistics are released in a way that commands trustworthiness. In addition, the role titles in the PRA lists should be clearer and less acronym-heavy so that it is clearer to all users who has PRA and why, and to improve transparency all statistics landing pages and/or bulletins should link to the PRA list.

Thank you to your team for their positive engagement during this review and again to the other teams with which we have engaged during this series of compliance checks. We look forward to continuing to engage with you and the team, and we hope our findings inform the development of the mental health statistics and the broader set of MOD statistics. Please do not hesitate to get in touch if you would like to discuss any aspects of this letter further or if we can offer further assistance as these statistics continue to develop.

Yours sincerely

Mark Pont

Assessment Programme Lead