Data gaps

Official statistics should support society’s need for information

At present, users cannot fully answer questions they have around mental health. This has been highlighted as an area of concern by a number of different organisations over recent years. The Northern Ireland Commissioner for Children and Young People (NICCY) concluded in the 2018 ‘Still Waiting report’ that there were ‘alarming gaps’ in operational data to efficiently plan, commission and deliver mental health services for children and young people in Northern Ireland.  Furthermore, in 2016, the Mental Health Foundation concluded as part of their research that mental health data in Northern Ireland was far more limited than in England, Wales and Scotland.

The Mental Health Policy Group (NI), an umbrella policy group consisting of Action Mental Health, Inspire, the Royal College of Psychiatrists and the British Psychological Society in 2019 called for urgent action to develop a new mental health strategy for Northern Ireland following the progress made in response to the Bamford Review. They also highlighted the need for improved data and better analysis of data, with the Trusts working together as one mental health service in Northern Ireland.

This position was reflected in the conversations we had as part of our engagement for this review. We heard that the existing statistics and data do not provide the answers to what many felt were basic questions. There is no way at present to fully evaluate the effectiveness of services in improving people’s mental health, with no regional evidence base for what works. It is difficult to measure outcomes and this is not a unique finding in NI, with our review of mental health statistics in England also highlighting this as a data gap.

Below are specific examples of unanswered user questions.

Statistics and data should provide insight and form the evidence base for important decision making, policy direction and to prioritise resources. They are used for a wide variety of purposes, by a ranging user community. Without answers to questions, such as the ones highlighted here, users have difficulty in forming that evidence base, to understand what works, what needs to improve and where preventative initiatives should be targeted.

A complete picture of mental health care provision across Northern Ireland is currently missing; with data from community-based services a huge gap

The provision of mental health services straddle the acute (hospital setting), community and the voluntary sectors. The Northern Ireland Statistics Research Agency (NISRA) routinely publish the following outputs containing statistics on mental health using data collected in hospital settings;

• Hospital statistics: inpatient and day case activity NI (National Statistics) – includes mental health programme of care (POC), statistics on total admissions to HSC hospitals, by trust and average availability of beds (National Statistics).
• Hospital statistics: outpatient activity (National Statistics) – includes outpatient activity for mental health programme of care (POC) by HSC trusts, hospital and speciality (National Statistics).
• Hospital statistics: Annual Mental Health and Disability tables (includes Mental Illness and Learning Disability Census completed every February and compulsory admissions under the Mental Health (NI) Order 1986 (National Statistics).

As a result of the Bamford review in 2009, more treatment and services are community-based. Despite this shift in service delivery, there are limited official statistics capturing mental health service activity in the community and voluntary sectors – meaning part of the picture is missing. Some prevalence data is collected as part of the QOF (Quality and Outcomes Framework) by GP practices and presented at an aggregate level for individuals on patient registers. QOF disease registers for mental health, depression and dementia are published and updated annually on the Department of Health (NI) website. These data are primarily collected to inform QOF payments to GP practices, rather than to provide clinical prevalence indicators, and therefore the data have limitations. Whilst these data provide a broad overview of prevalence for those people attending GP practices, they are not granular or disaggregated to patient-level characteristics (e.g. age, sex), therefore meaningful insight is limited. Existing data collection frameworks, such as the QOF, could be configured to collect more useful data from GP’s, and we would encourage the Department of Health (NI) to consider this as part of their development process to improve mental health statistics and data.

Statisticians in the Department of Health (NI) are looking to develop experimental statistics for activity within the community. We heard that there are plans to incorporate statistics based on the NI psychological therapy manual return. However the significant impact of the COVID-19 pandemic over the last 18 months on resource has led to many plans being paused.

Statistics users are turning to other data to answer the questions that they have. We heard through our user conversations, that academic research is the primary source of mental health information in Northern Ireland. For example, the new 10 year Mental Health Strategy by the Department of Health (NI) is primarily based on academic research due to limited official data.

There are positive developments to highlight here, with work ongoing to fill some information gaps. IMPACT, a psychological research centre, unique to the Northern Health and Social Care Trust, was established in March 2019 to promote research activity within psychological services. The work of the centre is adding huge value to the evidence base with a key aim of improving the effectiveness and efficiency of services. Users we spoke to were very positive and complimentary about the work of the psychological centre, referring to it as the ‘gold standard’ – they also highlighted that it was the exception and were hopeful about the plans to create a centre of excellence for mental health research in the Mental Health Strategy 2021-2031. We also heard about the ‘10,000 more voices’ project led by the Public Health Agency which aims to capture patient and client experiences of healthcare services.

Users of statistics and data should be at the centre of statistical production, their needs should be understood, their views sought and acted upon, and their use of statistics supported. We recommend that the Department of Health (NI), as part of the ‘New Ways of Working’ theme in the Mental Health Strategy 2021-2031, use and build upon our findings in this report to augment and improve the availability of official data on mental health.

Statistics should be granular with insightful commentary to support user understanding

Public health, charity and third sector organisations use data and statistics as an evidence base to inform public health campaigns, target interventions effectively and to help them prioritise their resources. They want to understand how to prevent and tackle mental health before people reach crisis point and identify vulnerable groups who may be more susceptible to poor mental health in local communities. In order to provide insight for those sub-populations at higher risk, demographic data needs to be recorded and captured accurately.

Under Section 75 of the Northern Ireland Act 1998, there is a statutory duty for public bodies to promote equality of opportunity and good relations across a range of categories, including age, sex, racial group, disability, marital status, sexual orientation, political opinion and religious belief.  Often referred to as the Section 75 indicators, the recording of key equality data by public authorities is in line with the requirements of these duties. Robust equality data is needed to support evidence-based policy making and, in this case, also has an impact on the granularity of statistics produced using this data.

Users told us that Section 75 indicators are very poorly collected in health data, therefore it is really difficult to identify the prevalence of poor mental health in sub-population groups, e.g. BAME and LGBT+ groups, older adults, homeless people, refugees and asylum seekers. Disaggregated age bands for those aged 60 and over was also raised to us as an area for development in health data, replacing 65 and over with more specific categories to recognise the diversity of experience in the third age. The absence of research and data addressing mental health for different groups and populations in Northern Ireland was recognised by the Mental Health Foundation in 2016. Furthermore, the Equality Commission for Northern Ireland’s most recent review of the implementation of the s75 duties identified data development as an issue. We are encouraged to see that the Equality Commission, in partnership with NISRA, has recently published a signposting guide to encourage and support public authorities to collect and use equality data in line with the requirements of the duties.

Users expressed a specific interest in more localised data, e.g. Super Output Areas/Council level. Official statistics and data are presented at a high-level, with little detail. Statisticians we spoke to were aware of this user demand, however due to sample sizes, disaggregation beyond HSC Trust level was difficult due the robustness of the data.   We also heard that more recent publications were much less insightful with more data presented in excel spreadsheets and less statistical commentary. Users we spoke to want to know more about what the statistics mean and how they fit into the bigger picture. To support user understanding, the Information Analysis Directorate (IAD) in the Department of Health (NI) should seek to provide more insightful commentary for existing mental health publications to explain key statistical messages.