In our latest guest blog Gary Childs, Head of Analytical Delivery at NHS England, discusses steps NHS England has been taking in recent years to improve the quality of mental health data…
Context
Within NHS England we have been striving to improve the quality of mental health data for a number of years. In this blog we will focus on the Mental Health Services Dataset (MHSDS), but the methodologies and principles apply equally to datasets reporting on topics such as NHS Talking Therapies or Learning Disabilities and Autism (LDA).
Clinical Data in National Collections
Charles Babbage once said: “Errors using inadequate data are much less than those using no data at all”. However, there must be a threshold as poor-quality data can lead to inaccurate analytics, bad decisions, and in health, have an impact on patient care. The Government Data Quality Hub states that: “Good quality data is data that is fit for purpose. That means the data needs to be good enough to support the outcomes it is being used for”.
In health, one would automatically assume that data would be of the highest quality as it is captured in clinical and operational systems for the purpose of direct patient care, and that assumption is probably true. The problem comes when you need to use this data to get a national picture of performance and for secondary uses.
This requires data to be extracted from these clinical and operational systems in a standardised format for aggregation within a national setting. Taking mental health as an example, there are probably over 500 providers (many being relatively small) of mental health services (excluding NHS Talking Therapies); the actual number is unknown. Of those providers, we have identified at least 30 distinct IT systems in use (such as SystmOne, Epic and RiO), as well as many in-house systems. The data within these systems is held in differing structures and formats and key information will be captured as free text.
To create the national collection for Mental Health (MHSDS – Mental Health Services Dataset) requires providers to make monthly record level submissions to NHS England. The provider must use the technical output specification (TOS) and user guidance to understand the scope and definition of each data item to be submitted. In addition, they have to familiarise themselves with the MHSDS intermediate database to understand how data items are grouped for the data submission file. To achieve this, providers have to carry out a ‘data mapping exercise’ to understand how well their existing systems align to the MHSDS TOS and take appropriate action to ensure that the standard is fully met. As Mental Health is a multifaceted service covering many policy areas, the data is complex and the submission process can be arduous, especially for smaller providers.
Supporting Providers to Submit Data
A key focus has been on increasing the number of mental health service providers submitting to the MHSDS, which now stands at over 370 providers on a monthly basis and an estimated 99%+ of all NHS funded activity. This improvement from 85 providers in 2016 has been achieved through a variety of initiatives.
Understanding who should be submitting is key and once they are submitting, knowing what services they provide and therefore the data they should be submitting. A Master Provider List is maintained that identifies all known providers in scope of the MHSDS together with their submission behaviours. Data submissions are tracked throughout the submission window, which together with historic submission behaviours result in tailored communications being sent to providers to encourage more positive behaviours.
Ensuring All Data is Submitted
In collaboration with the CQC, regional leads and providers, it has been possible to identify the services that are being delivered by most providers. This has allowed us to assess whether providers are submitting all relevant data. This has been a particular problem with Independent Sector Service Providers (where they are delivering NHS funded services) and has required the intervention of DHSC and Health Ministers.
Improving the Quality of Data
Once providers are submitting data across the service lines that they provide, we can assess the quality of that data and support providers to improve it. This starts with self service tools, at the point of submission providers receive a line-by-line data quality assessment. This can be a daunting report, hence a Validation and Rejection Submission Tool was developed that converts the record level submission report into an easy to understand summary of the issues, with instructions on how to fix them.
Providers can resubmit the data as many times as they want within the submission window, improving the data each time. However, there are occasions where data issues are identified after the submission window is closed that can affect the quality of the data for that month but also have a knock-on effect on future monitoring, such as for 3 or 12 month rolling metrics. To address this issue a multiple submission window model (MSWM) was implemented to allow providers to address data quality issues throughout the financial year. Use of the MSWM is closely monitored and reported upon to avoid abuse of the facility, as it should be a last resort.
To illustrate the quality of the data and the compliance of that data it is surfaced within a data quality dashboard that reports on the quality of each data item submitted by each provider, allowing for comparisons at a provider, regional and system supplier level. In addition, to promote the compliant use of SNOMED (a structured clinical vocabulary for use in electronic health records) relevant data items are reported upon within a SNOMED dashboard. The dashboard assesses how much SNOMED is flowing to MHSDS, to which data items and tables, and by which providers, there is also a focus on correctly identifying procedures and assessments.
To reinforce these tools, providers receive an automated data quality report by e-mail each month. These reports summarise the key issues with a provider’s data and strategies to fix them. Providers can also access policy specific guidance in the form of workshops, webinars and documentation. This has previously focussed on topics such as eating disorders, restrictive interventions, problem gambling, perinatal and maternal services and memory clinics. In addition, providers have received questionnaires to better understand where they need more support.
Talking About Data Quality
While all these tools facilitate better data it is the direct engagement with providers by the data liaison team that can have the biggest impact. The data quality analysis will identify the providers experiencing the biggest challenges, which is used by the data liaison team to provide tailored support on a 1-2-1 basis. This was particularly successful during the recent Advanced Cyber Incident that impacted the data of a variety of mental health service providers for almost 9 months.
Next Steps in Improving Mental Health Data
At first sight all these solutions may seem excessive. However, the data is the foundation for decisions relating to commissioning, service improvement and service design, it supports research and innovation, and helps understand the impact of mental health care on patient outcomes and experiences. Through improvements in data quality, we have been able to close several duplicate collections and are now able to move to a single window for data submission. This will soon allow insights to be delivered a whole month earlier, making decisions more relevant and timelier.
At the start of this blog, I stated that the data we are referring to is secondary uses data, but that data did originally come from clinical and operational systems that are used for direct patient care. As we know that this data is of a higher quality than that within MHSDS, we must find a way to mitigate the degradation in quality that we are currently seeing. Initiatives of this nature are currently being explored within NHS England in the hope that we can improve data quality further, make the data even timelier, as well as easier to collect.