Executive Summary: Our Recommendations
To enable greater data sharing and linkage for the public good through tackling current barriers we make the following recommendations. High-level findings are also provided for context.
Public engagement and social licence
Key Findings
- There is a need for more public engagement about data sharing and linkage, to improve both transparency of work that is being carried out, and public confidence in data sharing and linkage more generally.
- There is growing evidence that some people in the UK want and expect their data to be used when it is done securely and transparently.
- While there are examples of public engagement is being done well, there can also be a lack of understanding about how to do public engagement effectively.
- The Public Engagement in Data Research Initiative (PEDRI) is a new sector-wide partnership looking to bring together organisations who work with data and statistics to collaborate and embed meaningful public involvement across the data ecosystem. This initiative could strengthen the public engagement landscape, sitting alongside other existing centres/initiatives that already support specific communities.
- The amount of social licence for a data sharing or linkage project is likely to be related in part to data security. The Five Safes Framework is a set of principles employed by many data services that enable them to provide safe research access to data. Assurance that the Five Safes Framework continues to support the appropriate level of security would be helpful.
- Privacy Enhancing Technologies (PETs) are newer technologies that can help organisations share and use people’s data responsibly, lawfully and securely. There is growing interest in PETs and the benefits their use could bring.
Recommendations
Recommendation 1: Social Licence
The government needs to be aware of the public’s views on data sharing and linkage, and to understand existing or emerging concerns. Public surveys such as the ‘Public attitudes to data and AI: Tracker survey’ by the Centre for Data, Ethics and Innovation (CDEI) provide valuable insight. They should be maintained and enhanced, for example to include data linking.
Recommendation 2: Guidelines and Support
When teams or organisations are undertaking data sharing and linkage projects, there is a growing practice of engaging with members of the public to help identify concerns, risks and benefits. To help teams or organisations who are undertaking public engagement work, best practice guidelines should be produced, and support made available to help plan and coordinate work. This should be produced collaboratively by organisations with experience of this work for different types of data and use cases and brought together under one partnership for ease of use. We consider that, given its current aims, the Public Engagement in Data Research Initiative (PEDRI) could be well placed to play this role.
Recommendation 3: The Five Safes Framework
Since the Five Safes Framework was developed twenty years ago, new technologies to share and link data have been introduced and data linkage of increased complexity is occurring. As the Five Safes Framework is so widely used across data access platforms, we recommend that the UK Statistics Authority review the framework to consider whether there are any elements or supporting material that could be usefully updated.
Recommendation 4: Privacy Enhancing Technologies
To enable wider sharing of data in a secure way, government should continue to explore the potential for Privacy Enhancing Technologies (PETs) to be used to enhance security and protect privacy where data are personally identifiable. The Office for National Statistics (ONS) Data Science Campus is well placed to lead and coordinate this work.
People
Key Findings
- At every step of the pathway to share and link data, the people involved, and their skills and expertise, are instrumental to determining whether projects succeed or fail.
- A big barrier to data sharing and linkage for some organisations is whether it is a priority for the Accounting Officer. Making secure data sharing and linkage a strategic priority at the level of the Accounting Officer in more organisations would enable more joined up approaches across government. To achieve this, an appreciation of the potential benefits needs to be more widely held.
- Recruiting and retaining people with the skills needed to link, maintain and analyse data was a significant challenge raised by many of our interviewees.
Recommendations
Recommendation 5: Data Literacy in Government
To gain the skills to create and support a data-aware culture, it is important for senior leaders to have awareness of and exposure to data issues. One way to raise awareness and exposure would be for senior leaders to ensure that they participate in the Data Masterclass delivered by the ONS Data Science Campus in partnership with the 10 Downing Street (No10) Data Science Team.
Recommendation 6: Data Masterclass Content
The Data Masterclass could expand its topics to include sections specifically on awareness of data linkage methodologies, the benefits of data sharing and linkage and awareness of different forms of data. This would fit well under the Masterclass topics of ‘Communicating compelling narratives through data’ or ‘Data-driven decision-making and policy-making’.
Recommendation 7: Arbitration Process
To facilitate greater data sharing among organisations within government, a clear arbitration process, potentially involving ministers, should be developed for situations in which organisations cannot agree on whether data shares can or should occur. Developing such an arbitration process could be taken on by the Cabinet Office, commissioned by the Cabinet Secretary and delivered working with partners such as No10 and ONS.
Recommendation 8: Career Frameworks
To enable more effective and visible support for the careers of people who work on data sharing and linkage, those responsible for existing career frameworks under which these roles can sit, such as the Digital Data and Technology (DDaT) career framework and the Analytical Career Framework, should ensure skills that relate to data and data linkage are consistently reflected. They should also stay engaged with analysts and professionals across government to ensure the frameworks are fit for purpose. These frameworks should be used when advertising for data and analytical roles and adopted consistently so that career progression is clear.
Processes
Key Findings
- There is variation within government over how much data holders and researchers understand the process to share data under different legal bases.
- When applying for data through a secure data platform, the process is often lengthy and can appear overly burdensome to researchers.
- For every data share there will be many teams involved, within the same organisation or from many different ones. Not getting these teams together at the very start can cause major delays to data sharing.
- When researchers have a question about a dataset or process, it can be a challenge to find the right person who can help.
- Funding structures across government tend to be set up so that each department controls its own spend, making successful funding highly dependent on the priorities and vision within each department. This siloed approach is hampering efforts of collaboration and means projects with external funders are often more successful.
Recommendations
Recommendation 9: Overview of Legislation
To help researchers understand the legislation relevant to data sharing and linkage and when it is appropriate to use each one, a single organisation in each nation should produce an overview of legislation that relates to data sharing, access and linkage, which explains when different pieces of legislation are relevant and where to find more information. This organisation does not need to be expert in all legislation but to be able to point people to those that are. The Office for Statistics Regulation (OSR) will help convene those in this space to understand more about who might be best placed to take this on.
Recommendation 10: Broader use cases for data
To support re-use of data where appropriate, those creating data sharing agreements should consider whether restricting data access to a specific use case is essential or whether researchers could be allowed to explore other beneficial use cases, aiming to broaden the use case were possible.
Recommendation 11: Communication
To ensure data application processes are fit-for purpose and well understood, those overseeing accreditation and access to data held in secure environments should prioritise ongoing communication with users, data owners and the public to explain and refine the information required. Wherever possible, they should offer face-to-face or virtual discussions with those applying to access data early in the process, to ensure clarity around both the data required and the process to access it.
Recommendation 12: Checklists
To ensure all necessary teams are involved at the outset of a data sharing and linking project, organisations should consider the use of a checklist for those initiating data sharing. The checklist should contain all contacts and teams within their organisation who need to be consulted to avoid last minute delays.
Recommendation 13: Transparency
Every organisation within government should be transparent about how the data they hold can be accessed and the process to follow. This guidance should be presented clearly and be available in the public domain with a support inbox or service for questions relating to the process.
Recommendation 14: Funding Structure
To allow every organisation a consistent funding stream for their projects, a centralised government funding structure for data collaboration projects across government, such as the Shared Outcome Fund, should be maintained and expanded.
Technical Challenges
Key Findings
- It can be a challenge for those linking data to get enough information about the data to provide a high-quality linked output with a measurable rate of error.
- While we heard many positive reflections on the effectiveness of current data linkage methodologies, and the way that these are being developed, it was also acknowledged that methodological challenges do still exist, which can also themselves lead to issues with the quality of linked data.
- Variation in data standards and definitions used across government is making linking harder.
Recommendations
Recommendation 15: Sufficient resources
To enable effective, efficient, and good quality data linking across government, senior leaders should ensure there are sufficient resources allocated to developing quality metadata and documentation for data held within their organisations.
Recommendation 16: Standardisation
Many departments are looking to standardise government data and definitions, but it is unclear whether or how these initiatives are working together. Those working to standardise the adoption of consistent data standards across government should come together to agree, in as much as is possible for the data in question, one approach to standardisation which is clear and transparent. Given the work done by the Data Standards Authority, led by the Central Digital and Data Office (CDDO), the CDDO may be best placed to bring this work together.