Executive Summary
We have published a follow-up report that assesses the progress that has been made towards the recommendations made in this report.
Data sharing and linkage in government stands at a crossroads.
Since the Office for Statistics Regulation (OSR) last reported on this, there has been some excellent progress in creating linked datasets and making them available for research, analysis and statistics. The pandemic provided a particularly strong impetus to share data for the public good. But, despite the value of sharing and linking data being widely recognised, there remain areas of challenge and uncertainties about the public’s attitude to, and confidence in, data sharing. Unless significant changes are implemented, we are concerned the progress that has been made could be lost and the potential for data sharing and linkage to deliver public good will not be achieved.
Our report focuses on data sharing and linkage within central government and the devolved administrations of the UK, but most of our findings could also be relevant for the wider public sector. We plan to use the report as a platform to engage across and beyond government, working with others to help realise our recommendations and to help government enhance the public good of data and statistics.
During our report we use the term ‘government’ to refer to the UK government and the devolved administrations of Wales, Scotland and Northern Ireland. Where we wish to refer to a specific administration, this is made clear.
Our Review
Our review takes stock of data sharing and linkage across government. It points the way to build on recent successes and to confront the more ingrained challenges. To help others within government see how barriers can be overcome, and to enable positive action, we draw on inspirational examples of data sharing and linkage. Our findings and recommendations relate to four overarching themes:
1. Public engagement and social licence:
The importance of obtaining a social licence for data sharing and linkage and how public engagement can help build understanding of whether/how much social licence exists and how it could be strengthened. We also explore the role data security plays here.
2. People:
The risk appetite and leadership of key decision makers, and the skills and availability of staff.
3. Processes:
The non-technical processes that govern how data sharing and linkage happen across government.
4. Technical challenges:
The technical specifics of datasets, as well as the infrastructure to support data sharing and linkage.
Social licence, in this context, refers to the level of acceptance or approval by the local communities of interest and/or stakeholders for a data sharing and/or linkage project. The importance of obtaining a social licence for data sharing and linkage and how public engagement can help build understanding of whether/how much social licence exists and how it could be strengthened. We also explore the role data security plays here.
Learning from future scenarios for data sharing and linkage
To help individuals and organisations within government explore the possible implications of the choices they make now about whether to share or link data, we consider four possible ‘future scenarios’ for data sharing and linkage, set five years from now. They are not predictions but stylised versions of possible futures, which help to bring out the impact on public good of acting on (or not acting on) the current barriers that exist to data sharing and linkage. The scenarios are:
1. Data sharing and linkage for the public good:
This is the best-case scenario we currently see. Under this scenario, data sharing and linkage is a priority across government and many of the barriers identified in our report have been removed. This is achieved through high levels of collaboration among organisations within government, as well as strong partnerships with external researchers and organisations. In this scenario, opportunities to enhance the public good of data and statistics are fully realised and missed data use is very rare.
2. Data sharing and linkage in silos:
In this scenario public good is being realised in certain topic areas, but opportunities are being missed in other areas. In the pockets where things are going well, senior leaders are proactive and engaged, collaboration is high and consistency of practices helps things run smoothly. However, there are pockets where little to no progress is being made.
3. Data sharing and linkage for government:
This scenario features high levels of cooperation across government organisations but low engagement with researchers and organisations beyond government. As a result, the value of providing access to data to external researchers, and of sharing outputs of analysis beyond government is not being considered or realised. In this scenario, public good is not being realised and there is considerable missed use of data.
4. Data sharing and linkage deprioritised:
This is the worst-case scenario we envisage, where there has been a breakdown in support for data sharing and linkage and progress previously made across government has been lost. There are many examples of missed opportunities where data could have a real impact and, consequently, the potential for data sharing and linkage for the public good has not been realised.
The contrasts between these scenarios bring into greater focus the costs associated with not sharing and linking data, as well as the benefits when it is done. Based on this exploratory thinking and interview findings, we make 16 recommendations that, if realised, would move government away from the three less desirable scenarios and towards enabling greater data sharing and linkage for the public good. To help generate and maintain momentum on these recommendations, OSR will review and publicly report on progress towards them between six months and one year after this publication.
Back to top