People: Update on recommendations

OSR has previously highlighted that ensuring effective data sharing and linkage for statistics and research in government goes beyond technological advancements. At every step of the pathway to share and link data, the people involved, and their skills and expertise, are instrumental to projects’ success or failure. A lack of awareness, priorities and capability of people involved in decision making and development, including senior leaders, analysts and those in data governance and guardianship roles, can impose barriers to successful data sharing and linkage. In our previous report, we highlighted barriers caused by lack of leadership, low risk appetites, lack of data literacy, low awareness of the benefits of data sharing and linkage, and staff retention and skills. Based on these observations, we made recommendations that the continued attendance and development of the ONS Data Science Campus’s Data Masterclass for senior leaders in government be supported; that an arbitration process be used to help resolve disputes over data sharing; and that relevant analytical career frameworks be updated to ensure skills that relate to data and data linkage are consistently reflected.

During our research for this report, we found that progress relating to data literacy and skills has been slow. The ONS Data Science Campus has informed us that the development of new Data Masterclass content has been paused due to uncertainty around resource allocation. Furthermore, it is restricting enrolment in the current Masterclass to existing stakeholders. The Analysis Function (AF) and Government Digital and Data (GDAD) career frameworks have not been updated to include any data sharing and linkage-specific content, despite the AF career framework being refreshed in late 2023.

There has been growing recognition within government of the importance of data literacy across the Civil Service, not just among senior leaders. Last year, this recognition led to the One Big Thing initiative, which promoted analytical training for all civil servants. It will take sustained investment in such initiatives to ensure lasting change.

There has been progress, however, in work related to arbitration. Since our previous report, the CDDO has established the Data Sharing Network of Experts (DSNE), which helps organisations identify legal gateways to data sharing and linkage whilst also helping departments resolve questions or any differences of opinion around the appropriateness of data sharing. As such, we have updated our recommendation to draw attention to this good work and to encourage more development.

Recommendation 5: Data Literacy in Government

Original recommendation:

To gain the skills to create and support a data-aware culture, it is important for senior leaders to have awareness of and exposure to data issues. One way to raise awareness and exposure would be for senior leaders to ensure that they participate in the Data Masterclass delivered by the ONS Data Science Campus in partnership with the 10 Downing Street (No10) Data Science Team.

Key findings

• A lack of data literacy and awareness of the benefits of data sharing and linkage among senior leaders continues to lead to overcaution and to be a barrier in some cases.
• The Data Masterclass, an initiative delivered by ONS Data Science Campus to strengthen data awareness and literacy among senior leaders across government, is currently restricted in terms of enrolment and content. The development of similar programs could help fill this gap in some parts of government.
• It is not just senior leaders who need awareness and understanding of the benefits of data sharing and linkage. All civil servants need some level of data literacy.
• One Big Thing is a positive example of government trying to raise data literacy across the civil service. Sustained investment in such initiatives will be needed to ensure lasting change.

Summary of findings

In our previous report, we spoke about the roles Accounting Officers, Chief Data Officers (CDOs) and Data Protection Officers (DPOs) have in progressing or hindering the adoption of data sharing and linkage for statistics and research. We found that low risk appetites and varying levels of understanding of the challenges and opportunities associated with data sharing and linkage are major barriers to adoption. Our recommendation aimed to encourage senior leaders and decision makers in data to upskill through resources like the ONS Data Science Campus’s Data Masterclass.

Through conversations with stakeholders, we learnt that data literacy among senior leaders in government is still perceived as an important factor in enabling data sharing and linkage. Despite a growing focus on data literacy and commitments from the UK Government to put data more fully at the heart of government decision making, there is more to be done.

In particular, attitudes towards data sharing and perceptions of the potential risk–reward trade-off can still be a barrier to collaboration. The benefits of sharing and linking data may be widely spread, but if something goes wrong, the effects can be close to hand and potentially difficult for individual organisations. Senior leaders can come from backgrounds where little knowledge of data is required, which can cause them to be overly cautious. Greater understanding of the potential benefits and perceived public perceptions of data sharing and linkage would enable organisational change and remove barriers. There is a leadership role for the analytical services in championing the benefits of data reuse and linkage. Developing a shared narrative on areas where linking data could really benefit decision making would be useful to help departmental leadership to see the benefits of linking data around a person, place or business.

Our previous report highlighted the Data Masterclass as an effective way of introducing senior leaders across government to the power and potential of data, and to different ways of working with data. Between July 2023 and March 2024, the Data Masterclass for Senior Leaders was started by approximately 5,000 public sector staff, bringing total enrolments since its launch in 2020 to 15,000 individuals from more than 50 public sector organisations. These data are impressive and demonstrate the reach and potential of the Masterclass. Furthermore, we heard that similar programs are being developed to support this agenda. These include the Digital Excellence Programme, which is part of the Digital and Data Civil Service Reform Missions and aimed at training SCS in data literacy and AI, and the Welsh Government internal policy essentials course for senior leaders to help improve their skills in relation to data.

Of course, it is not just senior leaders who are important in enabling data sharing and linkage. Individuals from many professions and at different levels of seniority within the Civil Service all have a role to play in understanding and promoting opportunities for sharing and linking data. This is not least because those operating at the working level are often those who brief and advise senior decision makers. Since 2020, the Data Linkage Champion Network has provided a forum for civil servants of all grades and from across government to discuss and promote data linkage. Looking more widely at data literacy across the Civil Service, in 2023 the Civil Service launched One Big Thing, a new annual initiative led by the Data Science Campus, to help civil servants upskill on topic areas relevant to a modern Civil Service. The first topic of this initiative was data skills: 212,000 people across the Civil Service took part over four months, undertaking at least one day’s worth of analytical training. This course is now available for all civil servants to access on Civil Service Learning.

Initiatives like the data-focussed One Big Thing event help to promote a data-aware culture within government. This is a positive step forward, although it will take sustained investment in such initiatives to ensure lasting change. Data awareness will be essential for all civil servants given the push towards a modern Civil Service; our recommendation has been updated to reflect this.

Recommendation 6: Data Masterclass Content

Original recommendation:

The Data Masterclass could expand its topics to include sections specifically on awareness of data linkage methodologies, the benefits of data sharing and linkage and awareness of different forms of data. This would fit well under the Masterclass topics of ‘Communicating compelling narratives through data’ or ‘Data-driven decision-making and policy-making’.

Key findings

• The Data Masterclass continues to be delivered, but only to existing stakeholders.
• Development of the Data Masterclass has been paused.

Summary of findings

In our last report, we recommended that the content of the Masterclass be expanded to include more sections on awareness of data linkage methodologies. This recommendation aimed to reinforce the effectiveness of recommendation 5 through ensuring that the Data Masterclass content is fit for purpose.

From our engagement with the Data Science Campus (DCS), we found that development of the Data Masterclass has been paused while decisions on resource allocation are made within ONS. This means that, despite agreeing in principle, DSC has been unable to commit to developing the Masterclass in line with our recommendation. Furthermore, we heard that the Data Masterclass will only be delivered to existing stakeholders, reducing the number of senior leaders who would benefit.

Strengthening data literacy among senior leaders is vital to furthering data sharing and linkage, and other innovations related to data or analysis, across government. Buy-in from leaders is often required to progress positive data sharing and linkage initiatives. The Data Masterclass is an excellent tool for improving data literacy among senior leaders in central government and across the wider public sector. To continue to do this job well, the Masterclass needs to evolve to reflect developments in the data space, including data sharing and linkage. We are disappointed to hear that future investment in the development of the Masterclass is not ensured and would encourage ONS to prioritise this investment.

Given the importance of increasing awareness among senior leaders of the potential benefits of data sharing and linkage for the public good, we consider that our previous recommendation remains important and should be stricter.

Recommendation 7: Arbitration Process

Original recommendation:

To facilitate greater data sharing among organisations within government, a clear arbitration process, potentially involving ministers, should be developed for situations in which organisations cannot agree on whether data shares can or should occur. Developing such an arbitration process could be taken on by the Cabinet Office, commissioned by the Cabinet Secretary and delivered working with partners such as No10 and ONS.

Key findings

• There has been progress in this space since our previous report. The Central Digital Data Office (CDDO)’s establishment of the Data Sharing Network of Experts (DSNE) to help departments deal with questions or differences of opinion around data sharing is a positive step.
• In the year since our last report, Professor Denise Lievesley’s Independent Review of the UK Statistics Authority was published. Within this report, a recommendation was made to progress work on data sharing and linkage.
• Stakeholders have raised concerns with us that any arbitration process could have difficult consequences for data owners, including decisions being made on their behalf while risk remained with them.

Summary of findings

We found in our last report that hesitancy around data sharing and linkage due to potential overcaution was and remains a major barrier. Our aim with this recommendation was to encourage the development of a process that would enable departments to work through potential disagreements over data linkage that stem from security and legal concerns. The inclusion of a third party to advise on potential risks and benefits would hopefully mitigate over-worrying and better enable productive data linkage.

In line with our 2023 recommendation, the Independent Review of the UK Statistics Authority, published in March 2024, clearly reinforced the need for better processes to resolve data sharing challenges. Within this review, Professor Lievesley recommended that “the Cabinet Office, HM Treasury and No. 10 must actively work to resolve the systemic, often cultural barriers to data sharing between departments. All government departments, particularly those who own significant amounts of data, must prioritise data sharing for statistics and research purposes”. The government response to this independent review welcomed the review’s focus on data sharing within government, and committed to setting out a more detailed response to this recommendation later this year. We hope the government adheres to these commitments.

The issue is not always that government departments disagree over whether to share data. It can also transpire that everyone agrees to a data share in principle, but it does not advance because it becomes encumbered by the complexities of the process, which can involve multiple steps and people. In these situations, collaborative processes, rather than arbitration, are required to enable sharing initiatives to proceed.

Since our last report, positive steps have been taken towards establishing routes for departments to resolve questions about or conflicts over whether or how they can share data.

The Central Digital Data Office (CDDO) has established a Data Sharing Network of Experts (DSNE), which includes data protection leads and legal experts. This network helps organisations that want to share data identify legal gateways within the Digital Economy Act 2017 and helps departments deal with questions or differences of opinion around data sharing. The network also helps government departments to develop potential new data sharing objectives that can be put to the Public Service Delivery Information Sharing Review Board to expand the Digital Economy Act and enable new data sharing initiatives that would previously have been blocked by the act. The CDDO is doing helpful work in this area: guiding government departments through the legal challenges of sharing data and resolving disputes, offering the opportunity to mitigate a lot of the nervousness around data sharing and linkage, and leaving more room for optimism about future efficiencies and research. Due to the promise of DSNE, resources should be allocated to enable sustained activity in this space. However, the CDDO could make the DSNE more widely known, both within government and to the public. This would raise awareness that this service is available, as well as demonstrate transparency and offer reassurance that the process of sharing and linking data in government is rigorous. We have updated our recommendation to reflect the CDDO’s work on the DSNE.

The issue of arbitration and how this can most effectively be overseen and delivered is a complex one. Our stakeholder engagement revealed a variety of views about the potential benefits and risks of introducing a third-party arbitrator to the data sharing process. Data owners need significant reassurance that any data they share will be handled correctly, and that the benefits of sharing outweigh the potential risk of security breaches. Some of the stakeholders believed an arbitration process may not be able to address these issues. Concerns were also raised around whether an arbitrator could override data owners’ concerns about data security and take the control out of their hands. Stakeholders expressed worry that reputational damage could result from decisions made on their behalf. Others felt that the involvement of a third party, through an arbitration process, could be positive, as the responsibility for making decisions around data security, which would usually fall entirely on data owners, could be shared. However, with this view came the appreciation that finding an arbitrator willing to take on this level of responsibility could be difficult and require legislative changes around the accountability of data controllers.

Though we understand these concerns, overcaution around data sharing is a major barrier to realising its benefits, from both a research and operational perspective. It is important for government to continue to invest in processes that could help resolve disputes around data sharing and linkage. On reflection, collaborative processes wherein data owners and researchers could discuss terms and arrangements for data sharing with a third-party providing mediation to aid resolution may be less intimidating and more productive. To reflect this, we have updated our recommendation to focus less specifically on a formal arbitration process.

Recommendation 8: Career Frameworks

Original recommendation:

To enable more effective and visible support for the careers of people who work on data sharing and linkage, those responsible for existing career frameworks under which these roles can sit, such as the Digital Data and Technology (DDaT) [now known as Government Digital and Data, GDAD] career framework and the Analytical Career Framework, should ensure skills that relate to data and data linkage are consistently reflected. They should also stay engaged with analysts and professionals across government to ensure the frameworks are fit for purpose. These frameworks should be used when advertising for data and analytical roles and adopted consistently so that career progression is clear.

Key findings

• Though the Analysis Function (AF) career framework has been refreshed since our last report, it does not include specific material on data sharing and linkage.
• The Government Digital and Data (GDAD) Profession Capability Framework similarly has not been updated to include data sharing and linkage content.

Summary of findings

Research for our previous report highlighted a demand for data roles within both the public sector and the private sector. Due to the competitiveness of salaries in the private sector, the public sector is struggling to attract skilled individuals. Furthermore, we found that retention of talent is also a common problem, as individuals with specialised knowledge of processes move to other government departments, chasing higher salaries at the same grade. Based on these findings, we recommended that the most prominent career frameworks in government, the GDAD and AF career frameworks, be updated to include data sharing and linkage specific content.

Since our report, there has been little progress towards realising this recommendation. The AF career framework was last refreshed in late 2023. The AF career framework outlines core skills needed to work successfully across analytical roles and links to a complementary product (the technical learning curriculum), which sets out the wider learning opportunity for analysts and includes data sharing and linkage learning. Disappointingly, due to wider streamlining across the AF, workplans have shifted to a maintenance-only position for this financial year with no further developments to career framework or other currently scheduled products. The GDAD career framework, similarly to the AF framework, contains a breadth of information on what data skills are useful for analysts but does not contain any specific detail around data sharing and linkage. In addition, there is a lack of consistency in how these frameworks are used in government, which we highlighted in our previous report.

The work of the Cabinet Office and the development of the IDS illustrate that government statistics are moving towards an environment where data sharing and linkage are more commonplace. Therefore, more work is needed to improve these career frameworks by including this type of content explicitly, so that analysts in government develop the appropriate skills to enable this transition. It is counterproductive to have positive initiatives in the space of data sharing and linkage without equipping analysts with the skillsets needed to make use of these advancements. We therefore consider our original recommendation is still fit for purpose.