Reproducible Analytical Pipelines: Overcoming barriers to adoption

Executive Summary

Introduction to the review

Official statistics produced by governments should uphold the highest standards of trustworthiness, quality and value in order to serve the public good. In 2017 we championed the Reproducible Analytical Pipeline (RAP), a new way of producing official statistics developed by the Department for Culture, Media and Sport and the Government Digital Service. This approach involved using programming languages to automate manual processes, version control software to robustly manage code and code storage platforms to collaborate, facilitate peer review and publish analysis.

Since then, we have seen some excellent examples of RAP principles being applied across the Government Statistical Service (GSS), the cross-government network of all those who work on official statistics. However, through our regulatory work we have seen that there are often common barriers for teams and organisations wishing to implement RAP. These include access to the right tools and training and statisticians having the time and support to carry out development work.

In Summer 2020 we set out our intention to further advocate for RAP principles in government statistics as part of our Automation and Data programme. We consider that RAP principles support all three pillars of the Code of Practice for Statistics: trustworthiness, quality and value.

In Autumn 2020 we launched this review. Our aim was to explore the current use of RAP principles across the GSS, identify what enables successful implementation and to understand what prevents statistics producers implementing RAP. We spoke to a variety of organisations that produce official statistics. This included the Office for National Statistics, UK government departments, devolved administrations, arms-length-bodies and voluntary adopters of the Code of Practice for Statistics. We also engaged with users of official statistics and stakeholders with a supportive or leadership role in this area, such as the GSS Best Practice and Impact Team and the office of the National Statistician. Finally, we drew on other available sources of evidence. These included Civil Service and GSS surveys and findings from our previous regulatory work. More information about how we carried out the review is provided in Annex 1: Approach to the review.

Our findings and recommendations

To enhance the trustworthiness, quality and value of official statistics through increased use of RAP principles and see RAP become the default approach to statistics we make the following recommendations.

Show entries

Search:

Finding	Recommendation
A consistent shared understanding of RAP and RAP principles is needed across the GSS.	Building on their previous work to promote RAP, the Best Practice and Impact Team and RAP champions network should ensure that there is widespread awareness within the GSS of the recently developed minimum standard of RAP.
RAP is not only a change in tools – it involves a cultural change to the way that analysis is approached and carried out.	The Analysis Function board and Directors of Analysis should consider how best to foster a culture where reproducible analysis is prioritised across government.
RAP principles support the highest standards of trustworthiness, quality and value and should be used as a way to enhance compliance with the Code of Practice for Statistics.	The leadership of the GSS, including the National Statistician, should set a strategic direction for the use of RAP principles in official statistics.
Support and encouragement from senior leaders allows statistics producers to successfully and sustainably implement RAP.	Organisations in the GSS should ensure that RAP principles are included in their analytical strategies. Senior leaders responsible for strategies in their organisations must develop a good understanding of what RAP is, why it is required, and support an open culture of innovation.
The implementation of RAP principles is most successful when producers carry out their own development work and when a planned approach is taken – for example having a good understanding of skill levels, training needs and existing processes.	Statistics producers should take a managed approach to implementing RAP. Projects should be underpinned by senior support, sufficient resource and the required skills, training and mentoring support.
RAP is not all or nothing: implementing just some RAP principles will result in improvements.	Statistics producers should consider what can be achieved easily and build on developments iteratively over time.
Programming and code management skills are essential for modern statistical analysis.	The GSS People Committee should ensure that RAP-related skills such as coding and code management are considered core skills for statistics producers and included in future career frameworks, such as the competency framework.
Bespoke and targeted training is most successful. Statistics producers need access to advanced training on programming, as well as introductory courses.	The GSS should invest in advanced and bespoke training on RAP and RAP-related skills through the Analytical Learning Team. This should build on existing resources and be developed in collaboration with the Best Practice and Impact Team. Availability of training must be effectively communicated across the GSS so everyone is aware of it.
Support from experts has a significant impact on the success of RAP projects.	The GSS needs to invest in expert mentoring, for example, through the Best Practice and Impact Team. Organisations that have the required skills and knowledge should support those that don’t.
Access to the tools required for RAP, such as programming languages, version control software and code storage platforms, varies across organisations. Organisations are tackling the same technical problems with different results.	A strategy for implementing RAP principles across the GSS should recommend tools which should be available to statistics producers. It should also provide guidance on the best approaches to solving common technical problems.

Showing 1 to 10 of 10 entries

Previous Next