Systemic review outline: Reproducible Analytical Pipelines

What we are doing and why

At OSR we have previously endorsed the use of RAPs and our automation and data science programme aims to further advocate for the use of RAP within official statistics. However, through our regulatory work, we know that departments and teams have varying degrees of success in implementing RAP.

To better understand this area, we have launched a review which will explore the use of RAP across government statistics in more detail. We are particularly interested in what enables departments to successfully implement RAP and what issues prevent producers either implementing RAP fully or applying elements of it. In doing this we want to identify common themes and differences between departments, explore current barriers and hear the success stories.

Background

More and more, open source programming languages are being used to automate previously manual processes in statistics production. Version control software and good practice in working methods, such as code sharing and peer review, are making collaboration, code access and code re-use easier.

This approach to statistical production is commonly referred to as a Reproducible Analytical Pipeline (RAP). Adopting this type of approach reduces the risk of errors, improves the quality of analysis and makes production more efficient, freeing up time for statisticians to add more value to statistical outputs.

The Quality Centre within the Best Practice and Impact Division recommend that official statistics should be made reproducible through the use of RAP. This recommendation is underpinned by one of the four goals in the Government Statistical Service (GSS) Quality Strategy: to “implement automated processes to make our analysis reproducible”.

How we are doing it

We are currently gathering and reviewing evidence which is already available on this topic. This includes examples identified by regulators through assessments and compliance checks and the results of a survey on coding carried out by the GSS Good Practice Team last year. We plan to use the results of the second wave of this survey, which will be carried out in Autumn 2020, to further develop our understanding.

Taking part

Most importantly, we want to speak to producers of official statistics and others interested in this topic. We want to hear from a wide variety of people, including those who are applying RAP techniques and those who are not.

If you are interested in contributing to our review, please get in touch.

Contact for more information: Anna Price