Listen to your enthusiasts: Implementing RAP at Public Health Scotland

This is a guest blog from Scott Heald following the launch of our new report: Reproducible Analytical Pipelines (RAP): Overcoming barriers to adoption.

Firstly, let me introduce myself. I’m Scott, the Head of Profession for Statistics at Public Health Scotland (PHS). PHS is a new body, formed in April 2020 (to lead on tackling the challenges of Scotland’s public health but dominated by the COVID-19 pandemic in our first year). Our RAP journey started when we were in NHS Scotland’s Information Services Division (ISD), the health and care statistics body which now forms part of PHS.

PHS, and ISD before it, has been a big fan of RAP from the beginning. I wanted to share our story, from a bunch of enthusiastic statisticians who convinced me it was the right thing to do (I didn’t need much convincing!), to embedding it within our organisation in our reporting on COVID-19.

Our RAP journey began with a programme of work to transform how we published our statistics.  It quickly became clear that the programme had to be as much about our processes for producing statistics, not just the final published output. More automation was key – to speed up processes, eliminate manual errors, and to release capacity to add value to our statistics. Greater job satisfaction for our statisticians was a welcome impact too.

“We don’t use that here”

Up till this point our software of choice was propriety standards. More and more graduates were joining our organisation and wondering why we weren’t using open-source software like R, having been taught it at university. I guess, in those early days, I was probably part of the “we don’t use that here” (partly out of fear as I had not personally used any of the new software they were talking about).

However, I was persuaded (and willing) for a group of our statisticians to show us what could be done using R. Long story, cut short, is that our group of enthusiasts showed the power of what could be done and PHS is now making the strategic shift to follow RAP principles as a way of working.

The art of persuasion

I’d describe our RAP journey as “bottom up”, with support from the “top down”. When we started seeing the results, we didn’t need much convincing. OSR’s report showcases our work on Hospital Standardised Mortality Ratios – a quarterly publication which used to take five days to run (lots of detached processes, lots of room for error). I remember vividly the first time the team ran it the RAP way. Five minutes after the process started, the finished report was in my inbox. We couldn’t believe it! And, to be sure, spent the next five days running it the old way to make sure we got the same answers (we more or less did; the RAP way was more accurate, highlighting a few errors we fixed along the way!).

Our learning is that it’s a relatively easy shift for more recent graduates because they already know R. The focus for our training had to be on members of staff who have been with us for longer and weren’t familiar with it. And that can take some persuading – team leaders finding themselves managing teams who are using software they have never used themselves. We had to support and train at all levels (with tasters for managers who themselves may not need to delve into the finer details of R, but know they would know how to do it if they were doing their time again).

So, what have we learnt?

  1. Be prepared to try new ways of working
  2. Listen to your staff, who have a different perspective and fresh take on ways of working
  3. Start small – it’s easy to make the case when you can showcase the benefits of the RAP approach
  4. RAP improves the quality of what we do and eliminates errors
  5. Be prepared to invest in training – and recognise your staff will be in different places
  6. Use buddies – our central transformation team certainly helped with that, creating capacity in teams to RAP their processes
  7. Be open and share your code – we publish our code on GitHub, a great community to share ideas and approaches

Listen to your enthusiasts

OSR’s report highlights that we have a small central transformation team to support teams with their RAP work. This is crucially important as the initial work to RAP your processes can take time, so the additional capacity to support our teams to enable this to happen is a must. This initial investment is worth it for the longer-term gains. It’s not all about making efficiencies either. It’s about streamlining processes, reducing error, and giving our analysts job satisfaction. They are now able to add more value because they have more time to delve into the data and help users understand what the data are telling them.

Our focus on transforming the processes for our existing publications has stalled due to many of our staff being redirected to supporting PHS’s key role in Scotland’s response to the COVID-19 pandemic. However, a success of our RAP work is that many of our new processes required to produce our daily COVID-19 statistics are done using RAP principles – they had to be as we’re producing more statistics, more frequently than ever before. Our earlier use of RAP meant we were in a good place to apply the techniques to our new ways of working.

And my final bit of advice? Listen to your enthusiasts – I’m glad I did.