In 2020 we assessed estimates of station usage produced by the Office of Rail and Road (ORR) and designated them as National Statistics in December. In this blog, Lyndsey Melbourne, Head of Profession for Statistics at ORR, and Anna Price, the lead regulator for the assessment, talk about their experience and why assessments of official statistics are so valuable.
Where it all began…
Lyndsey: Most of our statistics were designated as National Statistics in 2012. In 2019 OSR carried out a compliance check and confirmed they continued to uphold the high standards expected. At the time we also discussed future assessments – in particular our most popular set of statistics, estimates of station usage, had never been assessed. These statistics provide unique information about each of the 2,500+ mainline rail stations in Great Britain. The granularity of data is one of the main reasons that these statistics are of interest to a very broad range to users: they are relevant to anyone no matter where they live. We were keen to further promote the quality and value of the statistics by gaining National Statistics status.
Anna: This assessment was a bit different to others. Usually we do our review, publish our findings and requirements, and then give producers a few months to meet these requirements. But when we first met with Lyndsey and Jay, the lead statistician for estimates of station usage, in April 2020 they told us they were keen to get National Statistics status in time for the next statistical release in December.
The assessment process
Anna: To support ORR to achieve this ambition, we adapted our usual process, for example sharing our findings and requirements as we developed them. This let the statisticians at ORR start on improvements while we worked on more complex findings and wrote our report, instead of waiting until the end. We had lots of meetings with the team during the project and were really impressed with the ideas they came up with each time we raised an area for improvement. I think the flexibility and enthusiasm of both teams was the reason that the project was so successful.
Lyndsey: Throughout the assessment, OSR were flexible and happy to work with us to agree timescales to fit in with our publication plans and around our day jobs. We were keen to work towards achieving National Statistics designation of the statistics in time for our next annual publication planned for December 2020. Otherwise, it would be up to 20 months before we could publish designated statistics!
OSR were very accommodating to this request and we worked closely during the following eight months to review and improve our statistics. OSR’s flexible approach allowed emerging requirements from their assessment to be addressed during the production process of the next set of statistics.
It’s fair to say that producing the annual publication at the same time as addressing OSR requirements was a challenge, but being able to confirm to users that our statistics had been successfully designated as National Statistics on publication day was very satisfying.
The value of this assessment
Lyndsey: During the assessment OSR spoke to a range of users and the feedback they obtained was extremely valuable. We have continued to speak to these users to understand their use of our statistics and how they could be improved further.
The improvement plan we developed to address OSR requirements and other feedback from users was a really useful tool for us. Sharing ideas and drafts with OSR along the way and getting their feedback was another valuable part of the process. We published this improvement plan on the user engagement page of our data portal to keep users up to date on the changes we were making.
Anna: The users of these statistics are passionate about them. So it was a lot of fun to hear about how they use the statistics, what they like and what would make using them even better. Seeing the variety of people who use the statistics, for a variety of purposes, was really motivating – it made it even more satisfying when we saw changes to the statistics in December which met the user needs we had identified.
At OSR we like to champion good practice, as well as areas for improvement. So it was nice to highlight the great work that ORR were already doing on these statistics – like the Twitter Q&A that ORR host on publication day, this year accompanied by a launch video and a live YouTube Q&A. It’s great to see statisticians putting themselves out there to talk about their statistics directly with users.
To keep up to date with our latest work, you can follow us on Twitter and sign up to our monthly newsletter.