Dear Neil
NATIONAL ACHIEVEMENT RATE TABLES (NARTs)
I am writing to you following the FE Week article[1] published on 5th April 2019 – ‘Investigation by statistics watchdog after ESFA lists “unreliable” providers’. The article outlined the reporting of the 2017/18 National Achievement Rate Tables (NARTs)[2] and questioned the decision to include 34 providers with ‘unreliable data’ in the aggregate headline figures.
We have previously written to you regarding these tables in 2017[3] when we undertook an investigation into the revised methodology. As a result, we recommended that you published not only historical national figures but also individual provider figures for previous years to allow for comparison. I am pleased to see that this data has now been published[4] and that similar data will be available for users on an annual basis.
We have investigated the most recent concerns raised by FE Week and noted that the 34 providers redacted in the formal performance tables have been redacted due to quality issues, which are the result of operational, rather than statistical, issues. We are aware that you work with the providers on an individual basis to try to resolve these issues.
Overall, we are content with the publication, and the approach taken to include all data in the aggregate, headline position, particularly given that excluding it risks over-estimating the national position. The publication of transparency tables helps users to understand the incomplete providers’ performance outcomes. However, it would be helpful to give more prominence to this in the headline publication and explain the context around the reasons for incomplete individual learner records. In particular, the fact that these provider outcomes can be very unpredictable from year to year could be emphasised as one of the key limitations of the data.
We also recognise that the work done around the Data Dissemination project[5] may ultimately improve these statistics which would allow users such as FE Week to customise the data.
Your sincerely
Ed Humpherson
Director General for Regulation
[1] ‘Investigation by statistics watchdog after ESFA lists ‘unreliable’ providers‘, FE Week, 5 April 2019
[2] National achievement rates tables 2017 to 2018, Department for Education
[3] Letter – Ed Humpherson to Paul Offord (FE Week), 17 July 2017
[4] National Achievement Rates Tables: 2017/18, Department for Education
[5] DfE Data Dissemination Discovery Report, July 2018