Executive summary
Why we carried out this assessment
ES.1 This assessment considered the Key Stage 4 (KS4) Performance statistics for England published by the Department for Education (DfE). It also considered the elements of the Compare School Performance website that relate to KS4 performance.
ES.2 Our assessment found widespread good practice in the production of the KS4 statistics. We have identified four actions for DfE to fulfil in order for the statistics to continue to be designated as National Statistics.
ES.3 DfE has committed to making improvements to the statistical bulletin on the impact on COVID-19 for the 2021/22 release (requirement 2) and on the distinction of the accountability measures for the 2022/23 release (requirement 1). The team will also increase the breadth of its stakeholder engagement and share more information publicly on the KS4 development plans (requirement 3) as well as publishing an update on the ICO audit before the 2022/23 release (requirement 4).
Key findings
ES.4 Overall we found good compliance with the Code of Practice for Statistics. Aspects of the statistics demonstrate strong examples of good practice. For example, user engagement, commentary, quality and data linkage and sharing. We found that there are a few areas where improvements could further support the Trustworthiness, Quality and Value of these statistics.
ES.5 Ministers monitor the education system and hold schools to account through the use of accountability measures. These accountability measures are reported with other performance and contextual information in the KS4 statistical bulletin. Although the statistics are reported on impartially, it may not always be clear to users within the statistical bulletin which statistics relate to the accountability measures and which statistics relate to KS4 performance more broadly. The Head of Profession for Statistics has stated that he would be willing to consider alternative ways of presenting the KS4 statistics to make the responsibilities clearer and support trust in the statistics.
Requirement 1: Improve the distinction between the KS4 accountability measures and the wider KS4 statistics by investigating and implementing changes to the presentation of the statistics in the statistical bulletin.
ES.6 Users reported that the level of quality of the statistics was very good and were generally happy with the level of information provided about the limitations, coherence and comparability in past releases. Where users did have concerns, these mostly related to the upcoming 2021/22 release. There have been a number of significant changes to the KS4 statistics over the last few years due to the effects of COVID-19 which mean that not all of the data can be compared over time. It is essential that the 2021/22 release contains comprehensive information on the strengths and limitations of the KS4 statistics and that it is clear which comparisons can be made.
Requirement 2: Ensure users are appropriately informed about the impact of the COVID-19 pandemic on the upcoming 2021/22 release and that they are able to draw their own reliable conclusions from the KS4 performance data by providing comprehensive information on the strengths and the limitations of the statistics and the appropriate comparisons that can be drawn.
ES.7 DfE has a comprehensive understanding of the different users of the KS4 statistics. DfE aims to meet the needs of these different users through providing different statistical products, such as the statistical bulletin, data downloads and the Compare School Performance website. DfE is in regular contact with its key users and receives feedback from them regarding the statistics. However, it is not always clear what the level of engagement is beyond these key users. It would be of benefit to the statistics if a wider range of users was consistently given the opportunity to input into proposed changes and priorities. Most users reported that they were able to get requested data from DfE relatively easily, however a small number reported struggling with this and that they felt that they had to raise formal requests under the Freedom of Information (FOI) Act in order to get the data rather than being able to engage with DfE less formally to get their needs met.
Requirement 3: Ensure users are at the centre of the statistical production process by engaging effectively with the widest possible audience through sharing more information publicly on the development plans for the KS4 performance statistics and by working closely with users to meet their needs.
ES.8 Data sharing and data linkage are particular strengths of the KS4 performance statistics in terms of providing value. The data are uploaded to the National Pupil Database which allows for further linkage and analysis by expert users. DfE provides many shares of these data for multiple purposes. In October 2020 the ICO published a summary of its Data Protection Audit for the Department for Education. Adherence to these recommendations will support DfE with maintaining compliance with the Code of Practice for Statistics. In January 2021, DfE published an initial response to the audit and a further update is currently awaited.
Requirement 4: Ensure public confidence in its data governance procedures, by publishing an update on progress with meeting the recommendations of the ICO audit that relate to the Code of Practice for Statistics and providing assurance that all data governance requirements are being adhered to.