Compliance check of Phonics Screening Check and Key Stage 1 Assessments in England

Dear Mike

Compliance check of Phonics screening check and key stage 1 assessments in England

We recently carried out a compliance check against the Code of Practice for Statistics[1] on Phonics screening check and key stage 1 assessments in England[2]. I am pleased to confirm that these statistics can continue to be designated as National Statistics.

We met with your lead statistician, who was very open and engaged. She told us about forthcoming plans for the statistics and also about your organisation-wide project (DfE dissemination discovery project) to present all DfE’s statistics in a more innovative way including a greater focus on interactive maps.

Value:

We were encouraged to hear about your plans to improve the statistics to give greater insight around attainment in the phonics check. For example, your plans to introduce time-series back to 2012 for more measures will be particularly valuable for users given that the time-series for other key stage assessments only begins in 2016 due to changes in methodology.
Users have shown an interest in attainment in the phonics check by different characteristics over time. Disadvantaged children’s attainment is of interest and it was positive to hear that the team has plans to improve the publication to give more focus on this in the future.
We welcome the fact that for the first time since the introduction of the phonics check in 2012, it has been possible to gauge the potential impact of teaching phonics on children as they move through the school by comparing their phonics check results with their key stage 2 reading attainment (KS2). The KS2 assessments release[3] included information on this and, as discussed with the lead statistician, we think that it would also be useful to see this referenced in the phonics statistics.

Quality:

Some academies can submit data through a different local authority to the one in which the school is geographically based. This can lead to quality assurance issues in particular when validating data against the appropriate school cohorts as there is then a separation between the local authority responsible for monitoring the phonics checks and the local authority submitting the data. We would encourage statisticians to work better with colleagues across local authorities and within the Standards and Testing Authority (STA) to ensure appropriate quality assurance can be carried out for all data. A good, ongoing working relationship between the teams would also ensure that the impact on the statistics of any other change to the format or running of the check could be considered well in advance.

Trustworthiness:

The expected standard threshold for the phonics check is not disclosed to schools in advance. However, it has remained the same since 2012 and the statistics draw attention to this point. The publication includes a graph showing the distribution of marks, which shows a noteworthy change in the distribution at the expected standard every year. Given that these statistics are not published at school level, it is not clear what the incentive would be for schools to unduly influence outcomes. However, the distribution does raise questions about whether knowing the expected standard threshold can impact the results. We were reassured to learn, however, that your statisticians had investigated this phenomenon and did not identify evidence of schools changing their behaviours to influence results. You continue to publish this graph and the expected standard threshold to enable users to understand the full distribution of marks. Education statistics producers elsewhere in the UK face similar issues and there might be value in sharing experiences with them.
We are content with your arrangements for regularly reviewing the pre-release access list, and that there is a clear process in place both for the maintenance of the current list and also for requests to be added to the list.

We look forward to hearing more about DfE plans on the changes in the dissemination of its statistics. We will follow up with the statistics team on the detail behind the points in this letter.

I am copying this letter to Rebekah Edgar, Acting Deputy Director, Education Data Division.

Yours sincerely

Mark Pont
Assessment Programme Lead

[1] https://osr.statisticsauthority.gov.uk/code-of-practice/

[2] https://www.gov.uk/government/statistics/phonics-screening-check-and-key-stage-1-assessments-england-2017

[3] https://www.gov.uk/government/statistics/key-stage-2-and-multi-academy-trust-performance-2018-revised