06 May 2025

Dear Paul

Thank you for your recent emails to the Office for Statistics Regulation (OSR) regarding your concerns about the available statistics on graduate outcomes in the UK and for your patience while we reviewed your most recent enquiry. We have considered the information you provided, including your summary of analysis and research on the Graduate Premium, your feedback from other interested parties, and your email interactions with the Department for Education (DfE). Thank you for sharing these with us.

Several of the statistical publications to which you refer fall within OSR’s remit as the independent regulator of official statistics, specifically the LEO Graduate and Postgraduate Outcomes, HESA Graduate Outcomes, and Graduate labour market statistics. OSR conducts periodic reviews of such statistics to ensure they comply with the Code of Practice for Statistics; pertinently, in April 2024, OSR assessed that the HESA Graduate Outcomes statistics met the required standards of accredited official statistics.

One of these publications, the Graduate Labour Market Statistics, makes specific reference to a Graduate Premium, reported as an overall average. Having reviewed the information you provided, I recognise that the current framing of the Graduate Premium has limitations and could potentially be misleading for users, especially when presented in isolation from prior academic attainment.

The Graduate Labour Market Statistics publication already contains a caveat warning against the attribution of overall income outcomes solely to graduate qualifications. However, it is not made clear in the publication that the caveat relates to how users should interpret the Graduate Premium statistic. I understand that DfE has acknowledged that the concept of a Graduate Premium could be misleading at face value. On this basis, we expect the DfE to make changes to the publication to more explicitly link its caveat to the Graduate Premium, and to reflect that the statistic should not be used to compare the outcomes of graduates and non-graduates in isolation from prior academic attainment. We would also expect a more comprehensive and accurate definition of the Graduate Premium to be carried forward to any future official statistics publications which make use of the term.

More broadly, I was pleased to see that the DfE has welcomed and encouraged further feedback from you about these statistics, in line with our expectations of producers around user engagement. We will continue to discuss developments in this area through our regular engagement with DfE. We will also consider the quality and value of these statistics in any future formal reviews, which are undertaken periodically.

I am also aware that you would like the Graduate Labour Market Statistics to be broken down by prior academic attainment. We understand that as this publication uses the Labour Force Survey for which prior academic attainment is not available, and so the producer unfortunately is not currently able to produce statistics broken down in this way. Separately, the LEO Graduate and Postgraduate Outcomes publication does present this breakdown for graduates as well as other disaggregations which influence employment and earning outcomes, although I recognise it does not compare comparative earnings for graduates and non-graduates by prior academic attainment.

Regarding the other statistics you reference, the HESA and LEO publications do not refer to the Graduate Premium and do not compare graduate outcomes with those of non-graduates. Prior academic attainment is also not considered in the statistical analysis. I agree with your assessment that it would therefore be inappropriate to calculate a Graduate Premium based on these statistics alone, and without acknowledging the role of prior academic attainment. As you acknowledge in your own research, however, these statistics do have public value by providing insight into graduate outcomes.

More widely, OSR is not in a position to comment on the public discussion about the financial benefits, or otherwise, of higher education. In reviewing your case we did not find any recent examples of official statistics being misused by government to influence the debate; if you are aware of any, please let us know. Nonetheless, you have highlighted an important issue about how the Graduate Premium is presented by statistics producers and understood by the public, for which I am grateful.

I will share a copy of this letter with the relevant team at the Department for Education.

Thank you for taking the time to contact OSR about this matter.

Yours sincerely  

Ed Humpherson
Director General for OSR

 

05 June 2025

Dear Paul

Thank you for your further emails to the Office for Statistics Regulation regarding your concerns about the available statistics on graduate outcomes and the concept of a ‘graduate premium’.  

Since receiving your initial correspondence, we have continued to engage with the Department for Education (DfE) regarding their work and plans for the statistics concerned, particularly the graduate labour market statistics. We understand that DfE’s work includes both immediate improvements to the publication and inviting public views on the longer-term plans for the statistics.  

The latest release of the graduate labour market statistics, published on 5 June, contains several positive changes based on our recent feedback. In addition to the previously stated caveats around the limitations and use of the statistics, these changes include the explicit linking of those caveats to all sections of the release, where relevant, to remind users how they should interpret the statistics, and again caution them against comparison due to the statistics not controlling for external factors such as prior attainment. DfE have also removed the term ‘graduate premium’ from this release to avoid any direct comparisons in salary between postgraduates, graduates and non-graduates being made.  

The Code of Practice for Statistics sets out clear requirements for producers in relation to the Trustworthiness, Quality and Value of official statistics. In particular, the Code establishes that statistics producers should maintain and refresh their understanding of the use and potential use of the statistics and data. They should consider the ways in which the statistics might be used and the nature of the decisions that are or could be informed by them. To do this effectively, the views of a range of users should be sought and, where practical, addressed. We are pleased to see DfE taking your views on board and we recognise that your requests will need to be balanced against the needs of wider users.  

I have been pleased by DfE’s engagement with us on this matter thus far, and we expect this to continue to be mirrored in the department’s interactions with users. The most recent release of the graduate labour market statistics shares DfE’s plans to further improve these statistics, looking at alternative, more robust sources to better meet user needs which the public are invited to provide feedback on. I encourage you to continue to engage with DfE as this work progresses. 

I will share a copy of this letter with the relevant team at the Department for Education.  

Thank you for contacting OSR about this matter. 

Ed Humpherson
Director General for OSR 

---

22 March 2025

My case has been open for several months now and I would like to now focus in on one issue only in an effort to achieving a positive result.

By far the biggest issue is that the Graduate Labour Market Statistics publishes a Graduate premium as an overall average only instead of breaking it down by prior academic attainment, which would be an effective way to measure marginal graduates premium.

The RSS (in their comment in my report) and the IFS agree with me that we need better data regarding returns for marginal candidates.

Just publishing the overall average Graduate Premium is dangerous and completely misleading and is resulting in serious harm to society with hundreds of thousands of young adults enticed by the HE Sector into getting degrees that only end them up in debt with no better career prospects.

My published report should give you all the evidence that you need. However, I now present fresh evidence of a ‘statistical anomaly’. This is explained in the below email to the IFS along with the attached spreadsheet, which very clearly illustrates that this statistic should no longer be published until a way is found to publish the grad prem broken down by prior academic attainment and not just overall average.

I call upon the Office for Statistics Regulation to intervene in this serious matter by requesting that the Graduate Labour Market Statistics in future should include a breakdown of the Grad Prem broken down by prior academic attainment.

Annex A – Extract from email to IFS

The phrase ‘Graduate Premium’ is used quite extensively, and I am sure has a different definition for different people. The only Govt stat that coins the phrase is the Graduate Labour Market Stats based on the LFS data. And it defines it as the difference between the Median average pay of Graduates less the Median average pay of Non-grads. I have been working on my report for 9 months now and I remain astounded that I appear to be the only person who has noticed how staggeringly flawed this method of defining the graduate premium is, and by publishing it this way, how wide open it is to willful misrepresentation.

I have attached a spreadsheet showing a couple of examples of how misleading this method of defining the Grad prem can be if you only report the overall average that is highlighted when you breakdown the Grad Prem into 10 bands of prior academic attainment *

Scenario A – This shows the outcome if you have a population where the graduate pay is the same as non-graduate pay within ALL of the bands of prior academic attainment. i.e. A graduate with 3 A’s earns the same as a Non grad with 3 A’s , a graduate with 3 C’s earns the same as a Non-graduate with 3 C’s etc. etc. So even though there is ZERO graduate premium for every single one of the prior academic attainment bands, then you still get a ‘healthy’ looking overall average Graduate Premium – in this case £10.8k .

Scenario B – This shows the ridiculous situation that even if you have a population where the graduate pay is 10% LESS than non-graduate pay within ALL of the bands of prior academic attainment, then you still get a positive overall average Graduate Premium – in this case £6.8k .

Scenario C- This shows a more realistic hypothetical scenario where grad pay is higher than non-grad pay in the higher prior academic attainment bands, but then Grads & Non grads pay converges as you drop down the academic bands and after around 30% they converge. It shows an overall grad prem of £11.2k , but only the top three bands actually are getting any premium at all. This shows that you need this 10 band breakdown (and not just the overall average) because this will show where the convergence point occurs, and thus can be used to inform policy makers where line should be drawn in terms of %HE participation rates.

* NB – For calculation purposes, this is based on the Mean average and not the median, but there will be a similar phenomenon if the median was used.

So the problem lies in that if only one figure of the overall average grad prem is published then the mere existence of this being a positive figure is interpreted /mis-interpreted as follows : –

1 – Correct interpretation – People with higher academic ability (who on average will attend HE in higher numbers) on average earn more than those with lower academic ability (who on average will attend HE in lower numbers)

2 – Incorrect Interpretation – Everybody should enter HE no matter what their level of prior academic attainment as they should all anticipate improving their pay in accordance with this average premium . i.e. Act as if everybody is going to be able to tap into this average grad prem no matter what their prior academic attainment and no matter how high the %HE participation rate rises.

This incorrect interpretation completely ignores the inevitable consequences of diminishing marginal returns, and by publishing only the overall average, then the Graduate Labour Market Stats is effectively condoning this concept to be ignored.

In my individual efforts to correct this situation, I have an open case raised at the moment that has been running since last summer with the Office for Statistics Regulation where I am attempting to get them to intervene and suggest to Govt policy makers that we need to urgently improve the data for marginal candidates by also publishing the Grad Prem by prior academic attainment (Case ref No 2024 / 101 – The most consequential Statistical Misunderstanding of our generation).

Related attachment:

Graduate_Premium_Statistical_Anomaly_Prior_Academic_Attainment_Overall_Average

---