UK Statistical System UK Statistics Authority Code of Practice
UK Statistics Authority Correspondence

Professor Dame Carol Propper to Seamus Logan MP: Statements on water quality

This letter was sent from Professor Dame Carol Propper, and is also available on the UK Statistics Authority website.

Dear Mr Logan,

Thank you for your email of 5 September to the Acting National Statistician regarding comments by the former Secretary of State for Environment, Food and Rural Affairs, that drew comparisons between England and Scotland’s water industry regulation.

I respond on behalf of the UK Statistics Authority’s Regulation Committee, which oversees the independent regulator, the Office for Statistics Regulation (OSR). Whereas normally the Chair of the Authority would respond to your concern, she has instead asked me to respond in her place, to avoid the risk of a perceived conflict of interest with another role.

Judgements about regulation of the water industry and broader environmental policy are rightly for you and your colleagues in the UK Parliament and devolved assemblies to consider. However, we have looked at some of the statements made as part of our role enforcing the Code of Practice for Statistics; a full list of these is enclosed in Annex A.

In brief, we consider that these statements lacked enough transparency about their sources to be verified, and that the broad evidence did not support them. Without appropriate discussion of the limitations of some of the more specific figures quoted, they run the risk of misleading the public.

As you point out, the then-Secretary of State said several times that water pollution and water quality are worse in Scotland than in England. He made these claims repeatedly in the House of Commons, as well as in broadcast interviews and online. We also note his letter of 24 July to Stephen Flynn MP published on X, which cites a range of statistics in response to the dispute over his claims.

The table of figures appended to the letter is not relevant to water pollution and water quality, but in the body of his letter he further claimed, without providing a source, that in 2023-24, there were 35.8 pollution incidents per 10,000km of sewerage network in Scotland, and 35.4 in England and Wales combined.

The Department for Environment, Food and Rural Affairs told us that its source for Scotland was an analysis of Scottish Water’s annual report, and for England and Wales, Ofwat’s water company performance report, wherein pollution incident rates are broken down by company. The sector average given for both England and Wales combined is 35.42. These water companies’ territories do not neatly match national boundaries, so it is not simple to calculate pollution incidents in England alone.

Underlying regulatory data on pollution incidents are not yet available for Wales or Scotland, although an Environmental Standards Scotland report claims Scottish Water has a lower incident rate (36) than English water companies (41), when including incidents from water supply assets. However, there are many differences in monitoring and reporting of pollution incidents data, so the Independent Water Commission (IWC) cautions against making these comparisons between countries. It is important to provide this context when referring to them in public debate.

The IWC concluded its review of the water sector on 21 July, and in its final report there are several sets of statistics directly relevant to water quality and other environmental data. It would have greatly improved public understanding to draw upon these figures, given their relevance, quality, and importance to the Government’s stated ambitions. The figures show broadly that Scotland has a similar or better share of bathing and surface water sites that meet good or excellent standards for water quality. The OSR has published a further analysis of this topic, which I enclose in Annex B.

The Authority expects that ministers take care to avoid using data that is overly selective or missing appropriate context. Based on the statements made without discussion of their context, sources, and limitations, there is the potential for people to be misled about English and Scottish water quality and infrastructure. As former Chairs have said, omitting this kind of information can damage public trust in the data, so we encourage those speaking on behalf of Government to ensure statistical statements are presented clearly and transparently, in a way that supports public understanding.

 

Yours sincerely,

Professor Dame Carol Propper
Chair of the Regulation Committee

ANNEX A

  • “[…] you were just hearing about Scottish Water, they’re nationalised, pollution in rivers in Scotland is worse than in England”
    Sunday with Laura Kuenssberg, BBC One, 20 July
  • “In Scotland they have a nationalised water company, but pollution levels in Scotland are worse than they are in England“
    Channel 4 News, 21 July
  • “Scotland has a nationalised water company… and water pollution is worse than in England as a result”
    House of Commons, 21 July and republished online at X.com
  • “… under the nationalised model in Scotland, pollution is worse, not better.”
    House of Commons, 21 July
  • “Official statistics […] show several areas where Scotland’s water quality underperforms relative to England.”
    Letter to Stephen Flynn MP, 24 July
  • “There were 35.8 incidents per 10,000km of sewer in Scotland versus 35.4 incidents per 10,000km of sewer in England and Wales reported in 2023-24.”
    Letter to Stephen Flynn MP, 24 July
  • “OFFICIAL: The SNP are managing water pollution in Scotland even worse than the Tories did in England”
    @SteveReedMP on X.com, 25 July
  • “Levels of pollution in England are bad enough, but under the SNP in Scotland they are even worse”
    House of Commons, 4 September
  • “I have published the data and I stand by it: pollution under the SNP in Scotland is even worse than it was under the Tories in England.”
    House of Commons, 4 September

Annex B

OSR statement on statistics concerning aspects of water in Scotland and England

Letter from Sir Robert Chote to Jeremy Balfour MSP – Health and Wellbeing Census Scotland

This letter was sent from Sir Robert Chote, Chair of the UK Statistics Authority, and is also available on the Authority website.

Dear Mr Balfour

Thank you for your letter raising your concerns about the Scottish Health and Wellbeing Census (HWBC).

Many of the issues about data governance and sharing procedures discussed by your constituents are out of remit for the Office for Statistics Regulation (OSR) and fall within the remit of the Information Commissioner’s Office (ICO). However, I have addressed your comments relating to the OSR’s letter to the Scottish Government, published in July 2022.

This letter set out that the Scottish Government needed to review the approach it took to question development including the “legal and ethical governance arrangements that are in place for asking questions of each age range”. Please note that the OSR did not request the Scottish Government to undertake a review of the survey and data governance processes as a whole.

It is regrettable that the outcomes of this review have not yet been made public and the OSR has now written to the Scottish Government to set the expectation that this is rectified within the next 30 days.

Yours sincerely,

Sir Robert Chote
Chair

 

Related links

Ed Humpherson to Alastair McAlpine: Scottish Health and Wellbeing Census

Jeremy Balfour MSP to Ed Humpherson – Health and Wellbeing Census Scotland

Letter from Sir Robert Chote to Paul O’Kane MSP: Scottish Government child poverty statistics

This letter was sent from Sir Robert Chote, Chair of the UK Statistics Authority, and is also available on the Authority website.

Dear Mr O’Kane,

Thank you for your letter expressing concern about statements made by both the current and previous Scottish First Minister that Scottish National Party (SNP) policies are “lifting” an estimated 100,000 children out of poverty.

Under the principles of intelligent transparency, it is always advisable to think how an average person would interpret a high-profile quantitative claim of this sort so as to minimise the danger of them being misled and thereby trust in similar statements being undermined.

In this instance, the First Ministers were referring to a modelled estimate of the difference between the level of relative child poverty expected in 2024/25 and the level that we would have seen in the absence of a number of policy measures (as set out in footnote 4 of the updated Cumulative Impact Assessment). The Scottish Government set out the methodology behind this comparison in an annex to its Cumulative Impact Assessment in 2022 and the 100,000 estimate comes from the update published in 2024.

This kind of analysis is a reasonable way to estimate the impact of Scottish Government policies on child poverty, even though, just like alternative estimates, the calculations are bound to be uncertain and dependent to some degree on methodological choices. But the average person hearing such a statement might well assume that the First Ministers were claiming that child poverty is 100,000 lower than when the SNP took office. And, as you point out, the Scottish Government’s official statistics on Poverty and Income Inequality in Scotland conclude that the proportion of children in Scotland living in relative and absolute poverty remains broadly stable. Comparing the number of children living in relative poverty from 2004-07 (pre-SNP) to the latest datapoint of 2020-23 shows a decrease of 10,000 children (250,000 to 240,000). For children living in absolute poverty the decrease is 40,000 children (250,000 to 210,000).

Given this potential confusion, Ministers would be well advised from time to time to accompany this type of claim with a reminder of the methodology underpinning it so that they are not suspected of making an unduly flattering comparison.

Yours sincerely,
Sir Robert Chote
Chair

Related links:

Paul O’Kane MSP to Sir Robert Chote – Scottish Government child poverty statistics

 

Sir Robert Chote to Rt Hon Richard Holden: Party spending claims

This letter was sent from Sir Robert Chote, Chair of the UK Statistics Authority, and is also available on the Authority website.

Dear Mr Holden,

Thank you for your letter of 6 June regarding the Labour Party’s analysis of Conservative Party commitments and its own plans.

In my recent letter to political parties, I asked that parties and candidates use statistics appropriately and transparently during this general election campaign and made clear why these expectations are in the interests of the public and those campaigning. These expectations were echoed in a statement published by the Office for Statistics Regulation regarding the claim that “a Labour government would mean £2,000 of tax rises per working household”. Many of the principles set out in that statement apply also to the claim you highlight, that the Conservatives have “£71 billion of unfunded spending plans”.

This figure derives from Labour’s 25 May document Conservatives’ Interest Rate Rise which sets out their costings of nine future ‘policy decisions’ and refers to roughly £71 billion of net extra spending in fiscal year 2029-30. In another document, Tory Manifesto Costings published on 13 June, the Labour Party claimed that Conservative manifesto plans would amount to net extra spending of roughly £71 billion over the next five fiscal years put together and “raise people’s mortgages by £4,800” cumulatively over that period.

Future costings are always subject to uncertainty and dependent on choice of methodology. To help people understand the assumptions that have gone into costing models, it is essential that the underlying calculations, data sources and context are provided alongside the figures. When distilling these claims into a single number, there should be enough context to allow the average person to understand what it means and how significant it is. Omitting this information can damage trust in the data and the claims that these data inform.

To safeguard trust in official statistics, we encourage that statistical claims are presented clearly and transparently so that the public can test the arguments, and descriptive statements, that political candidates make about them.

Yours sincerely,
Sir Robert Chote

Letter from Sir Robert Chote to Lord Bailey of Paddington AM: Housing statistics

This letter was sent from Sir Robert Chote, Chair of the UK Statistics Authority, and is also available on the Authority website.

Dear Lord Bailey,

Thank you for your letter to the National Statistician regarding the Mayor of London’s use of housing statistics in a LabourList article from March 2024. This was passed to the Office for Statistics Regulation (OSR), the regulatory arm of the UK Statistics Authority.

The Mayor’s comments appear to draw from the Greater London Authority’s (GLA) Affordable Housing starts and completions statistics. The statistics are broken down into house building ‘starts’ and ‘completions’. These definitions are clear and describe a real-world picture of what the statistics represent: whether a dwelling has started construction or whether construction has finished.

There were 25,658 affordable housing starts in the period April 2022 to March 2023, and 13,954 affordable housing completions during this time. The Mayor’s use of the word ‘delivered’ could be misinterpreted by the average person to mean the housing had been completed rather than started. While the Mayor would not have had access to the statistics for the equivalent period in 2023/24 at the time of writing the article, the number of affordable housing starts and completions for 2023/24 was 2,358 and 10,949 respectively. The reference to ‘within the last year’ may have been intended to reflect the latest year of statistics that were available, but this would not be clear from the statement alone.

In line with the principles of intelligent transparency, when making numerical claims, public figures should be clear what they are referring to and consider how a reasonable person would interpret the claim. To help the public understand statistical claims made during the General Election debates, the OSR has recently published a series of explainer articles including on housing supply and affordability statistics.

Yours sincerely,
Sir Robert Chote
Chair

 

Related links

Lord Bailey of Paddington AM to Sir Ian Diamond – housing statistics

Letter from Sir Robert Chote to Karin Smyth MP: NHS waiting lists

This letter was sent from Sir Robert Chote, Chair of the UK Statistics Authority, and is also available on the Authority website.

Dear Ms Smyth,

Thank you for your letter of 10 May regarding a statement made by the Secretary of State for Health, the Rt Hon Victoria Atkins MP, who said “we’ve cut waiting lists for five months in a row”. This was made in a video posted by the Department of Health and Social Care on X on 20 April. You raised a concern that this statement appeared to overlook a change to the data collection.

The guidance on the recording and reporting of Referral to Treatment (RTT) data was updated on 2 February 2024 to say that community service pathways should, from February 2024, no longer be reported in RTT datasets and should instead be captured in community health services data collections. These changes were announced appropriately by NHS England, with the estimated impact published in an ad-hoc management information excel file. The Referral to Treatment (RTT) statistics for 2023/24 clearly indicate a break in the data series, with a dotted line appropriately separating out February and March 2024, where the changes to submitted data started.

In the statistics release, NHS England estimated that the discontinuity between January and February 2024 was ‘around 36,000’. As the reported waiting list decreased by 36,198 between end-January and end-February, this results in a small decrease of around 198 once the discontinuity is taken into account. This followed falls in each of the four months from end-September to end-January. The Secretary of State’s statement was supported by the data, but greater transparency around the discontinuity in the time period being referred to would have better supported understanding.

Yours sincerely,
Sir Robert Chote
Chair

 

Related links:

Karin Smyth MP to Sir Robert Chote – NHS waiting lists

Letter to leaders of political parties in advance of the July 2024 general election

This letter was sent from Sir Robert Chote, Chair of the UK Statistics Authority, and is also available on the Authority website.

Sir Robert Chote, Chair of the UK Statistics Authority, has written to the leaders of the UK’s political parties reminding them of their duties to use statistics responsibly during the election period, 4 June 2024.

To: Leaders of UK political parties (full list of recipients at the bottom of this letter)

I am writing to ask for your support and commitment in ensuring the appropriate and transparent use of statistics during this general election campaign.

The work of the UK Statistics Authority is underpinned by the conviction that official statistics should serve the public good. This means that when statistics and quantitative claims are used in public debate, they should enhance understanding of the topics being debated and not be used in a way that has the potential to mislead.

To help government departments, public bodies and others do this day-to-day, the Office for Statistics Regulation has set out principles of ‘intelligent transparency‘. These demand that statistical claims and statements are based on data to which everyone has equal access, that are clearly and transparently defined, and for which there is appropriate acknowledgement of any uncertainties and context that people need to be aware of if they are realistically to interpret what they mean.

Adhering to these principles is just as important when parties are using statistics and making quantitative claims to help their candidates seek public office. Needless to say, the use of statistics in political communication is often necessarily succinct and devoid of lengthy explanation. Under these circumstances, a good rule of thumb is to consider how a reasonable person would interpret the statement being made and ensure that this is not likely to be misleading in the absence of additional information.

In the case of the governing party, it is also important that any public statements made during the pre-election period refer only to official statistics and data that are already in the public domain and not those to which ministers have privileged access.

Adhering to the principles of intelligent transparency is in the interests not just of the public but campaigners themselves. It avoids the need for subsequent clarifications and possible loss of trust in future claims, either later in the campaign or in government. The Office for Statistics Regulation will be monitoring the use of statistics in public debate through to polling day and we will be willing to highlight publicly where statements draw on statistics and data that are not published or are presented in a misleading way.

I am sending identical letters to the leaders of the main political parties, with a copy to the Cabinet Secretary. The Head of the Office for Statistics Regulation has written to Permanent Secretaries of Civil Service departments in similar terms.

Yours sincerely,

Sir Robert Chote
Chair

Full list of recipients:

Rt Hon Rishi Sunak (Conservative);
Rt Hon Sir Keir Starmer (Labour);
John Swinney MSP (Scottish National Party);
Rt Hon Sir Ed Davey (Liberal Democrats);
Rt Hon Gavin Robinson (Democratic Unionist Party);
Mary Lou McDonald TD (Sinn Féin);
Rhun ap Iorwerth MS (Plaid Cymru);
Rt Hon Alex Salmond (Alba);
Colum Eastwood (Social Democratic and Labour Party);
Naomi Long MLA (Alliance);
Carla Denyer and Adrian Ramsay (Green);
Nigel Farage (Reform UK);
George Galloway (Workers Party of Britain); and
Simon Case (Cabinet Secretary).

Related Links:

Press Release: UK Statistics Authority urges party leaders to use statistics responsibly in the General Election

Sir Robert Chote to Andrew Gwynne MP: Social care funding

On 01 May 2024, our chair,  Sir Robert Chote wrote to Andrew Gwynne MP regarding comments made by the Chancellor of the Exchequer on 7 March, during an interview on ITV’s Good Morning Britain. During this interview, the Chancellor said: “On social care, in my first Autumn Statement I put up the budget by nearly 40 per cent, over £5 billion or nearly £5 billion.”

Read the full letter here: Sir Robert Chote to Andrew Gwynne MP – social care funding

Sir Robert Chote to Dame Angela Eagle MP: statements on tax changes

On 19 February 2024 our chair, Sir Robert Chote, wrote to Dame Angela Eagle MP regarding comments made by Treasury ministers on changes in personal taxation.

The Office for Statistics Regulation is increasing its engagement with government departments, including HM Treasury, to ensure future communications do not have the potential to mislead and comply with the principles of intelligent transparency.

Read the full letter:

Letter from Sir Robert Chote to Dame Angela Eagle MP – statements on tax changes