‘Outlines’ set out the ‘concern’ to which a Monitoring Review relates, describe the issues to be explored in the Review, the geographic coverage of the Review, a broad timetable, and the broad methodology that we expect to follow in conducting the Review. Outlines are prepared before the main work on a Review is undertaken. They serve both as a guide for the team doing the research and as an indication of work in hand for others who might wish to make a contribution to the Review.

The Concern

Hospital Waiting Times statistics have, from time to time, been subject to critical comment by Parliament, media and the public. In 2011, public concerns were raised about the accuracy of waiting times data and subsequent investigations reported misuse of the recording methods. In addition, there has been public comment about the confusing nature of the published statistics. There are a number of high profile statistical releases for Hospital Waiting Times and we are concerned to ensure that these releases are sufficiently clear and well understood to support public choice and public debate.

This Monitoring Review will examine the consistency and reliability of Hospital Waiting Times statistics across all four UK administrations. It will consider the production and publication of the statistics, focusing on elective waiting times collated from hospital data, such as outpatient referrals and subsequent inpatient or day case treatment.

In doing so, this Review will comment on further steps which might be taken by producers of official statistics to:

• assure themselves and the public about the quality of the data
• publish advice and explanations about quality and reliability of the statistics
• present trends and patterns in hospital waiting times, especially around policy targets
• improve coherency of the statistical releases across different health departments
• present comparable statistics across the four UK administrations

Background

A ‘waiting list’ is the number of people waiting for a planned appointment or procedure at an acute or community hospital, whilst ‘waiting time’ is the period of time for which a patient waits for an appointment or other intervention. Patients generally wait to access health care at many stages in the course of their diagnosis and treatment.

There is no single definitively right way to measure waiting times. Every measure is likely to have advantages and disadvantages and to capture different aspects of waiting. For example, it may be argued that the waiting time of the patient on the list is desirable as it includes all the patients waiting – both those who will receive the treatments and those who will not. Patients may not receive treatment for a variety of reason such as choosing alternative treatment, opting for private sector treatment, or to give up the treatment and therefore these patients may never be included in the waiting time of the patients admitted. The waiting time of the patients admitted has the advantage of providing a measure of the completed waiting (as opposed to the uncompleted waiting of the patients on the list).

Waiting times for elective surgery are a significant health policy concern in approximately half of all OECD countries . Long waiting times are correlated with:

• a lower level of government spending on health
• a lower number of acute care beds per head of population
• a lower number of practising physicians per head of population
• the use of salaried remuneration, rather than fee-for-service, for physicians
• level budgets, rather than activity-based funding, for hospitals
• general practitioners (GPs) acting as gate-keepers.

It has been argued that rather than measuring how well hospitals are responding to need, current measures of waiting times tend to reflect capacity within the whole health economy.

Statistics can either be derived from administrative data from admitted patients (throughput data) or from administrative data on patients on the waiting list (census data). The data are recorded by hospital trusts and submitted to the relevant health department for the preparation of waiting times statistics releases. Robust quality assurance procedures, both in hospital trusts and government departments are vital to ensure that the statistics are accurate and reliable.

Depending on their level of need, patients can experience very different waiting times for given procedures and it is important that this variety is captured by, and reflected in, the statistics. Waiting times statistics have become increasingly available on internet websites. It is important for producers of official statistics to provide explanatory notes outlining the characteristics – strengths and limitations in relation to use – of the statistics.

Structure of the Monitoring Review Report

1. Findings and Conclusions
2. The procedures that the statisticians use to assure themselves of the quality of the statistics
   • Outlining the scope of the waiting times data for the purposes of this Review with a focus on the quality assurance processes.
   • Describing how statisticians ensure that the data collected have not been subject to manipulation, the systems in place to warn of potential manipulation and the steps taken to address concerns raised.
3. Informing users about the quality of the statistical outputs.
   a. The uses made of hospital waiting times statistics.
   b. The advice and explanations provided to the public about the quality of the statistics published, in relation to their different uses.
4. The presentation of the statistics
   a. A brief overview of the statistical products with a focus on whether the statistics are presented in such a way that they sufficiently inform public and parliamentary debate.
   b. The promotion of comparability across the UK administrations in the presentation of the statistics.
5. The Authority’s perspective and conclusions

Aspects of the Code
Some of these statistics have previously been assessed or are subject to ongoing assessment; this section will also review actions taken to address recommendations from previous assessments.

Principle 1: Meeting user needs/Protocol 1: User Engagement
P1.2 Investigate and document the needs of users of official statistics, the use made of existing statistics and the types of decision they inform.
P1.3: Adopt systematic statistical planning arrangements, including transparent priority setting, that reflect the obligation to serve the public good.
P1.3 Publish information about users’ experience of the statistical services, data quality, and the format and timing of reports.

Principle 4: Sound methods and assured quality
P4.1 Ensure that official statistics are produced according to scientific principles. Publish details of the methods adopted, including explanations of why particular choices were made.
P4.2: Ensure that official statistics are produced to a level of quality that meets users’ needs, and that users are informed about the quality of statistical outputs, including estimates of the main sources of bias and other errors, and other aspects of the European Statistical System definition of quality.
P4.6 Promote comparability within the UK and internationally by, for example, adopting common standards, concepts, sampling frames, questions, definitions, statistical units and classifications (including common geographic referencing and coding standards). Make the reasons for any deviations from standard models publicly available.

Principle 8: Frankness and Accessibility
P8.1 Provide information on the quality and reliability of statistics in relation to the range of potential uses, and on methods, procedures and classifications.
P8.2: Prepare and disseminate commentary and analysis that aid interpretation, and provide factual information about the policy or operational context of official statistics. Adopt formats for the presentation of statistics in graphs, tables and maps that enhance clarity, interpretability and consistency.

Protocol 3: The use of administrative sources for statistical purposes
Protocol 3.5: Prepare in consultation with the National Statistician a Statement of Administrative Sources.

(It may be that compliance with other aspects of the Code is considered based upon the evidence gathered which could be worthy of discussion within the report)

Process and Methods

Desk research:

• review of statistical releases and metadata
• review of publications by public and parliamentary bodies, other expert groups and media reports (including medical and nursing media)
• review of published hospital inspection reports about the collection of information used to compile waiting times statistics (Care Quality Commission, Healthcare Inspectorate Wales, DHSS in NI, NHS Scotland, NAO)

Collation of documentation from producers:

• service level agreements with data providers
• most recent Statement of Administrative Sources
• latest user engagement action plans and outcomes

Seek views of experts (via email and in person):

• statisticians responsible for producing statistics (in each Administration)
• Key users and stakeholders such as Health Statistics User Group, Monitor, the NHS Confederation, Healthwatch, the Nuffield Trust, the King’s Fund and relevant Patients’ Associations.

Timetable

We are currently gathering and reviewing evidence, and plan to report to the Authority Board in summer 2013.

Your views

We would welcome your views on any of the issues to be covered in this review. Please send any comments to: assessment@statistics.gsi.gov.uk by 23 April 2013 if possible but late responses would still be welcome.