Dear Mr. Humpherson,

I am writing on behalf of the Campaign for Better Statistics, possibly better known now as Better Statistics CIC (BSC). We are a Community Interest Company formed just over a year ago with the objective of promoting public awareness of, and interest in, the production of accurate and relevant statistics. Full details are available on our website, including the video of our successful launch meeting last November.

On this occasion we are primarily concerned with the Coronavirus Infection Survey (CIS) and asking whether the OSR considers that the approximate £1 billion cost of the study, combined with the 13% response rate, provides justification for an urgent review of the Code of Practice to include a specific reference to Value for Money (VFM). We believe that in future those responsible for conduct of work of this kind should report on VFM as part of the study protocols and that VFM should also be covered in the procurement process for such work.

Secondly, the Code of Practice does not provide any specific guidance with regard to the methods to be employed on surveys of this kind. For example, there are no ‘good practice’ guides as to how response rates might be improved, nor advice as to the minimum reporting details on response rates that should be provided. Also, there are no guidelines as to how potential biases resulting from uneven response rates should be investigated and how they might be resolved through the weighting used in the estimation.

Our suggestions for adding to the code in this way have arisen from extensive correspondence with the CIS team and others over the past 8 months. In our opinion that correspondence provides a dossier of evidence that the ONS may have ignored your advice to Ian Bell of 17th March 2021 and may also be in direct breach of various aspects of the existing code across all three pillars.

We are therefore reaching out to you for advice on the above matters, including how we (and others) might constructively work with the OSR to improve matters in future.

Although it is our opinion that there should be a detailed review of the CIS and of the Code of Practice, possibly to be undertaken by PACAC and supported by the NAO, such a review would take some time and meanwhile we consider that the importance of measuring the incidence of the disease is such that the CIS should nevertheless continue. However, we believe that its continuation for any length of time (e.g. beyond this June or July) should be subject to an urgent methodological review, possible focused on V5.3 of the code which states: “The suitability of existing data, including administrative, open and privately-held data, should be assessed before undertaking a new data collection.”

Of course, the proposed continuation of the study would not actually represent a new data collection, but we now know a great deal more about the characteristics of the disease and therefore the circumstances are very different from when the CIS methodology was first stablished. Accordingly it is our opinion that a more conventional nationally representative survey design should be considered as an alternative to the existing study; using Hospital Admissions data as a ratio estimator for the prevalence of the disease in the population, both locally and nationally.

We would, of course, be available to contribute to any methodological review, if that would help.

Yours sincerely,

Tony Dent
Director, Better Statistics CIC