M&A Note 1/2008

This Monitoring and Assessment Note follows discussion in the media and elsewhere of unexpectedly large estimates of month-on-month changes for the volume of retail sales in Great Britain in May and June 2008.

The Office for National Statistics (ONS) published results of its investigation into the retail sales figures for May and June 2008 in Special Background Notes in June, July and September 2008. We agree with ONS’s conclusion that “estimates … of retail sales in May (and the subsequent decrease in June) reflect the actual changes in the turnover reported by businesses”.

We also note, and regard as good practice, the fact that ONS has published detailed comparisons of the Retail Sales Index (RSI) with the British Retail Consortium’s Retail Sales Monitor. These comparisons conclude that the two sets of statistics differ mainly because of the recent growth in sales by small and medium businesses: these are largely excluded from the British Retail Consortium’s sample in contrast to ONS’s use of a comprehensive and representative sample to measure the activity of the retail sector.

However, we have two suggestions for improving the reporting of the results of the ONS surveys, both of which take account of the fact that all such figures derived from sample surveys are estimates.

Firstly, none of ONS’s estimates of the level or change in the RSI is accompanied by guidance to users on the accuracy of the published figures. We recognise the difficulties in calculating estimates of sampling variability for complex series of this kind, but we consider that even an informal indication of reliability, in numerical terms, would minimise misinterpretations. We note that ONS publishes a revisions analysis for this series, which we regard as helpful, although there is no reason to suppose that the low revisions reported correlate with low sampling variability.

Secondly, we think that the presentation of the various estimates of change in retail sales could be improved to make their usability and interpretation clearer to users. For example, we believe that some users place too much reliance on month to month changes; in a highly volatile situation such monthly estimates are not good guides to longer term trends. This tendency to place too much reliance on a single set of figures strengthens the argument for a numerical indication of reliability. We welcome the publication of underlying trends in the September Special Background Note as a step towards informing users about underlying patterns in data.

We will continue to monitor developments, and we would be pleased to hear the views of users.

Monitoring and Assessment, UK Statistics Authority