Dear Andrew
Income estimates for small areas in England and Wales
We recently completed our compliance check of the Income Estimates for Small Areas statistics for England and Wales against the Code of Practice for Statistics.
The statistics have a wide range of users and uses, including supporting local government bodies in service planning and delivery, and supporting research into income-based poverty and deprivation. With the UK Government’s levelling-up agenda developing, the user interest in these small area income statistics is likely to grow, as they provide a valuable insight into the current state of small area variations of income.
We were pleased to learn about the collaborative approach between the small area income estimates (SAIE) team and the ONS team currently researching alternative administrative data sources for income measures. Integrating more administrative data including at the unit level with survey results will enable production of more timely and precise statistics that better meet user needs.
As you develop these statistics, particularly working towards using more administrative data, we consider that the following points would add further value:
- You should look to establish a good understanding of the strengths and weaknesses of each of the administrative data sources used to produce these statistics, through an assessment of risk to quality using the Quality Assurance of Administrative Data (QAAD) toolkit. Doing so will help in your own development of methods to manage the risks associated with the quality of the data and provide reassurance to users about the steps that you have taken.
- You should engage proactively with a wide range of users throughout the development process in order to ensure that their requirements are well understood in order to maximise public value. For example, understanding the needs of current non-users who require income estimates at lower levels of geography would be helpful to understand how public value could be best added following the expected increase in the sample size of the Family Resources Survey.
Regarding the presentation of the statistics, we consider that:
- The bulletin explains the methods in appropriate detail and includes a link to a helpful technical document that provides a more in-depth insight into the methodology, including a warning about appropriate use of the statistics to reduce the risk of misuse.
- The bulletin also uses engaging interactive graphics which illustrate the data in a more accessible form for lay users and enables them to make comparisons between incomes estimates before and after housing costs.
- While the estimates are published along with confidence intervals in the datasets showing the range of uncertainty in each area, no such indication of uncertainty is shown in the main bulletin. Some of the confidence intervals are quite large, and this uncertainty around estimates needs to be clearly shown to ensure that the statistics are not misused. For example, the graphic allowing a user to enter a postcode which places the selected area into a percentile plot does not indicate confidence intervals and risks users assuming the estimate is made with absolute certainty.
- It would be helpful to provide information about the comparability and coherence of SAIE with other similar statistics and clarify the distinctive purpose that SAIE serve, to support understanding and interpretation of the data, as well as to prevent its misuse.
We have agreed that you will respond formally to us on progress towards meeting the recommendations in this letter around the time of your next publication. We will review the progress made, with a view to confirming the National Statistics designation then.
We look forward to hearing more about your plans for estimating income at lower-level geographies. Please do not hesitate to get in touch if you would like to discuss any aspects of this letter further or if we can offer further assistance as these statistics continue to develop.
Yours sincerely
Mark Pont
Assessment Programme Lead