Administrative Data Quality Assurance Toolkit

Annex A: QA Matrix

Level of Assurance	Areas of practice related to quality assurance of administrative data regularly provided for producing official statistics
	Operational context & administrative data collection	Communication with data supply partners	Communication with data supply partners	Communication with data supply partners
A0: No assurance	Context and data collection by supplier not investigated, managed or documented	No communication	No description of suppliers' QA procedures and standards	No description of own QA checks
A1: Basic assurance Statistical producer has reviewed and published a summary of the administrative data QA arrangements	Consider the following types of activities: Producer has provided users with an outline of the administrative data collection process, Illustrated the administrative data collection process and main stages, Outlined the operational context, Identified actions taken to minimise risks to quality, Identified and summarised the implications for accuracy and quality of data, including the impact of any changes in the context or data collection	Consider the following types of activities: Producer has outlined the data provision arrangements including: annual statement of needs, timing and format of data supply, coordination of data sign-off from supplier, Fed back identified errors to data suppliers and recorded their response, Sought the views of statistics users about the data and resolved any quality issues reported	Consider the following types of activities: Producer has knowledge of suppliers' QA checks and published a brief description, Identified whether audits are conducted on the admin data (such as internal or operational audits, external audit such as by regulator), Described the implications for the statistics	Consider the following types of activities: Producer has established regular QA checks on the received admin data, Published a description of its own QA checks on the admin data, Outlined general approach and findings, Identified the strengths and limitations of the admin data, Explained the likely degree of risk to the quality of the admin data
A2: Enhanced assurance Statistical producer has evaluated the administrative data QA arrangements and published a fuller description of the assurance	Consider the following types of activities: Producer has provided users with a fuller description of the operational context and administrative data collection arrangements, eg: a process map detailing data collection processes, explanations for classifications, Identified and summarised potential sources of bias and error in administrative system, Identified and described safeguards taken to minimise risks to data quality, Provided a detailed description of the implications for accuracy and quality of data, including the impact of any changes in the context or collection arrangements	Consider the following types of activities: Producer has agreed and documented: data requirements for statistical purposes, legal basis for data supply, data transfer process, arrangements for data protection, sign-off arrangements by data suppliers, Established an effective mode of communication with contacts (eg with data collector and supplier bodies, IT systems, operational/policy officials) to discuss the ongoing statistical needs in the data collection system and quality of supplied data, Sought the views/experiences of statistics users and resolved any quality issues reported	Consider the following types of activities: Producer has provided a fuller description of the main QA principles, quality indicators and checks used by the data suppliers, Described the role of relevant information management or governance groups in data quality management, Described the role of audit of the admin data within the collection and operational settings, Described the implications for the statistics for the quality issues identified by data supply bodies and regulators	Consider the following types of activities: Producer has provided a fuller description of its own QA checks on the admin data, Detailed the general approach and findings for specific quality indicators, Identified the strengths and limitations of the admin data, Explained the likely degree of risk to the quality of the admin data
A3: Comprehensive assurance Statistical producer has investigated the administrative data QA arrangements, identified the results of independent audit, and published detailed documentation about the assurance and audit	Consider the following types of activities: Producer has provided users with a detailed description of the administrative system and operational context: explained why the data are collected, who by and how, identified differences across areas in the collection and recording of the data, identified issues for individual data items, such as whether objective or based on subjective recording, missing and/or imputed, poorly recorded, Identified issues in design and definition of performance measurements and targets, Identified and described potential sources of bias and error in the administrative system, Identified and explained any safeguards used to minimise the risks to data quality, Provided detailed description of the implications for a quality of the data, including the impact of any changes in the context or data collection	Consider the following types of activities: Producer has established/maintained collaborative relationships, Has a written agreement specifying: roles and responsibilities, legal basis for data supply, data supply and transfer process, security and confidentiality protection, schedule for data provision, content specification, Used a change management process, Regularly communicated with the data collector and supplier bodies, IT systems, operational/policy officials eg newsletters, conferences, attending data supplier/IT system group meetings, Regularly engaged statistics users, resolved any reported quality issues, and held user group conferences	Consider the following types of activities: Producer has described the data suppliers' principles, standards (quality indicators) and quality checks, Reviewed quality reports for the received data (such as input quality indicators for data accuracy, coverage and completeness), Identified and documented the findings of investigations and audits conducted on the admin data and associated targets (such as internal and operational audits, and external audits by regulators and professional bodies), Described the implications for the statistics and determined whether the data continue to be satisfactory for official statistics purposes	Consider the following types of activities: Producer has provided a detailed description of its own QA checks on the admin data (including validation, sense and consistency checks), Given quantitative (and where appropriate qualitative) metrics for specific quality indicators (such as input, process and output quality metrics), Undertaken comparisons with other relevant data sources (such as survey or other admin data), Identified possible distortive effects of performance measurements and targets, Identified the strengths and limitations of the admin data and any constraints on use for producing statistics, Explained the likely degree of risk to the quality of the admin data

« Previous

Download PDF version (243.41 KB)