Findings

Actions in response to the Professor Sturgis review

The Gambling Commission has drawn on expert input in developing the Gambling Survey for Great Britain (GSGB). It commissioned the National Centre for Social Research (NatCen) and the University of Glasgow to design and collect the data for the GSGB. NatCen is the largest independent and not-for-profit social research organisation in the UK and designs and delivers several survey data collections on behalf of government bodies.

The Gambling Commission has also sought independent expert input to review the GSGB methodology. The Gambling Commission commissioned Professor Patrick Sturgis to undertake an independent assessment of the GSGB methodology and subsequently published his review and findings. This independent expert review provides a valuable external perspective and identifies key areas of focus in the next phase of the survey development. The Gambling Commission has been transparent about the findings of this report, by publishing it in full; this approach supports the Code expectation that producers should be open about identified areas for improvement.

The Commission has published a survey improvement plan to help users understand its intended approach to address Professor Sturgis’ review recommendations. We welcome the publication of this plan, but consider that it would benefit from more detail on how each recommendation will be addressed, the respective timescales and how the learning from this research will be embedded in future waves of the survey. As the Gambling Commission undertakes this work, or commissions external organisations to undertake this work on its behalf, any updates or changes to the survey improvement plan, or the GSGB methodology, should be published and promptly communicated. This approach would enable users to monitor progress on the development of the survey, building user confidence in the GSGB and ensuring openness and accountability.

Quality assurance

Following data collection, the GSGB data undergo quality assurance by NatCen. The data are then passed to the Gambling Commission, who undertake further quality assurance steps. The Gambling Commission provides a high-level overview of the quality assurance processes that it undertakes and also provides information about NatCen’s approach to quality assurance within the technical report. For complete transparency, and to aid user understanding, the Commission should consider bringing this information together into one place.

As part of its quality assurance processes, the Gambling Commission also cross-references the final GSGB statistics with some other relevant statistics and data sources. For example, it has collaborated with the Bingo Association to validate GSGB data about bingo participation. It has compared GSGB data with the Bingo Association’s industry data to understand where there might be differences and why.

As a result of this work, the Commission designed a question in collaboration with the Bingo Association, which was added to the GSGB for 2025, to better understand the places where people are playing bingo and gather more information about this sector.

Information on this validation process is not currently available in the public domain. To enhance user confidence in the GSGB estimates, the Gambling Commission should be transparent about how the data are validated against other data sources to support the accuracy and reliability of the GSGB statistics.

Supporting appropriate use

As with any survey, there are limitations and caveats that users need to be aware of when using and interpreting the statistics derived from it. The Gambling Commission provides information on the survey design, methodology, and data analysis and reporting in the GSGB technical report. We found that the technical report is clear in communicating that GSGB statistics are estimates and are subject to a margin of error. The Gambling Commission has also produced separate guidance on using statistics from the GSGB which provides clear information and examples on how the GSGB statistics can and cannot be used.

The statistical bulletins state that “All surveys have strengths and limitations and we have outlined the strengths and limitations of our approach in the data analysis and reporting section of the technical report.” It is positive that this is acknowledged upfront with a link to the supporting technical report.

The Gambling Commission consulted with three primary stakeholder groups in developing the GSGB guidance: the gambling industry, academics and individuals with lived experience through the Commission’s ‘Lived Experience Advisory Panel’ (LEAP). Several iterations of the guidance were developed based on feedback from these groups before the final version was published.

Despite these efforts, the feedback we received during our user engagement regarding the GSGB guidance was varied. While some users found the guidance to be clear and valued the accompanying technical report, other users felt that the documentation lacked clarity and that the number of caveats undermined their confidence in the data.

The GSGB estimates of gambling prevalence and harm are higher than those from previous face-to-face interview surveys. The reasons for this difference are still being explored and are discussed in more detail in the Professor Sturgis review. For example, Professor Sturgis notes that this issue has been the subject of two investigations, Sturgis and Kuha (2022) and Ashford et al (2022). These studies considered the impact of potential biases; in particular non-response bias and social desirability bias. However, Professor Sturgis notes that “neither study was able to come to a definitive conclusion about the relative magnitudes of these errors nor, as a consequence, which estimates are closer to the truth”. Professor Sturgis concluded that “until there is a better understanding of the errors affecting the new survey’s estimates of the prevalence of gambling and gambling harm, policy-makers must treat them with due caution, being mindful to the fact there is a non-negligible risk that they substantially over-state the true level of gambling and gambling harm in the population”.

As part of this review we have not sought to determine the potential impact of errors, by which we mean potential sources of bias, that affect the GSGB estimates and whether as a result, these estimates are an over- or under-estimate of gambling prevalence and harm. Implementing Professor Sturgis’ recommendations will help the Gambling Commission to understand more about these errors and their impact. However, until the Gambling Commission understands more, users should be made aware of the risk raised by Professor Sturgis that the GSGB may be an over-estimate. Some information is provided on this within the technical report and the guidance for users; however, the Gambling Commission should further support the appropriate use of these statistics by including a clear statement, at the start of each statistical release, on Professor Sturgis’ conclusion that there is a risk that they over-state the true level of gambling and gambling harm in the population. This would ensure that all users are made aware of this limitation without needing to navigate to the technical report or guidance for users; this approach will provide consistency in how this information is explained to users and where it is included.

There have been several instances of misuse of the GSGB statistics in the media, which the Commission has addressed in line with its regulatory responsibilities. This misuse concerns use of the data in ways which are inconsistent with the Commission’s GSGB guidance, such as converting the percentage of survey respondents who scored 8+ on the Problem Gambling Severity Index (PGSI) into equivalent numbers in the population, using the PGSI as a measure of addiction, and directly comparing the GSGB statistics to previous telephone surveys or the Health Survey for England (HSE).

The Gambling Commission’s guidance for users provides key information to support the use and interpretation of the GSGB statistics. However, this guidance is only available through the GSGB landing page and the Power BI report, and is not linked to within the other statistical releases or the technical report. As such, users may not be aware of this guidance, which may contribute to the misuse of these statistics.

To support users, the Gambling Commission should make it easier to find this guidance by providing links in all statistical releases and the technical report. The Gambling Commission may also want to consider other ways to support users, such as the media, in other ways, for example, by providing notes to editors where appropriate.

The Gambling Commission told us that in response to this misuse, it will carry out some research to understand how users interpret the guidance and how it can be improved in future; for example it will seek views via its new statistics user group. We welcome this exercise and encourage the Gambling Commission to seek views from a range of users in carrying out this evaluation to ensure that their views feed into development of the guidance.

The GSGB technical report provides information on the strengths and limitations of the GSGB. We note that there is a section on ‘Non-response and/or selection bias’, which discusses the possibility that the survey under-selects those with lived experience of gambling harms, as they may be unlikely to participate in a survey when they are experiencing gambling difficulties. Elsewhere in the technical report, the Gambling Commission notes that “As the GSGB is ‘gambling focused’, it is possible that the survey disproportionately attracts those who gamble, so that this group may be over-represented”. We consider that it would benefit users to bring these two points together clearly in one section to provide a balanced picture for users.

From our recent conversations with the Gambling Commission, we understand that it has already made progress in addressing some of these findings, and we welcome this further development of the guidance.

Coherence and comparability

The GSGB differs from other surveys previously used to produce official statistics about gambling in methodology, collection mode, questionnaire content, age coverage, sample size and geographic breakdown. Within the GSGB guidance, the Gambling Commission outlines the differences between the GSGB, the HSE and the Gambling Commission’s quarterly telephone survey.

The Gambling Commission is very clear that direct comparisons between the GSGB and these other surveys should not be used to assess trends over time, stating that “Due to differences in the way data for the GSGB is collected in comparison to prior gambling or health surveys, the GSGB is not directly comparable with results from previous surveys and direct comparisons should not be used”. Instead, the Commission explains that the first annual GSGB publication, published in July 2024, represents the first year of a new baseline, against which future annual data from the GSGB can be compared.

However, users will naturally want to compare findings from different surveys on the same topic, especially where the estimates are different. During our user engagement, we found that users sought to understand these differences and their causes. The work that the Gambling Commission is undertaking to address Professor Sturgis’ recommendations will provide more information on how the move to self-completion has affected estimates of gambling behaviours. As such, the degree of comparability between GSGB and the HSE will become clearer over time.

While work is ongoing to determine how the GSGB compares to other surveys that provide information on gambling, it is important that the GSGB estimates are not treated in isolation. There are other statistics and sources of data on the broader topic of gambling that can help to set the GSGB estimates in context. For example, the Gambling Commission itself collects and publishes other statistics and research on gambling behaviour and the gambling industry.

The Gambling Commission should understand and explain the consistency and comparability of the GSGB with other related statistics and be transparent about the potential reasons for these differences; implementing Professor Sturgis’ recommendations will support this understanding. Comparing GSGB estimates with other sources of information, both from within the Commission itself and from sources broader than the previous interview-led surveys, and publishing the findings from this exercise, will help to set the GSGB statistics in context.

Communicating how the GSGB fits within the broader gambling data landscape will help users gain better insight, make effective use of related data and understand which statistics to use in which circumstances. This resonates with the findings from our 2019 review of the Gambling Commission’s Industry Statistics. Within this review, we explained that “You also publish other interesting statistics outside of the main Industry Statistics such as ‘National Lottery statistics – Funds raised for good causes’. In terms of value, these offer interesting supplementary material. The Department for Digital, Culture, Media and Sport (DCMS) also publishes gambling statistics. We consider that integrating your data, and looking to link more closely with the DCMS statistics, could lead to extra value being added through providing a more coherent and thorough narrative”.

The Gambling Commission informed us that as part of its rolling research programme, it has been undertaking a broader piece of work to identify and address gaps in gambling data, and will continue to explore how the GSGB fits into the wider evidence base, as part of its long-term objectives. It is good that the Commission is taking this forward, and we would encourage it to publish its findings.

We also note that Professor Sturgis recognised the value of this type of validation and recommended that the Commission benchmark the estimates from the GSGB against a contemporaneous face-to-face interview survey. In its survey improvement plan, the Gambling Commission explains that this would be unaffordable as a standalone exercise, but we welcome its intention to benchmark estimates from the GSGB against the 2024 HSE when it is published at the end of 2025 and the Adult Psychiatric Morbidity Survey (APMS) in summer 2025. In addition to these surveys, and as well as data provided by the Bingo Association, the Gambling Commission should also consider other potential data sources that could be used to validate the GSGB.

We note that when the HSE is published later in 2025, this will provide an opportunity for the Gambling Commission to openly discuss the findings from the two surveys, the potential reasons for any differences, and the work that is being done to further understanding in this area.

User engagement

In developing the GSGB, the Gambling Commission has convened consultations, forums, workshops and webinars to gather users’ opinions and feedback, some of which are detailed in the development timeline. The Commission also told us about events such as its evidence conference, where the GSGB has been a main item on the agenda for the past three years and has helped it to identify gaps and priorities.

Having an open and effective dialogue with users and stakeholders is vital to demonstrate that the Gambling Commission is a trustworthy organisation that actively listens and responds to users’ views. Regular communication also helps to manage user expectations and build trust. The Commission should build on the user engagement that has been carried out to date and develop a broader strategy to proactively consider how to interact with users and understand their needs. Alongside other development work that the Commission is taking forward, this strategy can help to shape and steer the GSGB’s development.

The Gambling Commission told us that, together with NatCen, it has designed a survey with questions and modules that can be adapted to reflect the evolving gambling landscape. Therefore, it will be essential that the Commission seeks and incorporates user views to enable the survey to remain relevant and reflect how needs may change over time.

The Commission’s engagement strategy should define user groups, how the statistics might be used, and where needs can and cannot be met. In developing this strategy, it will be helpful for the Commission to consider OSR’s framework of user engagement. Additionally, the Government Analysis Function offers comprehensive information on user engagement, which could also aid the Commission in formulating its user engagement approach for future developments.

In developing the GSGB, the Gambling Commission has engaged with a range of users including the gambling industry, academics and individuals with lived experience through the Commission’s ‘Lived Experience Advisory Panel’ (LEAP). Whilst this engagement is good, during our own user engagement opinions varied on the degree to which users felt involved, and on how the Commission has handled feedback.

Some users spoke positively about the time and effort that the Commission invested in the consultation process. Conversely, some expressed a desire for more-frequent stakeholder engagement opportunities and increased transparency about how their feedback is used. Some users reported that they had submitted feedback during the development phase or post-publication but did not receive a response, or felt that the response they received was not sufficient. We are also aware that in some instances, users have requested meetings to discuss their own analysis, but these offers have not always been taken up by the Gambling Commission.

The Code states that feedback should be provided to users about how their needs can and cannot be met, being transparent about reasons for the decisions made and any constraints. The Commission should adopt this approach to help users feel listened to and to understand how their views have been addressed.

Stakeholder engagement

In designing and developing the GSGB, the Gambling Commission has collaborated with several organisations, including NatCen, the University of Glasgow and Bryson Purdon Social Research. The Commission also identified three stakeholder groups to consult on the survey’s developments: an industry group, an academic group and a lived experience group.

By engaging with a variety of stakeholders in developing the GSGB, the Gambling Commission has built a strong foundation for future relationships. To support the ongoing development and production of the GSGB, we encourage the Gambling Commission to collaborate with stakeholders and experts more widely. For example, we noted that whilst there had been some engagement with stakeholders in Wales and Scotland, this could be broader, and should be addressed in future work.

During our user engagement, some users told us that they had not been approached for their input, but they would have liked to participate in the stakeholder engagement process. Therefore, the Commission should more generally review its stakeholder engagement network and ensure it covers a broad range of perspectives.

In discussions with the Gambling Commission, we heard that it has engaged with other official statistics producers who have changed their methodologies. It is good that the Gambling Commission is building partnerships and collaborating with other statistics producers to strengthen good practice within the statistical community, and we would encourage the Commission to continue this approach.

Communicating with users and stakeholders

The Gambling Commission has been transparent in publishing detailed information about the background and development of the GSGB, including its decision-making processes and rationale. There have been significant efforts made in establishing a new GSGB statistics and research landing page which brings together all of the documentation about the survey in one place to help aid navigation. This page contains comprehensive information detailing the planning and development of the GSGB, the survey methodology, the process in developing the questions that measure the adverse consequences from gambling, the guidance on using the GSGB statistics, updates on planned survey improvements, the GSGB data from the most recent survey collection waves and Power BI dashboards to visualise these statistics. The survey publications and development have been communicated through a number of blogs and reports, and a webinar was organised to discuss the publication of the annual statistics.

However, during our review we noted that information was published, and updates were made to documentation on the website, without any communication of these changes. Whilst it is good that the Gambling Commission has produced and published a wealth of material concerning the planning, design and development of the GSGB to date, it needs to ensure that users are informed of its work, aware of its release calendar and alerted to publications, as well as new developments to the GSGB, in a more frequent, timely and transparent way.

To communicate more effectively, the Gambling Commission should develop and implement a communication strategy. This strategy could include methods of communication as well as timelines and any dependencies. For example, an initiative such as a regular newsletter would enhance engagement and transparency and ensure users remain well informed of the latest available information. We understand that, as a result of our findings, the Commission has implemented more regular communication of information related to the GSGB via an e-bulletin, and we welcome this development.

Accessibility and usability

The GSGB statistics are disseminated through a variety of methods to accommodate different user needs: a detailed text-based summary with ‘Key Facts’, an Excel spreadsheet and visualisations using Power BI dashboards.

During our engagement for this review, many users referred to the GSGB statistics as detailed, valuable, clear and easy to understand, describing the summaries and headline statistics as “comprehensive”. Users of the statistics appreciated the questions related to gambling harms, and the adaptive nature of the questions based on topics of interest. The production of timely data, at the level of Great Britain, was seen as useful in identifying trends and capturing a broader scope of gambling information. Users responded positively to the Power BI dashboards and the use of graphs to visualise the statistics. They also noted potential for further research as more data becomes available.

Conversely, some users expressed concerns about the limited data available on the Power BI dashboard and the constraints on comparing data, which they felt reduced their overall usability. Users also indicated the need for more-granular data, the ability to cross-tabulate different datasets and understand the results at a smaller geographic level.

The Gambling Commission explained to us that the dashboard published in July 2024 was the first iteration, and that it intends to develop and improve the dashboard over time. We would encourage the Commission to consider user needs regarding the presentation of these statistics in future developments. Enhancing navigation and usability could be achieved by incorporating hyperlinks to and from the Excel contents page. Additionally, expanding the GSGB Power BI dashboards by increasing the volume of available data and cross-tabulating datasets would offer more in-depth insights.