Communication

This section summarises the evidence identified on the topic of statistical communication. In particular, on how to communicate effectively to non-specialist audiences to maximise understanding and engagement.

Due to the breadth of this evidence, it has been broken down into categories covering a wide range of factors, from considering the target audience, to how to communicate uncertainty within statistics.

4.1 Audience

The first factor to consider when developing statistical communication materials is the target audience (UNECE, 2009). In a review of risk and uncertainty communication published by David Spiegelhalter in 2017 (Spiegelhalter, 2017), it was recommended to consider the target groups of the communication and identify their needs, beliefs, and skills. This recommendation followed a recurring finding within the review that the best approach to communicating information can vary substantially depending on the characteristics of the audience.

A study comparing understanding of health-related statistical information, communicated using either numerical or graphical representations, found that the result was dependent on the graph literacy of the participant. In a study by Gaissmaier et al., (2012), each participant’s graph literacy was assessed using a scale which assessed an individual’s understanding of health-related information conveyed using graphical representations (Galesic & Garcia-Retamero, 2011). Participants with high graph literacy demonstrated greater comprehension and recall of the statistical information when presented with a graphical representation, whereas the reverse result was observed for adults with low graph literacy scores (Gaissmaier et al., 2012). Similarly, another study found that levels of numeracy had a significant impact on how participants perceived risk communicated either via frequencies or percentages (Peters et al., 2011). In particular, less-numerate participants presented with risk information in a percentage format perceived medication as less risky than when presented with the same information as a frequency. Whereas highly numerate participants perceived similar risks in both formats. These studies highlight the importance of considering the characteristics of the target audience when designing statistical communication materials.

Although some research has already been conducted on characteristics to consider, such as graph literacy (Gaissmaier et al., 2012), and numeracy (Kreuzmair et al., 2016; Peters et al., 2011), it is also recommended to carry out your own development process of testing and evaluating materials within target groups (Spiegelhalter, 2017). This process was undertaken in the development of infographics to communicate personal risk from COVID-19 (Freeman et al., 2021). This began with an initial round of qualitative interviews both with members of the general public and with primary care physicians, followed by further rounds of refining the material design in qualitative interviews as well as quantitative experiments in large samples of participants.

This process provided valuable insight such as finding out that the general public does not consider their risk from COVID-19 in a numerical way. A consequence of this finding was a recommendation to translate numerical risk to match more closely with the user’s own subjective experience. This included a more qualitative description of risk including a reference point of a person with known health risk factors of a particular demographic. Subsequent research further highlighted that using illustrative personas with clear risk factors provided useful context to enable users to understand their level of risk from COVID-19 (Freeman et al., 2021).

Following a series of workshops involving economists and the general public, it was also recommended to conduct future public engagement studies with economists and the general public to discuss economic issues and gain information on how to improve communication with the public (Runge & Killick, 2021). This research focussed on economic statistics, but this recommendation may be generalisable to other areas of statistics.

As well as aligning communication with the audience’s thinking on the topic, it has also been recommended to use language suitable for the target audience (Spiegelhalter, 2017). Understanding may also be improved by emphasising the relevance of the information to the audience’s lives. Public understanding of Economics and Economic Statistics was captured in a 2020 report published by the Economic Statistics Centre of Excellence (Runge & Hudson, 2020). A series of studies involving 12 focus groups and an online survey of UK adults explored understanding in areas such as inflation, unemployment, and GDP. Overall, the results indicated that public understanding was greatest for areas where personal relevance is perceived to be higher such as inflation and interest rates rather than GDP (ESCoE, 2020). One approach to enhancing the relevance of statistics could be breaking down high-level (e.g. country-level) statistics into local or demographic-based statistics so that users can access those most relevant to them. The tailoring of statistical information to enhance relatability was shown to be effective in improving public comprehension and trust in research conducted by the Bank of England and the Behavioural Insights Team. This study is discussed further in subsequent sections (Bholat et al., 2018).

4.2 Context

The second theme that emerged from this part of the review was the importance of providing contextual information with statistics. Broadly this involves ensuring statistics are always contextualised so that audiences can comprehend their significance (BBC Trust, 2016). This also encapsulates more specific recommendations such as including baseline risk when discussing changes to risk (BBC Trust, 2016), emphasising personal relevance to the audience (Bholat et al., 2018; Runge & Hudson, 2020), and establishing a narrative within the communication materials to improve understanding (Runge & Hudson, 2020; Spiegelhalter, 2017; UNECE, 2009). This narrative should be vivid but should not cause undue affective responses (Spiegelhalter, 2017).

Another important aspect of a statistic’s context is its source. In a series of recommendations, developed by authors at the Norwegian Institute of Public Health on how to communicate evidence-based information about the effects of healthcare interventions, it was advised to provide relevant background information (Oxman et al., 2020). This could include how the information was put together, what it is based on, the people who put it together and whether they have any conflicts of interest. It was stated that this would allow users to understand why, and if, information is trustworthy (BBC Trust, 2016; Oxman et al., 2020; Runge & Hudson, 2020).

Lastly, the importance of choosing appropriate comparators was highlighted by several sources. This includes Spiegelhalter (2017), who highlighted that comparators are useful when communicating risk to people with low numeracy, but some comparators can be associated with an emotional response. For example, the risk of being struck by lightning is a poor comparator because it is newsworthy and is therefore perceived as more likely than it is. Therefore, its use as a comparator would lead to misperception.

4.3 Language

The next topic surrounds language use within statistical communication. Overall, the message observed consistently across several articles is to use simple and easy-to-understand language (Bholat et al., 2018; Oxman et al., 2020; Spiegelhalter, 2017; The Plain Numbers Project, 2021; UNECE, 2009). In particular, one should avoid using technical language or ‘jargon’. The level of technical language that should be used should be determined by the intended target audience. Language use should also be consistent (Oxman et al., 2020) and only necessary information should be included (Spiegelhalter, 2017).

4.4 Format

Another aspect of statistical communication that should be considered is the format of the statistical information itself. One area in which there is some debate is the use of frequencies versus probabilities, particularly when communicating risk.

In the context of diagnostic or screening tests, the evidence seems to be in favour of natural frequencies (Akl et al., 2011; Hoffrage et al., 2000).  Here, a natural frequency refers to the joint frequency of two events (e.g., the number of people who have a disease and the number who would have a positive test result using a particular screening tool). Findings from a systematic literature review (Akl et al., 2011) suggested that natural frequencies are better understood than probabilities, this result was also argued in an earlier article published in 2000 (Hoffrage et al., 2000).

This result may be specific to the context of presenting joint probabilities. In an aforementioned study developing infographics to convey the personal risk of COVID-19 (Freeman et al., 2021), users showed lower variation in their estimation of risk when shown percentages. Probabilities also had the consequence of lowering the user’s perceived level of risk than when shown the same information as a frequency. Overall, this study concluded that percentages conveyed risk most clearly but resulted in an underestimation of risk. Therefore, the recommendation was to present both percentages and frequencies for balance.

Spiegelhalter (2017) argues that being clear is the most important consideration above the choice of format. If using a frequency format, then he argues that there are choices to be made about the denominator due to ratio bias (a larger numerator suggests a larger risk). The author further recommends choosing a frequency format with a clear reference class, using “1 in X” can be seen as suggesting higher risk when expressed with a higher numerator particularly when the user has a low educational level.

Additional, format-based recommendations include reporting absolute effects (Oxman et al., 2020), and being cautious of using percentages when the numbers are small to avoid misinterpretation (Home Office, 2013).

4.5 Framing

Minor changes in the wording of information can have important implications on how it is perceived. Framing is an important issue in healthcare and one clear example of this is the choice of “survival rate” versus “mortality rate” when conveying risk information. One study using this language to investigate the impacts of positive and negative framing, found that negative framing (“mortality rate”) was associated with greater perceived uncertainty in the risk information as well as greater perceived risk (Freeman et al., 2021). Positive framing (“survival rate”) was, perhaps unsurprisingly, relatively well-liked by users and perceived as less concerning than negative framing, although it was, in some cases, associated with poorer comprehension of risk. Though this is described as an exploratory finding and should therefore be treated with caution before this result has been fully replicated (Freeman et al., 2021). Another study similarly found that positive framing of risk information (“90% of patients do not get a bad blistering rash”) resulted in a lower perceived risk than negatively framed information (“10% of patients get a bad blistering rash”; Peters et al., 2011).

It was recommended in all of the studies that discussed framing within this review (though with some common authors), that both positively and negatively framed text should be presented to avoid unintended biases (Freeman et al., 2021; Peters et al., 2011; Spiegelhalter, 2017). It has also been recommended to use visualisations of part-to-whole comparisons of both positive and negative outcomes (Spiegelhalter, 2017).

4.6 Trust

Trust is another consideration which emerged from the review, specifically the recommendation to include information in statistical communication that helps users to understand whether it should be trusted (Oxman et al., 2020). This has clear links to the prior topic of context as knowing how the information was put together is an important factor when judging its reliability. As well as contextual information, detail should also be provided on any limitations or quality issues affecting the data (Home Office, 2013).

Relevant information may include who produced the statistical information and how it was produced (Oxman et al., 2020; UNECE, 2014). The potential implications of a lack of clarity on the source of statistics were highlighted in the previously mentioned report on public understanding of Economic Statistics (Runge & Hudson, 2020). The focus group research conducted in the development of that report revealed that the participants often erroneously associated economic data with the government and the politicians they perceived presenting them in the news. This association resulted in some of the focus group participants communicating a lack of confidence in the accuracy and reliability of the economic figures presented. Economic statistics, such as unemployment and inflation rates are produced by the Office for National Statistics (ONS) the UK’s national statistical institute which is independent of government. If this was expressed more clearly perhaps confidence in these statistics would be raised, highlighting the importance of this information. In a series of workshops involving economists and members of the general public, economists also argued that distrust, due to suspicion that the government manipulates unemployment statistics, could be somewhat addressed by highlighting the ONS’ independence of government (Runge & Killick, 2021).

The focus group involved in ESCoE’s 2020 report expressed that distrust in economic statistics was also caused by the view that economic topics were discussed in an inaccessible way, using economic jargon (Runge & Hudson, 2020). This further highlights the importance of using language suitable for the target audience which was first considered in the “Language” section of this report.

Research conducted jointly by the Bank of England and the Behavioural Insights Team (BIT), found that using more relatable language in their communication of information related to inflation led to greater understanding (Bholat et al., 2018). Relatable language included using more first and second-person pronouns and using more “day-to-day” terms rather than technical language. Using relatable language also led to small but statistically significant increases in the participant’s trust in the information and perception of the bank.

Another connection between language choice and trust is deciding whether to use numbers or words. In a review of risk communication, Visschers (2009) included the recommendation that both numerical and verbal probability information should be included when communicating risks. This is because people prefer the accuracy of numerical information, trusting and understanding it more, but pass it on using evaluative words. Therefore, people require both formats to be fully equipped with the information they will need to engage with the statistics (Visschers et al., 2009).

As well as providing information to allow users to judge whether evidence is trustworthy, the quality of evidence can also be stated explicitly. One study explored the effects of communicating different levels of evidence quality on the perceived trustworthiness of the evidence (Schneider et al., 2021). Results indicated that when participants were told that evidence was of low quality it was perceived as less trustworthy. When users were told the evidence was of high quality, the perceived trustworthiness of the information was not significantly different than when no information at all was provided on evidence quality. This may indicate that when no information is provided regarding evidence quality users assume that the evidence is of high quality. This has clear potential negative implications of providing no information about evidence quality, particularly when presenting evidence of low quality.

Lastly, evidence suggests that trust is detrimentally impacted when an outcome is different from the expectation (Runge & Killick, 2021; Vicol, 2020). This highlights further, the importance of providing sufficient information so that the reasons underlying an unexpected result are understood.

4.7 Uncertainty

Communication of uncertainty is a major topic in the literature and admitting uncertainty in statistics is one of the recommendations made by David Spiegelhalter in his 2017 review of risk communication (Spiegelhalter, 2017).  Oxman et al., (2020) also recommended that the certainty of evidence should be explicitly assessed and reported.

An ESCoE survey regarding the communication of uncertainty, particularly around the U.K GDP, found that communication of uncertainty information was useful in ensuring that the public does not take GDP estimates at point value while not decreasing trust in the data. In particular, it is recommended to communicate uncertainty quantitatively (e.g. using intervals, density strips and bell curves; Galvão et al., 2019).

More recent ESCoE research, based on 20 qualitative interviews with members of the UK public, found that perceptions of uncertainty can vary dependent on the statistic being communicated (Runge, 2021). In this study, uncertainty was presented around GDP and unemployment statistics, and these were discussed in semi-structured interviews. Participants engaged more with uncertainty information about unemployment statistics. Whereas some respondents stated that, for GDP, they would have preferred just the statistic without the uncertainty information. This is aligned with recommendations to tailor messaging for audiences with different information needs and interest levels. The interviews also covered how participants felt about the underlying causes of uncertainty. For unemployment statistics, information around the method of collecting these data challenged common assumptions. In this instance, a greater amount of method-based information may be useful to aid understanding.

Further evidence on how uncertainty should be communicated was summarised by FullFact in a 2020 briefing “How to communicate uncertainty”. They collated several recommendations including being transparent, being specific about what exactly is uncertain, indicating uncertainty in existing data using numerical ranges in brackets after the main value, and when making future predictions using verbal expressions supplemented with numerical probability ranges citing underlying data where possible (Vicol, 2020). Furthermore, they identified research indicating that verbal indicators of uncertainty are more open to interpretation (see also: Dhami, 2018) and can be interpreted cumulatively. For example, if an outcome is described as “likely” across multiple sources it may be perceived by some as “very likely” (Mislavsky & Gaertig, 2022; Vicol, 2020).

A comparison of verbal and numerical uncertainty indicators was also included in a recent study investigating outcomes such as cognition and trust (Van Der Bles et al., 2020). The results indicated that the effects of communicating uncertainty were highly dependent on the format of the uncertainty indicator. Communicating uncertainty with verbal quantifiers (e.g., “it could be somewhat higher or lower”) led to small significant declines in the perceived reliability of the message and the trustworthiness of the source of the information. Conveying uncertainty numerically was not associated with these outcomes (Van Der Bles et al., 2020).

Overall, it has been recommended to use both verbal and numerical indicators alongside each other and there is even advice on how they should be ordered. Guidance on Communication of Uncertainty in Scientific Assessments from the European Food Authority recommends that when presenting both verbal and numerical indicators of an approximate probability, numerical indicators should be presented first as in English this order is interpreted more consistently (EFSA, 2019). They also recommended providing information on sources of uncertainty estimates for an informed audience.

Beyond the question of numerical or verbal uncertainty indicators, how should uncertainty be presented? One study recommended using a probability function to communicate uncertainty, where relevant, to non-experts (Greis et al., 2015). This finding was based on experimental data with a relatively small sample, investigating behaviour based on expected rainfall statistics with uncertainty communicated in a variety of ways.

For additional information regarding the communication of uncertainty, the OSR has recently undertaken a review into “Approaches to communicating uncertainty in the statistical system”. This review captures the guidance available to statistical producers, past recommendations made by the OSR, and examples of good practice when presenting uncertainty.

4.8 Visualisations

Visualisations have also been discussed more broadly, including recommendations from Spiegelhalter (2017) in the aforementioned review of risk communication.  This includes the recommendation to avoid “chart junk” such as three-dimensional bar charts (Spiegelhalter, 2017; see also: UNECE, 2009). This recommendation was made when considering uncertainty communication but appears to apply to other statistical communication. Similarly, Spiegelhalter advises caution when incorporating interactivity or animation into statistical communication as it may introduce unnecessary complexity rather than being beneficial to comprehension of the statistical message.

Further recommendations regarding visualisations, from the same publication, include using multiple formats to suit different audience groups, adding numbers and words to graphs to aid comprehension, as well as useful and clear narrative labels.

Overall, the use of visualisations was shown to be potentially beneficial in a study involving the Bank of England (Bholat et al., 2018). Their visual summary of statistical information related to inflation, including engaging icons and charts, was related to higher comprehension compared to their traditional text-based communication. Their visual summary also used simpler language so the sole effect of using visualisations was unclear.

4.9 Summary

Overall, this section of the review identified a wide range of recommendations on how to best communicate statistics to non-specialist audiences. The basis of these recommendations varied from smaller-scale research studies to large-scale surveys, and recommendations from statistical bodies. When deciding whether to apply these recommendations to statistical communication the evidence base should be considered, with more weight given to recommendations when the study sample matches the target audience of the communication and when a finding has been replicated across multiple studies.

This section also highlighted examples of when OSR have published correspondence in the areas being discussed. Where examples were identified, these appeared to be aligned with the findings of the review. Examples were more available on broader recommendations (e.g., uncertainty should be reported) and were not always found for more specific recommendations (e.g., reporting both verbal and numerical uncertainty indicators). This may be because these more specific recommendations have less of an evidence base.