Evaluating the effectiveness and efficiency of risk communication for maps depicting the hazard of COVID-19.

COVID-19 maps convey hazard and risk information to the public, which play an important role in the risk communication for individual protection. The aim of this study is to improve the effectiveness and efficiency of communicating the specific risk of COVID-19 maps. By testing 71 subjects from Wuhan, China, this study explored how color schemes (cool, warm, and mixed colors) and data presentation forms (choropleth maps, graduated symbol maps) influence visual cognition patterns, risk perception, comprehension, and subjective satisfaction. The results indicated that the warm scheme (yellow/red) has significant strengths in visual cognition and understanding, and the choropleth map (vs. the graduated symbol map) has significant strengths in risk expression. On subjective satisfaction, the combination of the mixed scheme (blue/yellow/red) and the choropleth map scored highest mean value. These results have implications for enhancing the focused functions of COVID-19 maps that fit different terms: in the early and medium terms of disease transmission, choropleth maps with warm or cool colors should be considered as a priority design for their better risk perception. When the epidemic conditions are on the upturn, a better reading experience combination of choropleth maps with mixed colors can be considered.

INTRODUCTION

Currently, COVID‐19 has proved a worldwide threat to public health. As a pandemic, it involves an extensive area and a great number of people, lasting for a long time with many uncertainties. In such breakouts, grasping hazard information helps form individual risk consciousness and drive precautionary decision‐making, such as purchasing additional supplies, maintaining social distance, limiting outdoor time, and voluntarily getting vaccinated (Tan, Li, Wang, Chen, & Wu, 2004). Conventional mapping, and more recently GIS, has long been considered critical in tracking and combating contagion (Boulos & Geraghty, 2020). A COVID‐19 map, showing the inherent infectious distribution with geospatial locations, is a convenient tool for communicating real‐time COVID‐19 information between risk managers and the public. Some researchers have proved that valid information acquisition and risk perception are important factors in individual preparedness as to epidemic control (Kiss, Cassell, Recker, & Simon, 2010; Poletto, Boëlle, & Colizza, 2016). However, prior studies about infectious disease maps were mostly targeted at experts as reader groups, such as hospital staff, risk managers, and visualized system developers (Alcíbar, 2018; Yoon, Cohen, Cato, Liu, & Larson, 2016). So, there are still insufficient public‐oriented infectious map studies and corresponding evaluation approaches. Whether the existing diverse COVID‐19 maps meet the urgent needs of valid risk communication needs to be studied further.

Maps depicting hazards and reflecting risks can highlight geospatial features in various scenarios. Personalized risk perception is their most substantial functional difference from ordinary maps (Cao, Boruff, & McNeill, 2016; Hervás & Bobrowsky, 2009; Kaufmann & Ramirez‐Andreotta, 2020; Leedal, Neal, Beven, Young, & Bates, 2010; Louis et al., 2014; Thompson, Lindsay, & Gaillard, 2015). In the domain of hazard and risk maps, Kostelnick, McDermott, Rowley, and Bunnyfield (2013) used an example of mapping sea‐level rise to outline a cartographic framework for visualizing hazard and risk. A common issue cited by them is that hazard and risk maps, and geovisualizations generally, are prone to a “one‐size‐fits‐all” approach, which means that target user, type of hazard, and risk characteristics need to be more carefully considered. In a study of wildfire warnings, Cao et al. (2016) defined the effectiveness dimensions of hazard information as accuracy comprehension, risk perception, and attractiveness, and they tested and compared the texts and maps through online surveys. Thompson et al. (2015) summarized the main factors of risk communication as data classification, color scheme, content, and key expression in hazard maps. Furthermore, Fuchs, Spachinger, Dorner, Rochman, and Serrhini (2009) proposed a model of flood hazard mapping for both expert and non‐expert readers, with eye‐tracking and task experiments. From the research above, three primary dimensions can be extracted to present the validity of hazard and risk maps: risk perception, comprehension effectiveness, and individual preference, which are affected by visual variables.

Since COVID‐19 is a novel infectious disease, the information depicting this hazard translated infected numbers and spatial distribution into visual variables on maps. A variety of cartographic research on visual variables for encoding risk has been explored (Cheong, Kinkeldey, Burfurd, Bleisch, & Duckham, 2020). However, it is difficult for some elements (hatching, pattern, texture) to be intuitively recognized in the corresponding magnitudes, especially for some small‐area units on the maps (Koo, Chun, & Griffith, 2018). Transparency, fog, texture, and grain are strong candidates for representing uncertainty information using static methods to give readers a sense of uncertainty (Bostrom, Anselin, & Farris, 2008; Cheong et al., 2016, 2020). Therefore, this study chose color schemes and data representation forms as two main contributing factors of COVID‐19 maps to focus on. Based on the perspectives and methods of previous research, this study used the eye‐tracking technique and task experiments to explore how color schemes and data representation forms affect the risk perception, comprehension, and individual preference of COVID‐19 maps. This research seeks to provide design strategies for promoting risk communication of COVID‐19 maps, helping to optimize the efficiency of information acquisition and the validity of risk perception. By empirically measuring and analyzing readers’ visual cognition patterns and behavior characteristics, the study enriches the theories and empirical evaluation methodologies in the area of risk and hazard maps, especially for the public. Although the implications discussed in this study are based on COVID‐19 maps, the principles can be applied to the cartographic design of mapping hazards and risks in broader scenarios.

RELATED WORKS

Color schemes in hazard information

In early research, color was defined as a primary visual variable by Bertin (1983). It has become a consensus in cartography that color is the most significant element affecting map reading (Cai, Mao, & Zhou, 2000; Jan, 2010; Ling, Wang, & Ding, 2017; Robinson, 1960). In thematic maps, color schemes encode and translate complex statistical data into visualizations (Ling et al., 2017), affecting readers’ visual strategies and cognitive patterns (Bostrom et al., 2008), which is reflected in reading interests, memories, and emotions (Elliot & Maier, 2014; Nicholson‐Cole, 2005; Thompson et al., 2015). A color‐enhanced image could transmit more threat than gray‐scaled images for readers (Ash, Schumann III, & Bowser, 2014). By using high‐contrast color schemes which clearly distinguish from the background in hazard maps, readers are better able to perceive and understand hazard and risk information (Kaufmann & Ramirez‐Andreotta, 2020). Among all colors, red is the commonest for warning and the most salient color that stands out (Bostrom et al., 2008). Certain research has empirically shown that reading information with cool or warm colors leads to different emotions and task performances, but the conclusions were inconsistent (Bartram, Patra, & Stone, 2017; Cyr, Head, & Larios, 2010; Madden, Hewett, & Roth, 2000; Thompson et al., 2015). Mehta and Zhu (2009) studied the cognitive and behavioral influence owed to different colors, implying that red and blue could activate different motivations, and the red (vs. blue) color evokes a higher level of emotion. Elliot, Maier, Moller, Friedman, and Meinhardt (2007) got a reverse result; they found that red (vs. green or blank) color has a negative effect on information access because people consider red a sign of failure and danger, leading to cognitive avoidance. Though red is widely used as a warning color, Elliot and Maier (2014) later reviewed a considerable number of empirical research articles and proposed that the meanings and effects of colors are closely bonded to the context, which means that even the same color may indicate converse semantics. Kwallek, Lewis, and Robbins (1988) drew a similar conclusion that red would evoke anxiety. In addition, some researchers thought that mixed color schemes containing two main feature dimensions (hues of both warm and cool, and lightness) have advantages in visual searching (e.g., Shortridge, 1982). This opinion is also supported by Mersey (1990) and Breslow, Ratwani, and Trafton (2009), with the latter finding that a multicolored scheme performed best in identification tasks; the former study concluded that individuals can distinguish a greater number of different hues than different shades of a single hue in maps. So, we could assume that the COVID‐19 maps with mixed color (vs. warm or cool color) have a richer visual feel and superior reading experience.

According to the literature cited, which color scheme can balance risk warning and accuracy comprehension for COVID‐19 maps needs further study. We propose the following hypothesis:

Different color schemes have significant effects on risk perception, comprehension, and satisfaction in COVID‐19 maps.

Data presentation forms in maps

External forms of data presentation in hazard maps sort data into nominal or quantitative attributes and visually display them through thematic maps (Dent, Torguson, & Hodler, 2008; Slocum, McMaster, Kessler, & Howard, 1999). A thematic map is a type of map designed to describe a scenario emphasizing specific geographical features in a certain area. It encodes and maps the numerical levels into specific visual parameters, such as color, size, shape, and geometry, which impact readers in the perceptual and cognitive domain (Dent et al., 2008). In terms of psychology, Hegarty and Just's (1993) work pointed out that a schematic representation with features and relationships helps readers to grasp integrated formation and problem‐solving, and they identified three phases of such cognitive possessing: (1) visual perception of primary reading; (2) interpretation of schematic characteristics; and (3) comprehension of indications with internal experience. In terms of cartography, Fairbairn, Andrienko, Andrienko, Buziek, and Dykes (2001) implied that appropriate methods of data classification are crucial to the depiction of visualization. Cheong et al. (2020) found a relationship between different cartographic representations of risk and readers' decision‐making, and they indicated a pattern of less‐preferred graphical symbolizations associated with slower responses and lower‐risk route choices. Based on cognitive map theories, Hou, Rashid, and Lee (2017) compared different types of thematic maps and arrived at a result that the key to improve information transmission effectiveness and user experience is ensuring the validity of representation.

Thematic symbology (e.g., choropleth, graduated/proportional symbol), for example, is used in existing maps to visualize quantitative data that summarize potential impacts of hazards (Kostelnick et al., 2013). An earlier study by Brewer and Pickle (2002) proposed that choropleth maps describing statistical data classification with color schemes and representation forms have been a common expression method in epidemiology. However, there are still shortcomings to the choropleth method when its application is to disease maps. Barrozo, Pérez‐Machado, Small, and Cabral‐Miranda (2016) proposed that a choropleth map may mislead readers on a uniform distribution, with its colors filling entire regional areas. Similarly, Calka, Nowak Da Costa, and Bielecka (2017) implied that choropleth maps in natural hazards risk assessment have a defect in the incompatibility of mapping units used for data analysis and visualization. Shortridge (1982) inferred that graduated symbol maps are more suitable for presenting differences between levels, using diagrams with separate visual dimensions such as size and shape. Meanwhile, Wang, Sun, Wang, Jiang, and Lv (2003) referred to choropleth maps expressing characteristics and distributions of phenomena more vividly. The opinions of Chen and Jiang (2001), along with Liao (2003), agreed with the above; that is, choropleth maps have advantages in terms of expressing distribution range and quantity difference, while graduated symbol maps are more appropriate and accurate to describe dynamic changes and magnitudes of a single unit. Thus, which representation form is the most suitable for COVID‐19 maps needs further research, and a hypothesis is proposed as follows:

Different data presentation forms have significant effects on risk perception, comprehension, and satisfaction in COVID‐19 maps.

In thematic maps, data presentation forms and color schemes have a joint effect on the visual expression. Taylor (1991), who established the core and main body of contemporary cartography, connected information transfer theory with visual cognition theory in a visualized theoretical system, which mainly consisted of data presentation and color scheme. Carswell, Frankenberger, and Bernhard (1991) also indicated that inappropriate color and data presentation would make key message lost in a redundant visual, resulting in overloaded and disturbed cognition. So, a hypothesis is proposed as follows:

Different color schemes and data presentation forms have significant interaction on risk perception, comprehension, and satisfaction in COVID‐19 maps.

Eye‐tracking technique implemented in hazard and risk maps

Eye movements, reflecting the processing of visual cognition and mental activity (Çöltekin, Fabrikant, & Lacayo, 2010; Fabrikant, Rebich‐Hespanha, Andrienko, Andrienko, & Montello, 2008), have been acknowledged as an efficient tool, along with task experiments, to evaluate the effectiveness and efficiency of information delivery in the domain of geovisualization (Brodersen, Andersen, & Weber, 2002; Jacob & Karn, 2003; Koua, MacEachren, & Kraak, 2006; Li & Chen, 2012). Some examples of the eye‐tracking technique assisting in hazard/risk map design and evaluation are as follows. Zheng, Li, Fang, and Qian (2016) examined how users translate and comprehend information in maps by collecting fixation and sequence data with an eye‐tracking method to design hazard emergency maps. Meyer et al. (2012) interviewed users and tested them with a compound eye‐tracking task so as to provide suggestions to improve flood maps for oriented groups. Brychtova, Paszto, Marek, and Panek (2013) assessed the validity of a risk map website with eye‐tracking for the purpose of optimizing the layouts of websites and maps for emergency events. Golebiowska, Opach, and Rød (2017) used hazard maps as experimental material and analyzed the eye‐tracking parameters of subjects to evaluate the usability of geovisualization tools. In addition, eye‐tracking is also applied in studies of hazard maps to evaluate comprehension and decision‐making; for example, Korporaala and Fabrikant (2019) explored how risk uncertainties and time limitations influence decision‐making in different representation maps with behavioral tasks and eye‐tracking.

In conclusion, eye‐tracking parameters in hazard and risk map reading tasks help us compare differences in visual strategies and attentive patterns between users. This approach helps to formalize and quantify the framework of hazard maps, optimizing the layout of visual elements and improving the effectiveness of risk communication. Thus, this study used the eye‐tracking technique to measure and compare visual cognition patterns in COVID‐19 maps.

METHOD

Subjects

There were 71 subjects randomly recruited via cellular phone text messaging in Wuhan, of which 23 (32.4%) were males and 48 (67.6%) were females. They ranged in age from 20 to 45 and their careers were wide‐ranging, including teaching staff, administrative staff, graduate students, interpreter, security personnel, and so forth, with the majority above college degree. All the subjects lived long‐term in Wuhan, Hubei Province, China and had no familiarity with areas other than Wuhan. The majority of subjects had experience of browsing COVID‐19 maps via mobile phones. All subjects participating in this experiment declared that they had normal eyesight (or corrected‐to‐normal eyesight) and normal color sense, and each was paid ¥20 as a reward.

Levels of independent variables

We selected 37 well‐known online media sources (e.g., JHU CSSE, mapbox, dxy.cn, and Tencent News; https://www.arcgis.com/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6; https://www.mapbox.cn/coronavirusmap/#3.91/27.16/113.28; https://ncov.dxy.cn/ncovh5/view/pneumonia; https://news.qq.com/zt2020/page/feiyan.htm?from=timeline&isappinstalled=0#/global) containing COVID‐19 maps by rank on research engines (Baidu and Google). By analyzing the color schemes and data presentation forms of these COVID‐19 maps, three types of color schemes were determined: cool, warm, and mixed. Cool color schemes mainly used blue as a filler color, ranging in lightness from light to dark as the magnitude increases. Warm color schemes mainly used a brighter yellow, red, or orange for lower levels of magnitude and a darker red for higher levels of magnitude. A mixed color scheme filled the map with both warm and cool color, always ranging from low‐brightness blue (lower magnitudes) to low‐brightness red (higher magnitudes). Therefore, we used sequential yellow/red as the warm color scheme, sequential single‐hued blue as the cool color scheme, and diverging blue/yellow/red as the mixed color scheme. All the color schemes used in experimental materials were selected from ColorBrewer, an online tool providing effective color schemes for cartographers. The convention of “light is less, dark is more” (Garlandini & Fabrikant, 2009; He, Song, & Li, 2016) was followed in the coding relationship between magnitude and luminance to ensure colorblind safeness.

The variable levels of data representation forms were also extracted from COVID‐19 maps of the 36 media sources selected, and the statistical tallies showed that choropleth maps accounted for 29 (78.38%) and graduated symbol maps accounted for 19 (51.35%); among these, 12 (32.43%) used both choropleth and graduated symbol maps. Other forms accounted for 1 (2.7%). Therefore, choropleth and graduated symbol maps were taken as two variable levels of data representation form. The choropleth map generally divides the area into regional or natural units, filling with colors or textures which map the statistical classifications. This representation method emphasizes depicting the phenomena and developments with large‐area colors. In comparison, the graduated symbol map generally uses administrative regions as units, using charts, tables, and other forms of statistical graph in each unit, focusing on the total amount, composition, and variables within units (He et al., 2016).

Dependent variables

Each subject in this study needed to accomplish a three‐part measurement, including eye‐tracking, behavioral tasks, and subjective scales. Seven metrics of dependent variables were measured in total: (1) first fixation duration; (2) fixation count; (3) average saccade amplitude; (4) accuracy rate; (5) response time; (6) risk perception; and (7) subjective satisfaction.

Eye‐tracking metrics

First fixation duration, fixation count, and average saccade amplitude were taken for eye‐tracking metrics.

First fixation duration records the time of the eyes’ first fixation before gazing to the next point, which generally has a timespan of 200–250 ms. It validly indicates the primary processing of the early stage of visual stimuli (Dong et al., 2019; Li & Chen, 2012). Fixation count is the total number of fixations during the eye‐tracking experiment. A higher fixation count reflects more complicated experimental materials to be processed or a higher reading interest of readers. Average saccade amplitude is the average distance of gazing from one focus to another, and implies the breadth of visual search and the difficulty of the task (Yan et al., 2013). A larger fixation count presents a larger amount of information perceived by the reader at each gazing point, and vice versa, so it is an indicator of reading efficiency and complexity (Zhang & Ye, 2006).

Behavioral metrics

Accuracy rate and response time of tasks were behavioral metrics to measure the comprehension accuracy and efficiency in the COVID‐19 maps. According to Amar, Eagan, and Stasko's (2005) “low‐level mathematical and cognitive actions” on task classification, three types of task questions were designed to present the fundamental process of information retrieval from COVID‐19 maps. Examples of task questions are given in Table 1. The within‐group questions are the same, while the between‐group questions are different. The complexity between the three types of task questions increases sequentially, but the questions are still low‐level in mathematical cognitions and operations. Moreover, in order to avoid the familiarity effect and experience interference, Wuhan was excluded from the task objects for its well‐known condition. All these questions were verified via a pre‐test.

TABLE 1

Task questions for measuring accuracy and efficiency of comprehension

Task	Question descriptions (examples)	Task types	Task features
T1	What magnitude does the number of infected in Xiaogan have on this map?	Comprehension (retrieve value)	Search one object and retrieve value
T2	Whose magnitude is higher on the number of infected in this map between Tianmen and Yichang?	Comprehension (retrieve value and low‐level comparison)	Search two objects, retrieve values and make comparison
T3	Which magnitude in this map contains the largest number of cities, and what exactly is the number?	Comprehension (filter and compute derived value)	Filter several objects by criteria and count their number

T1 is the easiest question for subjects to answer, so as to motivate them to complete the remainder of the experiment. Xiaogan was chosen for the question as it adjoins Wuhan and is easy to locate. The two cities of Tianmen and Yichang in T2 have unequal magnitudes of data in maps and are not bounded by Wuhan. Answers to T3 are designed to be different within group in choropleth and graduated symbol maps.

Metrics of subjective scales

Risk perception and the satisfaction of subjects were measured by seven‐point Likert scales, as in Table 2. Arranging all the cities in Hubei Province according to their order of magnitude, Xiangyang (T4, Table 2) was one of the cities in the median level, so we chose it as the question‐oriented city of risk perception.

TABLE 2

Questions of subjective scales

Task	Question descriptions (examples)	Task types	Question features
T4	If you were in Xiangyang, what extent of risk would you feel? Please choose your score of 1–7 (from low to high)	Risk perception	A higher score means higher risk perception
T5	What extent of satisfaction do you feel with this map? Please choose your score of 1–7 (from low to high)	Subjective satisfaction	A higher score means higher satisfaction

Materials

According to the levels of independent variables referred to above, we designed experimental materials with three types of color scheme (cool, warm, and mixed) and two forms of data presentation (choropleth maps and graduated symbol maps), displaying the distribution of COVID‐19 infections in Hubei Province. The statistical data of infected people were obtained from the website of the Chinese Center for Disease Control and Prevention (http://www.chinacdc.cn/jkzt/crb/zl/szkb_11803/jszl_11809/). In order to avoid the familiarity effect, datasets used in choropleth and graduated symbol maps were different within groups.

The choropleth maps (a1, a2, a3) were based on the number of people diagnosed with COVID‐19 in Hubei Province on February 5, 2020, and the data were classified into five levels (10–99, 100–499, 500–999, 1,000–4,999, 5,000–50,000). Graduated symbol maps (b1, b2, b3) were based on the number of people diagnosed with COVID‐19 in Hubei Province on March 1, 2020, and the data were classified into five levels (10–99, 100–499, 500–999, 1,000–1,999, 2,000–11,000). All the material maps (Figure 1) were made in ArcMap 10.2 software.

FIGURE 1

Experimental materials: (a1) warm color and choropleth map; (a2) cool color and choropleth map; (a3) mixed color and choropleth map; (b1) warm color and graduated symbol map; (b2) cool color and graduated symbol map; and (b3) mixed color and graduated symbol map

Devices

In the eye‐tracking part of the experiment, a Tobii Pro Fusion eye‐tracker was connected to a computer with a sampling rate of 600 Hz, accuracy of 0.3°, and precision of 0.06° RMS (Figure 2). The stimuli were presented via Tobii Pro Lab software. In the behavioral task and scale parts of the experiment, stimuli were displayed via the E‐prime software program.

FIGURE 2

Experimental devices (including eye‐tracker, recorder, and demo program)

Experimental design and procedures

This experiment adopted a mixed design, using color schemes as between‐group variables and data representation forms as within‐group variables. A mixed design considers both interactive correlation and single variable effects for multiple‐factor experiments, composing between‐group and within‐group designs. When an experiment has multiple variables, a mixed design has advantages in terms of validity and efficiency, and could avoid errors caused by repeated measurements in a within‐group design (Cai & Yan, 2011; Yuan & Wang, 2002).

In total, 71 subjects were randomly allocated in three groups: 24 in the cool color group, 24 in the warm color group, and 23 in the mixed color group. Each subject would view two COVID‐19 maps, which were the same in terms of color scheme and different in terms of data presentation form.

The experiment was conducted from June 5 to 15, 2020. Human ethics has been approved by the Research Ethics Board of China University of Geoscience. The laboratory was located in a quiet and well sound‐insulated room. We sanitized the experimental devices and ventilated the room in the interval between participants. Researchers wore masks and reminded subjects to wear a mask during the experiment. Light was mainly provided by the computer screen. The range of head movement allowed was within a three‐dimensional range of 50 cm W × 36 cm H × 70 cm D, and the allowed operating distance from the monitor was within 45–70 cm.

Eye movement measurement

The procedure of the eye‐tracking experiment is presented in Figure 3 (warm color group) as example. First, an introduction is shown on the screen along with a verbal explanation to inform the subjects about the experiment's aims and requirements. After the subjects sit in front of the screen, the distance to the screen is adjusted. Then they are required to keep their heads as still as possible and gaze directly at the screen for calibration in the Tobii Pro Lab program. After the calibration is finished, the subjects are told to browse the following pages as per their normal habits. A fixation point presented in the center of the screen lasts for 500 ms, followed by one map lasting for 20,000 ms. Afterwards, the screen goes blank for a 1,000 ms rest. This is then repeated, presenting a fixation point for 500 ms and another map for 20,000 ms. In order to eliminate the order effect, the display order of the two maps is random. It takes about 1 min to finish this part of the experiment for each subject.

FIGURE 3

Procedures of eye‐tracking experiment (warm color group for example)

Behavioral and subjective scales measurement

After completing the eye‐tracking experiment, the subjects rested for about 1 min while the researcher quit the Tobii Pro lab software and entered the E‐prime program. All the stimuli in this part of the experiment are presented in the E‐prime program, and the procedure is shown in Figure 4. First, the subjects read the introduction presented on the screen to know the steps of the procedure, then they enter the practice program before the formal experiment. COVID‐19 maps of the China region are used for viewing. Through answering the province‐oriented task questions with clicks on the maps, subjects can familiarize themselves with the operations until they are ready.

FIGURE 4

Procedures of behavioral and scale measurement

In the formal experience, a corresponding introduction is also presented. In the behavioral measurement, the questions with options attached are shown below the map, three pages in total. Participants are first required to answer each question according to the map. The response time is unlimited, but the subjects are asked to choose options in as short a time as possible. After completing three task questions, the screen layout still shows the map, but the task questions are replaced with a seven‐point risk scale. Subjects respond with their risk perception of cities whose quantitative magnitudes are in the median level of the map, choosing and clicking the extent to describe the risk extent they feel. The following page is a seven‐point subjective satisfaction scale, independently presented (without maps) in the center of the screen. Subsequently, there is a blank page for 3,000 ms. Finally, the above process is repeated, with another representation form map with a set of questions. The order of the two maps is random. It takes about 5 min to finish this part of the experiment for each subject.

Data analysis

The eye‐tracking data was exported in TXT format from Tobii Pro Lab software and imported into Microsoft Office Excel 2010, sorting the data into a format compatible with SPSS software. The behavioral data were merged with E‐prime and imported into SPSS. Two persons' data were skipped because they were invalid—one whose eye movements failed to be recorded because his/her head position moved; the other because they did not complete the behavioral tasks. Therefore, 69 valid subjects remained. All statistical data were analyzed with SPSS 18.0 software.

RESULTS

Eye‐tracking data

The results in Table 3 demonstrate the differences in individuals' eye movement data when they use COVID‐19 maps.

TABLE 3

Multi‐factor ANOVA of eye‐tracking data

Dependent variable	Independent variable	F	p	η²
First fixation duration	Color scheme	6.977	0.002**	0.175
	Form of data presentation	5.954	0.017*	0.083
	Color scheme × form of data presentation	4.957	0.01*	0.131
Average saccade amplitude	Color scheme	1.571	0.215	0.045
	Form of data presentation	4.954	0.029*	0.07
	Color scheme × form of data presentation	3.780	0.028*	0.103
Fixation count	Color scheme	1.856	0.164	0.053
	Form of data presentation	4.768	0.033*	0.067
	Color scheme × form of data presentation	0.714	0.494	0.021

F (F‐distribution) is a continuous probability distribution that arises frequently as the null distribution of a test statistic. The F‐distribution has two parameters: degrees of freedom numerator (dfn) and degrees of freedom denominator (dfd). “The dfn is the number of degrees of freedom that the estimate of variance used in the numerator is based on. The dfd is the number of degrees of freedom that the estimate used in the denominator is based on” (Freund, Wilson, & Mohr, 2010). t is an inferential statistic; p is the probability of obtaining test results as extreme as the results actually observed during the test; η² is the strength of association.

p <0.05

p < 0.01

p < 0.001.

First fixation duration

The multi‐factor analysis of variance (ANOVA) results of the first fixation duration are shown in Table 3. The interaction effect between color schemes and data presentation forms was statistically significant, F(2, 66) = 4.957, p <  0.05, and the comparisons between six variable combinations are described in Figure 5. The results of a simple effect test following significant interactions (Table 4) showed that the first fixation duration of choropleth maps was significantly longer than that of graduated symbol maps under the premise of mixed color (p <  0.05), and the first fixation duration of mixed color was significantly longer than that of warm color or cool color under the premise of choropleth maps (p <  0.001).

FIGURE 5

The interaction on first fixation duration

TABLE 4

Results of simple effects test following significant interactions

Dependent variable	Independent variable	Type of map	M	MD	SE	p
First fixation duration	Mixed colors	Graduated symbol	233.652	4,291.551	1,851.970	0.000***
		Choropleth	480.130
	Choropleth maps	Cool colors	297.000	8,497.754	2,428.879	0.035*
		Mixed colors	480.130
Saccade amplitude	Warm colors	Graduated symbol	3.323	0.610	0.173	0.001***
		Choropleth	2.714
	Choropleth maps	Warm colors	3.323	0.543	0.252	0.035*
		Mixed colors	2.780

p < 0.05

p < 0.01

p < 0.001.

The main effect of data presentation forms on first fixation duration was significant, F(1, 66) = 5.954, p < 0.05. Testing the results with post‐hoc multiple comparisons (Table 5) on the variable of data presentation forms, the first fixation duration of choropleth maps was significantly longer than that of graduated symbol maps (p < 0.05). The main effect of color schemes was significant, F(2, 66) = 6.977, p < 0.01. The results of post‐hoc testing (Table 5) show that on the variable of color schemes, the first fixation duration of warm color was significantly shorter than that of cool color (p < 0.05) or mixed color (p < 0.001).

TABLE 5

Results of post‐hoc test following significant main effects

Dependent variable	Independent variable	Type of map	M	MD	SE	p
First fixation duration	Form of data presentation	Graduated symbol	286.942	89.043	36.491	0.017*
		Choropleth	309.406
	Color scheme	Warm colors	328.435	107.700	50.127	0.041*
		Cool colors	281.435
		Warm colors	328.435	186.826	50.127	0.000***
		Mixed colors	284.652
Saccade amplitude	Form of data presentation	Graduated symbol	2.841	0.223	0.100	0.029*
		Choropleth	3.064
Fixation count	Form of data presentation	Graduated symbol	222.609	2.768	1.268	0.033*
		Choropleth	311.652

p <  0.05

p < 0.01

p < 0.001.

Average saccade amplitude

The multi‐factor ANOVA results of average saccade amplitude are shown in Table 3. The interaction between color schemes and data presentation forms was significant, F(2, 66) = 3.780, p < 0.05, and the comparisons between variable combinations are shown in Figure 6. The results of simple effects on interaction showed (Table 4) that the average saccade amplitude of choropleth maps was higher than that of graduated symbol maps under the premise of warm color (p < 0.001), and the average saccade amplitude of cool color was significantly lower than that of warm color under the premise of graduated symbol maps (p < 0.05).

FIGURE 6

The interaction on average saccade amplitude

The main effect of data presentation forms was significant on average saccade amplitude, F(1, 66) = 3.780, p < 0.05. Testing the results with post‐hoc multiple comparisons (Table 5) on the variable of data presentation forms, the average saccade amplitude of choropleth maps was higher than that of graduated symbol maps (p < 0.05).

Fixation count

The multi‐factor ANOVA results of fixation count are shown in Table 3. There was no significant interaction between color schemes and data presentation forms. The main effect of data presentation forms was significant, F(1, 66) = 4.768, p < 0.05. Testing the results with post‐hoc multiple comparisons (Table 5), the fixation count of the choropleth maps was significantly more than that of graduated symbol maps (p < 0.05).

Behavioral data

The multi‐factor ANOVA results of accuracy rate and response time are shown in Table 6. The interaction between color schemes and data presentation forms on response time was significant, F(2, 66) = 3.939, p < 0.05, and the comparisons between the variable combinations are shown in Figure 7. Testing the simple effect of interaction, the response time of choropleth maps was significantly longer than that of graduated symbol maps under the premise of mixed color (p < 0.05). The main effect of color schemes was significant (p < 0.01) on the response time. Post‐hoc testing results (Table 5) show that the response time of mixed color is significantly longer than that of warm color (p < 0.01) and cool color (p < 0.05).

TABLE 6

Multi‐factor ANOVA of behavioral data

Dependent variable	Independent variable	F	p	η²
Accuracy rate	Color scheme	5.148	0.008**	0.135
	Data presentation form	0.594	0.444	0.009
	Color scheme × data presentation form	0.309	0.735	0.009
Response time	Color scheme	4.915	0.008**	0.046
	Data presentation form	0.003	0.956	0
	Color scheme × data presentation form	3.939	0.021*	0.037

p < 0.05

p < 0.01

p < 0.001.

FIGURE 7

The interaction on response time

On the variable of accuracy rate, the main effect of color schemes was significant (p < 0.01). Post‐hoc testing (Table 5) shows that the accuracy rate of mixed color is significantly higher than that of warm color (p < 0.05) and cool color (p < 0.01) (Tables 7 and 8).

TABLE 7

Results of post‐hoc test following significant main effects

Dependent variable	Independent variable	Type of map	M	MD	SE	p
Accuracy rate	Color scheme	Warm colors	70.291	15.271	6.581	0.024*
		Mixed colors	85.508
		Cool colors	65.218	20.290	6.581	0.003**
		Mixed colors	85.508
Response time	Color scheme	Warm colors	15,905.920	6,666.986	2,233.553	0.003**
		Mixed colors	22,572.906
		Warm colors	15,905.920	5,188.548	2,233.553	0.021*
		Cool colors	17,384.058

p < 0.05

p < 0.01

p < 0.001.

TABLE 8

Results of simple effects test following significant interactions

Dependent variable	Independent variable	Type of map	M	MD	SE	p
Response time	Mixed colors	Graduated symbol	18,591.604	4,291.551	1,851.970	0.021*
		Choropleth	18,650.319

p < 0.05

p < 0.01

p < 0.001.

Subjective scale data

The multi‐factor ANOVA results of risk perception are shown in Table 9. There was no significant interaction between color schemes and data presentation forms. The main effect of data presentation forms on risk perception was significant, F(1, 66) = 22.901, p < 0.001. Testing the main effect with post‐hoc multiple comparisons (Table 10), the risk perception of choropleth maps was significantly higher than that of graduated symbol maps (p < 0.001).

TABLE 9

Multi‐factor ANOVA of scale data

Dependent variable	Independent variable	F	p	η²
Risk perception	Color scheme	0.56	0.574	0.017
	Data presentation form	22.901	0***	0.258
	Color scheme × data presentation form	0.212	0.81	0.006
Satisfaction	Color scheme	2.045	0.137	0.058
	Data presentation form	4.067	0.048*	0.058
	Color scheme × data presentation form	0.916	0.405	0.027

p < 0.05

p < 0.01

p < 0.001.

TABLE 10

Results of post‐hoc test following significant main effects

Dependent variable	Independent variable	Type of map	M	MD	SE	p
Risk perception	Form of data presentation	Graduated symbol	3.638	0.794	0.167	0.000***
		Choropleth	4.435
Satisfaction	Form of data presentation	Graduated symbol	5.203	0.319	0.158	0.048*
		Choropleth	5.522

p <  0.05

p < 0.01

p < 0.001.

The multi‐factor ANOVA results of satisfaction are shown in Table 9. There was no significant interaction. The main effect of data presentation forms on satisfaction was significant, F(1, 66) = 4.067, p < 0.05. Testing the main effect with post‐hoc multiple comparisons (Table 10), the satisfaction of choropleth maps was significantly higher than that of graduated symbol maps (p < 0.05).

DISCUSSION

The experimental results show that both color schemes and data presentation forms influence risk communication of COVID‐19 maps, but the aspects they impact are not exactly the same. Color schemes and data presentation forms have interaction effects on the comprehension efficiency and visual cognition patterns. In terms of the main effects of single variables, the color schemes have significant effects on initial visual processing, comprehension effectiveness, and efficiency, and the data presentation forms have significant effects on visual patterns, subjective satisfaction, and risk perception.

Color schemes

Color schemes influence visual cognition patterns

The eye‐tracking results show the impact of color schemes on the early stage of cognitive processing. According to the indications about eye‐tracking parameters (Yan et al., 2013), a shorter first fixation duration means the visual object is more attractive. As warm color had a significantly shorter first fixation duration than cool or mixed colors in all maps, and warm and cool colors had a significantly shorter first fixation duration than mixed color in choropleth maps, the warm color was the most attractive, while the mixed color was the worst. Attention theories declare that only a few stimuli can be caught and sequentially processed with limited resources (Treisman & Gelade, 1980). There are two ways to guide attention: one is a down‐top pattern driven by stimuli, which means that the salient objects more easily obtain attention (Johnston, Hawley, Plewe, Elliott, & DeWitt, 1990); the other is a top‐down pattern driven by working memory, which means that attention automatically matches the external properties of objects with a similar internal experience of working memory (Olivers, Meijer, & Theeuwes, 2006; Soto, Heinke, Humphreys, & Blanco, 2005). Severtson and Vatovec (2012) referred to the yellow/red scheme as better for attention‐getting in risk expression, whether in the top‐down or down‐top patterns. Moreover, red is the commonest warning color (Griffith & Leonard, 1997; Kaufmann & Ramirez‐Andreotta, 2020), so red has high familiarity in the mental representation of working memory. Considering the results, the warm color scheme (vs. the cool and mixed color schemes) has an advantage in terms of visual salience and leads to a shorter time taken by users to grasp the information characteristics needed to complete initial processing.

The interaction on average saccade amplitude shows that the warm color scheme in the choropleth maps leads to better reading efficiency than the cool color scheme, while mixed color schemes are more complex to process. However, with the warm color scheme, the graduated symbol maps have better readability than the choropleth maps. This distinctness may be impacted by the avoidance effects of a large area of red (Elliot & Maier, 2014). Griffith and Leonard's (1997) study found that 98% of subjects bind the red color with non‐context semantics of “stop.” Moreover, Soto et al. (2005) proposed that the attention‐guiding effect of color is superior to shape in top‐down processing, so avoidance effects did not appear in the graduated symbol maps.

Color schemes influence comprehension

The interaction and main effect on behavioral metrics indicated that color schemes affect comprehension efficiency and accuracy. The mixed color maps had statistically worse performance on efficiency but best on accuracy versus those of the warm and cool color maps. The response time reveals the complexity of information processing. The disadvantage in terms of complexity of the mixed color scheme is in line with other results discussed above. A number of studies suggested that a single hue with brightness ratio changes is generally appropriate for quantitative data visualization (Kelleher & Wagener, 2011; Sherman‐Morris, Antonelli, & Williams, 2015). However, the strength of understanding of the mixed color scheme on experimentally observed accuracy might result in its distinctive visual salience of magnitudes, which means it is easier to identify and fix (Treisman & Gelade, 1980). Lewandowsky et al. (1993) used cognitive theory to explain the reasons that contrasting colors in statistical maps having stronger variable perception than single‐hue schemes. In the experimental material, the mixed color scheme had distinct characteristics in hue and luminance, intact for cognitive identification (Shortridge, 1982). Besides, some research could provide possible mechanisms and high‐consistency explanations to support our results. Gao, Guan, and Wang (2008) concluded that thematic maps with mixed colors have prominent features but can easily cause visual fatigue. Cheong et al. (2020) suggested that a visually complex representation may consume more time and attention to understand, but may promote more detailed considerations and better decision‐making.

Data presentation forms

Data presentation forms influence risk perception

The results show that data representation forms significantly affect the risk perception of COVID‐19 maps. The choropleth maps had better risk expressiveness than the graduated symbol maps. According to the holistic model of Gestalt theory proposed by Koffka (1935), such difference might be due to distinct diagrams. As the outlines of diagrams in choropleth maps use specific regional shapes, this makes it easier for users to bring their experience of real and integral administrative areas into the maps. Graduated symbol maps use separate circles of varying sizes as diagrams, and their risk expression via geometrical features is inferior to that of choropleth maps (Mehta & Zhu, 2009). Moreover, the studies of Xie, Li, and Yu (2008) and López‐Vazquez and Marvan (2003) mention that vividness of information is one determining factor evoking and improving risk perception. Choropleth maps with concrete visual features of “figure” have a higher level of risk perception.

Data presentation forms influence visual cognition patterns

The first fixation duration of the graduated symbol maps was significantly shorter than that of the choropleth maps in both interaction and main effect, showing the better attractiveness of the graduated symbol maps. The average saccade amplitude of choropleth maps was significantly greater than that of graduated symbol maps in both interaction and main effect, indicating the wider searching scope and higher efficiency of choropleth maps. The fixation count of choropleth maps was significantly lower than that of graduated symbol maps, which also shows that choropleth maps have better readability and less cognitive load than graduated symbol maps.

These three eye‐tracking metrics consistently indicate that the information of choropleth maps (vs. graduated symbol maps) is easier to process. Shortridge (1982) summarized the process of map reading into three basic steps: identifying symbols by visual search, comparing symbol differences, and integrating information to understand. They also mentioned that the size, color, and shape of diagrams in a map should be regarded as separate dimensions that are sequentially processed. So, it takes more time to process objects with more visual dimensions. In this study, color is the main dimension to recognize units and magnitudes of choropleth maps, while graduated symbol maps using more dimensions (color and size, along with similar shape) need more time to be processed. In addition, Leclercq (2002) proposed the same sensory process consuming the same type of cognitive resources, leading to competition. Therefore, diagrams in graduated symbol maps need more resources of attention than those in choropleth maps.

Data presentation forms influence reading experience

The response time of graduated symbol maps was significantly shorter than that of choropleth maps for the interaction with mixed color, that is, the comprehension efficiency of the graduated map was higher. Some cartographers pointed out that using a choropleth map for visualizing the absolute magnitude, like the mass of existing COVID‐19 maps available in the media, would distort or mislead the interpretation of the map, because map readers automatically consider the size of the areas (Dent et al., 2008). Compared to graduated symbol maps, this may result in lower comprehension efficiency of choropleth maps. Moreover, the fact of visually salient color performing better in relatively small‐sized maps supports this result (Severtson & Vatovec, 2012). The discussion above has referred to the mixed color scheme having the highest cognitive load among the three types of color schemes (Gao et al., 2008; Kelleher & Wagener, 2011; Lewandowsky et al., 1993; Sherman‐Morris et al., 2015). So, larger color‐filled areas in choropleth maps would enlarge the visual interference between adjacent units.

The choropleth maps have better satisfaction than the graduated symbol maps. Consistent with the cartographic studies referred to in Section 2.2, a possible explanation for this result is the characteristics of the data. Choropleth maps and graduated symbol maps each have their own advantages in expressing different types of dataset (Calka et al., 2017). Chen and Jiang (2001), Liao (2003), and Wang et al. (2003) arrived at a consensus that choropleth maps perform better when datasets reflect a distribution range and quantity difference. Considering the datasets used in the experimental materials display the static distribution of the infection on a single day, the choropleth maps are more intuitive than the graduated symbol maps on expressiveness. Moreover, Lewandowsky et al. (1993) suggested that graduated symbol maps cannot establish “Gestalt” for their independent symbols in cognition. Meihoefer (1973) implied that it is difficult for readers to accurately identify magnitudes by similar‐sized symbols in a small area, so graduated symbol maps have lower readability than choropleth maps when the size of the device display is relatively small. For the reasons above, choropleth maps are more suitable for presenting numbers of COVID‐19 infections.

CONCLUSIONS

This study explored whether color schemes and data representation forms influence risk communication of COVID‐19 maps. Through a compound task containing measurement of eye‐tracking, behavior, and scale, the analyzed results revealed the relationship between variables. The main conclusions are as discussed: the warm color scheme (yellow/red) has significant strengths in visual cognition and understanding, and the choropleth map has significant strengths in risk expression. But in terms of participants' satisfaction, the combination of mixed color scheme (blue/yellow/red) and choropleth map scored highest. What was unexpected in the experiment was the warm scheme not having a significantly higher risk perception than the cool or mixed schemes.

Based on the conclusions above, certain design strategies can be proposed: to determine which combination of color scheme and data presentation form to use for visualizing public health hazard or risk, the main function of the maps should be considered. For example, in the early and medium terms of disease transmission, risk perception is the primary function, so the design should give priority to choropleth maps with warm or cool color schemes. In urgent situations, warm color schemes can perform best in attracting readers' attention. When the situation of the epidemic is on the upturn, a better combination of choropleth maps with mixed color can be considered.

COVID‐19 maps translated invisible phenomena and ubiquitous geospatial information into visualization. Although it has become common to use maps for risk communication in all kinds of hazards, systematic methodologies for validity evaluation are still insufficient. This study aimed to improve the risk communication of COVID‐19 maps, trying to find suitable solutions in terms of cartographic design. The eye‐tracking method was used to capture visual cognition patterns and provide cognitive explanations of users' strategies that supported the hypotheses. Through discussing the further mechanism behind the results, relationships between visual presentation and usability were observed. This study enriches the perspectives and theories in the general area of hazard and risk mapping, especially with themes of public health events. Moreover, this study could help build the context of hazard scenarios in virtual geographic environments (VGEs) as well. As a VGE is an integration of geographic information and human behavioral models, it should consider the usability of its user interface settings to determine users' strategies, especially user effectiveness and efficiency (Ugwitz et al., 2019). It contributes by providing an empirical multidimensional approach to evaluate hazard visualization, but also provides practical strategies for the design and optimization of related infectious disease maps.

However, this study also has some limitations. First, a conventional principle of thematic map design divides the data into four to seven levels (Field, 2018; Olson, 1972), and the data in the experimental materials have been divided into five levels. Whether the number of data levels affects the performance of information transmission needs further study. Second, readers' perception and cognition are also related to the perceivers' age, cultural background, individual knowledge, and graphic literacy (Freedman & Shah, 2002; Sherman‐Morris et al., 2015). Graphic literacy refers to one's ability to understand graphically presented information and includes general knowledge about making inferences from different graphic formats (Freedman & Shah, 2002). Higher graphic literacy is generally related to higher education level or graphic experience (Galesic & Garcia‐Retamero, 2011); therefore, different readers reading the same map may have different cognitive patterns. As the majority of subjects recruited in this study have a Batchelor degree, the sample may not be sufficiently representative. Third, maps used in the experiment were displayed statically on a computer screen and could only be manipulated with a mouse. In a real scenario, COVID‐19 maps are usually presented using a web‐based mapping application with interactive manipulation and other kinds of visual features, such as 3D, textures and hatching, animation, zoom in and out. Therefore, the participants may have different performance and emotion in a real scenario compared to viewing maps in the experiment. In follow‐up research, the scope of the subjects will be expanded to make the research more general.