Data practices during COVID: Everyday sensemaking in a high-stakes information ecology.

How do people reason with data to make sense of the world? What implications might everyday practices hold for data literacy education? We leverage the unique context of the COVID-19 pandemic to shed light on these questions. COVID-19 has engendered a complex, multimodal ecology of information resources, with which people engage in high-stakes sensemaking and decision-making. We take a relational approach to data literacy, examining how people navigate and interpret data through interactions with tools and other people. Using think-aloud protocols, a diverse group of people described their COVID-19 information-seeking practices while working with COVID-19 information resources they use routinely. Although participants differed in their disciplinary background and proficiency with data, they each consulted data frequently and used it to make sense of life in the pandemic. Three modes of interacting with data were examined: scanning, looking closer and puzzling through. In each of these modes, we examined the balance of agency between people and their tools; how participants experienced and managed emotions as part of exploring data; and how issues of trust mediated their sensemaking. Our findings provide implications for cultivating more agentic publics, using a relational lens to inform data literacy education. Practitioner notes: What is already known about this topic Many people, even those with higher education, struggle with interpreting quantitative data representations.Social and emotional factors influence cognition and learning.People are often overwhelmed by the abundance of available information online.There is a need for data literacy approaches that are humanistic and relational. What this paper adds Everyday data practices can be variable and adaptable, and include engaging with data at different levels: scanning, looking closer, and puzzling through. Each of these modes involves different data practices.People, independently of their quantitative interpretation skills and disciplinary backgrounds, may engage differently with data (eg, avoiding versus delving deeper) based on their emotional responses, level of trust or interpersonal relationships that are evoked by the data.These everyday data practices have implications for people's sense of their own agency with data and involve emotional and trust-based relationships that shape their interpretations of data. These relational aspects of data literacy suggest productive directions for data literacy education. Implications for practice and/or policy Data literacy can be taught as a process that is inherently relational, for example, by discussing the ways in which learners are personally connected to different data, what emotions these connections evoke, and how that affects the ways in which they attend to, trust and interpret the data.Data literacy education can cultivate a wider range of data practices at a variety of depths of interaction, rather than prioritizing only in-depth inquiry.It may be helpful to include complex experiences with data sources that require learners to go beyond a binary "trustworthy/untrustworthy" distinction, so that learners can become more strategic, nuanced and intentional in forming a variety of trust relationships with different sources.Discussing how learners' everyday data practices interact with different data representations and tools can help them become more critically aware of the possible purposes, values, and risks associated with their everyday data practices.

INTRODUCTION

There have been calls for a rethinking of data literacy from new perspectives that have been called “humanistic” (Lee et al., 2021) and “relational” (Wilkerson & Polman, 2020). From such perspectives, learning to reason with data involves learning to navigate interactions with data, with tools, and with other people. This learning process has aspects that are not only computational but also social, civic and political (D'Ignazio & Bhargava, 2020; Garcia et al., 2021; Philip et al., 2016). This work points to the need for research to examine the ways people interact with data in their everyday, non‐professional lives (Kennedy & Hill, 2018; Masson & van Es, 2020; Pink et al., 2017; Tissenbaum et al., 2021). Focusing on everyday settings can surface aspects of reasoning with data that may not be salient in professional data science contexts, due to contextual differences, and to the under‐representation of groups of people in these settings (D'Ignazio & Bhargava, 2020; Kennedy & Hill, 2018; Pinney, 2020).

In this paper, we examined how people of varied backgrounds sought and interpreted COVID19‐related data and information in their day‐to‐day lives. The publicly available information about COVID‐19 has engendered a complex, multimodal and dynamic ecology of information resources (Cuan‐Baltazar et al., 2020) concerning a complex phenomenon that has impacted the lives of most people. This offers a fertile ground for gaining insight into non‐professional data practices, because people are engaged with the data out of their own volition (Bowe et al., 2020), rather than as a study manipulation (Kennedy & Hill, 2018).

Moreover, studying data practices during the pandemic can help us understand how to incorporate educational experiences with complex and equivocal datasets. Recent studies suggest that such experiences are important, because the structured, circumscribed datasets that often dominate educational experiences do not allow for developing the full range of analysis and critique required in out‐of‐school contexts (Chinn et al., 2021; Rubin, 2020).

In working toward better understanding humanistic and relational aspects of data practices in everyday settings, we draw on sociocultural theory, and on conceptualizations of literacy as social practice (Street, 1984). In this study, we asked the following research questions: We first present our theoretical framework relating conceptualizations of literacy as social practice and mediated action to data literacy. Next, we present a qualitative analysis of participants' accounts of their COVID‐19‐related data practices. We conclude with a discussion of implications for data literacy education, suggesting ways in which instruction and assessment can better connect with out‐of‐school practices in a datafied society.

RQ1: What publicly‐available COVID‐19 data did people seek?

RQ2: How did they go about accessing and interrogating this data?

RQ3: How did these data practices play a role in the ways people made sense of their lives in the context of the pandemic?

THEORETICAL FRAMING

Following Radinsky (2020), we operationalize data as “representations of quantity, space and time—numbers, charts, graphs, maps—as they are mobilized for inquiry and argument” (p. 375). Our approach to data literacy, like others' (D'Ignazio & Bhargava, 2020; Lee et al., 2021; Wilkerson & Polman, 2020), is based on the conceptualization of literacy as social practice (Street, 1984), in which literacy is seen as a process of making use of sign systems to create meaning and act. To be literate involves familiarity with particular sign systems and the conventions of their use but also the ways in which people mobilize resources to achieve goals, and how they position themselves and are positioned in terms of access or inaccess to particular ways of using sign systems (Lewis et al., 2007). From this perspective, to gain insight into how to create learning environments that cultivate literacy, we need to understand how texts naturally play a part in human activity (Street, 1997).

Data literacy, therefore, has to do with the ways people make sense of the world using representations of quantity, space and time (AUTHOR, 2020; D'Ignazio & Bhargava, 2016). This includes analytic practices using numerical, informational, graphical, statistical, computational and digital sign systems, as well as epistemic awareness (Lee & Wilkerson, 2018; Pedersen & Caviglia, 2019; Wilkerson & Polman, 2020), and discursive moves that consider the context within which the data are embedded (Bhargava, 2019; Lee & Wilkerson, 2018). Thus, data literacy has been described as developing within data ecosystems (Bhargava et al., 2015), within which people interact with data and with one another, bringing to bear multiple resources and sign systems, each of which mediates their agency (Wertsch, 1998).

Our analytical focus here is on seeking, interpreting, and acting on representations of quantities within such data ecosystems. However, these processes are inextricably intertwined with oral, textual, pictorial and other forms of non‐quantitative, multimodal information (Prado & Marzal, 2013). In the present analysis, where practices did not involve representations of quantity per se but were part of an information‐seeking process that would appear to mediate data sensemaking—such as when a participant described their relationship with a news anchor whom they trusted to provide accurate information—we included these interactions in our analysis, in order to better attend to the contextual and relational dimensions of data literacy.

Agency and data literacy

Literacy is key to human agency, which we define as the capacity to act (Ahearn, 2001). If, within a specific context, people know how to use sign systems to accomplish their goals well, then they are highly agentive in that context. Developing particular skills, and engaging in certain practices, can increase agency: it can enable people to accomplish things and assume social positions that were not possible previously. For example, Ahearn (2004) describes the ways in which developing reading and writing skills and adopting the practice of love‐letter writing opened a pathway for young Nepalese women to self‐initiated (rather than arranged) marriages.

From a sociocultural perspective, we do not assume that agency is a binary attribute that one does or does not possess. Rather, in the work of data interpretation, agency is a variable capacity to act and make meaning, which is distributed among agents and their cultural tools (Wertsch, 1998; Wertsch et al., 1996). In the example of love‐letter writing above (Ahearn, 2004), it was not only developing the individual ability to read and write that increased women's agency, but rather a constellation of interacting practices that afforded these new pathways in life: the love‐letter genre being taken up by local youth, and changing norms that valued emblems of modernity, influenced the inclination to write letters, the way the letters were perceived and the subsequent alternative paths to marriage.

Similarly, agency in the interpretation of data is distributed across interpreters and their tools (Hutchins, 1995). These tools can, for example, include tabular representations of quantifications of social or natural phenomena (eg, number of people in a region testing positive for COVID‐19), and a method for interpreting such tables that can be conveyed in a passing conversation with a friend or colleague (eg, noting the importance of considering number of cases with respect to total number of people in the region's population). For a relational understanding of data literacy, we need to identify the ways agency is distributed across people, their tools and data sources, and their relationships with others, in the contexts in which they encounter and use data in their everyday lives.

This distribution of agency is considerably more complex than it was even a decade ago. Garcia et al. (2021) argue that the contemporary digital world is increasingly mediated by online entities which have shifted people's agency with respect to information. Ironically, public access to data which is touted as enabling more agentic publics, can result in experiences of reduced agency for both individuals and social groups (Bhargava et al., 2015; D'Ignazio & Bhargava, 2020; Kennedy, 2018; Kennedy & Hill, 2018).

Management of resources in data literacy

Although agency is distributed, the agent (ie, people utilizing data) plays a crucial role in mobilizing, coordinating and managing resources, both external (eg, online sources or knowledgeable others) and internal (eg, understandings, repertoires of practices, emotions) (Stetsenko & Arievitch, 2004; Valsiner, 2002). For example, seeking advice after encountering an impasse can be a way to create more agency, by importing into the situation the practices needed to overcome the impasse. The agency resides in the interaction between the more‐knowledgeable advising other and the impassed agent, but the agent's management of resources—seeking the advice—plays a key role in changing the balance of agency.

Therefore, in our analysis, we examine the moves that participants make in managing external and internal resources to understand the dynamic distribution of agency in everyday data literacy practices. This management, especially of internal resources, is akin to the psychological notion of self‐regulation (eg, Azevedo et al., 2017). In particular, in the present analysis we highlight two resources emphasized in research on public engagement with data and online information: trust (Hendriks et al., 2016, 2020; Tynes et al., 2021), and emotion (Garcia et al., 2021; Kahan, 2017; Sinatra & Seyranian, 2016). Each is briefly reviewed below.

Trust, agency, and data literacy

Consistent with our relational approach to data literacy, we draw on literature that emphasizes the social aspects of epistemic trust, that is, trust that pertains to knowledge. In this view, trust is a relationship between a trustee (a person or institution that is relied upon to provide accurate and true knowledge) and a trustor (the person who relies on the trustee) (Irzik & Kurtulmus, 2019; McCraw, 2015; Rolin, 2020). In the practice of data literacies within everyday information ecologies, trust/mistrust relationships mediate what we do with data, because all agents, both experts and laypeople, rely on others as purveyors of data and data‐interpretations in their own meaning‐making and knowledge construction (Miller & Freiman, 2020). The role of trustee extends beyond human agents, such as our reliance on our watches to enable us to coordinate activity by accurately telling us the time (Goldberg, 2020).

Garcia et al. (2021) illustrate the ways that falsehoods are widely understood to circulate regularly online, while trust is declining in institutions, government, media, schools, medicine, and political culture (Price & Peterson, 2015). However, research also points to considerable variability in these trust relationships among individuals, social groups and the objects of trust (Evans & Hargittai, 2020; Funk et al., 2020). In this context of problematized trust in information, data literacy must involve more than simple admonitions to rely on trustworthy sources (AUTHOR, 2020; Wineburg et al., 2020). Participating with agency in online information ecologies can involve forming relationships specifically to combat untrustworthy or nefarious actors, including by organizing with trusted others against racist, politically manipulative or unjust deployments of online data (eg, Tynes et al., 2021).

Emotion, agency and data literacy

Literate practices involve transitions between emotions as readers interact with texts. Texts can spark an emotion, which triggers actions, which can change the mode of interaction between reader and text, which in turn can spark a different emotion. These transitions can either enhance or impede sense‐making activity (Graesser & D'Mello, 2012; Trevors et al., 2017). For example, encountering an unfamiliar word can lead to confusion, which can lead to re‐reading the passage, which can facilitate inferring the meaning from context. Similarly, in seeking, interpreting and acting on data representations, people may feel wary, encouraged, enlightened or confused. Increasingly, psychological research points to the advantages of recognizing, analysing and integrating emotion in sense‐making activity (eg, Suri et al., 2018).

While psychological research tends to emphasize an individual's well‐being, here we emphasize agency in sense‐making activity. In this framework, emotions are the inner outcome of evaluating one's own or others' actions with respect to the goal of activity, and as such, serve as devices to coordinate action (Holodynski, 2013). When agents attend to emotions and use them to coordinate action, they can increase their agency in that activity (Benesch, 2018), although the actions they take may or may not lead to normatively better outcomes.

Several studies point to the public's reliance on emotion as preventing them from engaging deeply with information and exploring alternatives, or distracting them from evidence that disconfirms their current positions (Garcia et al., 2021; Kahan, 2017; Sinatra & Lombardi, 2020). Yet, research on the role of emotion in regulating cognition (eg, Suri et al., 2013; Trevors et al., 2017) suggests that it may be inattention to one's emotions, rather than the experience of emotion, that is detrimental. These dynamics, increasingly evident over time as data ecosystems become more complex and consequential, suggest the urgency of understanding the ways people manage their emotional responses during everyday interactions with data.

The present study

The present study responds to recent calls for gaining a deeper understanding of humanistic and relational data literacy, in order to design learning environments that prepare learners to negotiate an increasingly datafied world (D'Ignazio & Bhargava, 2020; Garcia et al., 2021; Lee et al., 2021; Wilkerson & Polman, 2020). One understudied way to gain such an understanding is to examine how people use publicly available data sources in non‐professional, everyday contexts (Kennedy & Hill, 2018). Such contexts can include routine and mundane practices, as well as uncommon practices that arise under crisis or unusual circumstances, both of which provide productive fodder for understanding human sensemaking activity (Neal & Murji, 2015).

METHODS

We conducted semi‐structured, task‐based interviews as a window into six participants' routine data practices. The interviews, conducted in Zoom, asked for background and demographic identifiers; asked each participant to describe the ways they sought and interpreted information about the pandemic; and included a concurrent think‐aloud protocol (Ericsson & Simon, 1998; Goldman et al., 2012) in which they talked through their step‐by‐step use of one or more routinely used online COVID data sources with a shared screen. Where practices were described, we asked for elaboration on when, where, how and why they engaged in these practices, seeking an understanding of the relational context in which the data were used. Such narrative accounts are well suited to identifying relational aspects of participants' data practices, since they reveal the ways people position themselves with respect to others, in relationships with information and sources, and with respect to the represented world (AUTHOR, 2020).

For our sample, we sought diversity in people's likely relationships with data, so we recruited adults of different ages, educational experiences and disciplinary backgrounds. We combined a convenience sample with a snowball strategy (Crabtree & Miller, 1999; Marshall & Rossman, 1989), starting with three participants known to us, and recruiting three additional participants through them. Although six lies on the lower end of qualitative sample size conventions for maximum variation strategies, it is not rare (Marshall et al., 2013; Vasileiou et al., 2018), and is considered sufficient for studying highly specific experiences (Malterud et al., 2015).

An overview of the demographic characteristics of the six participants is included in Table 1. Participants ranged in age from their 20s to their 50s. Four identified as women and two as men. Four were residents of Israel, one of Canada and one of the United States.

TABLE 1

Participants

	Nationality	Racial/ethnic identification	Age	Gender identification	Disciplinary background/profession
P1	Chinese	East Asian	20s	Male	Researcher
P2	Israeli	White	20s	Female	Biology BA, theatre background, paramedic
P3	Israeli American	Jewish	50s	Female	Literature BA
P4	Israeli American	Iraqi, Romanian	30s	Female	Photography, film technical degree
P5	American	Bi‐racial	40s	Female	Geography MA, Creative Writing MA, doctoral student
P6	Israeli	Jewish	30s	Male	Physics BA, self‐taught computer scientist, open‐source programmer, start‐up founder

Coding and analysis

As noted above, the research questions guiding this study were: In answering research question 1, concerning what data resources people used, and research question 2, concerning participants' practices with these resources, we analysed the interview transcripts using content analysis (Mayring, 2000) and an immersion approach (Crabtree & Miller, 1999). From a transcript of each interview an initial round of coding identified sources of COVID data used (RQ1). Through line‐by‐line coding of transcripts, distinct practices were identified from each participant's narratives and think‐alouds (RQ2).

RQ1: What publicly‐available COVID‐19 data did people seek?

RQ2: How did they go about accessing and interrogating this data?

RQ3: How did these data practices play a role in the ways people made sense of their lives in the context of the pandemic?

To answer research question 3, we used the practices identified for RQ2 as the unit of analysis, and conducted a second round of coding examining participants' narrations of each practice, looking for themes related to making sense of their lives in the context of the pandemic. The themes that emerged in this secondary coding, related to agency, managing emotions and trust relationships, are presented below in the Findings section. In the Discussion section that follows, we sketch a relational understanding of the data literacies that emerged as participants made sense of the COVID pandemic.

FINDINGS

RQ1 and RQ2: What data did people seek, and how did they access and interrogate them?

In answering RQ1 and RQ2, we identified a wide range of routine practices that were narrated by participants and demonstrated in the think‐aloud activity. In these routine practices participants used multiple resources for making sense of the pandemic (RQ1) in diverse ways (RQ2). Examples of these resources and practices are listed in Table 2, and described in detail below.

TABLE 2

Data sources and practices for each participant

	Example data sources used	Example practices narrated in the interview
P1	Crowd‐sourced COVID‐data website created by people from Wuhan W.H.O., C.D.C. (USA) and C.D.C. (China) as untrusted comparisons Information and links shared in WeChat groups	Daily review of data dashboards Re‐check crowd‐sourced site at different times of day Compare data from official websites with crowd‐sourced data Click through crowd‐sourced links to news stories about cases of interest
P2	News articles, accessed through a search engine Social media sites Data dashboards	Browse news article headlines in a search engine Daily scan of data websites Social media sites “for a break” Look for locations where “in a very bad place” Compare dire places to locations of friends and family Appraise whether likely to be needed for emergency paramedic work
P3	Articles on news websites Phone news app Numerical data on new cases in last 24 hours	Skim headlines, click on articles of interest (“impact on my daily life”) Check phone app daily, sometimes before out of bed Scan daily data tables for “huge numbers” Avoid graphs or complicated metrics Sometimes scan virology or epidemiology reports
P4	Phone app pop‐ups News articles accessed through a search engine W.H.O. Instagram page CNN television broadcasts Facebook posts by trusted news anchors	Monitor phone pop‐ups, sometimes waiting until end of day “Obsessively checking the new numbers” on news infographics “Play with” “all the information” on the W.H.O. Instagram page Scan trusted anchors' Facebook posts, but “not … as a news outlet” Cross‐check people and sources against CNN reporting
P5	Data dashboards accessed through a search engine Infographics in general Facebook, Instagram posts Twitter app on phone News headlines (“conservative stuff”)	“Play with” Twitter world data app Scan news headlines from opposite political viewpoint Avoid news that invokes emotional response “Put news [reports] and data sets [from dashboards] … together”
P6	Instagram feed accessed from phone Johns Hopkins website Ministry of Health telegram channel Open source data resources WhatsApp messages	Monitor open source data resources Avoid news sites for data Daily fact‐checking of information across sources Monitor WhatsApp groups (up to “a thousand messages a week”)

Working inductively in coding the practices across participants, three categories of practices emerged in the analysis: We found versions of each of these three categories of practices across all six participants. The similarities and differences in how they described their practices reveal different relationships to information, information sources, tools and other people. These similarities and differences, described below, are useful for developing a relational understanding of data literacy practices along three dimensions: agency, management of emotions and trust. In the Discussion that follows, we examine the implications of participants' data practices for data literacy education.

Scanning: browsing, checking, glancing, looking at information sources;

Looking closer: choosing particular things to inspect or seek more information; and

Puzzling through: crafting an intentional inquiry, examination or thought experiment to explore the meaning of the information.

Scanning

Each participant described information‐seeking practices that involved scanning the information environment, trying to get a quick and easy sense of available information. In their Scanning practices, participants set up tools and routines to do much of the work for them, allowing them to engage easily without exerting great effort. Scanning practices involved a range of information sources (see Table 2). Participants scanned in search of different things: numerical data, particular kinds of content in news items, or information about places or subjects of personal importance. Although “Scanning” is a term that suggests seeking out information, there was also a converse function: to filter out a good deal of the information inundating participants during these practices. In some ways Scanning limited their engagement with information, allowing them to find relevant things while avoiding an undesired level of engagement.

Looking closer

This is a practice in which the person breaks out of a scanning routine to intentionally seek some greater level of understanding. From scanning headlines of news stories or a table of data, they might choose to click on a link to read more, or to view the data differently; or they might go to a different information source to pursue the same topic. In this process of shifting from scanning to looking closer, the person applies some criteria for selecting information from their scanning routines that they deem worthy of understanding more. These criteria were different for each person. The shift from scanning to looking closer might be indicated by a decision to switch from one information source or tool to another, or from one mode of browsing with a tool to a different mode of more intentionally looking for something with the same tool—sorting a list differently, switching to a different view, perusing a clicked article or watching a clicked video.

Puzzling through

In the shift to Puzzling Through, the person constructs comparisons or examples with the information, applying different conceptual categories, or proposing a thought‐experiment in which the information requires explanation. Comparisons could be between different places (countries, states, cities, neighbourhoods, social settings), policies (shutting down, opening up, masking), populations (racial, ethnic, age or nationality groups), or points in time (periods of the pandemic, months, times of historical events). The construction of these comparisons is mediated by the information sources, such as data dashboards with tables or maps that present the user with ready‐made comparisons. However, in puzzling through the choice of which things to compare, and the sense that is made of the comparisons, reflects a different stance than simply clicking for more information. In constructing comparisons or examples, participants articulated the relationship between data and a represented world—sometimes explicitly mapping out what particular interpretations would mean in that world. This could also involve talking through their computational sense‐making about statistical, spatial, or distributional properties of data. The ways people narrate their thought experiments or illustrative scenarios while Puzzling Through provide a glimpse into the ways data are deployed in sensemaking.

RQ3: How did these data practices play a role in the ways people made sense of their lives in the context of the pandemic?

As noted above, to answer this question we used the practices identified for RQ2 as the unit of analysis, and conducted a second round of coding examining participants' narrations of each practice. In this secondary coding the following themes emerged as valuable comparisons across participants' practices: These sub‐questions structure the section of the Findings below.

What forms of agency are constructed, with what tools and agents, in the narrations of participants' practices?

What emotional engagements and reactions are narrated?

What relationships of trust/mistrust are constructed in the narration of these practices, and how are the narrator and others positioned in these relationships?

RQ 3.1. Agency in making sense of the pandemic with data

Agency in scanning

For some participants scanning practices helped them maintain a sense of agency in a situation in which there was little ability to control events. P5 described Scanning world news sites “just to kind of try to keep aware of, like, the big things going on … like big picture stuff” [43]. She valued a Twitter “World Roundup” app where “you could see actual numbers” [113], and used graphs and interactive data maps “probably more than anything else … because I can pretty quickly see the trend, and I can see the actual numbers from day to day like going across” [306]. P3 described a practice of “skimming”: “go[ing] over the headlines first primarily just see if there's been any change or if anything has happened … that's critical for me to know about” [76]. She would check “to see how many new cases have been reported in the past 24 hours—it's a gauge for me to get a sense of how prevalent the virus is currently in the general public” [348].

Some participants' scanning practices involved giving over agency for their information search to a tool or source, rather than trying to control it themselves. For example, P2 and P4 relied on pop‐up prompts on their phones to alert them to news items. P4 would get notifications from the L.A. Times that “would take me to the page … I would check like the latest news on the updates” [191]. This giving over control could sometimes lead to a sense of losing agency, becoming dependent on the tool or even obsessed. P4 described a time when “every day I was like obsessively checking the new numbers” [354]. P5 described herself losing her own agency with respect to a Twitter phone app, such that “I hid the app for myself on my phone because I felt like I was getting a little obsessed with it” [113].

Some used scanning to filter out some of the overwhelming deluge of information, thereby maintaining a sense of agency. Sometimes this filtering involved limiting the amount of time spent on scanning. P5 described how she would scan quickly, to avoid looking at too much information: “if I do a search for news and I can't find it pretty quickly … I don't necessarily wade through the trash to find it” [279]. P4 described how sometimes instead of opening a pop‐up notification she would “just like wait another two hours and then read all the notifications, because sometimes you get so many notifications that you're just like, ‘I have to do this [just] once a day’” [76].

Agency in looking closer

The decision to take an additional step into the information, looking closer, was another practice through which participants expressed their agency as finders and interpreters of data. One reason for going beyond scanning was to find particular information with which to make decisions in their own lives. P3 would Look Closer “if anything has happened … that's critical for me to know about … if there's anything really urgent,” in which case she would “start clicking on articles” [76], looking for “anything that I think might have an impact on my daily life.” This might include “more diagnosed cases of the coronavirus in my area, or … if changes are being contemplated in restrictions on travel, or things like that. Those things will definitely get my attention right now” [84]. For P2, “because I'm a paramedic … if tomorrow there'll be 1,000 new cases, I'll call them up and be like, ‘Hey, do you need me?’ So, for me personally it's very, very relevant to my life, because if it gets really bad then I'm out of my home and like I'll probably go and help” [243].

Another reason for looking closer was to seek out voices or perspectives that might be missing on first glance. For example, P2 would sometimes look into social media posts, despite her general sense that they were not reliable, because she found it “important to hear it out [because] in the end of the day, social media is the outlet for the people” [130]. This was a way of exerting agency over a news environment that did not provide “outlets for the people.” Similarly, in P5's words, “I know that what I'm seeing is not always everything that's out there. So I try to dig a little bit past that sometimes, if I can” [54].

Looking closer was also a way to maintain agency in an environment where information might be misleading or false. P6 would generally seek a second source for most information: “If I tend to see something, I try to see if there's anything to confirm that information” [42]. P5 would Look Closer when data representations might be systematically misleading, such as reporting total numbers (of infections, hospitalizations or deaths) for each area, rather than numbers per population: “Because I have a geography background I knew that I would need to look at data, you know, per population to really get a more accurate picture” [479]. She described how data representations could be misleading when not accounting for different denominators (either spatial distribution or rate per population):

I can look at this map, and I can see, like, ‘Oh, Michigan numbers look a little bit smaller than Illinois’ … like the bigger dots are the bigger numbers, but that's also where there's bigger population centers … So that's kind of annoying. [298]

These practices maintained participants' agency with respect to information sources that might mislead, rather than accepting their face value implications.

P2, the paramedic, described her practices for determining how much closer to look:

If we had 400 ventilated people and they would say only 20 ventilators available, then maybe I would read up a little about the different criteria for when do you choose not to ventilate a patient so you can save someone else. Like, it would be more relevant to hear about all these ethical arguments. But right now it's just not relevant, so I feel like it's kind of pointless. [254]

Thus looking closer, like scanning, was also a practice for filtering unnecessary information.

Agency in puzzling through

Puzzling through involved constructing comparisons, thought experiments or reality checks. These practices could position participants as agentic consumers of information, not at the mercy of their sources, tools, or other people. For example, P5 would regularly compare mapped or graphed data with narrative news reports, “digging” past either text's initial presentation of information to construct an understanding: “I think together, maybe you get a better picture when you put news and data sets like this together” [455].

Comparisons are often pre‐packaged in COVID‐19 data resources, such as data dashboards that offer comparison tables by country, state, city, time period or sub‐population. Simply reading these sources did not necessarily prompt puzzling through practices, and some participants simply read off numbers from such tables. In contrast, P5's Puzzling Through practices enabled her to go around those provided comparisons when they were not useful, and construct her own comparisons:

Like just the other day I was trying to compare Canada and the US. And I was like, “That's not very helpful because their populations are different.” So unless I can find a source that shows the percentage, or unless I calculate percentage myself, which I've done, the population is so different where it's not worth comparing. But then I noticed that the population of California is not that different from the population in Canada. So I've been comparing that and looking at, OK, so Canada had 761 cases I think the other day, California had … something like 9000 [cases]. [208]

Similarly. P4 constructed a thought experiment when sense‐making with data on the rates of COVID‐19 cases on a timeline:

When I would like look at all the information, what I would look at is how many tests, how many tests they are doing. And the day they did tests. Because something I was also thinking about during Corona is, there is, like, a time that people like left the house. Everybody was going to the beaches and you would know that, you would see it. Like there was always a backlog, like you would see the numbers rising like two weeks after. So I also, like, would take that another week for sure to not like leave the house. I'm like, OK, everybody went to the beach. Let us see. A lot of people are out. Let us take another half a week, work and stay home. [581]

P4's puzzling through practices here mediated her agency in making decisions, by constructing a hypothetical world of tests, reported test results, and public activities that might produce those data.

P5 discussed an extended example of puzzling through. She sketched out a thought experiment that might surface realities that could be hidden in data reported at the state level, such as racial disparities that might be hidden in data that are not disaggregated by race:

I think that what's interesting about the per 100,000 is, it—I wonder if this is reflecting more of the, like the sort of racial—socio‐economic and racial aspects of it. Like, these are states with poorer people, and lots of ethnic minorities. It did not stand out so much in the total cases … I'm just wondering if like Mississippi, Louisiana, Florida, Arizona … I'm wondering if these states' numbers per 100,000, if this is more revealing of like, say, Black populations, and Mississippi being more impacted than other parts of the country. [400]

… But I just wonder if those places also, since they have a racial minority, it's maybe like a majority population, and I'm wondering if that's—since these populations are reportedly being more affected, I'm wondering if that's what I'm seeing on these maps: are those populations being more affected per 100,000. I do not know if I'm being clear. … Cause it's scaled for population, and it would not be so, kind of, hidden in the overall population. I do not know though. [414]

Here P5 puzzles through the implications of the racial disparities she has heard of in news reports (“these populations are reportedly being more affected”) for interpreting the patterns in the visual data display (“I'm wondering if that's what I'm seeing on these maps”). Her thought‐experiment wonderments attempt to account for something that is noticeable in the proportional representation, moreso than in the total numbers representation (“it didn't stand out so much in the total cases”). In order to “see” the disproportionate impact of COVID‐19 on Black, Brown and poor populations, she looks at the rate of infection in states where she knows there is a concentration of poor people, and/or a “majority‐minority” population. In these places, the disproportionate impact “wouldn't be so … hidden in the total population.”

This approach to puzzling through evinces an agentic stance toward data, though this agency was also moderated by the emotional impact of this thought experiment, as described below.

RQ 3.2. Managing emotions in making sense of the pandemic with data

Managing emotions in scanning

One aspect of scanning that was evident for several participants was that it could become so automatic that it was like “playing,” even relaxing. P2 described looking through social media sites for information as something she would do for fun, when “taking a break” from more rigorous searching related to her work as a paramedic. P4 described how she would visit the W.H.O. website, where they had “really cute graphic designs” [593], and “I would just have all the information, I would just be out here [on the site] … I would just go through all the information and read everything and then just play with” it [626]. Similarly, P6 would regularly go to “open source data resources—it wasn't really the news sites, but sort of sources to play with” [162]. This playful, informal engagement with data appeared to have an affective component, helping to manage the stress of the pandemic.

P5's description of her use of a Twitter phone app (mentioned above) shows how “playing” with COVID data could verge into a troubling feeling of obsession:

In the beginning I was looking at Twitter a lot because it had a cool app that was showing—it would be like a world round‐up, and it would show you kind of what was going on in different parts of the world. And you could see actual numbers. But I hid the app for myself on my phone because I felt like I was getting a little obsessed with it. [P5, 113]

Scanning information sources also involved selectively filtering out sources that would trigger negative emotions. P3 avoided information from politicians who made her angry: “so I'm kind of … skimming over what the prime minister said, I don't really care” [337].

Managing emotions in looking closer

Looking closer was also a practice used for managing emotional responses, concern and stress. On one hand, P3 used Looking Closer to help her decide how concerned to be: after initially not “tak[ing] it as seriously,” she then started trying “to understand how dangerous it was, and how concerned I should be” [174]. On the other hand, when her level of concern abated, she would Look Closer less: “I'm still reading, but I've definitely relaxed more, both in terms of consuming information, and in terms of also the way that impacts my choices about what I do” [216].

P2 would regularly check a data dashboard for information about California “because of my relatives. Usually after I look at this [data] table I give them like a call or send them a message, because it reminds me how bad the situation is in the United States” [280]. Looking Closer here was part of managing emotions of concern. For P4, practices of Looking Closer brought with them feelings of stress and insecurity: “I try to follow everything, but there's so much information from everywhere that you're just like, ‘Did I finish reading everything?’ Sometimes, really, you feel insecure that you, like, missed something” [82].

On the other hand, P5 referred to the ways information sources might manipulate one's emotions by being overly sensational, and she used Looking Closer as an antidote to this emotional manipulation: “I do feel like you have to dig a little bit to get to the data that you want to see, or that maybe sometimes like the forward facing page will be something that's very dramatic, but not necessarily the most helpful” [469].

Managing emotions in puzzling through

The kinds of intentional comparisons and thought experiments we call Puzzling Through also were sometimes associated with managing emotions. For example, P5's thought experiment about investigating possibly disparate impacts of the pandemic in predominantly Black populations (described above under Agency) was something she had thought through and considered pursuing with a more structured inquiry. The interviewer asked:

Is this something you would probably investigate … or just …[keep in] the back of your mind?

Yeah, probably if I had time [CHUCKLES] I would probably do some investigation, but it would definitely be in the back of my mind … I do not know if I would actually do a search for it. Cause some of this is just, like, really upsetting. And so I think I get to a point where I cannot really take in any more of it anymore, the reality of it is so depressing.

Here we see that the upsetting and depressing experience of discovering the realities of the pandemic could stand in the way of seeking out that deeper understanding.

For P4, puzzling through her interpretation of reported test data seemed to contribute to a spiral of frustration and anxiety:

But what was worrying me is, everybody's like, “OK, the numbers are down, the numbers are down,” but they were not doing a lot of tests. And then one day there were no cases. And like, yeah, because they did not even have any tests—they did not even submit tests, or whatever. And so that I was really obsessed with, and I felt like they were doing like some kind of … experiment on us, to see what would happen if, slowly, they would open. Because I do not think anybody … nobody knew what would happen when we open. So … I did not want to leave the house for the first three weeks. [142]

P4's fear of the government doing “some kind of … experiment” can be dismissed as simple conspiracy theorizing, but it is important to note the way this line of puzzling through served as an attempt to explain a data discrepancy that was worrying her. Her decision not to leave the house shows how she sought ways to resolve this spiral of anxiety.

RQ 3.3. Trust/mistrust relationships in making sense of the pandemic with data

Trust in scanning

The main way trust was visible as an aspect of Scanning was in the selection of sources. Each participant described sources—whether news media outlets, data dashboards, or even television personalities—which they trusted to provide the best available information. P4 described such trust relationships: “So that's how I felt safe, like if they were like, if talking the same language as CNN. So like, okay, they're cool. They're fine. They know what they're talking about” [312]. She trusted her father as a source of information, noting that “basically, he would quote [CNN commentator] Sanjay Gupta a lot” [322].

Some participants also described scanning sources which they did not completely trust. This involved Scanning in a way that accounted for possible inaccuracies, such as P2's description of information from social media as something to be “taken with a grain of salt.” Similarly, P4 stated clearly that social media sources like Facebook were not to be trusted, and though she would see information that people posted, she would “just try not to use them as a news outlet” [247].

P1 relied on a crowd‐sourced COVID data website that had been created by people from Wuhan, China, as an alternative information source to official sites like the WHO or the Centers for Disease Control (CDC) in the USA or China, which were not trusted. He described Scanning routines that included attending to the time of day, since data derived from multiple sources would be corroborated and “cleaned up” as the day went on, suggesting a nuanced set of trust relationships involving awareness of sources of error and scepticism about governments' motives in data collection and reporting.

Trust in looking closer

Some participants would only Look Closer with sources they already trusted. For example, although P4 (like others) distrusted Facebook, she had two favourite news anchors on an Israeli news channel, and she would go on Facebook to “read what [either of those anchors] would write. … if I would read stuff on Facebook, it would be from people that I actually trust, that I listened to them on the news” [304].

P6 described Looking Closer as a regular practice with almost any information of interest, and scepticism of information was always part of it: “I've been reading a lot, … and in many cases it was just really hard to determine, if you have a source of information, if it's true or not” [63]. In contrast with P4's emphasis on trustworthiness of sources, P6 combined cautious trust of certain sources with triangulation across sources:

Wikipedia I think is a good source that, if something is in there, there's a high chance that it's right. Maybe multiple other sources, if one source [is not adequate] maybe try and find another source which is not necessarily biased or connected, to try and confirm that. [45]

Thus, trust could be either a criterion for sources when looking closer, or a more complex relationship to manage and balance when perusing multiple sources.

Trust in puzzling through

When P6 moved beyond simply cross‐checking information across sources, he would construct thought experiments and reality‐checks to judge whether a news story or datum made sense, or even if it “has any meaning.” This was a practice not only with everyday news reports, but “even, like, academic stuff” [63]. He offered examples of research reports the logic of which were suspect, and which defied even common sense scrutiny: “I had to add a sort of intuition, if it has any sense or not” [73]. P6 pointed out that this “gets harder when you go into the biological stuff,” but that even scientists “wrote a whole bunch of things which could be disproven” [74].

P6's high confidence in his ability to separate fact from fiction (or to know when he could not) was central to his COVID data practices. A profoundly critical stance toward news media accompanied this practice of constant fact‐checking and reality‐checking:

Journalism is a wonderful way to be in touch with the ignorance of the community. … Seeing what the media says, in many cases could help you understand where things might be forged [meaning falsified], not only here but in other countries as well. Seeing what interests people have was also something to factor in when you are reading stuff. [365]

P6 described processes for making decisions about how much to trust information from particular sources, and why. His Puzzling Through practices shared some aspects of P4's sceptical stance, but were articulated less in terms of anxiety and more in terms of strategies for sense‐making about when information might be “forged” [falsified]:

So what made you suspect that something is forged? Or, what kind of clued you in to interests?

So, for example, China, I think, was being fairly accurate in the beginning, of reporting who is sick and who is not. Because you could deduce from what's going on that they wanted to signal to the world that something is wrong. Now … it's harder to tell, because everybody knows that there's something wrong, but it may not be China's interest to show that. … Thinking through these kind of strategies would tell me that maybe the reports of what's happening in China now might not be 100% accurate anymore.

P6's puzzling through practices did not seek to simply decide whether a source was trustworthy or not, but explored more nuanced issues of how their particular interests might be expected to skew their information in predictable ways:

So knowing what's happening in the media, and who has what kind of interest, would be relevant to understanding what's going on. Sometimes, for example, if you could see over‐optimistic reports in the media, you would see that maybe the information that you are getting is not accurate because it seems that they are hiding something. Or weird reports: I think Iran had way too many deaths per cases reporting. So you would know that there's more cases there, right? … So seeing what people are reporting, or what's in the news, or seeing that there's information missing, would tell you that someone's hiding something, which is also interesting when you are trying to make sense of what's going on.

In these ways, a lack of trust was not simply a reason to discount a source, but rather a lens through which to interpret data.

DISCUSSION: IMPLICATIONS FOR DATA LITERACY EDUCATION

This analysis of participants' data practices during the COVID‐19 pandemic suggests a useful set of considerations for data literacy education, not just in light of the pandemic, but as a way to understand everyday data practices more broadly. From a relational perspective, we can think of different modes of engagement with data (practices of Scanning, Looking Closer and Puzzling Through) as ways of managing the balance of agency among the reader, their tools and information sources, and a wider network of people and tools that make up their data ecology. In a time of “data deluge” (The Data Deluge, 2010) and widespread uncertainty about information in the environment, purposeful data practices offer ways to exert one's own agency as a reasoner and participant in the data‐mediated world.

Tools and agency

Data literacy education can help learners develop tools for engaging in these practices purposefully—including technical tools, and also cultural and cognitive tools like statistical concepts and critical habits of mind. All tools have a double function, both facilitating and obfuscating our awareness of the world that is represented with data. The tools we use shape the things we are likely to notice when scanning, or choose to examine when Looking Closer. We can submit to these tools' mediation of our agency, and/or work around it (P1 cross‐checking crowd‐sourced data; P5 constructing her own comparisons). We can seek out the tools that afford the kinds of data practices we prefer, and/or use tools that prompt and tweak us to look at things. We can give over more agency to our tools, obsessively checking data when we do not want to (P4 checking in bed); or we can exert control over such obsessions, regulating our use of them (P5 hiding her Twitter app).

All of these can become valuable targets for data literacy education. Through teaching and assessment, we can raise learners' awareness of this balance of agency as it plays out in their own data practices, and model practices that increase their agency. As learners become more conscious of their own practices and the balance of agency among themselves, their tools and sources, and the designers of their tools and sources, they can become more attentive and agentic, and more critically aware of the possible purposes, values, and risks associated with their practices.

Sorting, filtering, comparing, seeking—these tool features enable practices through which people figure out what they are seeing in the data, allowing them to scan or look Ccoser at only certain things. These practices are a way of exercising agency with respect to the data representation, but are also dictated by the design of the tool. One can “dig deeper” (P5, P6) only to the extent that one claims their own agency as a knower, and as a user and critic of the tools and the data. “Click‐bait” is the quintessential struggle for agency when looking closer. One should not be made to care about something enough to Look Closer; but also, one does not have total control over what information is available for that closer look.

For teaching and assessment, we can help learners recognize the range and limits of the tools they are using—for example, for sorting, filtering or comparing—and become aware of things they might want to do with a tool but cannot. They can learn to compare tools, and choose the tool that lets them look closer at the things they actually want to know about (versus those they have been baited to attend to). They can learn to more clearly articulate what they are Scanning for, the thing that made them click or sort—and also to be aware when they are not sure what they are scanning for, recognizing how that can also be a generative mode of search.

Puzzling through in this study included constructing extended comparisons, thought experiments, imagined scenarios, or examples to illustrate a larger point. These practices involve assuming a stance of agency over the information. Though both scanning and looking closer can involve making comparisons as well, they are more tool‐mediated (eg, scanning a table to see which state or country is highest on an index, P1; looking closer at the worst‐ranked states to notice their spatial area and population, P2, P5).

The stance of identifying something that requires further explanation is agentic. For teaching and learning, we can help learners develop this stance with respect to data. Learners can develop their voice and vocabulary for discursively negotiating a represented world with others—articulating what makes sense and what does not (P6's “has any meaning” filter), and how and why the available data make sense or do not in that light (P5's thought experiment), bringing data into conversation with prior knowledge, assumptions and concepts.

Managing emotions as a focus for data literacy education

At least in the context of COVID information, a big part of data practices is managing the emotional experience of the data, learning about COVID realities, and balancing the desire to know with the sense of personal well‐being. Our feelings of understanding or being lost, competence or incompetence, rage or relief are a major part of how and why we scan, look closer and puzzle through. When we manage an obsession with information, we are not only seeking a balance of agency with the tools but also seeking an emotional stability.

For data literacy education, we can incorporate the awareness of emotions into how we teach and assess data practices. Data are not detached from our emotional well‐being. If we design instruction to engage students by tapping into their real‐life experiences, they are likely to have greater personal stakes associated with what they learn. Learners can become aware of their own emotional stakes in data work, learn to communicate about them, and learn to identify them in others. Most participants looked closer for reasons with strong affective components: concern for loved ones, anxiety about their own health, deciding how worried to be. Looking Closer can itself be an index of emotion: when more anxious, one may look closer more often (P3). It might also be a way of inducing an emotional reaction, as when participants Look Closer to see “who is in a very bad place” (P2, P5).

Puzzling through likewise could be very emotional. P5's protection of her emotional well‐being put a limit on her puzzling through how racial disparities might explain Louisiana's data. In other cases Puzzling Through provided a way to distance oneself from emotional responses: by seeking out explanatory mechanisms or ways to explain outbreaks (P2, P6), we might do something more than marinate in the enormous suffering reflected in the data. P4 processed her anxiety and fear by constructing causal explanations for gaps in the data (“there were no tests”), and it seems likely that these explanations may have increased that anxiety and fear.

Rather than assume that reasoning with data should be detached from such emotions, data literacy education can help learners identify the affective aspects of their own desires to seek greater understanding. For teaching and assessment, data‐rich investigation can be taught as a process that is inherently emotional, as is regularly done when teaching the creative and language arts. When one pivots from Scanning to Looking Closer, noticing the emotions one experiences might provide important levels of understanding how and why one is engaging with the data, and the balance of agency among oneself and the tools. In situations where uncertainty and anxiety accompany looking closer or puzzling through, learners can develop skills and habits for noticing and interpreting these emotional cues as ways to better formulate their own data‐analytic purposes.

Implications of trust for data literacy education

When we are scanning or looking closer with tools, there are relationships built into that process, including relationships with the tools, the tools' designers, and the sources of information. Trust is a key element of these relationships. Scanning and looking closer can provide ways to filter out less‐trusted information, as all participants described. Based on the degree of trust, one may develop data practices to account for implicit bias, distortion, or uncertainty inherent in a tool or source, as P6 described. Trust relationships may also determine what, how, when and where scanning takes place, and whose scanning or looking closer practices are used as a model to follow.

Looking closer can be a way of testing or establishing trust—checking, verifying, determining what (trusted or untrusted) perspectives or affiliations are reflected in the data. The experiences we have when looking closer with a source might make us trust them more or less. Looking closer can be an indicator of trust (P4 only looking at trusted sources) or a sign of distrust (P6 always checking information).

There is a tradeoff between relying on trusted sources and learning from new perspectives. Social media sources can be notoriously untrustworthy, and at the same time they can be an “outlet for the people” that is not found elsewhere (P2). We might choose to Scan untrusted sources but not look closer with them, for fear of being enraged or contaminated (P5).

Puzzling through can be a mode of deciding whether and why to trust or mistrust information. This can involve developing a thought experiment or causal explanation to make sense of why something might logically be trustworthy or not. P6 articulates clear strategies for factoring mistrust into puzzling through, assuming “forging” and misrepresentation as logical likelihoods rather than ominous, amorphous threats.

For teaching and assessment, rather than simply teaching that some sources are more trustworthy than others, we can present a more complex reality in which learners become more strategic and nuanced in forming their trust relationships with different people and resources. One can be wary of a source and still learn a lot by Looking Closer at its contents—with appropriate guard‐rails (P2, P4, P5). Learners need to build on relationships of trust in their data practices, so learning these practices also involves learning to build trusting and trustworthy relationships—a valuable direction for a relational approach to data literacy education.

As Garcia et al. (2021) emphasize, online information exists within ecologies that are sometimes designed for nefarious and harmful purposes—racist, predatory, politically manipulative or extortive. Healthy data practices in this context include realistic caution and awareness of these nefarious possibilities. Beyond simple “mistrust,” these practices need to develop more robust modes of scepticism, self‐awareness, and proactive stances like those of P1, P5 and P6, and like the activist roles described by Tynes et al. (2021).

CONCLUSION

In order to understand how data literacy should be taught, we need to understand how data is used in everyday life. By understanding data practices as inherently relational, we can better design curriculum and instruction to impact the ways data practices play out in everyday contexts. Rather than sequestering computational and interpretive skills from those contexts, we can help learners see information sources and tools as meaningful resources for developing their own agentic identities, and managing emotional engagements in the world. By seeing data practices in the context of larger social and civic relationships, learners can go beyond a simple trust/mistrust heuristic to develop relationships that recognize both threats and possibilities to be more agentic participants in civic life.

CONFLICT OF INTEREST

No conflicts of interest were identified in this study.

ETHICS STATEMENT

All ethical and legal requirements for this research were approved by the Institutional Review Board (IRB), including anonymization of participants and informed consent.