Quantifying biopsychosocial aspects in everyday contexts: an integrative methodological approach from the behavioral sciences.

Contextual factors are crucial for evaluative research in psychology, as they provide insights into what works, for whom, in what circumstances, in what respects, and why. Studying behavior in context, however, poses numerous methodological challenges. Although a comprehensive framework for classifying methods seeking to quantify biopsychosocial aspects in everyday contexts was recently proposed, this framework does not contemplate contributions from observational methodology. The aim of this paper is to justify and propose a more general framework that includes observational methodology approaches. Our analysis is rooted in two general concepts: ecological validity and methodological complementarity. We performed a narrative review of the literature on research methods and techniques for studying daily life and describe their shared properties and requirements (collection of data in real time, on repeated occasions, and in natural settings) and classification criteria (eg, variables of interest and level of participant involvement in the data collection process). We provide several examples that illustrate why, despite their higher costs, studies of behavior and experience in everyday contexts offer insights that complement findings provided by other methodological approaches. We urge that observational methodology be included in classifications of research methods and techniques for studying everyday behavior and advocate a renewed commitment to prioritizing ecological validity in behavioral research seeking to quantify biopsychosocial aspects.

Introduction

It would be difficult to justify a behavioral science in which researchers are more interested in asking participants to tick a box indicating how they have behaved over the past month than in observing how these people actually behave in their natural, everyday environment. Studying everyday behavior in context, however, poses numerous methodological challenges. It requires: 1) reconstructing the “who,” “what,” “how,” and “when” for events of interest; 2) positioning the results within the social network of the individuals studied; 3) evaluating the impact of these events on the individuals’ ecological niche; and 4) evaluating the persistence of a reality reconstructed from a selection of everyday moments.1

The explosion of information and communication technologies in recent decades has led to methodological innovations seeking to quantify biopsychosocial aspects in everyday context. While a general framework has been proposed for classifying these innovations alongside more traditional approaches,2 it does not include observational methodology, despite similarities in basic methodological requirements.3–10 The aim of this paper is to review these basic requirements and show why criteria for classifying research methods and techniques for studying daily life should be modified to include contributions from observational methodology. We present our analysis within the framework of ecological validity, a concept refined by two important authors: Egon Brunswik and Urie Bronfenbrenner. In the first section of this article, we describe the ecological validity framework within the context of our study, and in the remaining sections, we characterize the research methods identified in our narrative review, highlighting their shared methodological requirements and reviewing the criteria used in their classification. We show that observational methodology deserves to be considered alongside these methods and advocate the use of mixed-method and multi-method research strategies to capture biopsychosocial aspects in everyday contexts.

Representativeness and ecological validity

The criteria or standards used to regulate and assess the quality of psychosocial research are the subject of intense debate. However, the concept of generalization or applicability of results to other participants, concepts, and times can be found in classic evaluation systems used in both quantitative11 and qualitative research models.12

Egon Brunswik (1903–1955) was instrumental in developing the concepts of representativeness and applicability. Creator of the probabilistic functionalism theoretical framework and an innovative methodological approach he called “representative design,” Brunswik offered coherent arguments challenging the methodological dogma of the time.13,14 His representative design proposal15 challenged the established concept of experimental design, reflected, for example, in Robert S. Woodworth’s seminal work on experimental design popularly known as the Columbia Bible.13 Experimental design focused on isolating and controlling variables, but Brunswik considered that this destroyed the natural causal fabric that existed in the environment of an adapted organism. He maintained that for valid inferences to be made, an experiment had to take into account the representativeness of two sampling dimensions: the participants and the situations to which they were exposed. The now consolidated concept of ecological validity is closely related to the work of Brunswik and his representative design approach, which, though highly relevant, is seldom cited in methodology texts.16

We embrace the concept of ecological validity as defined by Urie Bronfenbrenner (1917–2005) in his ecological model, namely, the “extent to which the environment experienced by the subjects in a scientific investigation has the properties it is supposed or assumed to have by the experimenter.”17 This model questions the scant attention paid to natural environments in many realms of psychological research where context is treated only as a control factor. Bronfenbrenner was interested in interactions between the different systems present in a person’s environment (microsystem, mesosystem, exosystem, and macrosystem) and in their potential impact on the person and his/her psychological development.

Program evaluation studies adopting an ecological approach are methodologically more complex.18 Contextual factors are crucial for understanding the effectiveness of an intervention, and furthermore, it has been questioned whether or not an intervention can actually be separated from its external influences. This same question has been posed by proponents of the realistic evaluation model, in which evaluative research should be able to explain what works, for whom, in what circumstances, in what respects, and why.19,20 Such an approach places context in a central position in which the concept of mechanism (defined as the underlying processes operating in a given context to generate outcomes of interest) can shed light on the relationship between context and expected outcomes. In the following sections, we describe numerous ways of observing, recording, and analyzing these underlying mechanisms and seek to demonstrate the benefits of quantifying biopsychosocial aspects in everyday contexts from an integrative methodological approach.

Research methods and techniques for studying daily life

Mehl and Conner2 proposed the term “research methods for studying daily life” to describe methods used to study repeated, real-time, biopsychosocial aspects in their natural settings. The aim of these methods is to “capture life as it is lived,”21 and they are highly sensitive to the concepts of representativeness and ecological validity. Their growth has undoubtedly been favored by the spread of information and communication technologies. Hogarth et al22 demonstrated the applications of these research methods in a study of perceived risk and self-reported emotional states related to everyday activities. Over a period of 10 days, 74 participants received phone text messages at random moments during the day asking them different questions to assess their perception of risk regarding different activities. They were asked to state what they were doing at the time they received the message and to describe their current emotional state, the possible consequences of what they were doing, and their perceptions of severity, vulnerability, and risk. A random sample of 30 answers was selected from each participant based on the principles of representative design related to situational sampling. Although there were just 74 participants, the study generated approximately 17,700 information units that could be analyzed from an individual or group perspective.

Everyday behavior can also be investigated in Electronically Activated Recorder (EAR) studies.23 The EAR is a small portable audio device that records short fragments of sounds produced at different moments during a day. In one EAR study of conversational styles, 30 seconds of sound were recorded every 12.5 minutes for 4 days in 79 participants, resulting in 23,689 recordings.24 Data collected using this device can be coded using both social environment criteria (eg, location, activity, and interaction) and social interaction criteria (eg, content, style, and emotional expression). EAR-based studies have produced information that helps to link psychological effects to observable frequencies in daily life. The device, for example, has been used to study the relationship between extraversion and time spent alone, and also to estimate the number of words spoken a day.25 It can also be used to analyze common everyday behaviors that are very difficult to report retrospectively, such as laughing or sighing.26

Mehl and Conner2 compiled a set of methods and techniques that, despite their different names (see Table 1, first column), have a similar general purpose and share certain characteristics with traditional methods used in the social and health sciences, such as the diary method21 and designs rooted in observational methodology.3–10 We propose using the broad – and fitting – label used by Mehl and Conner, “research methods for studying daily life” (hereafter RMS-DaLi), to refer to all such methods and techniques.

Table 1

An abridged list of research methods and techniques for studying daily life

Method/technique*	Reference†	Citation counts‡
		PubMed	PsycINFO§
Experience sampling method	Hektner et al56	213 (1981)	383 (1981)
Ecological momentary assessment	Shiffman et al27	501 (1996)	549 (1994)
Ambulatory assessment	Fahrenberg et al40	131 (1981)	107 (1995)
Intensive repeated measures in naturalistic settings	Moskowitz et al41	1 (2015)	1 (2009)
Naturalistic observation sampling	Mehl et al23	0	2 (2007)

Notes:

This column shows the literal terms used in our database searches;

one sample study is provided for each type of method;

accessed February 1, 2015. The number in parentheses refers to the year in which the first study of this type was published;

peer-reviewed journals only.

We performed a narrative review of the literature on research methods and techniques for studying daily life. We searched the PubMed and PsycINFO databases for articles using the search terms shown in the first column of Table 1 and the PsycINFO database for articles and chapters using the terms “observational methodology” and “observational method.” We additionally reviewed the reference lists of the articles retrieved to identify other relevant articles, chapters, and books. The search strategy generated more than 1,000 relevant articles and chapters, from which we selected the papers discussed below. As this is a narrative review, we are unable to discuss all relevant literature and have thus chosen publications that help to justify and illustrate the basic requirements of methods used to study everyday life and the rationale behind the classification criteria used.

Basic requirements of Research Methods for Studying Daily Life

The basic requirements of RMSDaLi are closely linked to the context and times at which the data are collected.4,27

The first requirement is that the data must be collected in a natural, real-world context. This favors ecological validity and permits the study of interrelations between experiences, everyday behaviors, and the setting in which these occur. In clinical settings, for example, experience-sampling techniques and ecological momentary assessment have provided new tools for studying the impact of social contact on the everyday lives of people with depression symptoms28 or anxiety disorders.29 Use of these methods during treatment can also increase understanding of how everyday life changes may promote or hinder clinical improvement.30 In the area of elite sport, observational methodology has provided innovative insights for linking critical behavior during a competition to physiological variations.31–34

The second requirement of RMSDaLi is that events must be recorded as soon as possible after they occur. This is particularly important when self-report methods are used, as the longer the interval between the event and its registration, the greater the risk of retrospective (recall) bias. Ideally, an event or behavior should be recorded in real time.35 Several studies have shown how individuals perceive experiences differently depending on whether they evaluate them contemporaneously or retrospectively. Redelmeier and Kahneman36 showed how patients’ memories of pain during a colonoscopy or lithotripsy (retrospective evaluation) did not match their judgement of pain during the actual procedure (contemporaneous evaluation). Studies of perceived risk have also shown that this is higher during than after an event.22,37

Reis and Gable38 described a series of threats or biases that can affect people’s reconstruction or interpretation of events or experiences, namely, recency, salience, sense-making, and state of mind. According to the recency theory, end-of-day reports on mood are influenced by the most-recent events in the day, because, logically, these are the easiest to recall.39 Salience bias, on the other hand, refers to the fact that events that are particularly intense, emotional, unusual, or significant from a personal perspective are more memorable. Sense-making refers to people’s tendency to reinterpret their lives in the light of more recent events or to confirm implicit theories and beliefs. Finally, state of mind, or retrospections (particularly of affect and attitude) can reflect a person’s mood at the time of the report.37 These biases can be controlled for by having participants report in real time, but this also has limitations, such as registration errors. It is therefore recommendable to use complementary observational techniques to study behaviors such as smoking that can be observed by others (hetero-observation).40

A third requirement of RMSDaLi is that information should be recorded over a period of time, following a strategy known as “intensive repeated measures.”41 Such an approach is crucial for studying within individual processes. Intensive recording across time offers a wealth of opportunities to study microprocesses, eg, interrelationships between cognitive, affective, behavioral, and physiological variables for short intervals of time in daily life.27,42–44 It has been established that behavior is made up of patterns that occur over time,45 and it would therefore seem only logical to use research designs that can help to uncover these patterns by describing observable moment-by-moment changes.46–52 In the clinical setting, for example, observational recording of behavior in everyday contexts, combined with the analysis of temporal patterns, has uncovered relationships between recurrent disruptive behaviors, physiological variables, and environmental factors that are undetectable with other methodological approaches.53

A fourth aspect that influences the validity of RMSDaLi is how the data collection times are selected.22 The minimum requirement is that data should be collected systematically, taking into account: 1) the study objectives, 2) the distribution of the target behaviors over time, 3) susceptibility to retrospection bias, 4) participant burden (eg, time needed, extent to which the protocol interferes with intimacy), and 5) expected adherence to protocol (eg, activation of recording device).

A range of protocols have been proposed to select data-collection times,38,40,54,55 and these can be simplified into three broad schemas: continuous monitoring, time-based monitoring, and event-based monitoring. The first schema, continuous monitoring of the event of interest during the study period, is particularly applicable to portable technology used to record physiological variables or observable experiences (eg, heart rate, movement, or extralinguistic behavior). The second schema is time-based monitoring with fixed or variable schedules. An example of a fixed timing schedule would be a system to measure blood pressure every 4 hours over a week. In a variable timing schedule, the researchers decide on a number of samples that must be obtained over a given period (eg, three samples from a working day). In this case, the moment of sampling is selected randomly or pseudo-randomly. Well-known examples can be found in the studies compiled by Hektner et al,56 that built representative samples, at the individual level, of everyday life experiences, such as happiness. In the third schema, event-based monitoring, recording starts on detection of a clearly distinguishable event. In a study on anxiety, for example, participants would be asked to answer a short questionnaire whenever they have a panic attack. Finally, in the case of observational designs it is necessary to specify the protocols used to select both within-session and between-session recording moments.4,7,10

Classification criteria

We have adapted the taxonomy proposed by Conner and Lehman57 to classify the methodological approaches used to quantify biopsychosocial aspects in everyday contexts, with the addition of observational methodology designs. This original taxonomy is based on two dimensions: measurement variable and participant involvement. The first dimension refers to variables that are to be measured, namely experiences (eg, pain), behaviors (actions that are observable to others, such as eating), physiology (internal workings of the human body, such as blood pressure), or a combination of these. The second dimension reflects the level of participant involvement in the data collection process. Conner and Lehman distinguished between active and passive methods, where active refers to participants providing measurements through self-reporting or other voluntary actions (eg, saliva sample), and passive refers to situations in which measurements are provided via a device that does not require any action by the participants, apart from wearing or carrying the device. We have extended this definition to include intensive measurements with minimally invasive devices, such as those used in observational methodology4–10,58 (Table 2, second row).

Table 2

Classification of research methods and techniques for studying daily life by type of variable and level of participation

	Type of variable57
	Experience	Behavior	Physiology
Maximum participation (Self-observation)	[1.1]Experiences are self-observed by participants (eg, perceived risk)	[1.2]Behaviors are self-observed by participants (eg, smoking)	[1.3]Samples are collected by participants (eg, saliva samples)
Minimum participation (Hetero-observation)	[2.1]*Experiences are inferred through observation (eg, sighing as an indicator of unease)	[2.2]*Behaviors are measured without the participation of the person being observed (eg, physical activity inferred from data from a pedometer, mother–child interactions inferred from a video recording)	[2.3]Samples are collected without the direct participation of the person being observed (eg, heart activity through a Holter monitor)

Note:

Optimal observational methodology profile shown cells [2,1] and [2,2].

Hetero-observation techniques offer several advantages over self-reporting methods: 1) they do not require participants to interrupt an activity to report something, thereby preserving spontaneity; 2) they are suitable for studying forms of everyday behavior that may go unnoticed by the participant or are prone to distortion (eg, sighing); 3) they place a lower burden on the participant; and 4) they are suitable for studying animals or people who cannot provide verbal reports (eg, infants). One limitation of these methods, however, is reactivity (ie, variations in behavior when a person knows he or she is being observed). That said, studies of the EAR, for example, have shown that carriers of this audio device become used to it within hours and that it is less intrusive than methods requiring active participation.19

The features of an ideal observational methodology profile are shown within dotted lines in Table 2. In such a case, the quantification process must be independent of the nature of what is being studied, whether this is a perceivable behavior, a text, an object, or a vocalization. The process has four key components:

Choice of appropriate observational design (of which there are eight);7,10,59

Use of an observation instrument (usually purpose-designed) that is perfectly adapted to the context in order to preserve the ecological validity of the data recorded;60

Use of recording software,61,62 such as HOISAN,63 SOF-CODER,50 and LINCE,64 to produce coded data that are amenable to strict quality controls to demonstrate the reliability of the data;

Analysis of data using appropriate methods, adapted to the design of the study.65,66

Conclusion: toward methodological complementarity for studying daily life

This paper proposes a general framework for integrating different methods and techniques for quantifying biopsychosocial aspects in everyday contexts, and it reviews their shared properties and requirements. We consider that building a bridge between methods is a prerequisite for designing tools that will facilitate the systematic analysis of the knowledge generated.

The methods reviewed in this article can provide empirical evidence to help answer questions that can only be answered by studying life while it is being lived (eg, are people who live in the present happier?).67 Ideally, RMSDaLi should be capable of capturing real-time data over time in a natural, everyday setting. Thanks to advances in information and communication technologies, these methods have gained prominence in recent years, albeit under different names. They are, however, costly to implement, and their use must be justified by the research question being addressed.21 Likewise, their omission from studies of everyday life must also be justified, and this is generally not done.

We should stress that we are not suggesting that RMSDaLi become the gold standard. A retrospective self-report design (eg, how much pain did I feel during a medical procedure) and a simultaneous self-report design (eg, how much pain am I feeling now), for example, will capture different aspects of the same experience and produce different predictors of behavior, as will an approach based on simultaneous self-reporting (how much physical activity am I doing now) versus one based on passive measurement (eg, data collected by a pedometer).68 Similarly, in a given study, it may be equally interesting to measure observable risk-taking by a worker and assess worker perceptions of risk, as both approaches can provide complementary information.69

The bridge that this paper proposes between observational methodology and innovative solutions such as EAR for studying daily life falls within the broader context of methodological complementarity. We strongly advocate the use of complementary methods to address major questions that arise in psychology.7,9,10 We are specifically referring to mixed methods (complementary use of qualitative and quantitative approaches) and multi-methods (complementary use of different methods, such as observational methodology and selective designs).70

The above examples illustrate the need for methodological complementarity and the need to adapt methodology to research questions. If we set out to quantify behavior by blindly opting for a well-established method that is a priori considered to be superior, we risk overlooking the virtues of methods whose goal is to preserve ecological validity, and this, combined with the additional cost of such methods, could lead to the start of their demise.71,72 It would be interesting for future research to explore this idea in greater depth and to determine just how much interest there is in the behavioral sciences in funding and publishing studies of behaviors in natural, everyday settings.