Switching from Telephone to Web-First Mixed-Mode Data Collection: Results from the Transition into Adulthood Supplement to the US Panel Study of Income Dynamics†.

We conducted an experiment to evaluate the effects on fieldwork outcomes and interview mode of switching to a web-first mixed-mode data collection design (self-administered web interview and interviewer-administered telephone interview) from a telephone-only design. We examine whether the mixed-mode option leads to better survey outcomes, based on response rates, fieldwork outcomes, interview quality, and costs. We also examine respondent characteristics associated with completing a web interview rather than a telephone interview. Our mode experiment study was conducted in the 2019 wave of the Transition into Adulthood Supplement (TAS) to the US Panel Study of Income Dynamics (PSID). TAS collects information biennially from approximately 3,000 young adults in PSID families. The shift to a mixed-mode design for TAS was aimed at reducing costs and increasing respondent cooperation. We found that for mixed-mode cases compared to telephone only cases, response rates were higher, interviews were completed faster and with lower effort, the quality of the interview data appeared better, and fieldwork costs were lower. A clear set of respondent characteristics reflecting demographic and socioeconomic characteristics, technology availability and use, time use, and psychological health were associated with completing a web interview rather than a telephone interview.

INTRODUCTION

Long‐running household panel surveys regularly face new challenges as their established data collection methods encounter secular social, technological, demographic, economic and political change. Recent challenges include declining response rates, increasing costs, and pressure for new modes of online data collection, which have led to a variety of adaptations and innovations. In this article, we focus on one specific adaptation—offering panel survey respondents the option to complete their interview on the web—that represents a key response to many of these recent challenges.

A self‐administered web interview offers numerous advantages to respondents compared to an interviewer‐administered interview, including: flexibility for when the interview is conducted, control over when to start and stop the interview, and greater privacy. At the same time, potential benefits to panel surveys from web interviewing include lower fieldwork effort, reduced costs, higher response rates, and maintaining or improving data quality. Especially attractive is the use of mixed‐mode data collection approaches that include both a web option as well as an alternative mode for respondents who are unable or unwilling to complete their interview on the web. As mixed‐mode interviewing has begun to emerge as the next major adaptation for panel surveys, important research questions have yet to be answered. We focus on two. First, does offering the web interview option lead to better survey outcomes based on fieldwork effort and cost? And, second, what respondent characteristics are associated with using the web interview mode compared to an established alternative mode?

Our analysis is based on an experimental mixed‐mode fieldwork design that offered a web interview option to a randomly sampled 80% of respondents in the 2019 wave of the Transition into Adulthood Supplement (TAS‐2019) to the US Panel Study of Income Dynamics (PSID), with the remaining 20% of respondents receiving the telephone mode used in previous waves.

BACKGROUND, THEORY, AND LITERATURE REVIEW

Web‐based data collection methods have been used in social science surveys for more than two decades (see Couper, 2000; De Leeuw, 2005; Dillman, 2000; Olson et al., 2021). The recent experiences have been largely positive for ongoing panel studies that are gradually adopting mixed‐mode (see Couper & McGonagle, 2019), including several panel studies focused on young adults. For example, the US National Survey of Adolescent to Adult Health introduced a mixed‐mode design in 2016 (Harris et al., 2019). Monitoring the Future, a nationally representative study of US high school graduates, experimentally evaluated a gradual rollout of web data collection compared to mailed paper questionnaires in 2014 and again in 2016 on an auxiliary panel sample of young adults aged 19–20 years. Results showed that respondents randomized to the web mode completed the interview at significantly higher rates and lower cost compared to the mail condition at baseline and again two years later (Patrick et al., 2017, 2019); based on these promising findings, the study instituted a steady‐state web‐push protocol beginning in 2018 (Patrick et al., 2020). In the UK, the Understanding Society study and its methodological innovation panel have provided experimental evidence on the gradual adoption of web data collection within an ongoing national longitudinal study of households. A recent Understanding Society study examined fieldwork outcomes during the shift over three waves to a mixed‐mode design (involving web and face‐to‐face data collection), and found no mode differences in attrition or sample composition while realizing cost savings (Bianchi et al., 2017; Carpenter & Burton, 2017).

Although the use of mixed‐modes in longitudinal studies is expanding, the methodological literature remains extremely limited, with the majority of experimental evidence based on cross‐sectional studies (see De Leeuw, 2018; Jäckle et al., 2017; Millar & Dillman, 2011; Olson et al., 2012; Smyth et al., 2010). Household panel studies, in particular, have adopted mixed‐mode designs cautiously, following extensive periods of development and testing due to concerns about the uncertain acceptance by panel respondents of new data collection methods and the need to maintain measurement consistency, achieve high data quality, and minimize panel attrition. To our knowledge, the only major household panel study that has experimentally tested the full effects of a transition from interviewer‐administration to mixed‐mode collection is the UK Understanding Society study (see Jäckle et al., 2015). Consequently, the evidence base for design and implementation of mixed‐mode survey administration in longitudinal studies across different settings remains underdeveloped (Couper & McGonagle, 2019).

Two reviews of recent evidence on outcomes of mixed‐mode designs in longitudinal studies in the US (Couper & McGonagle, 2019) and the UK (Jäckle et al., 2017) have identified several knowledge gaps and priority areas for future research. One priority is to study how the use of different modes affect primary data collection outcomes, such as unit nonresponse and fieldwork effort (e.g., interviewer contact attempts and days in the field), and also how different designs, such as assigning respondents to a preferred mode, affect these outcomes. Another priority is to examine the impact of different modes on data quality indicators such as item nonresponse rates. A final identified gap concerns the relative costs of mixing modes compared to single mode data collection, for which almost no evidence exists (Couper & McGonagle, 2019), and how different design choices affect costs. Thus, there remains a pressing need to identify the factors that lead to greater effectiveness and efficiency in the implementation of mixed‐mode data collection in longitudinal social surveys.

The introduction of online data collection in TAS‐2019 provides valuable opportunity to study the shift from a traditional interviewer‐administered study to a fully mixed‐mode design. Using an experimental design embedded in this ongoing, nationally representative panel study of US young adults, we address two research questions related to the implementation of a mixed‐mode design. Our first research question examines whether a mixed‐mode design that leads with an online data collection option results in better survey fieldwork outcomes and lower costs compared to a telephone only mode. The experimental design used random assignment of respondents to either a mixed‐mode or a telephone only mode, which allows us to make unbiased and generalizable comparisons between the two groups. We consider effects on response rates, the number of contact attempts, duration to interview completion, and item nonresponse rates as a measure of data quality. For TAS‐2019 panel respondents who were eligible for TAS in 2017, when the survey was telephone only, we compare fieldwork outcomes across the two waves; this comparison allows us to undertake a difference‐in‐difference analysis that assesses the effects on fieldwork outcomes of a mode switch.

Our second research question considers whether completing the interview using the web mode, rather than the alternative telephone mode, is associated with demographic, socioeconomic, and psychological characteristics of respondents, the availability of internet resources, and mode convenience. This question is key to determining the uptake patterns of mixed‐mode interviewing, which will guide future fieldwork protocols and enhance our understanding of potential mode effects. We draw on social psychological theories underlying survey participation to develop hypotheses about how respondent characteristics are related to interview mode. Leverage‐saliency theory suggests that survey participation may be enhanced by increasing the salience of certain survey design attributes, such as offering an accessible interview mode (Groves et al., 2000). Based on this theory, we hypothesize that having access to the internet and a personal computer or other online device, reporting a preference for the web mode, and previously using the web to complete a survey are each factors that will predict completion of the interview by web. Social exchange theory applied to survey design (e.g., Dillman et al., 2009) postulates that respondents are more likely to participate if the perceived rewards to participation—such as the token incentive and contributing to science—outweigh the costs, including the time, effort, and opportunity costs of participation. This theory suggests that individuals with high time demands, such as due to work, school or volunteering, will be more likely to complete the interview on the web, because this mode supports completion at any time of day, and does not require interaction with an interviewer or telephone calls, emails, and text messages to schedule an interview appointment. Finally, we draw on self‐presentation theory (Baumeister & Hutton, 1987) to hypothesize that high anxiety and low self‐esteem will be positively related to web interview completion. This theory suggests that evaluative interactions with others, such as during an interview with a stranger who is asking sensitive and personal information, may activate self‐presentational and socially desirable behaviors (Tourangeau & Smith, 1996). Individuals who are anxious or have low self‐esteem may be motivated to avoid evaluative situations and to take advantage of the privacy offered by completing the interview on the web.

METHODS

TAS is a national panel survey that provides data biennially about the experiences of young adults in the US—including employment and schooling transitions, family and peer relationships, and family formation. TAS participants are aged 18–28 years and are members of families that participate in PSID—a longitudinal household panel study that since 1968 has collected data on economic, social, and health behavior from a nationally representative sample of US families (see McGonagle et al., 2012). PSID follows the original panel members and enrols their adult children when they form their own economically independent families.

Eight waves of TAS have been collected since its launch in 2005, with response rates of 87% to 92% (based on definition RR6, AAPOR, 2016; Institute for Social Research, 2019; McGonagle & Sastry, 2015). A professional staff of interviewers employed by the Survey Research Center (SRC) at the University of Michigan has conducted PSID and TAS since the inception of each study. Computer‐assisted telephone interviewing was the primary mode of data collection for TAS through 2017.

In 2019, TAS made a shift to a mixed‐mode design following an extensive development process and drawing on new information about best practices for the successful design and implementation of mixed‐mode methods in longitudinal studies. Work towards the transition began in 2017 with significant revisions to the TAS questionnaire to make it more amenable for online administration. A major mixed‐mode pilot study in 2018 tested web and telephone questionnaires programmed in Blaise software, fieldwork protocols and procedures, and respondent engagement strategies and materials. A single version of the web questionnaire was programmed to adapt automatically to the type of device and display. TAS‐2019 goals were to maximize the number of cases completing their interview online for cost savings, while collecting high quality data and increasing respondent cooperation over the long term based on their preferences, elicited in TAS‐2017, for completing future interviews by web.

Data

Assignment to mode

The eligible sample of TAS‐2019 comprised 2,964 young adults. To investigate and document the effects of mode assignment on fieldwork outcomes and interview responses, we randomly assigned 80% of the TAS‐2019 sample, a total of 2,404 cases, to a sequential mixed‐mode data collection protocol (see De Leeuw, 2005; Dillman et al., 2009) that offered web first with telephone follow‐up. The mixed‐mode sample included 203 respondents who participated in a 2018 mixed‐mode pilot study. Respondents in the mixed‐mode treatment group were encouraged to complete the interview by web but eventually had an option to complete the interview with an interviewer by telephone. The remaining 20% of the sample, a total of 560 cases, was randomized to the telephone only mode with no option for completing the survey online. These telephone only control cases were worked concurrently with the mixed‐mode cases, allowing us to compare fieldwork effort and duration between the mixed‐mode treatment and telephone only control groups.

Sample release

TAS‐2019 was fielded over a 40‐week period between November 2019 and August 2020. By design, the field periods of TAS and PSID overlap, with interviewers contacting TAS study participants once their family completed the PSID interview. During TAS‐2019, the vast majority (87%) of eligible cases were released at the study launch in November 2019, with a small number of subsequent batches released through July 2020.

Data collection protocols

TAS‐2019 implemented an intensive respondent engagement strategy using telephone calls, email and text messaging, and postal mailings. Prior to study launch, advance notice letters were sent to both groups via postal mail describing the upcoming study and providing general information about participation, including the baseline incentive of $70 that would be paid upon completion of the interview. The letter for the mixed‐mode sample provided the web address of the online survey portal and unique login and password credentials. To maximize the number of respondents completing the web interview, no mention was made of the eventual possibility of conducting the interview by telephone with an interviewer. Telephone only respondents were told in the advance notice letter that an interviewer would be calling to conduct the interview by telephone. The same information was also conveyed to both groups through an email and text message sent at study launch. SRC interviewers then began contacting the telephone only sample to schedule and conducted interviews.

During the first eight weeks of fieldwork, email and text messages were sent regularly (i.e. at least once a week) to encourage all respondents to complete their interview. Mixed‐mode respondents were provided an authenticated link (with an embedded user name and password) to the web portal to complete their online interview. Telephone only respondents were encouraged to answer inbound calls from the interviewers or to call their interviewer directly to schedule an interview appointment. Two special mailings were sent to remind respondents of the survey request, including an email and text message sent during Thanksgiving week, and an end‐of‐the‐year holiday postcard mailing.

Mixed‐mode respondents who had not completed an interview after approximately eight weeks began receiving non‐response follow‐up telephone calls by interviewers encouraging them to complete the web survey and also offering to complete the interview by telephone. Email and text message reminders continued to be sent several times a week for the remaining field period to all respondents. After 24 weeks, respondents who were known to require greater project resources and effort to complete the survey (based on number of unproductive contact attempts) were offered an additional $30 for completing the interview. This increased incentive group comprised those who were new to the study and those who had not responded in the prior wave. With approximately one month remaining in the field period, a final postal mailing and email and text messages were sent to all remaining respondents describing an increased study incentive of $100. Final email and text messages were sent approximately two weeks before the end of data collection notifying respondents of the study's exact end date and encouraging their participation. At the end of the field period, 2,605 interviews were completed at an overall response rate of 87.9% (based on definition RR6, AAPOR, 2016).

Variables

Outcome measures

We compare six fieldwork outcomes for cases assigned to the mixed‐mode treatment group and the telephone only control group: (i) response rates, calculated as the percentage of all eligible cases completing an interview (based on definition RR6, AAPOR, 2016); (ii–iv) the number of interviewer contact attempts among all eligible cases required to finalize the case, with separate measures for telephone calls, email and text messages, and total contact attempts, constructed as the sum of telephone calls and email and text messages; (v) fieldwork duration, defined for all eligible cases as the number of days from the date of release to the date when a final fieldwork disposition status was assigned; and (vi) item nonresponse rate, calculated as the percentage of interview items left blank or skipped, among those who completed an interview, whether asked by an interviewer or visited in the web instrument; and (vii) fieldwork costs, calculated as the sum of costs across all cases, whether completed or not, for interview contact attempts, telephone administration of the interview, respondent incentives, and total costs. Note that non‐response cases are included in the calculation of response rates, interview contact attempts, fieldwork duration, and fieldwork costs, but are excluded from the analysis of the item nonresponse rate because this outcome requires a completed interview. For completed cases in the mixed‐mode group, we examine cases by their interview mode—that is, we compare web and telephone completed cases.

Respondent characteristics by treatment assignment

We examine differences between the mixed‐mode treatment group and telephone only control group to verify randomization of experimental assignment. We consider the following respondent characteristics: (i) sex, (ii) age, (iii) whether or not the sample member has ever completed an interview in a prior wave of TAS; (iv) residential independence, distinguishing among those living independently, living with their parents, or living away from home in an institutional setting; (v) an indicator of participation as a respondent for the Core PSID interview in 2019; (vi) Core PSID sample source, distinguishing among respondents originating from the SRC national probability sample, the low‐income oversample, or one of the two immigrant refresher samples.

Predictors of interview mode

We examine predictors of interview mode among TAS‐2019 mixed‐mode respondents using prior wave public use data from TAS‐2017 available through the PSID Online Data Center (http://psidonline.org). Based on theoretical considerations described above, we include predictors in the following domains: demographic and socioeconomic characteristics, technology availability and use, time use, and psychological health.

Demographic and socioeconomic variables include: (i) sex, (ii) self‐reported race/ethnic identity that distinguished between non‐Hispanic whites and all other race/ethnic groups, (iii) whether family income in 2017 was above or below the sample median, (iv) whether education attainment included at least some period of attending college or university.

Technology availability and use, variables include: (i) completing a web interview during the TAS‐2018 mixed‐mode pilot study, (ii) self‐reported interview mode preference, distinguishing a preference for a web interview versus a telephone interview or no preference, (iii) having high‐speed internet at home, (iv) having a desktop or laptop computer at home, and (iv) owning a cellphone or tablet computer.

Time use includes separate variables for the number of hours spent in the past week engaged in educational activities, volunteering, and employment.

Psychological health includes: (i) an indicator of being at the 90th percentile or higher in the distribution of the K6 six‐item screening scale for non‐specific psychological distress related to symptoms of depression and anxiety (Kessler et al., 2002), and (ii) a continuous summary score of global self‐worth as assessed by the 10‐item Rosenberg Self‐Esteem Scale (Rosenberg, 1965), a scale that assesses both positive and negative feelings about the self (e.g., ‘I have a number of good qualities’, ‘I do not have much to be proud of’) using a four‐point Likert scale ranging from strongly agree to strongly disagree.

Analysis methods and samples

Our analysis strategy begins by examining the balance between the mixed‐mode treatment group and the telephone only control group. To do so, we compare descriptive statistics between the two groups, using t‐tests for comparisons of continuous variables and chi‐squared tests for comparisons of discrete variables. We also consider the joint effects of these variables on treatment assignment by estimating a logistic regression model of whether a case was assigned to the mixed‐mode treatment group (y = 1) or the telephone only control group (y = 0).

To answer our first research question, we evaluate differences in fieldwork outcomes and costs between the mixed‐mode treatment group and telephone only control group using descriptive statistics, Kaplan–Meier estimates of the cumulative distribution function of time to a completed interview, and the estimated hazard curve for completed interviews. We conduct this analysis using an intent‐to‐treat approach, in which we compare outcomes based on treatment assignment (i.e. web‐first mixed‐mode vs. telephone only mode) rather than the actual mode used to complete the survey. This approach is appropriate because some cases assigned to the mixed‐mode treatment complete the survey by telephone, which is the mode assigned to the control group. Furthermore, some cases in both groups do not complete an interview, and the intent‐to‐treat approach allocates the associated fieldwork effort to their assigned group. By incorporating all treatment and control cases into our analysis, the intent‐to‐treat approach allows us to capture all fieldwork outcomes and provides an unbiased estimate of the treatment effect. Finally, our intent‐to‐treat results offer conservative estimates of fieldwork outcomes and cost savings of mixed‐mode interviewing because outcomes and costs for non‐compliers are included.

Our analysis of response rates and costs by treatment assignment is based on all TAS‐2019 cases, as is our analysis of total contact attempts, telephone contact attempts, and email or text message contact attempts, and of the number of fieldwork days. We compare mode of interview just for TAS‐2019 cases that completed an interview.

The most rigorous assessment of our first research question uses a difference‐in‐differences approach. We focus on fieldwork outcomes for cases that were eligible for both the 2017 and 2019 waves of TAS, regardless of their completion status, and we compare various measures of fieldwork effort between waves and, subsequently, between the treatment and control groups. All respondents in TAS‐2017 were interviewed by telephone; however, in TAS‐2019 the treatment group switched to a mixed‐mode assignment while the control group remained telephone only. This analysis provides a within‐subject control for fieldwork outcomes in the previous wave, thereby providing a clearer and potentially more convincing estimate of the effect on fieldwork outcomes of the mixed‐mode treatment compared to the telephone only control. This analysis is restricted to the subsample of TAS‐2019 cases that were also eligible for the 2017 wave of TAS. The sample is also age‐restricted among all TAS cases, because the oldest TAS‐2017 sample members (aged 27–28 years in 2017) were no longer age‐eligible for TAS in 2019, while the youngest TAS‐2019 sample members (aged 18–19 years in 2019) were too young to be eligible for TAS in 2017.

To answer our second research question, which focuses on the online versus telephone interview mode among cases assigned to the mixed‐mode treatment, we use logistic regression. The dependent variable is a binary indicator of whether the interview was completed by web (y = 1) or by telephone (y = 0). We estimate four logistic regression models that sequentially add variables describing demographic and socioeconomic characteristics, technology availability and use, time use, and psychological health. We report odds ratios along with robust standard errors that account for clustering of cases by family, and provide indicators of statistical significance for each parameter estimate. We use the same logistic regression modeling approach to examine the difference between respondents completing an online interview in the mixed‐mode treatment group and respondents completing a telephone interview in the control group. For both of these analyses, our sample is restricted to respondents who completed an interview in both 2017 and 2019. We focus our attention on the TAS sample from 2017 who were reinterviewed in 2019 because these cases have a rich set of covariates from the completed TAS‐2017 interview data.

RESULTS

We begin with results from analysing the balance between cases assigned to the mixed‐mode treatment group and telephone only control group (4.1). We next present findings related to our first research question, which is whether the online interview option leads to better survey outcomes, based on response rates (4.2), fieldwork outcomes (4.3 and 4.4), interview quality (4.5), and costs (4.6). We then present results for our second research question, which concerns respondent characteristics associated with completing a web interview versus a telephone interview (4.7).

Assignment to mixed‐mode treatment versus telephone only control

The mixed‐mode treatment and telephone only control groups in TAS‐2019 were well balanced. We compared the mixed‐mode and telephone only samples by sex, age, and several sample characteristics—being a new member of the TAS sample in 2019, residential independence (distinguishing between those in an independent household, those still residing in their parents' household, or being in an educational institution), being a respondent in Core PSID in the same wave, and sample origin (distinguishing between those from the original SRC sample, the low‐income sample, or an immigrant refresher sample). We found no differences for any of these characteristics between the two groups. The logistic regression model (results not shown) yielded an insignificant joint test for all model covariates of χ2(7) = 4.4 (p = 0.8) and no individually significant coefficients. In summary, assignment to the treatment and control groups was randomized appropriately and well balanced.

Response rates for mixed‐mode versus telephone only cases

The overall response rate for TAS‐2019 across both mode assignments was 87.9% (see Table 1). Cases assigned to mixed‐mode had a response rate of 89.3% while those assigned to telephone only had a response rate 7.5 percentage points lower (81.8%). Among TAS‐2019 cases assigned to mixed‐mode who completed an interview, 87.9% did their interview online while 12.1% completed their interview by telephone. Mode assignment was not conditioned on respondents' ability to complete the interview online by, for instance, determining if they had a device capable of connecting to the internet or a high‐speed internet connection. Hence, interview mode likely reflects respondents' capability to complete an interview online, mode preference, and eventual interaction with the interviewers—who contacted respondents and encouraged them to complete the interview by telephone. Finally, note that cases were assigned to only one or the other completion mode categories; although it was possible for a respondent to begin an interview online and then complete it by telephone, in fact this eventuality was not observed.

TABLE 1

TAS‐2019 response rates and completion mode, overall and by mode assignment: mixed‐mode versus telephone only

	Overall (n = 2,964)		Mixed‐mode (n = 2,404)		Telephone‐only (n = 560)
	N	%	N	%	N	%
Completed interview	2,605	87.9%	2,147	89.3%	458	81.8%
Completion mode
Telephone	717	27.5%	259	12.1%	458	100.0%
Web	1,888	72.5%	1,888	87.9%	0	0.0%

Fieldwork outcomes for mixed‐mode versus telephone only cases

Cases randomly assigned to mixed‐mode on average required substantially fewer contact attempts and completed their interviews much faster than telephone only cases. We show these results in Table 2. The mean number of total contact attempts was 8.6 fewer (p < 0.0001) for mixed‐mode cases (28.3 attempts) compared to telephone‐only cases (36.9 attempts); the gap in the median number of total interviewer attempts was of a similar magnitude (8.0 for mixed‐mode versus 17.0 for telephone only; p < 0.0001). Although interviewers contacted respondents by telephone or using email or text messaging, the difference in total contact attempts emerged entirely from a difference in telephone calls. For telephone only cases, the mean and median number of telephone calls (22.4 and 10.0 respectively) was substantially higher (p < 0.0001) than for mixed‐mode cases (10.9 and 0.0 respectively). The mean number of email or text messages was higher for mixed‐mode than for telephone only cases (17.4 vs. 14.4; p = 0.0045), although the median number of email or text messages was similar for both groups (7.0 and 8.0, respectively for mixed‐mode and telephone only cases; p = 0.15).

TABLE 2

TAS‐2019 fieldwork outcomes by mode assignment: mixed‐mode versus telephone only

Fieldwork outcome	Mixed‐mode (n = 2,404)	Telephone only (n = 560)	p‐value for test of equality
Number of total interviewer attempts
Mean	28.3	36.9	p < 0.0001
Median	8.0	17.0	p < 0.0001
Number of telephone attempts
Mean	10.9	22.4	p < 0.0001
Median	0.0	10.0	p < 0.0001
Number of email/text attempts
Mean	17.4	14.4	p = 0.0045
Median	7.0	8.0	p = 0.15
Number of fieldwork days
Mean	81.2	113.4	p < 0.0001
Median	31.0	68.5	p < 0.0001

We summarize the time‐to‐complete information in Figure 1, which shows the Kaplan‐Meier estimate of the cumulative distribution function for the duration to a completed interview by treatment condition. Mixed‐mode cases were completed considerably faster than those assigned to telephone only. The Wilcoxon test statistic for the equality of the cumulative distribution functions is χ2(1) = 65.73 (p < 0.0001). The median duration to completing the TAS‐2019 interview was 31.0 days for mixed‐mode and 68.5 days for telephone only cases (incomplete cases were censored at Day 282, which corresponds to the end of fieldwork).

FIGURE 1

Kaplan Meier estimate of the cumulative distribution function of duration to TAS‐2019 completed interviews: mixed‐mode versus telephone only cases

The estimated hazard function for completed interviews, by mode assignment, is shown in Figures 2. The hazard for completing an interview is extremely high for the mixed‐mode treatment at the beginning of fieldwork. All of these cases complete their interviews by web, because the telephone follow‐up option did not begin until approximately eight weeks after cases were released to the field. By the tenth day of the fieldwork period, one‐third of the mixed‐mode cases had completed their interviews online—a remarkably quick pace. The hazard for the telephone‐only cases peaked at around the same time (Day 10), after which the hazard of completing the interview was similar for both groups through the end of the fieldwork period.

FIGURE 2

Hazard function for TAS‐2019 completed interviews: mixed‐mode versus telephone only cases

Note: Kernel smoothing of estimated hazard contributions, specified as biweight with bandwidth of 10.

Differences in fieldwork outcomes by mixed‐mode versus telephone only assignment and year of interview

The results of our analysis of fieldwork outcomes for cases that were eligible for both the 2017 and 2019 waves of TAS are presented in Table 3. All TAS interviews in 2017 were completed by telephone and in 2019 were randomly assigned to either mixed‐mode or telephone only mode.

TABLE 3

Differences in fieldwork outcomes by year and by mode assignment and year among TAS cases eligible for both 2017 and 2019

	TAS‐2019 mode assignment
	Mixed‐mode (n = 1,729)			Telephone only (n = 392)
Fieldwork outcomes	2017   (Tel.)	2019     (Mixed)	Difference	2017   (Tel.)	2019   (Tel.)	Difference	Difference‐in‐ differences	p‐value
Number of total interviewer attempts
Mean	20.4	28.0	7.6	18.7	33.7	15.0	7.4	p = 0.0013
Median	10.0	8.0	−2.0	8.5	16.0	7.5
Number of email/text attempts
Mean	6.9	18.1	11.2	6.2	13.8	7.6	−3.6	p = 0.0054
Median	3.0	7.0	4.0	2.0	8.0	6.0
Number of telephone attempts
Mean	13.5	9.9	−3.6	12.5	19.9	7.4	11.1	p < 0.0001
Median	7.0	0.0	−7.0	6.0	8.0	2.0
Number of fieldwork days
Mean	79.0	70.8	−8.2	72.7	94.6	21.9	30.1	p < 0.0001
Median	51.0	24.0	−27.0	46.0	42.5	−3.5

Note: All cases in TAS‐2017 were interviewed by telephone.

The mixed‐mode and telephone only groups both had a similar mean number of total contact attempts in 2017 (20.4 for mixed‐mode and 18.7 for telephone only). While both groups had higher mean total interviewer contact attempts in 2019, the increase was substantially smaller for the mixed‐mode group. In 2019, the mean contact attempts for telephone only cases jumped by 15.0 attempts, while the mean contact attempts for mixed‐mode cases increased by just 7.6 attempts. The change from 2017 to 2019 in contact attempts was 7.4 attempts greater for telephone only cases than for mixed‐mode cases (p = 0.0013). Thus, the online option for mixed‐mode cases substantially dampened an otherwise large increase in contact attempts that was seen for the telephone only cases. The increase in mean total contact attempts for the telephone only sample was remarkably large, almost doubling from 18.7 to 33.7 attempts; even with this increase, the response rate for telephone only cases of 81.8% was considerably below that for the mixed‐mode cases of 89.3% and, indeed, the TAS‐2017 response rate of 87.2%.

Table 3 shows that the increase between 2017 and 2019 in contact messages by either email or text was larger for mixed‐mode cases than for telephone only cases. The number of contact messages for mixed‐mode cases was almost three times higher (18.1 vs. 6.9) but for telephone only cases was only about twice as high (13.8 vs. 6.2). The difference‐in‐difference for mean email or text contact attempts was –3.6 attempts (p = 0.0054), indicating a smaller increase for telephone only cases than mixed‐mode cases in 2019 compared to 2017. The difference‐in‐difference for total contact attempts was thus manifested entirely in telephone call attempts, for which there was an increase from 12.5 to 20.0 calls for the telephone only sample between 2017 and 2019 and a decrease from 13.5 to 9.9 calls for the mixed‐mode sample, for a net difference‐in‐difference of 11.1 telephone contact attempts (p < 0.0001).

Finally, the results for fieldwork duration show that the median number of days to complete the TAS interview for telephone only cases declined modestly between 2017 and 2019 by 3.5 days (from 46.0 to 42.5 days). In contrast, the median number of days to completion for the mixed‐mode cases fell by half—from 51.0 days in 2017 to 24.0 days in 2019. Mixed‐mode cases declined by 23.5 more days between 2017 and 2019 than the telephone only cases, which represents a large and dramatic relative change.

Interview quality for mixed‐mode versus telephone only interview cases

We examine a single main measure of interview quality among completed interview cases, which is the item nonresponse percentage—that is, the percentage of questions explicitly declined or answered with a ‘don’t know' in a telephone interview or skipped by clicking ‘next’ in a web interview. Although telephone interviews require an explicit response to skip a question, we found a significantly higher rate of skipped questions among the 458 telephone only cases (3.42%) compared to the 2,147 mixed‐mode cases (3.01%), with t(2,589) = 2.8 and p < 0.01 for the difference. Among the mixed‐mode cases, skipped questions were significantly higher (3.78%) among the 259 cases interviewed by telephone compared to the 1,888 cases interviewed online (2.91%), with t(2,134) = 4.4 and p < 0.0001 for the difference. There were no differences in the skipped question rate among online interviews completed by cellphone (2.96%; n = 1,235) compared to those completed by computer (2.82%; n = 646), with t(1,877) = 0.9 and p = 0.4 for the difference.

Cost comparison of mixed‐mode and telephone only interviewing

Table 4 presents results of a fieldwork cost comparison between the mixed‐mode and telephone only groups. We report the mean cost per case of interviewer contact attempts, telephone administration of the interview, and respondent incentives, as well as total costs per case and total overall costs. We focus exclusively on the variable fieldwork costs of conducting interviews and exclude fixed costs that are the same by mode, such as the cost of developing the questionnaire instrument, or one‐time costs associated with transition to mixed‐mode interviewing, such as new software and technical systems for web interviewing.

TABLE 4

Variable cost comparison of mixed‐mode and telephone only interviewing based on TAS‐2019 experience

	Mixed‐mode	Telephone only
Cases
Total	2,404	560
Completed	2,147	458
Response rate	89.3%	81.8%
Interviewer contact attempts
Mean contact attempts (non‐response cases)	82.0	86.9
Mean contact attempts (completed cases)	21.8	25.7
Mean total cost (across all cases) at $6.56/attempt a	$185	$242
Interviewer survey administration by telephone
Completed interviews administered by telephone	12.1%	100.0%
Mean telephone interview length (minutes)	95	84
Mean total cost (across all cases) at $26.24/hour b	$4	$30
Respondent incentives
Mean total cost (across all cases)	$66	$62
Grand total mean variable cost‐per‐case
All cases c	$256	$333
Non‐response cases d	$538	$570
Completed cases e	$222	$281
Total variable survey costs f	$758,303	$988,053
Mixed‐mode vs. telephone difference ($)	−$229,750
Mixed‐mode vs. telephone difference (%)	−23%

Note. Mean total cost for contact attempts is equal to mean contact attempts by response status (non‐response vs. completed) weighted by the fraction of cases in each group multiplied by the cost per contact. For mixed‐mode, mean total cost is ((21.8 × 89.3%) + (82.0 × (100% − 89.3%))) × $6.56 = $185; for telephone, mean total cost is ((25.7 × 81.8%) + (86.9 × (100% − 81.8%))) × $6.56 = $242.

Mean total cost for interview administration is equal to the average telephone interview length multiplied by the fraction of completed cases interviewed by telephone multiplied by the hourly interview rate. For mixed‐mode, mean total cost is 95/60 × 10.8% × 89.3% × $26.24 = $4; for telephone, mean total cost is 84/60 × 100% × 81.8% × $26.24 = $30.

Grand total cost is the sum of costs for contact attempts, interview administration, and respondent incentives. For mixed‐mode, total cost is $185 + $4 + $66 = $256 (does not sum to total due to rounding); for telephone, total cost is $242 + $30 + $62 = $333 (does not sum to total due to rounding).

Grand total cost for non‐completed cases is based on the cost of interview contact attempts. For mixed‐mode, cost per case is 82.0 × $6.56 = $538; for telephone, cost per case is 86.9 × $6.56 = $570.

Grand total cost for completed cases is the sum of costs for contact attempts for completed cases plus the sum of interview administration and respondent incentives divided by the response rate. For mixed‐mode, cost per case is 21.8 × $6.56 + ($4 + $66)/89.3% = $222; for telephone, cost per case is 25.7 × $6.56 + ($30 + $62)/81.8% = $281.

Total variable survey costs are equal to the total number of cases (2,404 + 560 = 2,964) multiplied by the grand total mean cost‐per‐case for each mode.

The mean total cost for interviewer contact attempts is calculated as the weighted average of contact attempts, across non‐response and completed cases, multiplied by a mean cost per contact attempt of $6.56. The cost per contact attempt is derived from actual interviewer hourly costs (wages plus fringe benefits) and the estimated productivity of four contact attempts per hour, which is independent of the contact type (telephone, email or text message) because each requires approximately the same amount of interviewer time. This comprises time spent to dial and leave a voice‐mail message or type, review, and send a text message or email message, and time to review summaries of the case history (e.g., descriptive respondent information and prior contact information, including the number, dates, and outcomes of prior contact attempts) and update case history notes for each new attempt. Mean total cost for interview contact attempts is $185 for mixed‐mode cases and $242 for telephone only cases. Approximately half of the $56 difference in cost by mode is due to the 7.5 percentage point higher non‐response rate among telephone only cases (non‐response cases required many more contact attempts). The remainder is due to the greater number of contact attempts required by telephone only mode compared to mixed‐mode among both non‐response cases (86.9 vs. 82.0) and completed cases (25.7 vs. 21.8).

The second cost component is interviewer time spent conducting the interview, which for each mode is based on the fraction of cases completing a telephone interview (12.1% for mixed‐mode and 100% for telephone only), the average interview length (95 min for mixed‐mode and 84 min for telephone only), and hourly interviewer costs of $26.24. Mean total costs for interviewer telephone administration is $4 for mixed‐mode and $30 for telephone only mode. The final cost component is respondent incentives for which the mean total cost is $66 for mixed‐mode cases and $62 for telephone only cases.

The grand total mean variable cost per case is $256 for mixed‐mode and $333 for telephone only. The $78 higher mean cost for the telephone mode is due to differences in mean total costs for interview contact attempts (+$56), interviewer survey administration by telephone (+$26), and respondent incentives (–$5). The mean total cost per case are shown separately in Table 4 for non‐response and completed cases by mode. The costs for each type of case are higher for telephone only mode than for mixed‐mode: modestly higher for non‐response cases (+6.0%) but substantially higher for completed cases (+26.7%). For both modes, average total cost per case is approximately twice as high for non‐response cases as for completed cases.

The final panel in Table 4 shows the hypothetical total variable fieldwork costs of conducting the survey exclusively using mixed‐mode ($758,303) or telephone only ($988,053). The savings from using mixed‐mode, compared to telephone only, would amount to $229,750 or 23%.

Interview mode among mixed‐mode cases

Our next analysis examines interview mode among the mixed‐mode cases. TAS‐2019 cases randomly assigned to mixed‐mode (80% of all cases) were strongly encouraged to complete an interview online, and for the first eight weeks of fieldwork were only offered this option. After Week 8, all cases that had not completed an interview were offered the option of completing the interview by telephone. We analyse characteristics associated with completing the interview by web rather than by telephone over the entire fieldwork period. We note that the web completion group includes respondents who completed their interview early, even before the telephone option was available, which may have reflected a preference for the web mode or, potentially, for completing the interview as quickly as possible. We purposely did not restrict our analysis of interview mode to those who completed an interview later in fieldwork when both modes were offered (i.e. after Week 8) because such an analysis would be based on a select subset of the web completes. Our goal was to learn about characteristics associated with web completion over the full fieldwork period for the entire analysis sample. Among the 1,464 cases randomized to mixed‐mode who completed TAS interviews in both 2017 and 2019, a total of 1,319 respondents completed interviews by web (90.1%) and 145 by telephone (9.9%).

We examine the effects of four sets of covariates drawn from TAS‐2017 survey data in our model of interview mode. First, demographic and socioeconomic characteristics, comprising race, sex, family income, and education; second, technology‐related characteristics, comprising mode preference for an online interview, participation in the TAS‐2018 web pilot, having high‐speed internet at home, having a computer at home, and owning a cellphone or tablet computer; third, time demands, measured by time spent at work, school, and volunteering; and fourth, psychological health, measured by psychological distress and self‐esteem. Descriptive statistics for technology‐related characteristics reveal that 85.4% of respondents had high‐speed internet at home, almost as many (78.1%) had a computer at home, and ownership of a cellphone or tablet computer was nearly universal (98.4%).

The interview mode logistic regression results are presented in Table 5. Model 1 shows that an online interview is more common among respondents who are of white race (an odds ratio of 1.94; p < 0.001), female (1.93; p < 0.001), higher income (1.61; p < 0.05), and with more education (1.89; p < 0.001). Adding indicators of technology‐related characteristics in Model 2 attenuates the parameter estimates for the demographic and socioeconomic characteristics, although all remain statistically significant except for family income. The technology characteristics reveal that respondents are more likely to complete the interview online if they have a stated mode preference for web or a high‐speed internet connection at home. Neither variable indicating ownership or access to technology at home is associated with web interview mode, nor is having prior experience completing a web interview. Model 3 adds variables describing respondents' time demands. The effect of greater time demands for volunteering activities is to reduce the likelihood of an online interview, while greater work time demands are associated with a higher likelihood of completing the interview online. This finding is likely related to the sizable incentive offered to respondents for completing the interview, which represents a greater monetary reward compared to earnings from working for an equivalent amount of time. The socioeconomic variables—income and education—are attenuated and no longer statistically significant in this model. The last model adds psychological health to Model 3. There are no notable changes to estimated parameters or standard errors for retained covariates. The main finding is that respondents with higher levels of self‐esteem are more likely to complete an interview online rather than by telephone.

TABLE 5

Estimated odds ratios from logistic regression models for TAS‐2019 interview mode (web vs. telephone) among respondents assigned to mixed‐mode

	Model
Variables	(1)	(2)	(3)	(4)
Race: white (vs. nonwhite)	1.939***	1.782**	1.805**	1.997**
	(0.390)	(0.365)	(0.377)	(0.422)
Sex: female (vs. male)	1.930***	1.803**	1.795**	1.830**
	(0.360)	(0.338)	(0.337)	(0.347)
Family income above median (vs. below)	1.609*	1.343	1.353	1.349
	(0.309)	(0.260)	(0.272)	(0.270)
Education: some college (vs. no college)	1.890***	1.546*	1.438	1.343
	(0.344)	(0.300)	(0.276)	(0.259)
Mode preference: web (vs. telephone)		1.560*	1.566*	1.565*
		(0.329)	(0.333)	(0.336)
TAS−18 web pilot participant		1.982	2.011	2.012
		(0.696)	(0.721)	(0.726)
High‐speed internet at home		1.804**	1.827**	1.733*
		(0.386)	(0.399)	(0.378)
Computer at home		1.448	1.569*	1.590*
		(0.317)	(0.353)	(0.354)
Own cellphone or tablet		1.254	1.245	1.283
		(0.656)	(0.675)	(0.663)
Education time (weekly hours)			0.995	0.994
			(0.008)	(0.008)
Volunteering time (weekly hours)			0.965*	0.964*
			(0.014)	(0.015)
Work time (weekly hours)			1.012*	1.010*
			(0.005)	(0.005)
Psychological distress				0.792
				(0.209)
Rosenberg self‐esteem scale				1.832**
				(0.405)
Constant	3.135***	1.225	0.996	0.155*
	(0.501)	(0.672)	(0.560)	(0.142)
Wald χ2 (df)	48.61*** (4)	71.63*** (9)	85.52*** (12)	93.23*** (14)
Observations	1,464	1,464	1,464	1,464

Note: Dependent variable y = 1 if interview mode is online (n = 1,319) and y = 0 if interview mode is telephone (n = 145); robust standard errors in parentheses; ***p < 0.001, **p < 0.01, *p < 05; analysis sample is restricted to TAS‐2019 respondents who were interviewed in TAS‐2017.

Overall, our model results identify a clear set of factors associated with completing the TAS‐2019 interview online rather than by telephone. Variables representing all four sets of factors (demographic and socioeconomic, technology‐related, time use, and psychological) are associated with the likelihood of completing an interview using the online mode. The latter three sets of factors most clearly account for the greater likelihood of respondents of higher socioeconomic status choosing to complete their interview online rather than by telephone.

We conducted a similar logistic regression analysis to compare characteristics of mixed‐mode cases interviewed online and cases assigned to the telephone only mode who completed their interview. We found no differences between these two groups (results not shown).

DISCUSSION

We examined fieldwork outcomes in the 2019 PSID Transition into Adulthood Supplement—an ongoing panel study of young adults in the US—that were associated with a switch from telephone to online data collection. A key design feature and an important strength of our analysis was the random assignment of cases to either a telephone only mode or a web‐first mixed‐mode design that encouraged participants to complete their interview online but offered a telephone interview mode for respondents who were unable or unwilling to complete an online interview. We addressed two research questions: first, whether offering the online interview option led to better survey outcomes and lower costs and, second, what respondent characteristics were associated with using the web interview mode.

Regarding the first research question, our results were unambiguous and convincing: among mixed‐mode compared to telephone only cases, response rates were higher, interviews were completed faster and with less interviewer effort, the quality of the interview data appeared better, and fieldwork costs were lower. Our results were sufficiently encouraging that we strongly considered, but eventually decided against, ending our randomized design early and offering the web option to the telephone only control group.

Prior research has not consistently found benefits to offering an online interview option. For example, a recent meta‐analysis of variously designed studies with experimental tests comparing response rates on web to other modes documented a 12‐percentage point advantage for traditional modes over web (Daikeler et al., 2020), although this difference was attenuated by various study characteristics, including whether the study was an ongoing panel rather than a single‐wave survey. In our study, we found the opposite—sample members randomized to the telephone only mode had a 7.5 percentage point lower response rate compared to those randomized to mixed‐mode. This finding is consistent with the broader literature on mixed‐mode survey design (e.g., De Leeuw, 2018) which suggests that cooperation with a survey request is affected by multiple factors, such as protocols for follow‐up contact with non‐respondents, a backup mode for completing the survey among those assigned the online option, the longitudinal data collection framework, and sample characteristics such as participants' ages. Young adults have high rates of internet use but are in a life stage with high mobility and frequent turnover of traditional sources of contact, including addresses and landline telephone numbers. Consequently, our findings may not generalize to studies of other age groups. As described in Lugtig (2014), panel studies such as TAS may have conditioned particular survey‐related behavior through repeated interaction with respondents (and their families) and may also benefit from higher levels of respondent trust and openness to trying new data collection methods. Finally, a limitation of our analysis of mode differences in interview data quality was restricted to a just a single outcome, namely the rate of item nonresponse.

Cost savings is a major motivation for surveys to adopt mixed‐mode data collection with a web interviewing option. We found that cases assigned to mixed‐mode had lower costs than those in the telephone only mode. Costs were lower for both non‐response cases and for completed cases among the mixed‐mode group; furthermore, a smaller fraction of mixed‐mode cases were in the non‐response group, which incurred very high costs for both modes. Mixed‐mode cases interviewed by web contributed to lower fieldwork costs by completing their interviews very early in the fieldwork period and hence required few contact attempts. Mixed‐mode cases that did not complete a web interview may have lacked a web‐friendly devices or internet access, or may not possess the technical skills to complete the interview on the web. In the panel study context, these cases could be targeted when the interviewer calling phase of data collection begins, which should reduce their costs. Mixed‐mode cases that had not completed a web interview after eight weeks of fieldwork were contacted by telephone and encouraged to complete an interview by telephone. The timing of this intervention, and the intensity of non‐response follow‐up, were important drivers of the total number of contact attempts and overall costs. Fieldwork for TAS‐2019 followed a structured contact protocol for all cases, but an adaptive approach to determine the timing, mode, and type of contact attempts could improve fieldwork efficiency and reduce costs. A promising technique to develop such an adaptive fieldwork approach is the use of sequential, multiple assignment, randomized trials (SMART; see Almirall et al., 2018; Murphy, 2005), which could offer improvements to sequences of contact attempts, through changes in type, mode, and timing, across subgroups based on their characteristics and prior contact attempts.

Our cost evaluation was based on variable costs. The captured costs were for contact attempts, conducting interviews, and respondent incentives, which together account for the majority of all fieldwork costs. A limitation of our cost evaluation is that we do not include fixed costs and variable costs that are difficult to quantify, including the initial investments made to develop, test, and implement an online data collection option during the transition to mixed‐mode survey administration (which can be substantial) and the costs of managing the fieldwork enterprise during and after the shift to mixed‐mode data collection (which are moderate).

Regarding our second research question about the factors associated with mixed‐mode respondents completing their interview by web rather than telephone, the availability of panel data allowed us to test several hypotheses about interview mode using a rich set of covariates. The findings are generally consistent with our hypotheses drawn from social psychological theoretical frameworks of survey participation and the small number of past studies. Consistent with leverage‐saliency theory and social exchange theory, which both emphasize increased participation when salient design features enhance perceived benefits, we found that accessibility to the internet and computer equipment are major predictors of completing the interview by web (Freedman et al., 2018; Smyth et al., 2014). We also found that individuals with high time demands in the form of weekly work hours, but not hours spent volunteering, were more likely to complete a web interview than to complete the survey by telephone with an interviewer. The results also generally support the finding that higher socioeconomic status is positively related to a web interview completion (e.g., Patrick et al., 2017), although the independent effects of education and income became attenuated with controls for internet access and computing devices. Our analysis of interview mode used a subset of respondents who participated in two consecutive waves, which is a potentially select subsample.

An especially attractive feature of offering online data collection is the very rapid pace of completing interviews at the launch of fieldwork. We completed an online interview with one‐third of mixed‐mode cases in the first ten days of the fieldwork period. At the same time, a back‐up mode to online data collection is essential. For TAS, this backup mode is to interview respondents by telephone—although other modes, such in‐person interviews, are also possible. These additional modes are essential because not all respondents will complete an online interview and our interview mode analysis shows clearly that those who completed a telephone interview rather than an offered online interview in TAS‐2019 had different characteristics, related to demographic factors, access to technology, time use, and psychology.

Another important caveat that tempers our enthusiasm for online data collection was the considerable duration needed to achieve our target response rates. Despite the rapid rate of interview completion in the early days of the study and the fact that two‐thirds of the mixed‐mode cases were completed without any telephone calls, the pace of completions quickly dropped and remained low thereafter. We expended a considerable amount of time, interviewer effort, outreach, and incentives to reach our response rate target for the study—which occurred 282 days after the launch of fieldwork.

One perplexing result was that the telephone only group did not achieve a higher response rate—and, in particular, a response rate comparable to the prior wave. This result is certainly not due to allocating a lower level of fieldwork effort to these cases. Rather, our analysis showed a substantial increase in contact attempts for cases that participated in both the 2017 and 2019 waves of TAS. It does appear that circumstances made fieldwork more difficult in TAS‐2019, and one obvious explanation—other than the secular trend of declining response rates (De Leeuw et al., 2018)—is that the completion of later cases was affected by the Covid‐19 pandemic, which unfolded while TAS‐2019 was in the field (see Sastry et al., 2020). The pandemic could have had both positive and negative effects on fieldwork. On the positive side, some respondents were at home and hence easy to reach and with few competing commitments, and the survey incentive could have been particularly attractive due to the adverse economic situation caused by the widespread lockdowns. On the negative side, respondents were undoubtedly anxious about the pandemic situation; in addition, many TAS respondents in the target age range were at home with young children whose schools and day care were closed due to the pandemic.

The findings from this study are informing future waves of PSID and TAS. We conducted the 2021 wave of PSID using a similar design that combined a mixed‐mode and telephone only approach, and the 2021 wave of TAS, which is currently underway, has switched to using the mixed‐mode design exclusively. Among the important remaining research questions concerning the adoption of web interviewing that can be addressed with data from the TAS‐2019 mode experiment are the effects of interview mode on questionnaire topics that may be sensitive to mode effects—such as income and wealth, sexual orientation and gender identity, and mental health. Data from the mode experiment can also be used to examine whether lower item nonresponse rates among mixed‐mode cases are related to lower levels of social desirability bias for interviews completed by web compared to telephone, as suggested by previous studies (e.g., Kreuter et al., 2008).

Our study design and results will also benefit other surveys considering the adoption of online interviewing and, in particular, the use of a mixed‐mode approach. For example, our results regarding the quick early pace of completion for web interviews and the long tail needed to achieve response rate goals is important for other studies to consider as they design their fieldwork plans and protocols. Our results regarding the overall cost differences between the mixed‐mode and telephone only designs will likely translate broadly to studies in other settings and with other study populations because the structure and nature of these costs is likely to be similar. Finally, our findings on the uptake patterns for web interviewing according to various respondent characteristics should help other studies to identify—and target—specific sample segments that are most amenable to conducing their interview online. The extent to which other studies adopt and adapt these design features, and document their experiences, will help advance research on, and the practice of, online survey data collection.