Conducting a Time Trade-Off Study Alongside a Clinical Trial: A Case Study and Recommendations.

Time trade-off (TTO) is an established method in health economics to elicit and value individuals' preferences for different health states. These preferences are expressed in the form of health-state utilities that are typically used to measure health-related quality of life and calculate quality-adjusted life-years in an economic evaluation. The TTO approach to directly elicit health-state utilities is particularly valuable when generic instruments (e.g. EQ-5D) may not fully capture changes in utility in a clinical trial. However, there is limited guidance on how a TTO study should be conducted alongside a clinical trial despite it being a valuable tool. We present an account of the design and development of a TTO study within a clinical trial as a case study. We describe the development of materials needed for the TTO interviews, the piloting of the TTO materials and interview process, and recommendations for future TTO studies. This paper provides a practical guide and reference for future applications of the TTO method alongside a clinical trial.

Key Points for Decision Makers

Introduction

Time trade-off (TTO) is an established method to elicit and value individuals’ preferences for different health states through asking participants to hypothetically trade between quality of life (QOL) and quantity of life [1]. These preferences are measured in the form of utilities that can be used to calculate quality-adjusted life-years (QALYs). QALYs are recommended by the National Institute of Health and Care Excellence (NICE) as the preferred measure of benefit when conducting economic evaluations of healthcare interventions for NHS England and Wales [2]. In an economic evaluation, QALYs are often calculated using generic preference-based QOL measures such as the EQ-5D. The responses to those questionnaires can be translated into QALYs using a pre-determined utility tariff [3-5]. TTO is a key component of health-state valuation protocols in developing utility tariffs for EQ-5D instruments [6, 7]. The EQ-5D is widely used in evaluations of healthcare and public health interventions with advantages of simplicity and brevity compared with other instruments [8]. However, there are instances where the EQ-5D may not capture all relevant gains and losses in health status; for example, within a clinical trial where the fixed frequency of data collection may not capture utility changes resulting from unexpected or short-term acute clinical events. Using a concurrent TTO exercise to capture the (dis-)utility of those clinical events provides a viable alternative. A recent review of the literature found a wide range of procedural considerations when performing TTO tasks [9]; this paper aims to provide a practical guide on the design, development and conduct of a TTO using a worked case study of a TTO alongside a clinical trial [10].

Time Trade-Off (TTO) Case Study Setting and Methods

The TTO exercise was conducted among a sample of consented eligible trial participants to enhance the economic evaluation within the OPEN (open urethroplasty versus endoscopic urethrotomy—clarifying the management of men with recurrent urethral stricture) study, a UK multicentre randomised trial to determine the clinical and cost effectiveness of open urethroplasty compared with endoscopic urethrotomy in men with recurrent bulbar urethral stricture [10]. The TTO exercise was conducted alongside the trial because the EQ-5D administered at fixed time intervals may not capture short-term but potentially significant decrements in patients’ QOL post-intervention, and they may experience multiple interventions given the likelihood of recurrence of the condition. Those decrements may have significant impact on patients’ QOL that should not be overlooked when conducting cost-utility analysis. The short-term utilities elicited by the TTO will inform the calculation of utilities provided by the EQ-5D.

TTO is typically used to measure the utility of chronic health states where participants remain in the impaired health state for several years. However, in this trial the impaired health states lasted for days or weeks before a return to baseline health. The suitability of the conventional TTO methodology in valuing short-term health states that typically last less than a year has been questioned [11, 12]. Thus, a variant of TTO—chained TTO—has been suggested using an anchor state as a bridge between the temporary states and death [13]. The chained TTO task has two stages: respondents are asked to compare the health-state profiles with the anchor state followed by returning to perfect health (instead of death in a conventional TTO) in stage one, and then compare the anchor state with perfect health state in the conventional TTO task in stage two. In this study, both the conventional and chained TTOs were conducted, and participants were randomly allocated to one of the methods. See Appendix 1 for technical details on the TTO analysis methods.

Development of TTO Materials

A face-to-face interview format was used because of its anticipated high response rate and the chance to better understand participants’ thought processes. Additionally, because of the complex nature of the chained TTO, it was deemed necessary to administer the study in person rather than use self-completion methods. The materials required to conduct a face-to-face TTO exercise include a TTO board, a number of cards describing health states to be valued and an (optional) interviewer script detailing the process of the TTO exercise. A ‘props method’ (referring to the use of physical interview prompts) [14] was followed to design the materials.

Decision Board

The decision board was made of an A3 foam board (chosen to withstand extensive use) representing two scenarios: Life A and Life B. The timeline for Life A was fixed while the timeline representing Life B had a movable marker to represent changing time lengths. The timelines enabled participants to visualise the lengths of time spent in each state. The health-state profile cards and the anchor health state were then placed next to Life A and Life B as appropriate.1

Health-State Profiles

The health-state profiles were developed to describe adverse effects of the trial interventions. These ranged from common and mild effects to rare but severe effects. Clinicians and a patient representative were involved throughout the development process to ensure health-state profiles were both medically accurate and understandable. Profiles representing two levels (mild and severe) of adverse effects for each of the two trial interventions were developed, resulting in a total of four health profiles to be valued. The burden on participants to value four health profiles was considered low compared with other studies [12, 15, 16]. These profiles were printed on laminated A6-sized coloured cards.

Anchor State

The anchor state profile required for the chained TTO describes a hypothetical situation that is intuitively worse than the most severe health profile being valued yet preferable to death, in order to enable trade-off in the first stage of the exercise. The suitability of the anchor state required substantial testing based on reported problems in previous studies [17, 18]. The anchor state was printed on a laminated A6-sized card.

Time Horizon

One of the challenges in the design of a TTO exercise is to decide on an appropriate time horizon. The condition-specific adverse effects in the OPEN study ranged from a few days to several months. A duration of 28 days was initially chosen based on the longest duration of short-term adverse effects (those lasting < 1 year). Within each health profile, specific lengths of time were stipulated for adverse effects that did not last for the entire 28 days. The intention was to give a realistic representation of the impaired health states immediately following the trial interventions.

Script and Training Video

We produced an interview script with instructions for each version of the TTO exercise. As study participants were recruited across the UK, it was decided to enlist research nurses (RNs) at participating trial sites to conduct a proportion of the TTO interviews. A training video was produced to provide RNs with an additional resource to supplement face-to-face training by the study team.

The interview script provided suggested wording for interviewers to use during the TTO exercise. Creating an interview script ensures all interviews are conducted consistently. We observed that after several interviews it was not necessary to read verbatim from the script, but it remained a useful point of reference and ensured fidelity of the interview process. The interview scripts and instructions are included in Appendix 3.

The training video was structured as a ‘mock’ interview, and covered common difficulties encountered in a TTO exercise (Appendix 4).

Piloting of TTO Materials and Process

We conducted three rounds of piloting of materials and processes; following each round, the project team discussed the issues noted, refining materials and processes where appropriate. This process is described in Fig. 1.

Fig. 1

Time trade-off (TTO) exercise development

Round 1

Piloting of the initial versions of the TTO materials was conducted with 17 male and female volunteers (researchers at Newcastle University); the aim here was to ensure plain language was used. During this round, it quickly became clear that participants often entered the TTO exercise without fully understanding the task ahead. A decision was made to add a practice task to ‘warm-up’ participants, as suggested in previous studies [6, 19]. Generic EQ-5D-3L profiles were used in the practice task. For the conventional TTO method, one practice health state was evaluated before the start of the main TTO exercise. For the chained TTO method, one practice health state was evaluated before each stage of the chained process. Alterations to the materials were made following this first round of piloting and tested in subsequent piloting.

Round 2

Piloting was conducted with 15 male staff members from a participating study site and a patient representative. This round was limited to men because we wanted to test the TTO materials with the gender of interest for this study. These pilots were used to assess whether the changes made in response to the first round of piloting were sufficient or if further amendments were necessary.

Round 3

Piloting was conducted with nine men eligible for the OPEN trial. Review of the resulting data assessed internal consistency of the health-state valuations and found evidence that participants might still not fully understand the TTO exercise. A major change to the TTO exercise was made to extend the practice period by adding an extensive practice exercise (described in the pre-interview practice section below).

Piloting was crucial to the development of the TTO exercise, reflected by the resulting feedback and revisions, and also served as a training exercise for researchers. Sufficient time should be allowed for piloting before embarking on the study data collection. Efforts should be made to ensure piloting includes individuals with similar characteristics to the target study participants, so that potential issues relating to the characteristics of study participants can be identified. Following piloting, the following revisions were made:

Piloting showed that the burden of valuing four profiles was low. With further clinical input it was therefore decided to further divide adverse effects into mild, moderate and severe, „ in total (Appendix 2). More profiles allowed a richer data set to estimate health-state utilities and the nuances of each health state were better expressed.

As noted above, a chained TTO requires an anchor state. The first version of the anchor state was titled ‘chronic pain’ and described a health state in which the individual experienced chronic, debilitating pain with no relief. Piloting identified two problems with this profile: firstly, ‘chronic’ implied long term, which was at odds with the short duration of the profile; secondly, the health state described was perceived as worse than being dead by a large proportion of piloting participants. The title of the anchor state was accordingly changed to ‘severe pain’ to imply less permanency and the severity was reduced: maintaining the impairment of usual functioning (i.e. working, leisure activities), but allowing basic self-caring activities. This was considered in most subsequent piloting as a state between the most severe of the health states being valued and death. This was because participants felt that maintaining basic functioning without complete dependency on others was important for a health state to be considered better than death. The final version of the anchor state is in Appendix 2.

Piloting identified problems in the original time horizon format where the entire health state lasted 28 days, but within this some components had their own specific timelines, such as an overnight hospital stay. These different timeframes appeared confusing for participants and re-evaluation of the profiles led us to implement a reduced time horizon of 14 days. Adverse effects that lasted fewer days were specified as such within the profiles and were defined in relatively non-specific terms within the descriptions as ‘a few days’ or ‘overnight’; the intention was to minimise cognitive burden on participants by providing only one numerical value (14 days) to concentrate on when considering each health state. Piloting confirmed that participants preferred this format. The more complex a health-state profile is, the more difficult it is for participants to visualise and evaluate. Piloting emphasised the need to ensure an appropriate balance between realistic and clinically accurate health-state profiles, and simple and clear scenarios.

Pre-interview Practice

Piloting demonstrated that an extended practice period was needed to improve participants’ understanding of the TTO exercise. In the extended practice, participants were asked to evaluate three health-state profiles (based on EQ-5D profiles) as examples after ranking them from the best to worst. Participants’ evaluations on the practice health profiles were then immediately compared with their rankings using a ‘Practice Sheet’,2 and they were also asked to verbalise their decision-making process. The researcher then talked through the practice results with the participants using the ‘Practice Sheet’ as a discussion tool, giving them a chance to reflect on their decisions and ask questions.

Steps of TTO Interviews

Following piloting, the TTO interview procedure was agreed (Fig. 2). Ethical approval was obtained as part of the OPEN study. Upon consenting to the TTO interview, participants were informed about the aim of the study and the interview process, following which they were asked to complete a short questionnaire regarding their sociodemographic details before the TTO exercise began. At the end of the TTO exercise, participants were asked to rate the difficulty of the task on a 1–5 scale (1 being ‘no difficulty at all’ and 5 being ‘very difficult’) and provide additional feedback if they wished. Following completion of the interview, interviewers noted their own reflections on the process.

Fig. 2

Time trade-off (TTO) exercise procedure

The TTO Exercise

An illustration of the interview process is provided in Fig. 3. In order to obtain a utility value for each health state, the TTO exercise aims to elicit a point at which a participant is indifferent between the health state being valued (Life A) and an alternative state (Life B). Initially the participant is asked to state a preference for Life A or Life B when the duration of each is 14 days (Fig. 3; Iteration 1). This is followed by an iterative process of varying the time a participant ‘spends’ in Life B from 14 days to 1 day to 13 days to 2 days, etc. (Figure 3; Iterations 2–4) in order to ascertain the duration of time in the alternative state which is equivalent to 14 days in Life A; this number of days is recorded by the researcher. This process will continue until the participant is indifferent between Life A and Life B. Figure 3 provides a visual representation of the interview process described in the full interview script, which is included in Appendix 3.

Fig. 3

Iteration process of varying time in Life B in the time trade-off exercise

Results of TTO Interviews

Overall, 40 participants were recruited to the study; 20 participants completed each TTO variant. Two participants did not complete the TTO tasks, and were excluded from the analysis. The median difficulty rating was 2 (range 1–5), suggesting the TTO exercise was viewed as reasonably easy. Participants described their decision-making process, which drew on personal life experiences, family situations and experiences of the interventions being evaluated. Adaptation was observed as some participants stated that after having experienced urinary symptoms and the use of a urinary catheter, they no longer felt they would be as negatively impacted by a recurrence. Interviewer notes generally covered how well the researcher felt the participant had understood the task. In all but a minority of cases, the researcher perceived that the participant had understood the task. Interviewers noted where participants had given apparently illogical answers and any possible reasons for this. Those qualitative elements of the TTO exercise were useful for interpreting the health-state valuations results.

Discussion and Reflections

This paper provides a practical guide on the development, design and conduct of a TTO study (Fig. 1). We believe it makes a valuable contribution to the literature given the paucity of practical descriptions of conducting TTO studies. The illustrative case study was part of a clinical trial comparing two surgical interventions. Both of the interventions led to short-term disutility that may not be captured by generic preference-based QOL measures. This setting represents an example of the value of a TTO exercise in such circumstances. This paper provides a detailed account of the TTO exercise, including how study materials were developed, how iterative piloting informed changes, and gives our reflections and recommendations for researchers planning to use TTO.

A number of recommendations for best practice in designing and conducting a TTO exercise are made based on study reflections. In the design stage of a TTO exercise, if health states are based on a clinical scenario, the involvement of clinicians in developing the profiles is essential. Clinicians are experienced at translating textbook clinical effects into lay language; therefore, their involvement ensures health-state profiles are accurate and realistic, yet simple and clear to participants. For the purpose of the case study, the descriptions of health-state profiles were based only on adverse effects of the interventions, however, it should be noted that in the design of health-state profiles, both negative and positive aspects of health may be included dependent on the study question. Additionally, patient and public involvement (PPI) throughout the design and piloting phases was essential to ensure the final product was fit for purpose. In the development of a TTO exercise, multiple rounds of extensive piloting are crucial to ensure the study materials and processes are fit for purpose. It is also essential to pilot using a sample with similar characteristics to the target population. A further key message from our work is the need to conduct an extensive practice task before the main TTO exercise. This was particularly useful in ensuring participants fully understand the logic behind the TTO exercise before engaging in the evaluation of study health-state profiles. Another point to note is that TTO interviews, if conducted face-to-face and at geographically distant locations, are resource intensive; sufficient time and funding should be allocated for both the development and conduct of the TTO study.

Conclusion

In summary, this paper provides practical guidance and recommendations for conducting a TTO study alongside a clinical trial. Given the value and potential broad usage of TTO in measuring QOL, we believe this paper will be a useful resource for researchers who wish to embark on designing a TTO exercise in their own studies.

A time trade-off study is a feasible method to elicit patient preferences and value short-term changes in quality of life alongside a clinical trial.

Extensive piloting and revisions are required when developing the time trade-off materials to ensure they are fit for purpose. The involvement of clinicians and patients during this process is essential.

The complex nature of time trade-off interviews is likely to require face-to-face interviews, and together with an extensive development process, it is important that sufficient time and funding is allocated for the process.

Urethrotomy: Mild

Discomfort in the penis and bladder from using a catheter for a few days

Brief discomfort on passing urine after the catheter is removed

A few drops of blood after you have finished passing urine

Mild urinary tract infection giving you mild fever-like symptoms

Urethrotomy: Moderate

Discomfort in the penis and bladder from using a catheter for a few days

Discomfort on passing urine after the catheter is removed

Moderate urethral bleeding which requires you to keep the catheter in longer or have a telescopic examination under anaesthetic

Serious urinary tract infection which makes you feel ill and requires you to stay in hospital overnight for antibiotics from an IV drip

Urethrotomy: Severe

Discomfort in the penis and bladder from using a catheter

Severe urethral bleeding which requires you to have a telescopic examination under anaesthetic

Serious urinary tract infection which makes you feel ill and requires you to stay in hospital overnight for antibiotics from an IV drip

Severe pain in the penis and bladder area requiring you to take regular painkillers

Difficulty getting and maintaining a penile erection for sex

Urethroplasty: Mild

Discomfort in the penis and bladder from using a catheter

Mild mouth pain or discomfort when you eat or drink

Mild urinary tract infection giving you mild fever-like symptoms

Mild swelling and wound pain in the area between the testes and back passage

Urethroplasty: Moderate

Discomfort in the penis and bladder from using a catheter

Moderate and constant mouth pain and scarring in the mouth needing regular painkillers

Serious urinary tract and wound infection which makes you feel ill and requires you to stay in hospital overnight for antibiotics from an IV drip

Moderate wound pain in the area between the testes and back passage needing regular painkillers

Urethroplasty: Severe

Discomfort in the penis and bladder from using a catheter

Severe and constant mouth pain and scarring in the mouth needing regular painkillers

Serious urinary tract and wound infection which makes you feel ill and requires you to stay in hospital overnight for antibiotics from an IV drip

Severe wound pain in the area between the testes and back passage needing regular painkillers

Leakage of urine from the area between the testes and back passage requiring you to wear incontinence pads

Difficulty getting and maintaining a penile erection for sex

Anchor State

Severe pain state

You have recently been injured and as a result of the injury:

You are able to do basic tasks (e.g. washing, feeding and communicating) but you have problems walking about

You have extreme pain and discomfort. No medication can completely alleviate the pain

You cannot take part in usual activities (e.g. work, social activities and exercise)