Strategies to exclude subjects who conceal and fabricate information when enrolling in clinical trials.

Clinical trials within the US face an increasing challenge with the recruitment of quality candidates. One readily available group of subjects that have high rates of participation in clinical research are subjects who enroll in multiple trials for the purpose of generating income through study payments. Aside from issues of safety and generalizability, evidence suggests that these subjects employ methods of deception to qualify for the strict entrance criteria of some studies, including concealing information and fabricating information. Including these subjects in research poses a significant risk to the integrity of data quality and study designs. Strategies to limit enrollment of subjects whose motivation is generating income have not been systematically addressed in the literature. The present paper is intended to provide investigators with a range of strategies for developing and implementing a study protocol with protections to minimize the enrollment of subjects whose primary motivation for enrolling is to generate income. This multifaceted approach includes recommendations for advertising strategies, payment strategies, telephone screening strategies, and baseline screening strategies. The approach also includes recommendations for attending to inconsistent study data and subject motivation. Implementing these strategies may be more or less important depending upon the vulnerability of the study design to subject deception. Although these strategies may help researchers exclude subjects with a higher rate of deceptive practices, widespread adoption of subject registries would go a long way to decrease the chances of subjects enrolling in multiple studies or more than once in the same study.

Introduction

One of the challenges to recruiting in any clinical trial is meeting targeted goals of recruitment while maintaining the quality of candidates. One of the more significant threats to the quality of research is sampling from the population of subjects who enroll in multiple clinical trials with the objective of generating income. These subjects, hereafter referred to as “professional subjects,” present a significant risk to the integrity of study designs by providing false information as a strategy for meeting inclusion and exclusion criteria for study enrollment [1], [2], [3] and by providing false information about their disease symptoms or medication compliance [4]. Enrolling subjects who use deception in research can substantially undermine the study design by increasing the sample size needed to detect a treatment effect [5]. The impact of the problem is not widely understood and it is unlikely that investigators make sample size assumptions that account for subjects who use deception to gain entry to a study and provide false information while enrolled in the study.

Although investigators may operate with the assumption that subjects are truthful when providing information to researchers, there is mounting evidence that study participants conceal recreational drug use [6], conceal tobacco use [7], lie when answering screening questions [8], enroll in the same study multiple times [9], [10], and enroll in multiple studies simultaneously [1], [10]. Professional subjects share strategies for evading the restrictive entry criteria of studies [11], share information about upcoming studies using centralized resources online [12], and even have their own smartphone App (“Study Scavenger recruitment App”) to help locate studies based on location, payment, and study topic [13]. It is clear from the literature that subjects use deception, but most of what we know about this is not revealed unless subjects are caught using deception. The true scope of the problem includes both the subjects who have been caught using deceptive practices as well as the subjects who have not been caught.

In an attempt to understand how common deception is used by subjects enrolling in research, Devine et al. [1] surveyed 100 “experienced research subjects” recruited from newsprint and online postings to estimate the proportion of subjects who employ deception in research. In this study, the majority of these “experienced subjects” (75%) reported concealing some information from researchers when screening for a study. A significant proportion of subjects reported high rates of concealing information that might exclude them from participation, including participation in more than one study concurrently (43%), health conditions (32%), use of prescribed medications (28%), recreational drug use (20%) and alcohol use (12%). Devine et al. [1] also reported that 33% of subjects admitted to using some form of fabrication to enroll in previous trials; 25% of subjects sampled admitted to exaggerating health conditions to qualify for a study, 14% pretended to have a health problem in order to qualify for a study, and 12% gave researchers false information about symptoms that were the primary focus of the study. Devine et al. [1] also asked subjects about earnings per year and number of studies per year And found that subjects who admitted to using deception averaged $141 US dollars of reimbursement per study in the past year, and reported an expectation of receiving a minimum of $20 (on average) for participating in a study. Professional subjects are known to be attracted to high-paying inpatient phase I studies [3], but these results suggest that studies with reimbursement as low as $20 are also vulnerable to professional subject enrollment. Not all studies are vulnerable to the risk of professional subject enrollment, but some study characteristics may increase the vulnerability including 1) lack of objective testing for primary inclusion criterion (e.g., depressive disorders, anxiety disorders, bipolar and related disorders, pain disorders, substance use disorders), 2) high rates of subject reimbursement, and 3) dispensing study medication that has an inherent potential for diversion.

Although some researchers have offered valuable guidance for reducing the impact of deception in clinical research including using centralized subject registries [14], [15], verifying identification through photo ID [16], and using more rigorous assessment [17], there is little published guidance that addresses this problem on an individual site level that may inform recruitment and screening practices. Although the best protection against professional subject enrollment may be widespread adoption of centralized subject registries, single-site investigator-initiated studies that have not enrolled in one of the commercial registries would benefit from building protections into the study protocol. The present paper is intended to provide investigators with a range of practical strategies and suggestions for developing and implementing a study protocol with protections to minimize the enrollment of professional subjects. Given the risk to study integrity that results from subjects concealing and fabricating information in order to qualify for study enrollment, it is important for researchers to have a diverse set of strategies for minimizing the chance of sampling this professional subject population.

Strategies

There is no single screening test or method that will likely eliminate the possibility of professional subjects concealing information and fabricating information in order to qualify for a study. A multifaceted approach may provide the best protection against deception in the absence of methods to objectively measure each entrance criterion. The approach described below includes recommendations to minimize deception in research using advertising strategies, payment strategies, telephone screening strategies, and baseline screening strategies. The approach also includes recommendations for attending to subjects' motivation and being alert to inconsistent study data (see Table 1).

Table 1

Strategies to exclude professional subjects.

Strategy		Outcome
Advertising	• Avoid mentioning payments in media campaigns. • Limit the detail of information provided on clinicaltrials.gov (e.g. exclusionary criteria)	• May attract subjects who are more interested in potential benefits than generating income. • Will provide some protection against subjects who wish to deceive researchers by eliminating the “study guide” they use before screening.
Payment	• Limit payments in studies with a high vulnerability to enrolling professional subjects (e.g., clinical trial of narcotic pain medication). • Set a low payment amount for the initial screening visit. • Withhold payment for screening if the subject reports a behavior or health condition that should have excluded them at the time of telephone screening.	• May help to limit the enrollment of subjects who are motivated solely by payments. • May deter subjects looking for a one-time payment. • Prevents slowing of recruitment and protects against repeated costly one-session screen failures.
Telephone screening	• Construct a telephone screening that disguises the criteria for entry. • Use non-leading questions that do not reveal the entrance criteria. • Ask all screening questions even after the point that it is known that the subject will be excluded. • Develop several versions of the telephone screening.	• Can help reduce the risk of professional subjects passing a screening interview by studying the questions. • Will make it harder for subjects to study the phone screening process and determine what the “correct” answers are.
Baseline screening	• Obtain consent to review the subject's clinical records at baseline. • Include objective measures to assess other inclusion and exclusion criteria. (e.g. to determine drug use, perform a urine drug screen prior to randomization)	• May decrease concealment or fabrication of information; deception will likely be revealed when the researchers review the clinical record. • May decrease the likelihood of the use of deception during the screening.
Assessing subject motivation	• Use some degree of clinical judgment when interacting with subjects during screening. • Train staff to recognize when subjects ask payment questions more frequently, in more detail, or with greater importance.	• May help to identify subjects whose motivation is to generate income.
Attending to data inconsistencies	• Use different assessments that ask the same questions in different ways. • Be aware of data inconsistencies that are signs of professional subjects attempting to use deception during screening (e.g. data that does not make sense or is an outlier relative to other subjects with the target disease)	• May provide a validity check on the consistency of data. • May help to identify when subjects are being deceptive, exaggerative, or are concealing, or fabricating information.
Subject registries	• Implement the use of subject registries, such as CTSdatabase, DUPcheck, Verified Clinical Trials, and ClinicalRSVP.	• Will help to identify possible professional subjects by seeing if they are enrolled in multiple studies simultaneously.

Advertising strategies

At the very outset of conducting a clinical trial, the strategy used for recruitment can have an impact on the rate of professional subject enrollment. Professional subjects are an organized group who search out studies to take part in and share information about trials with high rates of payment [3]. Advertisements, flyers, or other media that includes detailed information about study payments may be a draw for this group of subjects. In studies with a potential for direct benefit, a media campaign that does not mention payments may attract a population that has a more genuine interest in the benefits of research participation than those subjects whose intent is to generate income.

Although it is not intended as a method of advertising, compliance with section 801 of the FDA Amendments Act [18] requires that clinical trials completed in the United States be registered on clinicaltrials.gov. Investigators often include specific information about the inclusion and exclusion criteria for study entrance in the registry, and there is some evidence that professional subjects study this registry and answer screening questions to be consistent with study criteria [1]. Although compliance with section 801 of the FDA is required for many researchers in the US, it is not necessary to provide all of the exclusion criteria within this registry. Limiting the detail of information may increase the rate of screen failure as potential subjects are not aware of the exclusions, but this will also provide some protection against subjects who wish to deceive researchers by eliminating the “study guide” they use before screening.

Payment strategies

Minimizing subject payments is one strategy to reduce the likelihood of recruiting professional subjects. Models for determining appropriate subject payments have been discussed extensively [19]. However, there is some evidence that increasing payments are related to increasing willingness of subjects to conceal information that would exclude them from enrolling in a study [20]. Although limiting payments could have a negative impact on the rate of subject recruitment, the benefit of limiting the enrollment of subjects who are motivated solely by payments may be a reasonable tradeoff in studies with a high vulnerability to enrolling professional subjects (e.g., clinical trial of narcotic pain medication).

High payments for initial screening visits may make a study vulnerable to professional subjects who are looking for a one-time study payment. Some subjects know that they will not qualify but begin screening with the objective of making money for one visit before being excluded. These subjects may not be a threat to the validity of study data as they will be excluded, but there is a significant cost in staff time to screen them and they occupy screening slots that could be filled with better quality candidates. Over the course of a study, one-session screen failures can undermine the study objectives if they occupy a large proportion of the new subject screening visits to the point that resources are depleted before the recruitment goal can be met. At a minimum, screening these subjects slows recruitment and makes the recruitment phase more costly. Setting a low payment amount for the initial screening visit may deter some of these subjects.

As a further protection against professional subjects who enroll for a one-time payment, one strategy is to withhold any payment for screening if the subject reports a behavior or health condition that should have excluded them at the time of telephone screening. Some subjects may conceal information during the telephone screening so that they will qualify for the initial screening visit that includes a pro-rated payment. For example, in our own studies of medications to treat alcohol problems we exclude subjects who report recreational drug use during our preliminary telephone screening. If a subject in the baseline screening visit has a positive urine drug screen result, we withhold payment because the subject concealed their drug use during the telephone screening. Subjects are informed of this policy during the telephone screening so there are no surprises on the day of their appointment. Upon hearing this policy, some subjects decline to schedule a baseline visit.

Telephone screening strategies

Some subjects may share information with one another or complete a phone screening several times until they qualify for a study [1]. Part of the process involves subjects trying to determine what the “correct” answers are to be included in a study. Constructing a telephone screening that disguises the criteria for entry is a strategy that can help reduce the risk of professional subjects passing a screening interview by studying the questions. Using non-leading questions that do not reveal the criteria for entrance will make it harder for professional subjects to study the phone screening process and determine what the “correct” answers are. For example, asking “have you participated in any medication studies for alcohol problems in the past 7 years?” may leave the subject with the impression that participating in a clinical trial over the past seven years is exclusionary. An alternative non-leading question, which disguises the exclusion criterion, is to ask “What types of treatment and/or treatment research have you received in your lifetime?” followed by an assessment of dates of service for each reported type of intervention.

Tools from market research may have some value in designing screening measures for clinical trials. For example, one market research strategy to minimize dishonest answers is to include “dummy termination questions” to help disguise the reason for excluding somebody based on a screening interview [21]. Instead of ending a phone screening after hearing a disqualifying answer, the interviewer will continue with the interview and terminate on a later question that is unrelated to the real reason for exclusion. Another method for concealing the reason(s) for screen fail is to ask all screening questions even after the point that it is known that the subject will be excluded. The subject can then be excluded after the entire interview is complete. Depending upon the vulnerability of the study to being exploited by professional subjects, investigators might also want to develop several versions of the telephone screening to minimize the risk that subjects will study the screening questions until they learn the qualifying answers.

Baseline screening strategies

There are some screening strategies that may help exclude professional subjects who are using deception to qualify for a study. Although some studies may focus on a primary condition that is not easily assessed with objective measures, there may be opportunities to consult outside healthcare records to confirm the subject's report of their disease history. If the trial is being conducted at a medical center where the subject has received care, a standard line in the consent granting researchers access to view the subject's clinical record is the most efficient means of gaining access to this information. Obtaining consent to review the subject's clinical records may also decrease the subject's willingness to conceal or fabricate information as they are aware that this deception will likely be revealed when the researchers review the clinical record.

In studies where the primary condition is not readily confirmed with objective measures, investigators may still be able to include objective measures to assess other inclusion and exclusion criteria. For example, recreational drug use may be ruled out through self-report or even a diagnostic interview of DSM-5 substance use disorder criteria, but a subject who has the incentive to conceal recreational drug use can easily manipulate both of these methods. Including urine drug screening prior to randomization is one objective test that may increase investigator's confidence that self-reported data is accurate. Whenever possible, the use of objective measures to confirm other eligibility may decrease the likelihood of professional subjects using deception during the screening.

Assessing subject motivation

It would be challenging and possibly futile to attempt to characterize what a professional subject is like based on demographic data. Based on our own previous study of subjects who use deception [1], the only subject characteristics that were associated with higher rates of deception were age and gender; younger subjects had greater rates of deception than older subjects and men had greater rates of deception that women. This finding is of little value in determining who might be a professional subject as not all young subjects or all men should be excluded on the suspicion of using deception. Subject behavior is often more revealing than their characteristics and investigators can rely, to some degree, on clinical judgment when interacting with subjects during screening.

One of the most transparent subject behaviors that investigators might observe in subjects is a preoccupation with the schedule of payment, the amount of each payment, and the minimum effort required to earn each payment. Although it may be commonplace for most subjects to ask a question or two about payments, professional subjects often spend more time on this topic as they may see reimbursement as the primary direct benefit of participation. In contrast, subjects who are enrolling with the motivation to experience some direct benefit from a therapeutic intervention may be more focused in their questions about these direct benefits and have little interest in the payment schedule. Training staff to recognize when subjects ask payment questions more frequently, in more detail, or with greater importance, may help to identify subjects whose motivation is to generate income.

Attending to data inconsistencies

Investigators can also pay close attention to the quality and consistency of information that subjects report when screening for a study. Subjects who conceal and fabricate information sometimes have difficulty maintaining the consistency of their information across several screening measures or interviews with several staff members. Using different assessments that ask the same questions in different ways may provide a validity check on the consistency of data. While it is possible that non-deceiving subjects may also provide inconsistent information, professional subjects may present as more evasive when asked a question as they are cognizant of the possibility that their answers may disqualify them.

Another potential sign of professional subjects attempting to use deception during screening is data that does not make sense or is an outlier relative to other subjects with the target disease. For example, in a study of medications for alcohol dependence it is uncommon to have a subject report drinking in excess of 50 standard drinks each day and then provide an alcohol breath sample that is zero. A report of this level of daily drinking appears to be either a professional subject fabricating symptoms of the target disease, or a genuine subject exaggerating the disease symptoms for the purpose of qualifying for the study. In both cases, the data is not consistent with the usual presentation of alcohol use disorders and this could be interpreted as a sign of deception in the context of other data that suggests deception.

Discussion

Although there is evidence that some degree of deception is ongoing within clinical trials in the US, the scope of the problem is not fully understood and study design strategies to combat these problems have not been well articulated in the literature. The methods and strategies presented in this paper provide a multifaceted approach that may help reduce the rate of professional subjects using deception and fabrication to qualify for inclusion in a clinical trial. Implementing these strategies will certainly result in some increased costs including staff effort, money, and recruitment pace, but the risks of not implementing strategies to reduce deception may be much more substantial. Specifically, if the integrity of study designs and quality of study data is substantially undermined by subjects using deception then the entire expense of conducting the trial may be wasted if the study fails to accurately test the hypotheses it was designed to evaluate. Another possibility is a null finding that is the result of error variance in the data due to oversampling professional subjects who provide poor quality or false data. Populating the clinical trial literature with findings that are erroneous may undermine researchers' confidence in a medication that has real therapeutic value that has been masked in error variance.

These strategies may help researchers minimize the enrollment of professional subjects but will likely not prevent this in all cases. Significant efforts have been undertaken to implement subject registries within clinical trials in the US in order to minimize subjects enrolling in multiple studies concurrently. CTSdatabase, DUPcheck, Verified Clinical Trials, and ClinicalRSVP [10], [22], [23], [24] are all examples of how registries can be implemented to reduce the risk of subjects enrolling in the same study at multiple sites, or subjects enrolling in multiple unrelated studies concurrently. Widespread adoption of these tools on a national level would certainly reduce the risk of professional subject enrollments. Efforts to protect the integrity of research data from the deception used by professional subjects should include a range of tools including strategies described in this paper and widespread use of registries.

Limitations

Although many of the design considerations described in the present paper have been part of our own research methods for a series of medication trials to treat alcohol problems, these methods have not been systematically evaluated. Systematic testing of the strategies presented in this paper may inform future researchers as to the most effective methods for excluding professional subjects.

Funding

This work was supported by the National Institute on Alcohol Abuse and Alcoholism [Contracts HHSN275200900005C and HHSN275201400001I].