The Inconvenient Truth About Convenience and Purposive Samples.

Most research is conducted on convenience and purposive samples that may be randomly or nonrandomly drawn. A convenience sample is the one that is drawn from a source that is conveniently accessible to the researcher. A purposive sample is the one whose characteristics are defined for a purpose that is relevant to the study. The findings of a study based on convenience and purposive sampling can only be generalized to the (sub)population from which the sample is drawn and not to the entire population. This article explains the concepts involved with the help of examples of both good and bad sampling practice. Database studies and studies with enriched designs are cited as special examples of convenience and purposive sampling. Issues related to the internal and external validity of convenience and purposive samples are explained. The importance of good sampling techniques in the design and interpretation of research is understated; this must change.

We wish to study whether a new antidepressant drug is superior to placebo in patients with major depressive disorder (MDD). Our unstated desire is to draw conclusions about the efficacy of this drug in all patients with MDD, all over the world, now, and in the future, as well. This is because it is meaningless to do a study whose findings apply only to the sample that we recruit and to nobody else, anywhere else, anytime else.

In research, we therefore implicitly seek to generalize the findings from our sample to the entire population, present and future. However, this is possible only if our sample is representative of the population; a sample is likely to be representative of the population only if it is randomly drawn from the population.

As a side note, “population,” here and in the rest of this article, is used in its statistical sense to refer to the entire group of persons with the characteristics of interest. So, in a study of the safety and efficacy of escitalopram in MDD, the sample is the group of patients in the study, and the population, in principle, is everybody with MDD, all over the world. In practice, however, if the study is conducted on outpatients with MDD in a private hospital in India, the population shrinks to all outpatients with MDD in similar private hospitals in India. This population is effectively a subpopulation of “everybody with MDD, all over the world.” The concept is further explained in the rest of this article.

Unfortunately, it is rarely possible to draw a random sample from the population. For our antidepressant study, for example, it would be impossible to list every person on the planet who has MDD, and to draw a random sample from this list; of course, it would be impossible to sample persons with MDD who have not yet been born. Research, therefore, is almost always conducted on convenience samples.

Convenience Samples

A convenience sample is one that is drawn from a source that is conveniently accessible to us. This sample, however, may not be representative of the population at large. Thus, for example, a convenience sample of patients may be drawn from a hospital; but these patients may not be representative of all patients, such as patients in the community. Also, a convenience sample of students may be drawn from a nearby medical college; but these students may not be representative of all students, such as students in other professional and nonprofessional colleges.

Research that is conducted on convenience samples can only be generalized to the population that was conveniently accessible, from which the sample was drawn. As an example, a study on learning disabilities is conducted on a random sample of students drawn from a government school in a rural part of Karnataka, India. This is a convenience sample, and the findings from the study can only be generalized to the students of that school, and possibly to students of other government schools in that region. It would be imprudent to generalize the findings to city schools, private schools, and schools in other parts of India or the world.

A study conducted on a convenience sample can have high internal validity if the findings are trustworthy. This is possible if the study was methodologically sound and if the data were properly analyzed. However, a study conducted on a convenience sample will have limited external validity. This is because the findings cannot easily be generalized to populations with characteristics that differ from the population that was conveniently accessible, and from which the sample was drawn.[1]

Here is a further important limitation. Generalization from a convenience sample to its population is possible only if the sample was randomly drawn from that population. So, if a study on hospitalized alcohol-dependent patients in a deaddiction center recruited only those patients occupying beds assigned to the research student, or only on days on which the student was on duty, or only from the clinical unit in which the student was working, the sample may be biased in known or unknown ways and may not represent even the population of patients that attend the specific deaddiction center, let alone alcohol-dependent patients hospitalized elsewhere. This further compromises the external validity of the study. Research of this nature is compromised; yet, samples are perhaps often recruited in this manner.

Here is a particularly egregious example of a nonrandomly drawn convenience sample. In a hypothetical study of blood micronutrient levels in patients with schizophrenia, a healthy control sample was formed from friends and colleagues who volunteered to donate blood. Micronutrient levels were found to be lower in patients than in controls. The only generalized interpretation possible is that the population of schizophrenia patients who attend the researcher’s hospital have lower blood micronutrient levels than the population of friends and colleagues of the researcher. Research of such nature is therefore unhelpful to the cause of science. Readers may also note that when such a control sample is nonrandomly drawn, the researcher can “help” prove the study hypothesis by deliberately selecting controls who have a balanced diet and lead a healthy lifestyle.

Population-Based Convenience Studies

Some studies, such as those that extract data from healthcare or insurance databases in a state or country, claim to be population-based studies. Here, “population” does not mean “from the entire country” let alone from all over the world; “population” means that there is no sampling, and that the whole population of eligible subjects in that database is studied. There is, therefore, no need to generalize from the sample to the population when the population is itself the subject of study. However, this is still a form of convenience sampling because the database was conveniently available and only the subjects eligible to belong in that database were studied. So, the findings do not necessarily generalize to people in other databases, or to other people in that country, let alone to others in the rest of the world. Thus, even these population-based studies are a form of convenience sampling with limited external validity.

Internal validity in database studies may not be high because the databases may not contain all the information that is necessary for the study and the recorded information may not necessarily have been accurately obtained from subjects. That is, there may be unmeasured and inadequately unmeasured confounds. For example, in a database study of the influence of lifestyle behaviors on the risk of dementia, databases may record whether or not a subject is a smoker, but not how many cigarettes are smoked in a day, or whether the cigarettes are low or high in tar content; so smoking is an inadequately measured confound. The database may not contain any information about dietary habits, so diet is an unmeasured confound.

It is not common to cover the entire country for healthcare or insurance databases. However, in database studies, an example of exceptions could be national register-based studies in Scandinavian countries, where everybody in the country is recorded in registers and where different registers can be cross-linked. The external validity of these studies comes closest to the ideal.

Purposive Samples

Research is also almost always conducted on purposive samples. A purposive sample is the one whose characteristics are defined for a purpose that is relevant to the study. For example, a study may purposely examine the antidepressant benefits of fluoxetine in children and adolescents because we do not know whether the drug will work as well in children and adolescents as in adults. Also, a study may purposely examine smoking quit rates with varenicline in persons who have been smoking more than ten cigarettes a day for at least the past 1 year because patients with lower levels of smoking may be able to quit on their own (so drug may be no better than placebo in such patients). Also, a study may purposely examine attitudes toward ECT in depressed patients who have never received ECT because it is important to know what these patients think about a treatment that might sometimes be recommended to them.

The greater the number of inclusion and exclusion sample selection criteria set, each for a necessary purpose, the more purposive the sample becomes. Advantages of purposive samples are many. For example, they study only the population that is of specific interest, or they make the sample homogeneous (when between subjects variance is reduced, statistical significance is more easily obtained), or they exclude subjects who are at risk of serious adverse events. The disadvantage of purposive samples is the same as that of convenience samples: the more purposive the sample is, the more limited the external validity will be.

Random sampling is possible with purposive samples just as it is with convenience samples. However, even with random sampling, when the sample is purposive, generalization is only possible to the population defined by the sample selection criteria. So, the findings of a randomized controlled trial (RCT) that was conducted in adults cannot be generalized to children with the same diagnosis; or the safety profile of an antidepressant in an RCT that recruited nonsuicidal depressed patients cannot be generalized to depressed patients who are suicidal.

Enriched Samples Are Purposive Samples

Many acute phase RCTs use a placebo run-in phase as part of the design; patients who improve during this period are not randomized. Many maintenance phase RCTs use a maintenance treatment stabilization phase as part of the research design; patients who drop out or relapse during this period are not randomized. Such enriched samples are also examples of purposive samples; internal validity may be high, but external validity is low because of poor generalizability to patients in everyday practice.[2]

Recapitulation

Research is conducted on samples because it is rarely feasible or even necessary to study the entire population. However, because we want to draw conclusions about the population, and not just about the sample, the sample must be truly representative of the population. This is only possible if the sample is randomly drawn from the population. In a random sample, every member of the population has an equal chance of being selected. The greater the extent to which this criterion is violated, the less representative the sample is of the population, and the less is the external validity of the findings of the study.

Almost all research, including most research that claims to be population-based, is conducted on samples that are both convenience samples and purposive samples. The results of such research can only be generalized to the subpopulations with the characteristics that define and limit the convenience and purposive samples. As an additional concern, if such samples are not drawn at random from their respective subpopulations, then the research cannot be validly generalized to even the subpopulations, let alone to the entire population of interest.

Need for Convenience and Purposive Samples

Research based on convenience and purposive samples can be important and necessary, such as when sociocultural and other factors are expected to influence outcomes. Through convenience and especially purposive sampling, the findings relevant for subpopulations can be identified. In other words, there is nothing wrong with convenience and purposive sampling as long as readers are aware of the (sub)population to which the findings are relevant. In this context, readers may note that stress, support, nutrition, drug compliance, and a host of confounding variables could differ between different convenience and purposive samples, and could even influence response rates in psychopharmacology studies, making such samples necessary, but making generalization across subpopulations problematic.[1]

Parting Notes

Convenience and purposive samples are described as examples of nonprobability sampling.[3] A probability sample is one where the probability of selection of every member of the population is nonzero and is known in advance. So, strictly speaking, convenience and purposive samples that were randomly drawn from their subpopulation can indeed be probability samples if the findings are generalized only to the subpopulations from which they were drawn. They are nonprobability samples only if the results are sought to be generalized to the entire population.

Readers may find that convenience and purposive samples are defined in different ways in different reference sources. Usually, this is because research methods differ in different research disciplines.

Take-Home Message

If a study conducted on a convenience and purposive sample was methodologically sound, the internal validity would be good; but because the sample was both a convenience and purposive sample, the external validity would be limited by the restrictions defined by the convenience and purposive nature of the sample (generalization is possible only to the population from which the sample was drawn, and to those in the population who have the characteristics of the sample studied; the findings cannot be generalized to everybody).