Lost in translation: assessing effectiveness of focus group questioning techniques to develop improved translation of terminology used in HIV prevention clinical trials.

INTRODUCTION: Achieving participant comprehension has proven to be one of the most difficult, practical, and ethical challenges of HIV prevention clinical trials. It becomes even more challenging when local languages do not have equivalent scientific and technical vocabularies, rendering communication of scientific concepts in translated documents extremely difficult. Even when bilingual lexicons are developed, there is no guarantee that participants understand the terminology as translated.
METHODS: We conducted twelve focus groups with women of reproductive age in Mwanza, Tanzania to explore the effectiveness of four questioning techniques for: (1) assessing participants' familiarity with existing technical terms and concepts, (2) generating a list of acceptable technical and non-technical terms, (3) testing our definitions of technical terms, and (4) verifying participants' preferences for terms. Focus groups were transcribed, translated, and qualitatively analyzed. RESULTS AND DISCUSSION: A translation process that uses all four questioning techniques in a step-wise approach is an effective way to establish a baseline understanding of participants' familiarity with research terms, to develop and test translatable definitions, and to identify participants' preferred terminology for international HIV clinical research. This may help to ensure that important concepts are not "lost in translation." The results emphasize the importance of using a variety of techniques depending on the level of participant familiarity with research concepts, the existence of colloquial or technical terms in the target language, and the inherent complexity of the terms.

Introduction

Achieving participant comprehension has proven to be one of the most difficult, practical, and ethical challenges of HIV prevention clinical trials. Language can be a key barrier to participant comprehension due to the use of technical terminology which may be unknown or unfamiliar to populations who participate in international clinical trials [1].

Achieving comprehension becomes even more challenging when local languages do not have equivalent scientific and technical vocabularies, rendering communication of scientific concepts in translated documents extremely difficult [2]–[7] and resulting in miscomprehension, inaccuracies, and the inadvertent introduction of cultural beliefs [3]. To a large degree, HIV prevention research has been conducted in Sub-Saharan Africa, where HIV prevalence is the highest. Kiswahili dialects serve as the lingua franca for 40 to 100 million people in the region [8]. However, despite the large number of speakers, Kiswahili, like many African languages, lacks scientific technical terminology in many topic areas, making translation of technical documents problematic [5], [9]–[12].

Although processes for translating documents in clinical trials are not standardized [13], [14], they usually consist of a variation of the Brislin method – a general practice of forward and back translation [15]. This method involves several rounds of independent forward and back translation of documents from the source language to the target language, back to the source language, and back again to the target language until a satisfactory rendition is achieved. The main advantage of the Brislin method is that the checks and balances inherent in the multiple rounds of forward and back translation ensure that semantic incongruences between the two versions of a document are identified and resolved; this reduces the potential for translation to introduce inaccuracies into a document.

To address the need for consistent translation of key terms across different types of study documents, some clinical trial researchers have begun to add a lexicon development step to the Brislin-derived translation process [9]. A lexicon is a content-specific dictionary consisting of an alphabetized list of terms and their definitions. In this process, researchers develop study-specific English lexicons prior to translating study documents and then collaborate with local translators to translate and back translate the research terms. Once researchers and translators agree upon the translations developed during the iterative forward- and back-translation process, the bilingual translations in the lexicon are used consistently across study documents, such as informed consent documents, study instruments, and participant information sheets.

Using bilingual lexicons may lead to the consistent translation of study documents; however, the critical disadvantage is that it relies on the linguistic expertise and cultural experience of the translator rather than of the trial participant. A translator may not always have a nuanced understanding of language commonly used by the study participants [4], or may have an educational level higher than that of the study population [3]. There may also be a variety of people serving in the role of translators, including investigators, study staff, and professional translation companies. Because the backgrounds of different translators will inevitably vary, systematic use of the Brislin method to develop a bilingual lexicon may still result in translations that include terms which are unfamiliar or not commonly used by the study population, impeding participant comprehension [16] [9], [11]. Therefore, even when clinical research studies utilize bilingual lexicons, the risk remains that participants could misinterpret or not comprehend key terms essential to the clinical trial process.

We conducted research to assess the effectiveness of four focus group questioning techniques derived from field linguistics and social science research methods [16]–[18] for developing and verifying lexicons for use in HIV prevention clinical trials. The process consisted of a series of focus groups with members of future clinical trial study populations. Here, we describe our analysis of whether the questioning techniques that we adapted for the focus groups were effective for:

Learning participants' familiarity with existing technical terms

Generating a list of technical and non-technical terms known to participants

Testing our definitions of technical terms

Learning participants' preferred translations

Methods

Ethics Statement

This study was conducted by FHI360 (USA) and the National Insitute for Medical Research (NIMR) (Tanzania) according to the principles expressed in the Declaration of Helsinki, and was approved by the Protection of Human Subjects Committee (PHSC) of FHI 360 and the Tanzanian National Health Research Ethics Review Sub-Committee (NatHREC). Written informed consent was obtained from all study participants.

Study Design

We conducted two rounds of six focus groups with women from Mwanza, Tanzania in which we explored the effectiveness of four focus group questioning techniques in (1) assessing participants' familiarity with existing technical terms and concepts, (2) generating acceptable technical and non-technical terms, (3) testing our definitions of technical terms, and (4) verifying translation and term preferences.

Study Population

Tanzanian women from Mwanza were recruited from bars, guesthouses and similar venues using eligibility criteria analogous to previous HIV prevention clinical trials conducted with this population. Women were eligible to participate if they were between the ages of 18 and 45, had never participated in a clinical trial, and reported at least one vaginal sex act in the previous 14 days or more than one sexual partner in the previous 30 days.

Data Collection

Two rounds of data collection – the Development Phase and the Verification Phase – took place between October 2010 and May 2011. Six focus groups were conducted during each round of data collection. The number of focus groups was based on our assessment of what would be feasible to replicate in a clinical trial setting. Three questioning techniques were used in the first round, and a fourth technique was used in the second round only.

Development Phase

In the Development Phase, research staff created an English lexicon of technical terminology related to HIV prevention trials. The list was derived from lexicons from two HIV prevention trials [19], [20], as well as other guidance for the use of terminology in international HIV prevention research [21]. We selected 49 terms which we grouped into three broad topic areas: sexual behavior, reproductive health and infectious diseases, and research terms (see Table 1). For each term, we then developed English, plain-language definitions derived from the previous lexicons.

Table 1

Terms and questioning techniques, grouped by topic and phase.

TERM	TECHNIQUE
	Term Elicitation	Term Explanation	Definition Explanation	Verbal Multiple Choice
Abstain/ Abstinent	R1			R2
Anal sex	R1			R2
Benefits	R1			R2
Bisexual man	R1			R2
Blood test	R1*
Blood Draw	R1*
Casual partner	R1			R2
Cervix	R1*
Clinical trial		R1		R2
Concurrent sexual partner			R1	R2
Confidentiality	R1, R2
Effectiveness	R1, R2
Eligible	R1			R2
Enrol	R1			R2
Family planning method	R1			R2
Female sex partner	R1			R2
Genital wart/lesion	R1, R2
HIV test	R1*
HIV/AIDS	R1*
Informed consent			R1	R2
Injectables	R1
Intimate partner violence	R1, R2
Male condom	R1*
Male sex partner	R1			R2
Menstrual period	R1			R2
Microbicide	R1			R2
Monogamous	R1*
Oral contraceptives	R1*
Oral sex	R1			R2
Pap smear	R1			R2
Pelvic exam	R1			R2
Pre-exposure prophylaxis	R1			R2
Pregnancy test	R1*
Pregnant	R1*
Primary/Stable sexual partner	R1			R2
Randomization			R1	R2
Research study		R1		R2
Risks	R1			R2
Rounds [of sex]	R1			R2
Safe sex		R1		R2
Screening			R1	R2
Sexual intercourse	R1			R2
Sexually transmitted infection		R1
Side effects	R1			R2
Speculum	R1			R2
Transactional sex	R1			R2
Urine sample/test	R1*
Vaginal fluid	R1*
Voluntary	R1			R2

Term not included in Verification Phase.

A Tanzanian translator with expertise in translating research terminology was then identified. The local translator worked with local the FHI360/NIMR research team to generate translations of the lexicon terms and definitions in the dialect of Kiswahili spoken in Mwanza. For most words, the translator and Tanzanian principal investigator (PI) offered multiple translations that consisted of dictionary equivalents, words they considered most appropriate for a research context, and locally used words and expressions.

We developed three questioning techniques for use in the Development Phase focus groups. These techniques sought to (1) assess participants' familiarity with existing technical terms and concepts, (2) generate acceptable technical and non-technical terms, and (3) test our definitions of technical terms. Terms were matched with one or more techniques depending on the research objective (Table 1).

In cases where the local investigator and translator identified Kiswahili technical and non-technical equivalents, we used the Term Explanation technique, which is described below, to assess participants' familiarity with the suggested terms.

Term Explanation Technique: We asked participants to explain their understanding of translated terms to learn their familiarity with existing technical terms in Kiswahili. For example, for the term “sexually transmitted infection,” the following question and probes were developed:

I'd like to ask about sexually transmitted infections, sometimes also called sexually transmitted diseases (STIs). What is a sexually transmitted infection?

Probes:

How does a person become infected with a sexually transmitted infection?

What are the different kinds of sexually transmitted infections?

What are the symptoms of these infections?

For terms that the translator and Tanzanian PI identified as familiar concepts and for which there were no known local-language equivalents, we developed adefinition and used the Term Elicitation Technique, described below, to test the definition and generate a list of technical and non-technical terms.

Term Elicitation Technique: We presented participants with definitions of terms and asked them to provide terms that matched the definition. For some questions, scenarios were used to provide a context for the definitions. The technique was used to simultaneously test our definitions and generate a list of technical and non-technical Kiswahili terms known to participants. For example, for the term “ female sex partner,” the following question and probes were developed:

Next, I would like you to complete this sentence: When a man enters his penis into a woman's vagina, the woman is the man's ______________.

Probes:

To whom would you be talking when you say [insert elicited term]?

Does [insert elicited term] mean anything else? If so, what?

Are there any other terms you would use when you speak to a doctor? Your husband or boyfriend? Your sister? Your friends?

For terms, which were unfamiliar concepts and had no local language equivalents, we developed new definition and used the Definition Explanation Technique, described below, to test the definitions we created and generate a list of technical and non-technical terms.

Definition Explanation Technique: We provided an explanation of a term and then asked participants to explain it back to the group to learn whether our definition was easily understood. For example, we developed the following definition and question for the term “screening”:

In most cases, study staff will ask the person some questions to find out if the person has all the required characteristics on the list. The person may also need to have some laboratory tests to find out about her health. For the research study you are participating in today, we asked you some questions to make sure you had all of the characteristics necessary. Could someone explain to the group what that process was like? What were the characteristics necessary to be in today's research?

Focus groups were recorded and transcribed into Kiswahili by the moderators. The study translator, in consultation with the Tanzanian PI, then translated the Kiswahili transcripts into English. Researchers then reviewed the transcripts to assess whether participants understood the term or definition and to identify the elicited Kiswahili terms. We gauged comprehension of the term or definition by the similarity of terms elicited and participants' explanations of the term (Table 2).

Table 2

Criteria for evaluating effectiveness of the questioning techniques.

Technique	Objective	Effectiveness Criteria
Term Explanation	Assessing participants’ familiarity with existing technical terms	Were we able to ascertain, through an analysis of participant discussion, that participants were familiar with the technical/non-technical term?
		Did participants provide equivalent technical and non-technical terms?
		Did participants provide an explanation of the term that was technically correct?
Term Elicitation	Testing our definitions of technical terms and generating a list of technical and non-technical terms known to participants	Were we able to ascertain participants’ understanding of our translated definitions as well as participants’ familiarity with technical/non-technical terms?
		Were participants able to correctly answer questions about the definition content?
		Were participants able to correctly explain the term to the group after hearing the definition?
		Did participants provide equivalent technical and non-technical terms?
Definition Explanation	Testing our definitions of technical terms.	Were we able to ascertain participants’ understanding of our translated definitions as well as participants’ familiarity with technical/non-technical terms?
		Were participants able to correctly answer questions about the definition content?
		Were participants able to correctly explain the term to the group after hearing the definition?
Verbal Multiple Choice	Identifying participants’ preferred translations	Were we able to ascertain participants’ term preference?
		Did the questions get participants to identify a preferred term?
		Was there a consensus among the participants of the different focus group discussions?
		Did participants provide reasons for term preferences?

Verification Phase

In the Verification Phase, research staff sought to (1) determine participants' term preferences in cases where multiple term equivalents were elicited, and (2) assess participant familiarity with new term definitions in cases where the definition used in the Development Phase was unsuccessful.

For a subset of 13 terms, the techniques used in the Development Phase elicited a single term that was uniformly used by participants and matched the terms that the local translator and Tanzanian PI had suggested. These terms were not included in the Verification Phase (Table 1).

In four cases, techniques used in the Development Phase were re-employed in order to assess the new term definition. In this phase, we also used the fourth technique–Verbal Multiple Choice–for determining participants' term preferences. This technique, which is described below, was used for the 32 cases in which we elicited multiple technical and non technical terms.

Verbal Multiple Choice Technique: In order to identify term preferences, we asked participants to match our definition to the term that they liked best and that they felt most closely matched the original definition. We provided participants with term choices that included (1) words we had elicited in the Development Phase focus groups and (2) suggestions from the local translator and Tanzanian PI. Moderators read these choices aloud. We asked participants to vote on their favorite answer(s) and to explain the reasons for their selections. More than one answer was possible so that we could learn whether participants viewed multiple words as acceptable ways to communicate the definition. This technique was intended to learn participants' preferred terms in order to include them in the lexicon. For example, for the term “male sex partner,” the following question and probes were developed (translation notes included in brackets were not read to the participants):

The counselor asks the research participant, Limi, if a man has inserted his penis into her vagina during the past seven days. I am going to give you four choices of words that the counselor could use to refer to this act. Listen to all of the choices first. Then when I read them again, close your eyes and raise your hand for the best choices of words. Which of the following four choices would the counselor most likely use to refer to this act?

Kujamiiana [sexual intercourse – formal]

Kufanya mapenzi [sexual intercourse, vaginal sex – word used in research – translator's choice]

Kutiana [sexual intercourse – informal]

Kafanya tendo la ndoa [to have a sex act]

Probes:

Why do you think [most voted answer] is the best choice for a counselor to use?

Many of you also chose [next most voted answer]. Why do you think a counselor would use this word/phrase?

Moderators transcribed and translated the Verification Phase focus group discussions and the translator then reviewed the English transcripts. In real time, moderators also filled out a data extraction form documenting how many people had voted for each Verbal Multiple Choice answer.

Data Analysis

We developed criteria to evaluate the effectiveness of each questioning technique based on the objectives of each technique (Table 2), as the goal of data analysis was to evaluate the techniques' effectiveness rather than to conduct a thematic analysis. We then coded the transcripts using NVivo 9, a software program designed to assist in qualitative coding and analysis [22]. A two-step coding process was utilized. First, we applied codes representing the technique and specific term to each focus group transcript. We then individually analyzed segments of text and tagged them with codes that denoted the effectiveness of the technique (Table 2):

We assessed the Term Explanation technique's effectiveness for indicating participants' familiarity with existing terminology based on whether the questions got participants to provide meaningful explanations of the terms and technical and non-technical term equivalents.

We assessed the Definition Explanation and Term Elicitation techniques' effectiveness for eliciting a list of technical and non-technical terms known to participants and/or simultaneously evaluating the translatability of our definitions of terms based on whether the questions successfully generated equivalent terms and/or a correct explanation of the term from group members.

We evaluated the Verbal Multiple Choice technique's effectiveness in allowing us to identify participants' preferred translations based on whether participants identified a preferred term and whether a consensus was reached among all participants.

To identify trends in effectiveness among the different questioning techniques, we generated and reviewed code frequency reports and coded text reports.

Results

Sixty-one women participated in the 12 focus group discussions. There were an average of eight participants per focus group in the six focus groups of the first round –the Development Phase–for a total of 44 women, and an average of 10 women per focus group in the six focus groups of the second round–the Verification Phase.

Here, we describe the effectiveness of the four focus group questioning techniques, for use in developing and verifying lexicons for use in HIV prevention clinical trials, according to the criteria described in Table 2.

Effectiveness in assessing participants' familiarity with existing technical terms and concepts

The Term Explanation technique was used with four of the 49 terms for which the translator identified existing Kiswahili technical and non-technical equivalents. The Term Explanation technique to gauge participants' familiarity with existing Kiswahili technical terms. In all four cases (i.e., “clinical trial,” “research,” “safe sex,” and “sexually transmitted infection/STI”), participants were familiar with the term, as indicated by their ability to provide detailed and technically correct explanations and identify equivalent technical and non-technical terms.

For the term “sexually transmitted infection,” for example, participants provided extensive examples of symptoms, including “raised bumps with pus,” “rashes,” and “pain during urination.” Participants were also able to identify several specific STIs, including HIV, gonorrhea and syphilis. For the term “safe sex,” they explained their understanding of the widely used Kiswahili translation mapenzi salama as: “to have sex without causing bruising or scratching,” “to trust each other,” “honesty,” and “using protection during sexual intercourse.” For the two more abstract terms – “research” and “clinical trial” – our questions included probes in addition to the explanation of the terms. The term “research,” for example, included the following explanation:

Research is the process of carefully studying information to discover one or more facts. Let me give you an example. Scientists have developed a new drug. They want to know if it will cure malaria infection within one week. They must do research to learn the answer to this question. First they have to find 100 people with malaria who are willing to try the new drug. If it cures the malaria infections of most of the people within one week, scientists will learn the answer to the question.

We also included the following two probes: “Who does research? What is the purpose of research?” Reviewing the focus group discussion in response to the question and probes allowed us to ascertain the depth of participants' understanding of each term. It was evident from participants' initial responses that many participants had been exposed to these terms/concepts, as when one participant indicated that “research is to look for something, I mean to investigate on something until you get the solution.”

Effectiveness in generating a list of technical and non-technical terms and testing our definitions of technical terms

The Term Elicitation technique served the dual purposes of allowing us to test our definitions of technical terms identify equivalent technical and non-technical terms known to participants. Term Elicitation was used with 38 terms that the local investigator and translator identified as familiar concepts with no known Kiswahili technical or non-technical equivalents. In these cases, we translated the definitions and used the Term Elicitation technique to test the translated definition and generate a list of technical and non-technical terms. We assessed the technique's effectiveness by whether we were able to ascertain participants' understanding of our definitions and their familiarity with technical or non-technical terms (Table 2).

Overall, the Term Elicitation technique was used in 38 cases and was successful in all instances (38/38). Participants understood the definitions that were provided and during group discussion were able to identify complementary technical and non-technical terms. For example, as part of the discussion about the term “casual partner,” we provided a definition and several relationship scenarios and then asked participants to provide terms for the type of partner we had described. The discussion identified technical terms for a casual partner (mpenzi wa kupita), as well as local idioms and terms for this type of relationship such as sukuma (word for small tin, used to mean a secondary or casual partner), kibotorwa (word for small container, used to mean a secondary or causal partner), mchicha (a word for spinach which is used idiomatically to mean “secondary nourishment” or a casual partner).

Term Elicitation was also successful in indicating the lack of familiarity with a concept or a lack of equivalent terminology. In some cases, participants understood the definition they were provided but were not familiar with a corresponding term. For example, when provided with a picture and definition of genital warts, many women indicated that the picture showed a sexually transmitted infection. However, even though women were able to identify the ways in which genital warts differed from other STIs (i.e., raised bumps, warts) they were unable to provide a term that referred specific to this condition.” Even after a refinement of the definition for the Verification Phase, focus groups yielded similar results.

In other cases we defined procedures, objects, behaviors, or conditions which lacked equivalent terms in Kiswahili, such as “transactional sex,” “concurrent sexual partners,” “genital wart,” “pap smear,” and “pelvic exam.” The discussion indicated that participants understood the definition we provided but were not familiar with technical or non-technical terms to represent the concept. For some terms, such as “pap smear,” participants had no experiential frame of reference by which to identify equivalent terms. Participants had never undergone a pap smear, pelvic exam, or any similar gynecologic exam, thus they provided general terms such as “intensive check-up,” “pregnancy check-up,” or “cervix cleaning” that were based on their knowledge of medical procedures.

Term Elicitation was also successful in indicating when participants did not understand the definitions we provided. For the terms “confidentiality” and “effectiveness,” a technical definition was provided to participants and they were asked to provide an explanation and similar terms. The discussion indicated that participants only partially understood the definition, thus making it impossible to ascertain participants' familiarity with the concept. In the case of the term “effectiveness,” participants provided explanations such as “the medicine has been taken,” which did not include discussion of the extent to which a drug worked to prevent or treat a medical condition. Based on participant discussion, we then created a new definition that included a scenario in which a malaria treatment drug was tested and only found to cure a portion of the people who contracted the disease. The inclusion of a scenario with a familiar example facilitated participant discussion and allowed us to ascertain participants' understanding of the new definition as well as their familiarity with the technical concept and similar terms. Participants were able to produce non-technical phrases and descriptions to explain the concept, such as “successfulness of a drug,” “the drug treats,” and “the drug is reliable.”

We used the Definition Explanation technique with four of the 49 terms, which the local investigator and translator identified as unfamiliar concepts. For these terms we developed definitions and used the definition explanation technique to simultaneously test the definition. We assessed the technique's effectiveness by whether we were able to ascertain participants' understanding of our definitions (Table 2).

Overall, Definition Explanation was successful in all four cases. In two cases, “informed consent” and “screening,” participants were able to provide detailed explanations of the concepts, thus allowing us to ascertain their understanding of the definition. The definitions for these terms referenced participants' own experience enrolling in our study and gave participants a point of reference for their discussion of the concepts.

In the other two cases, “concurrent sexual partners” and “randomization,” the technique was effective in ascertaining that participants did not understand the definition we provided. When given a definition of the term “randomization,” participants had difficulty understanding how “random” assignment would take place, saying that they would “take a side,” “be appointed,” or “be selected.” In the case of “concurrent sexual partnerships,” participant discussion indicated that participants did not distinguish nuances in the definition related to partnership overlap. Participants provided terms that were used to identify someone involved in multiple, although not necessarily concurrent, sexual partnerships such as “prostitute,” “sex swindler,” and “player”

Effectiveness in identifying participants' preferred translations

The Verbal Multiple Choice technique was used exclusively in the Verification Phase to learn participants' preferences for terms. We evaluated the Verbal Multiple Choice technique's effectiveness based on whether participants were able to identify a term they preferred and on whether we were able to come to a consensus on a preferred term to use in the lexicon. We assigned the cutoff for a “consensus term” as a term that garnered more than half of the total votes across all focus groups and was at least 10 percentage points higher than the “runner up.” This cutoff was determined by reviewing the natural spread of points in the data. The Verbal Multiple Choice technique was used for 32 terms for which we had elicited multiple technical and non technical terms in the development phase; however, the responses for one term (“pap smear”) had to be discarded from the analysis due to incorrect translation of the question.

We found the technique to be overwhelmingly effective (28 of 31) for learning which terms participants preferred and for identifying instances in which more than one term was acceptable. Of the 28 cases in which the Verbal Multiple Choice technique worked, we found that there was a consensus on a preferred term in 24 cases. In some instances, terms elicited in the Development Phase were unpopular with the participants in the Verification Phase. For example, a picture of a speculum had as its choices: a technical translation, “duck's mouth” which had been suggested and agreed upon by participants in the Development Phase focus groups, and “cervix opener/expander,” a descriptive term the moderator had spontaneously proposed in the Development Phase focus groups to try to describe the unfamiliar object. Only two of the 31 participants chose “duck's mouth,” with the most popular answer being “cervix opener/expander.”

Although successful for identifying term preferences, the Verbal Multiple Choice technique was only moderately effective for learning why participants preferred certain terms. Some questions yielded useful information on usage, including which terms would be most widely understood, which terms were vulgar or not appropriate, and to some extent why some terms were not appropriate choices. However, for more than half of the terms, participants could not articulate why they felt a particular response worked better than another. For example, for the term “confidentiality,” almost three quarters of participants preferred the term “confidential,” whereas all others selected the term “privately.” In both cases, participants cited that their preferred term meant that individuals “do not share facts or details with anyone else” and “keep information secret.” However, when participants were specifically asked why the term that they had not selected was less appropriate, participants once again cited the reasons for their selection, rather than a rationale for its appropriateness.

Additionally, the Verbal Multiple Choice technique was ineffective in three cases (i.e., “concurrent sexual partners,” “randomization,” and “rounds of sex”). In these cases, we attempted to use a definition that had been revised following the Development Phase but not assessed using the Definition Explanation or Term Elicitation techniques. In all three cases, analysis of the discussion indicated that the participants did not understand the revised definition, and thus we were unable to ascertain term preferences.

Discussion

Regions of the world that bear the most disproportionate burden of infection, especially Sub-Saharan Africa, are frequently the context for HIV prevention trials. However, the successful and ethical conduct of these clinical trials rests on the ability of trial participants to clearly understand what they are being asked to do.

In this analysis, we sought to evaluate the effectiveness of four questioning techniques for gauging participants' familiarity with existing terms, generating a list of technical and non-technical terms known to participants, testing definitions of terms, and learning participants' preferences for translated terms. Our findings indicate that a translation process that uses all four questioning techniques in a step-wise approach is an effective way to establish a baseline understanding of participants' familiarity with research terms; develop and test translatable definitions; and identify preferred terminology for HIV clinical research. This could help ensure that concepts critical to participants' understanding of clinical trial procedures and their involvement in clinical trials are not “lost in translation.”

Furthermore, the results emphasize the importance of using a variety of techniques depending on the level of participant familiarity with research concepts, the existence of colloquial or technical terms in the target language, and the inherent complexity of the terms. The step-wise process we recommend is outlined below and diagrammed in Figure 1.

Figure 1

Recommended Guidelines for Using Techniques.

STEP 1: Gauge participant understanding of a term or existing definition

Definition Explanation was an effective first step for gauging participants understanding of a concept or plain-language definition. However, when participants were not able to explain the definition as part of this technique, it fell to the moderator to probe spontaneously in order to determine whether this was due to an inadequate definition or to participants' lack of familiarity with the concept. Therefore, focus group moderators need to be prepared for this possibility. Additionally, our findings suggest that even in cases where an existing technical term is thought to be well known among the study population, it is best to verify that participants are familiar with the term using Term Explanation prior to using it in a clinical trial setting.

STEP 2: Identify technical or non-technical terms that could be used to communicate the concept

Definition Explanation and Term Elicitation were both effective ways to elicit technical and non-technical terms that could be used to communicate a concept. Our results indicate that scenarios that referenced participants' own experiences enhanced the effectiveness of both of these techniques. We would also note that previous studies have highlighted participants' preferences for non-technical terminology [23]–[27].

STEP 3: Identify participants' preferred terms

As a final verification step, the Verbal Multiple Choice technique should be used to ascertain participants' word preferences and identify cases in which multiple terms are acceptable. Even though only one term might be used when applying the lexicon to clinical trial documents, including multiple verified terms as part of study staff training or staff scripts and job aids may help facilitate participant comprehension.

Limitations

Our study was preliminary in that it did not include an evaluation component that would have allowed us to determine whether the preferred translated terms identified by the study population, as well as our plain-language definitions for words that have no technical equivalent, led to improved comprehension when used in informed consent forms or other study materials. A critical next step is to assess participants' comprehension of informed consent forms that incorporate terminology and definitions elicited and/or verified in focus groups and compare them with participants' comprehension of informed consent forms prepared via the standard forward translation – back translation process. An assessment of the readability of the consent forms should also be part of this next research step.

The four questioning technique were evaluated with terms that are specific to HIV prevention research, but we believe that our findings may also apply to translation of materials used in other types of international clinical trials. Materials could include informed consent forms, but also quantitative data collection forms, qualitative instruments, study product adherence counseling, and community education.

In addition, the data collection methods we used were specifically chosen in an effort to identify processes for lexicon development within the context of HIV prevention clinical trials. Thus, this study sought to develop a process that could be integrated into existing research activities at clinical trial sites. Cross-cultural adaptation techniques, including cognitive interviewing and committee-based translation, which are utilized in other research settings for adapting study instruments, for example, may be able to address some of the limitations of the techniques described here, and should be explored.