Neuropsychological assessment of HIV-infected populations in international settings.

Resource-limited regions of the world represent the areas most affected by the global HIV epidemic. Currently, there are insufficient data on the neurocognitive effects of HIV in these areas and neuropsychological studies that have been carried out thus far are marked by inconsistent methods, test batteries, and rating systems for levels of cognitive impairment. These differences in methods, along with genetic variability of both virus and host, differences in co-infections and other co-morbidities, differences in language and culture, and infrastructural deficiencies in many international settings create challenges to the assessment of neurocognitive functioning and interpretation of neuropsychological data. Identifying neurocognitive impairment directly attributable to HIV, exploring relationships between HIV-associated neurocognitive impairment, disease variables, and everyday functioning, evaluating differences in HIV-1 subtype associated neuropathology, and determining implications for treatment remain complicated and challenging goals. Endeavors to establish a more standardized approach to neurocognitive assessments across international studies in addition to accumulating appropriate normative data that will allow more accurate rating of neuropsychological test performance will be crucial to future efforts attempting to achieve these goals.

Introduction

The establishment of highly active antiretroviral therapy as the mainstay of HIV treatment in developed nations has led to impressive reductions in the prevalence of severe HIV-associated neurocognitive disorders (HAND) and central nervous system (CNS) opportunistic infections. Accounts of HIV-associated dementia in these settings are now generally limited to patients who are either treatment naïve or are failing therapy due to viral drug resistance or problems with adherence. Milder forms of neurocognitive dysfunction, however, are still prevalent and continue to be under recognized in patients on antiretroviral therapy. Most of the studies to date investigating the action of highly active antiretroviral therapy at improving neurological and cognitive dysfunction have been carried out in the resource intense settings of the US, Europe, and Australia. Resource-limited communities in Sub-Saharan Africa, Asia, and the rest of the developing world, however, represent the areas most devastated by the HIV epidemic. These areas offer considerable potential for research and stand to gain the most from effective therapy.

Neuropsychological assessments are arguably the most important tools for diagnosing and categorizing HIV effects on the CNS. Especially in resource-limited settings, where sophisticated neuroimaging technology often is unavailable, characterization of neurocognitive functioning through neuropsychological assessments is crucial to successful diagnosis and treatment. When assessments are reliable and valid, and appropriate normative standards exist, they are quite sensitive to even milder forms of CNS compromise and also may provide valuable estimates of functional impairment. This article presents a review of the current status, as well as the potential, and some challenges to conducting neuropsychological assessments in resource-limited settings, with a focus on HIV-infected populations.

Global Epidemiology of HIV and HAND

HIV is a truly global disease, affecting roughly 33 million people all over the world. The number of people infected with HIV in the United States, Western Europe and Oceania however represent only 4% of worldwide infections (Hemelaar et al. 2006). Most of the people infected or affected by HIV live in developing countries where cultural values, social influences, educational opportunities and access to other resources are clearly distinct from those in the West. Africa and the Middle East account for over 66% of worldwide infections, Asia for over 20%, Eastern Europe and Central Asia for approximately 4%, and Latin America and the Caribbean for around 6% (Hemelaar et al. 2006).

In addition to the wide dispersion of HIV around the world, the rapid evolution of the virus itself has led to considerable genetic variation in a relatively short period of time. In West Central Africa, where the original cross-species transmissions are believed to have occurred (Gao et al. 1999), almost all of the nine major subtypes of HIV-1 Group M (A-D, F-H, J, and K), as well as strains of HIV-1 Groups N and O, and HIV-2 can be found. In other parts of Africa and other regions of the world however (Fig. 1), certain subtypes and recombinant forms such as CRF01_AE and CRF02_AG predominate over others (Hemelaar et al. 2006). The extensive genetic diversity that characterizes HIV along with the geographic compartmentalization of viral species raises interesting and challenging questions in regards to associated differences in disease progression (systemic and neurological), effectiveness of antiretroviral therapy, and the outlook of the constantly evolving pandemic.

Fig. 1

Regional distribution of major HIV-1 Subtypes. *Map of regional distribution of major HIV-1 subtypes borrowed from cover of August 2007 issue of Journal of NeuroVirology, used with permission from the publisher. (Liner et al. 2007)

Neuropsychology of HIV Infection

Neuropathology

HIV-1 enters the CNS early during the course of infection (An et al. 1999; Davis et al. 1992) and frequently results in neurological disease marked by a set of cognitive, motor, and behavioral symptoms (Chiodi et al. 1992; Navia et al. 1986b). The significant loss of neurons and neuronal processes (e.g. dendritic complexity) in people dying of AIDS clearly correlates with ante-mortem neurocognitive impairment (Masliah et al. 1992, 1997; Mattson et al. 2005). Cells primarily infected by HIV within the CNS are blood-derived macrophages, resident microglia, and perhaps astrocytes, but most studies suggest that neurons are not directly infected (Epstein and Gendelman 1993; Kure et al. 1990; Takahashi et al. 1996; Trillo-Pazos et al. 2003). The neuronal damage that occurs is likely caused by shed viral proteins such as gp120 (Dreyer et al. 1990; Lannuzel et al. 1995) and Tat (Behnisch et al. 2004; Jones et al. 1998; Maragos et al. 2003; Nath et al. 1999) or indirectly through the elevated production of neurotoxic molecules released by activated astrocytes (Levi et al. 1993; Merrill et al. 1992; Mollace et al. 1993; Nath et al. 1999; Patton et al. 2000), macrophages (Gendelman et al. 1994; Levi et al. 1993; Merrill et al. 1992; Nath et al. 1999), and microglia (Gendelman et al. 1994; Jones et al. 1998; Kramer-Hammerle et al. 2005; Levi et al. 1993; Mattson et al. 2005; Nath et al. 1999). HIV infection in the brain has widespread and variable effects but appears to preferentially cause damage to the basal ganglia and deep white matter (Navia et al. 1986a). However, damage to cortical and subcortical neurons (hippocampus and putamen) (Archibald et al. 2004; Everall et al. 1999; Moore et al. 2006), particularly dendritic pathology (Masliah et al. 1997) also are likely to play a role in CNS disease manifestations.

Nomenclature

The nomenclature of HIV related neurological diagnoses was recently revised and updated (Antinori et al. 2007), but there are a number of terms used in the existing literature worth discussing for purposes of clarity. Navia et al. initially recommended the term AIDS Dementia Complex to label the severe neurocognitive loss and neurological dysfunction associated with advanced immunodeficiency (Navia et al. 1986b). Later the American Academy of Neurology suggested HIV Associated Dementia as a better label (AAN 1991). AIDS Dementia Complex and HIV Associated Dementia have since been used interchangeably. Patients with some degree of cognitive impairment but whom did not meet criteria for dementia were classified as having Minor Cognitive Motor Disorder in the AAN nomenclature although, especially in the neurological literature, milder forms of neurocognitive disturbance sometimes have been classified as a lower stage of “dementia” (e.g. Memorial Sloan-Kettering (MSK) scale for dementia: 1-mild, 2-moderate, 3-severe, 4-end stage, mute) (Price and Sidtis 1990).

With true HIV Associated Dementia, patient profiles were marked by severe behavioral changes, attention and executive dysfunction, psychomotor slowing, and memory impairment (Bornstein et al. 1993; Stern et al. 2001). Minor Cognitive Motor Disorder patient profiles were characterized by impaired cognitive and motor speed, working memory, and new learning, but most aspects of language (except fluency) and long term memory (semantic) were relatively unimpaired. As the scope of cognitive dysfunction in HIV infection has become more evident, it has been suggested that for research and epidemiological purposes HIV-Associated Neurocognitive Disorders (HAND) be used as a more comprehensive title, with three broad subdivisions depending on the degree of cognitive impairment and associated changes in everyday functioning: Asymptomatic Neurocognitive Impairment, HIV-associated Mild Neurocognitive Disorder, and HIV-Associated Dementia. Asymptomatic Neurocognitive Impairment is defined by performance at least 1 standard deviation (SD) below the mean of demographically adjusted normative scores in at least two cognitive areas (attention-working memory, speed of information processing, language, abstraction-executive, complex perceptual motor skills, memory, including learning and recall, simple motor skills or sensory perceptual abilities), but without any apparent changes in activities of daily living. Mild Neurocognitive Disorder, previously referred to as Minor Cognitive and Motor Disorder, features the same test performance criteria as above, but with notable changes (at least mild) in activities of daily living. HIV-Associated Dementia requires performance of at least 2 SD below demographically corrected normative means in at least two different cognitive areas, as well as marked difficulty in activities of daily living due to cognitive impairment (Antinori et al. 2007). Diagnosis of all three forms of HAND also requires a determination that the observed neurocognitive impairment and/or functional disturbance cannot be explained by co-morbid (non-HIV related) conditions.

Now that the severe forms of HAND are far less common, this classification system has been geared more towards mild and even asymptomatic impairment. As a consequence, the new system may give better estimates of neurocognitive disorders than AIDS Dementia Complex and Memorial Sloan-Kettering ratings which have caused some confusion by using the term “dementia” to describe impairment on a number of levels. The AIDS Dementia Complex and Memorial Sloan-Kettering ratings were initially proposed at a time when it was not clear that the milder and more severe forms may be separate entities and that the milder forms do not necessarily progress to HIV-associated dementia. This updated classification system, based on both neuropsychological test performance and effects on activities of daily living, also represents a positive step towards more consistent comparison of individuals and groups across studies and cultures. We must keep in mind however that the use of specific neuropsychological tests and screening instruments still varies significantly from study to study (Butters et al. 1990; Clifford et al. 2007; Cysique et al. 2007b; Heaton et al. 1995; Maj et al. 1994; Robertson et al. 2007a, 2008; N. C. Sacktor et al. 2005; Wilkie et al. 2004; Woods et al. 2006; Yepthomi et al. 2006)

Patterns of Neurocognitive Dysfunction

HIV infection characteristically generates a “sub-cortical” pattern of neurocognitive dysfunction with deficits predominantly affecting executive functions, speed of information processing, attention/working memory, motor speed, new learning and retrieval of new information, while long term (semantic) memory, many language skills, and visuo-spatial abilities may remain intact (Dawes et al. 2008; Grant et al. 1987; Heaton et al. 1995). This average pattern of neuropsychological impairment, reported by numerous studies before and after the advent of highly active antiretroviral therapy, has shaped the development of current and future test batteries used across the US and internationally.

The pattern of neurocognitive dysfunction, however, is not consistent across individuals (Dawes et al. 2008) and may be even less consistent across individuals from markedly different backgrounds. Individuals exhibit considerable variation in strength and weakness of ability domains such as verbal memory, visual memory, processing speed, attention/working memory, executive function, and motor skills. Some may exhibit strong motor skills with weak executive functions and verbal memory, some may retain processing speed but show decreased visual memory and executive functions, and still others may feature strong memory but weak motor skills (Dawes et al. 2008). Test batteries used in international settings tend to be limited to fewer of the ability domains above, and to have a fixed focus on effects of fronto-striatal and white matter involvement. As such, a battery focused on motor skills and verbal learning may miss significant impairment in processing speed and attention, or vice versa. Differences in patterns of dysfunction may be due to differences in HIV neuropathology and co-morbid conditions. Distinct patterns of co-morbidities, such as higher rates of intravenous drug use in the US compared to high prevalence of Hepatitis C in rural China (Heaton et al. 2008) may have varying effects on the CNS and performance on neuropsychological testing. In some cases the impact of CNS opportunistic infections or prior injury or illness may entirely preclude the detection of any direct effects of HIV on the nervous system. Determining the biological and environmental sources of these different patterns of impairment, and how they affect activities of daily living, remain important goals of HIV research in both the US and internationally. In addition, any evolving changes in neurological outcomes may be missed by prior and even ongoing studies that use focused test batteries tailored to fronto-striatal and white matter involvement. For example, if older people are more represented in a study population, more “cortical” patterns may occur, but may be missed by assessment focused on sub-cortical dysfunction. Extensive batteries frequently used for neuropsychological assessments in the US may cover a wide range of abilities, but may not be an option for studies in resource-limited settings due to limitations in time, local expertise, and availability of valid test instruments and norms. It should be kept in mind however that, especially in this evolving epidemic, we cannot assume that neurocognitive abilities not assessed by the test battery being used are unaffected. Unfortunately, even if we change (expand) our batteries now, at the expense of considerable time, money and resistance from other colleagues in the field, there will be little previous data to compare with the new findings.

Aims of Neuropsychological Assessments

Some of the major goals of neuropsychological evaluations in HIV-infected populations include:

Finding neurocognitive impairment directly attributable to HIV

Determining if neurocognitive impairment is associated with co-morbid factors such as psychiatric illness, nutritional deficiencies, or co-infections

Exploring relationships between neurocognitive impairment and HIV disease variables such as history of immunodeficiency (current and nadir CD4 count), viral load, biomarkers of HIV neuropathogenesis, neuroimaging, and brain pathology

Exploring the relationship between HIV-associated neurocognitive impairment and everyday functioning within different populations around the world

Determining implications for treatment including adherence and use of CNS penetrating antiretroviral regimens

Determining when to start treatment to protect the CNS from damage and promote continued quality of life/productivity over the lifespan.

and Providing feedback to patients and clinicians on progress of disease and treatment effects.

Determining the relative impact of HIV and co-morbid factors on neurocognitive impairment is an important but very complicated enterprise. In most areas of the world it is rare to have HIV infected people without co-morbidities. A recent effort to examine the prevalence and nature of HAND in association with disease and treatment in a large unselected US sample found that fewer than 10% of patients were considered to have no co-morbidities and in fact most had 2 to 3 problems (Heaton et al. 2009). Clinicians and investigators must consider the advantages and disadvantages of screening and excluding co-morbidities (some or all) in studies of HIV-infected populations. Excluding co-morbidities, especially in resource-limited settings like sub-Saharan Africa, renders the sample less representative of the broader HIV-infected population, but engenders more confidence that any impairment discovered is directly due to HIV. In addition, the prevalence and impact of various co-morbidities may vary across international settings. TB, malaria, syphilis, Hepatitis C, and malnutrition for example, are much bigger problems in the developing world than in the US and other developed countries, and the impact of each can vary across regions. Where the neurological effects of opportunistic infections preclude determining the direct effects of HIV, researchers and clinicians should attempt to clearly distinguish patients with significant CNS opportunistic disease from those with HIV-associated neurocognitive disorders.

Another major problem is that clinicians have not been consistent or reliable in how they rate co-morbid factors (singly and in combination). The Frascati report (Antinori et al. 2007) attempted to provide better guidelines for classifying co-morbidities by specifying and giving rules for three levels of co-morbidity. In order to better guide “clinical judgment” about this important issue, however, there needs to be more research in both Western and non-Western regions regarding the neuropsychological impacts of different co-morbidities within the context of HIV infection. For example, determining how certain co-morbidities affect rates of neurocognitive impairment and the likelihood of being able to show relationships between neurocognitive impairment and HIV disease variables (e.g. nadir CD4, viral loads, treatment effects, biomarkers reflecting inflammation, etc.) are two areas of interest. Infectious Disease specialists may not be expert in all the conditions of importance in making co-morbidity classifications (e.g. psychiatric disorders, depression, and developmental conditions such as learning problems), but involving experts relating to every possible area of concern is not a practical option for most international HIV studies. People have assumed that depression can have a much more robust effect on neuropsychological impairment than the available data would support, and past histories of substance use disorders may not be as important as many clinicians assume (Heaton et al. 1995). Cysique et al. (2007a) reported that even incident major depression did not affect neuropsychological function in an ambulatory group of HIV-infected men (Cysique et al. 2007a). Goggin et al. (1997) did not find significant differences in global neuropsychological impairment between groups of depressed and non-depressed HIV-infected individuals and did not observe a relationship between severity of depression and neurocognitive impairment (Goggin et al. 1997). Authors reported that these findings may be the result of HIV disease overriding the effects of depression, but this does not explain normal cognition in many HIV-infected persons who are also depressed (Cysique et al. 2007a; Goggin et al. 1997). Possibly the biggest problem facing assessment of the impact of individual co-morbid conditions is the reality that in the world of clinical practice, one rarely sees people with just a single co-morbid issue, and gathering meaningful data on the significance of each issue may be difficult.

Exploring the relationships between neurocognitive impairment and everyday functioning presents another complex area of investigation. Clinicians must rely primarily on patient self-reports which may be influenced by depression, degree of insight, and complexity of everyday activities. The same neuropsychological deficits may have very different everyday consequences for different people and even different international populations with more, less or just different requirements for specific cognitive abilities in their everyday lives. For example, the exact same neuropsychological pattern which yields a Mild Neurocognitive Disorder or HIV-Associated Dementia classification in most areas of the US may not affect everyday functioning in Uganda or rural China. Even the same person with the exact same neurocognitive impairment may have no problems functioning in the rural area but may be seriously handicapped if he travels to the city for work. While depression symptoms can account for significant variance in self-reported neurocognitive complaints (Rourke et al. 1999), one should not entirely dismiss subjective cognitive complaints related to everyday function (Sadek et al. 2007). Biases or subjectivity involved in self-reports may be part of the issue in determining the impact of depression and other co-morbidities on everyday life in each setting, but the actual degree or severity of depression or other co-morbid condition may be more important. When classification systems factor in surveys of activities of daily living, seemingly different disease entities may end up with the same label and make comparing classifications across different populations very difficult. A positive step in this area was the Frascati conference consensus to acknowledge as a potentially significant condition, so-called “Asymptomatic Neurocognitive Impairment,” to classify individuals with HAND if they have deficits on neuropsychological testing but no reported difficulties with activities of daily living (Antinori et al. 2007; Cherner et al. 2007). As this classification gains more widespread use, research is needed to clarify the relative long term importance (especially biological significance, if any) of the distinction between “asymptomatic” versus symptomatic conditions. For now, maintaining a primary focus on impairment itself, based on objective neuropsychological test performance, may be more reliable and valid than including activities of daily living, especially for comparison across different international settings.

HIV-Associated Neurocognitive Disorders in Developed and Developing Settings

Prior to the era of highly active antiretroviral therapy, the cumulative risk of developing HIV associated dementia was estimated to be between 15–20% (McArthur et al. 1993). Incidence of HIV dementia in the MACS cohort was estimated to decrease by 53% from 21.1 per 1,000 person-years between 1990 to 1992, to 10.5 per 1,000 person-years between 1996 to 1998 (Sacktor et al. 2001). Although definitions used vary from the current HAND rubric, these estimates illustrate the decrease in the incidence of dementia with the introduction of highly active antiretroviral therapy, and the seemingly paradoxical finding of increased prevalence of dementia with patients surviving longer. The clinical spectrum of the disease has shifted from the severe and devastating form of dementia commonly encountered in association with advanced AIDS (typically end-stage disease) before the introduction of protease inhibitors and highly active antiretroviral therapy, to the milder and more manageable forms of HAND. More recent studies conducted in patients on highly active antiretroviral therapy, estimate that the prevalence of neurocognitive dysfunction (based on neuropsychological assessments) in HIV populations ranges from 20–37%, even with treatment (Robertson et al. 2007b; Sacktor et al. 2001; Sacktor et al. 2002). The CHARTER study group recently presented findings from comprehensive neuropsychological evaluations (lasting 2 to 2.5 h) on a large unselected population of 1,555 HIV-positive patients and reported that, overall, 45% of the cohort had neurocognitive impairment based on a global neuropsychological rating, although prevalence of impairment also varied considerably based upon levels of co-morbidity (Heaton et al. 2009).

Clinical accounts of sub-acute or progressive cognitive and motor decline are now uncommon and may be limited primarily to treatment naïve patients, patients with adherence issues, or those experiencing treatment failures (Liner et al. 2008; Price and Spudich 2008). Possibly the most remarkable outcome following the introduction of PI and combination therapy was the dramatic decrease in CNS opportunistic diseases such as cryptococcal meningitis, cerebral toxoplasmosis, primary CNS lymphoma, and progressive multifocal leukoencephalopathy (d’Arminio Monforte et al. 2004). In resource-limited settings these CNS opportunistic diseases as well CNS infection by Mycobacterium tuberculosis, Plasmodium species, and other pathogens, remain common. Major deficiencies in diagnostic technologies used to rule out these diseases confound diagnosis and management of CNS disease in these settings.

Reports of the prevalence and presentation of HIV-associated neurocognitive disorders have demonstrated remarkable variability across international studies and at present a very limited amount of data is available on HAND in resource-limited settings. For example, the prevalence of HIV-Associated Dementia in sub-Saharan Africa has been reported to be from as low as 3% (Belec et al. 1989) to as high as 54% (Howlett et al. 1989), at least in part to due differences in definition and ascertainment methods. In one of the earliest international studies of HIV-associated cognitive impairment, the World Health Organization found fairly consistent rates of impairment on a relatively small test battery, of 19.1% (Zaire), 15.3% (Kenya), 18.4% (Thailand), and 13.0% (Brazil), and more substantial neurological impairment rates of approximately 41% (Zaire), 40% (Kenya), 66% (Thailand), and 54% (Brazil) in symptomatic individuals (Maj et al. 1994). Clifford et al. (2007) report that the International HIV Dementia Scale, a brief screening tool, did not detect any significant differences in cognitive status between HIV positive and negative subjects in Ethiopia consistent with clinical impression (Clifford et al. 2007). Contrastingly, studies in India, China, and Uganda reported prevalence rates of 56%, 34%, and 31% respectively (Heaton et al. 2008; Robertson et al. 2007a; Yepthomi et al. 2006). HIV-1 subtype C predominates in India, subtype B and CRF01_AE in China, and subtypes A and D in Uganda. These discrepancies could be the result of varying neurovirulence of viral subtypes, different environmental factors, or a consequence of underappreciated cultural nuances and differences in the neuropsychological methods used in the different studies.

In terms of the patterns of neuropsychological effects observed across different countries, a pilot study investigating the neurobehavioral effects of HIV-1 infection in China reported a pattern of deficits in abstraction/executive function, information processing speed, and learning consistent with Western studies (Cysique et al. 2007b). This study found no significant country effects on the global neuropsychological score or measures of executive function, attention, learning, memory, or motor functions, although significant country effects on tasks of verbal fluency and speed of processing were reported. In addition, this study found that moderately high levels of depression did not account for neuropsychological performance in either the US or Chinese HIV positive groups, and that neuropsychological performance did correlate with complaints of cognitive difficulties in everyday life as well as with unemployment status.

In a larger study of former plasma donors in rural China, Heaton et al. (2008) administered the same international test battery to 203 HIV+ and 198 HIV− adults who were mostly farmers (mean education = 5.6 years) (Heaton et al. 2008). Results of uninfected controls were used to create demographically corrected neuropsychological norms, which classified 37% of the HIV+ group as impaired. The normed test results were sensitive to both HIV and Hepatitis C virus (HCV) effects, as well as HIV disease severity (AIDS status and history of severe immunosuppression). Finally, participants classified as impaired reported more cognitive difficulties in their everyday lives and decreased independence in performing instrumental activities of daily living (such as financial management, shopping, housekeeping, and cooking).

Another study evaluating the pattern of neuropsychological performance in a sample of HIV positive patients and HIV negative control subjects in Uganda revealed significant group differences on measures of verbal learning and memory, speed of processing, attention and executive functioning (Robertson et al. 2007a). Gupta et al. (2007) compared a sample of 119 adults in India infected with HIV-1 subtype C who were not on antiretroviral therapy, with normative data derived from an Indian sample of 540 healthy volunteers (with comparable gender distribution, age, and education) and with a matched cohort of 126 healthy, HIV-1-seronegative individuals (Gupta et al. 2007). They found a high rate (60.5%) of mild to moderate cognitive deficits in the HIV patients but no evidence of true dementia. The neuropsychological profile was characterized by deficits in fluency, working memory, and learning and memory, once again similar to patterns that have been observed in the West.

A study reporting HIV-1 subtype-associated differences in neurological disease was recently reported by Sacktor et al. (2007), in a small Ugandan cohort. These investigators found that subtype D was associated with higher dementia prevalence than those with subtype A at similar disease stage (Sacktor et al. 2007). Subtype D has also been reported to be associated with faster progression of systemic HIV disease (Kaleebu et al. 2002; Laeyendecker et al. 2006). Robertson et al. (2008) reported substantial differences in baseline neurocognitive test means across seven resource limited countries (Malawi, South Africa, Zimbabwe, Thailand, India, Peru and Brazil) between unmatched groups of patients with low levels of co-morbid conditions (Robertson et al. 2008). In the first randomized clinical trial observing neurocognitive effects of antiretroviral treatment in treatment naïve patients from multiple resource limited settings, Robertson et al. (2009) found substantial improvement across multiple time points of follow up from week 24 out to week 96 across seven RLS countries (Robertson et al. 2009). The analysis was limited to 293 participants who were randomized to treatment with didanosine enteric-coated (ddI) + emtricitabine (FTC) + atazanavir (ATV) in the AIDS Clinical Trials Group (ACTG) Study A5175 (PEARLS), and did not include groups on alternate treatment regimens or untreated control groups for comparison. Significant improvements in neuropsychological functioning after initiating antiretroviral therapy were determined after controlling for baseline function, age, sex, country, CD4, plasma HIV-1 RNA stratum, and years of education. Notably, the magnitude of improvement in neurocognitive functioning varied across the countries and could not be explained by systemic disease factors. Improved neurocognitive functioning may be due to control of HIV viral load through antiretroviral effects, uncontrolled practice effects on repeated test administrations, or both and demonstrate the need for further normative comparison data collection (including norms for change that consider practice effects and normal test-retest variability) in resource limited settings.

The studies above clearly demonstrate that HIV affects the CNS and that existing neuropsychological instruments can detect rather similar patterns regardless of country, culture, or HIV-1 clade. Determining the relative impact of these variables (country, culture, and clade) on severity of neurological disease and neuropsychological impairment however is not possible with available studies, due to differences in inclusion/exclusion criteria, test instruments and normative standards utilized, and systems used to classify impairment as mild, moderate, or severe. Another limitation of many previous studies of HAND in international settings has been the lack of control or untreated comparison groups. Investigators with limited time and resources often have declined to enroll these groups for comparison with infected and treated populations, but including these groups in future longitudinal studies (as well as cross-sectional) will add greatly to the interpretability and scientific value of the results.

The Frascati guidelines (Antinori et al. 2007) described above are a step in the right direction in terms of classifying impairment with regard to neuropsychological performance, everyday functioning, and presence of co-morbid conditions that may cause or contribute to neurocognitive impairment, but much work still needs to be done to establish greater consistency in instruments and other methods used across studies in order to approach accurate cross population comparisons. Infrastructural deficiencies and differences in assessment and diagnostic methods across these studies obscure the interpretation of results and impede international neuroAIDS research in general. Determining which tests have the broadest international applicability and what factors (cultural, educational, linguistic, etc) affect the generalizability of norms will be especially important to progress in this field.

Challenges to Conducting Assessments in Resource-Limited Settings

Some of the challenges to conducting assessments in resource-limited settings include: the overwhelming disease burden for clinicians in these settings, geographic factors and infrastructural deficiencies, and a lack of neurological and neuropsychological expertise. As an example of the disease burden, an estimated 25 million people in sub-Saharan Africa are infected with HIV and in areas of some countries prevalence of HIV infection exceeds 30% (WHO/UNAIDS 2008). The staggering levels of HIV disease create an immense load on healthcare systems in these regions that are also wrought with tuberculosis, malaria, and other diseases. These and other commonly encountered diseases which have their own CNS effects not only create problems for studies trying to tease out the effects of HIV on the nervous system, but can also overshadow the importance of neuropsychological studies in general. When so many people are in need of treatment for the direct effects and symptoms of disease, justifying studies without some direct benefit to patients can be difficult, especially under conditions of extremely limited funding.

Hospitals and clinics in developing countries tend to be available only in large cities and towns, forcing people to travel long distances for medical care or participation in studies. Deficiencies in paved roads and access to transportation mean that people frequently travel on foot to clinics in town. For longitudinal studies, this can have an impact on both study drop-out and accrual. People in many settings must weigh the potential benefits of temporary access to free medication and treatment against the effort to get to clinics and the expense of losing a valuable day of work.

The lack of neurological and neuropsychological expertise in many areas requires these specialists to either be brought in from other countries to conduct studies themselves, or to train local physicians and nurses to perform the neurological examinations and neuropsychological assessments. In order for neurocognitive tests and neurological evaluations to be effective research tools, and to promote valid and reliable assessments, administration of assessments must be standardized and remain consistent across sites and examiners. To achieve this goal, formal training of examiners and ongoing review of assessments and training procedures is required. This could include formal training of examiners at study sites, access to training videos and slides for the neurological exam and individual neuropsychological test instruments, and a set of on-line quizzes to certify examiners. Planned conference calls with investigators and team members from each site, as well as periodic visits to the sites by the primary investigators could be carried out for quality assurance. Taking these measures helps to ensure the quality of assessments and data and helps to ensure that examiner drift does not occur over time. Building capacity and infrastructure, and providing training to transfer neurological and neuropsychological expertise to healthcare providers in underdeveloped areas undoubtedly will yeild benefits well beyond HIV research, and will likely improve healthcare as a whole in these areas.

Challenges to Interpreting Assessments in International Settings

The challenges to interpreting neuropsychological assessments in international settings arguably are more difficult than those impeding the performance of assessments and conducting studies in general. For example, although psychometric properties of Western tests generally have been well established in the US and other Western countries, little is known about the reliability and validity of such instruments in the rest of the world. The validity of an assessment refers to whether or not the test, question, or skill at hand has a common, shared meaning or existence in the minds of both test-maker and test-taker. The reliability of an assessment refers to whether or not the results on a test or questionnaire (depression screen for example) remain consistent for an individual (or group?), either within an administration or over separate administrations. Some of the challenges to the validity and reliability of neuropsychological assessments in international settings include determining which skills are not “pan-human” (and therefore which tests of the skills may not be valid in a new setting), translating and adapting tests to be appropriate for a different language and culture, and gathering adequate normative and control data for the patient population in the new setting. In light of the variable pattern of neurocognitive impairment across HIV-infected individuals, group mean comparisons both across and within populations may not be sufficient. For many purposes (research as well as clinical) we need appropriate norms to accurately classify individuals.

The majority of the cognitive and motor skills targeted by neuropsychological assessments probably can be considered “pan-human,” but different cultural/educational backgrounds likely emphasize or de-emphasize specific abilities. Studies of normal populations from different cultures have revealed differences in cognitive abilities between ethnic groups on a number of standardized tests (including IQ tests, tests of learning efficiency and problem solving, etc.). The WHO-Neurobehavioral Core Test Battery (WHO-NCTB) study, for example, showed significant differences in performance on seven neurobehavioral tests (Digit Symbol, Digit Span, Benton visual memory test/recognition form, Santa Ana dexterity test, Simple Reaction Time, Pursuit Aiming II, and Profile of Mood States) between Western (North American and European) populations and people from cultures significantly different from North America and Europe (Anger et al. 2000). Other studies have observed performance differences across cultures in measures of language (Llorente et al. 2004), verbal memory and fluency (Razani et al. 2007a), simple attention (Anger et al. 1993), simple visual attention (Paul et al. 2007), information processing speed (Anger et al. 1993; Llorente et al. 2004), timed and sustained performance tasks (Agranovich and Puente 2007; Anger et al. 1993; Byrd et al. 2004; Chung et al. 2003), spatial based tests (Holding et al. 2004; Jahoda 1979; Mayes et al. 1988; Rosselli and Ardila 2003), digit based tests (Hedden et al. 2002), and even perception (Nisbett and Miyamoto 2005). On the other hand, similarities in performance on tests of non-verbal skills (Hsieh and Tori 2007; Razani et al. 2007b), simple reaction time (Anger et al. 1993), simple attention (Paul et al. 2007), visual retention (Anger et al. 1993), counting and summation (Hsieh and Tori 2007), visuo-spatial measures of speed of processing (Hedden et al. 2002), and motor based skills (Parsons et al. 2007) also have been reported. At the present it is difficult to determine whether group differences, if they exist, suggest that the test may not be valid or only suggest that different norms are needed. Groups of US patients in their 20 s and 80 s for example would perform differently on most tests, but this does not mean the tests are invalid for either group.

The concept of “Test-wiseness” incorporates important test taking skills acquired through incidental learning rather than being taught directly in educational environments. These skills are most powerfully acquired through experience in formal educational systems (Rosselli and Ardila 2003). Success on neuropsychological tests depends on many behaviors valued in classroom settings (sit still and be quiet, pay attention, use pen and pencil, take notes on verbal instruction, copy designs, solve problems, and work quickly to finish tests and keep up with class). In developed countries, a high school education (10–12 years) usually provides enough experience for well-practiced test-taking skills, but the same may not be true for developing countries. Schools in resource-limited settings are often under funded and have high pupil-teacher ratios. In areas where teachers are overburdened, school supplies are scarce, and testing practices are less rigorous, the Western phenomenon of “test-wiseness” may not be fully developed in most people. For example, the Trail Making Test B is easily performed by people from Western societies where most have at least 10–13 years education, but it may be too difficult for semi-literate populations or non-western societies with less experience working with numbers and letters (Mitrushina 2005; Spreen 1991). The Trail Making Test B measures task switching and executive functions by asking the test-taker to connect randomly placed numbers and letters in order while alternating back and forth between the two sequences. Although semi-literate people may be able to accurately recite numbers and the alphabet, they usually retrieve the sequences much more slowly and this effect may overshadow the so-called executive requirements that the test was designed to emphasize. In addition to these differences in educational environments, most students in Western educational systems are encouraged to “work as quickly and accurately as you can.” These seemingly contradictory demands reflect the philosophy of Western cultures, where assessment of cognition usually is associated with both speed and accuracy. In other cultures, however, being smart means being cautious and thoughtful which often results in slower, more deliberate test-taking. Slower test taking could result in the false attribution of cognitive symptoms if compared to Western normative data. Several authors have confirmed this finding that members of many cultures frequently perform slower in timed tasks compared with US populations. Zairian children performed more slowly on the Tactual Performance Test than American and Canadian children in one study (Boivin et al. 1995). Razani et al. (2007a) published findings that a normal group of monolingual English-speaking Anglo-Americans significantly outperformed an ethnically diverse normal group on the Trail Making Test part B as well as the Stroop Tests B and C (Razani et al. 2007a). In a comparison of US and Russian normal adults, Agranovich and Puente (2007) reported a significant effect of culture on the timed tests Color Trails Test and Ruff Figural Fluency Test.

It is unclear whether such differences affect test validity (sensitivity to disease or even relevance to everyday functioning) but these cultural differences definitely require different normative standards to accurately interpret performance. Again, due to differences in cultural values placed on concepts of efficiency, timeliness, and deadlines, tests of “processing speed” or timed tests in general tend to be performed very poorly by people in many developing areas of the world compared to Western standards (Agranovich and Puente 2007; Anger et al. 1993; Byrd et al. 2004; Cysique et al. 2007b). Nevertheless, in spite of the differences in performance across international populations, tests of processing speed clearly are quite sensitive to HIV disease in both types of areas (Cysique et al. 2007b; Heaton et al. 2008; Robertson et al. 2007a). Processing speed as well as other cognitive traits may not be emphasized as much in non-Western educational systems and in many non-Western jobs, but we cannot assume it is an unimportant (or nonexistent) human characteristic in non-Western populations.

Transferring existing neuropsychological instruments to new settings, particularly resource-limited settings, is not a simple task. In addition to cultural differences in skill sets, differences in and even a total lack of formal education can be factors especially in rural settings where lack of familiarity with writing instruments, much less computers, place limitations on assessments. Familiar stimuli in Western cultures such as subways, escalators, and even certain foods and animals, are unfamiliar and inappropriate for use in testing people in other cultures. Visual, auditory, and other stimuli therefore, may need to be redesigned for use in a particular setting. In settings where other alphabets are standard for example, the Trail Making Test has been replaced with the Color Trails Test which removed the English alphabet from the instrument. Investigators at the University of Miami altered word lists and phrases in the instructions and certain passages of six verbal learning, memory, and fluency instruments in an attempt to make these instruments valid for use in Hispanic populations (Wilkie et al. 2004).

Differences in language can be problematic in moving tests between settings. In terms of translating and administering assessments, even professional and accurate translation does not remove all problems. Standard instructions may not be easily translated verbatim and therefore should be carefully tailored to the common language and idioms of the target population so that accuracy of content and ease of understanding are maintained. Some languages have no words to describe a particular object and translation of verbal stimuli, such as word lists, can lead to differences in word length or difficulty which result in various items or even an entire test being fundamentally different, either more or less difficult. Differences in item difficulty may become problematic on tests that have discontinuation rules after a certain number of incorrect responses in a row, especially if “early” items are less familiar in the new culture. Nonetheless, since resources are usually strained and many tests have demonstrated some validity, existing tests will inevitably be translated and used in international studies. When planning neuropsychological evaluations of a group of people using tests developed in another culture and language, at the very least, it is important that tests be translated into the target language by a neuropsychologist or professional fluent in the target language, and then back translated by an independent professional translator with no knowledge of the test instruments as an extra precaution (Cysique et al. 2007b). The tests should then be administered by a neuropsychologist or trained psychometrician fluent in the target language and familiar with the culture of the participants, since informal rapport often is needed to maximize a patient’s motivation and cooperation. In any event, it is also important to pilot the new translations with the target population to ensure comprehension of the instructions’ and the test content’s meanings.

In some regions of the world such as India, Indonesia, and many African countries (Zambia and Nigeria for example) multiple languages and dialects are spoken by various subgroups of people. The Eighth Schedule of the Constitution of India recently increased the number of languages recognized by “The Official Languages Act, 1963” from 14 to 22 official languages (“The constitution of india: The official languages act 1963”). In fact, over 400 individual mother tongues are spoken across India (Gordon 2005). The existence of so many subgroups within some countries and regions of the world creates considerable impediments to translating tests and establishing normative data for those regions. To gather normative data for the WHO/UCLA Auditory Verbal Learning Test in India for example, translating and administering the test and word lists in Hindi dialects alone may be insufficient. When clinical study sites are dispersed throughout the various regions of the country, tests may need to be translated into any one of the other 21 official languages, depending on the dominant language of the region. Translating and adapting tests, then gathering normative data for each subgroup in a country based on differences in language and culture, would be very costly and time consuming, and therefore test publishers and other private organizations that collect neuropsychological data have been reluctant to undertake the task.

Collecting appropriate normative data for neuropsychological tests in new settings is often a daunting task, requiring large sample sizes consisting of hundreds of normal subjects to bring about the needed datasets. Factors such as age in years, education, gender, and language are essential for appropriate comparisons, but other differences such as urban versus rural settings, ethnicity, and socioeconomic factors often need to be addressed as well. Ideally, all demographic characteristics that relate to test performance in normal individuals will be considered (“corrected”) in any norms that are developed (Heaton et al. 2008). In addition to the cultural, linguistic, and financial obstacles to gathering normative data described above, issues with recruitment (clinics versus elsewhere), incomplete or inaccurate medical histories, and management of confounds (psychiatric, medical, and neurological) create added difficulties. When many demographic variables must be controlled, attaining adequate sample size with all levels of all factors well represented can be a very expensive undertaking. Funding agencies thus far have been unenthusiastic about supporting basic normative data collection, and the private sector which stands to gain little financial reward, has been even less enthusiastic. As a result, most studies attempt to accrue local control groups for study-specific comparisons; however, these may suffer from small sample sizes with unique characteristics, and often will be inadequate for developing standards for classifying impairment of infected individuals.

In the future, it would be useful to explore the generalizability of existing norms from other places (Western and non-Western), especially those with similar cultures and socioeconomic conditions, because available resources will not permit development of separate norms for every population on earth. In order to accomplish this, first we need to know what factors (cultural, educational, linguistic, economic, etc) affect generalizability, and then identify which populations share these characteristics. For example, tests and norms developed in the U.S. are commonly used throughout the Western world (Europe and Australia) (Bornstein et al. 1993; Paul et al. 2007; Portegies et al. 1993; Tozzi et al. 2005; Wright et al. 2008) apparently with similar validity, so similar generalizability across developing countries where people share similar cultural and experiential backgrounds may be possible.

Investigative Strategies

For maximum efficiency in identifying the presence or absence of impairment (but not necessarily characterizing its nature) tests included in international resource-limited settings should be as brief as possible and easily administered (such as by non-neurologist/psychologist), and test batteries should sample a wide range of cognitive abilities. Tests/test batteries must be chosen with consideration for the background and current life circumstances of the people with whom they will be used. As previously mentioned, some tasks and test stimuli may be unfamiliar, while others are not. People may have different cognitive approaches to questions and problems, which may change the inherent meanings of results. Factor analyses may be helpful in identifying constructs sampled by various tests in different places. Determining the influence of culture on test performance and validity of test instruments, however, is not a simple “yes” or “no” question. The psychometric appropriateness or adequacy of a test also incorporates test-retest reliability, relation of performance to demographics, relation to disease, and relation to everyday lives.

The NIMH Core Battery for Assessment of AIDS-Related Changes (Butters et al. 1990), for example, measures a wide range of abilities affected by HIV, but many of the tests are not well validated in cross-cultural settings and the length of the battery is a problem for many settings (especially those that are resource-poor). The WHO Neuropsychological Screening Battery is a bit shorter and still measures a variety of abilities; however, the lack of appropriate normative data tailored to different cultural settings is an issue. Brief screening batteries such has the International HIV Dementia Scale and the ACTG 5199 International Battery are very practical and measure mostly motor-based skills which translate well across cultures, but they are not very sensitive or specific to HIV related effects. Screening batteries may be highly efficient and cost effective for identifying potential study participants when a large pool of patients exists, but to identify milder impairment and characterize the pattern of neurocognitive disorder, longer batteries which tap multiple cognitive domains are necessary. During the early stages of the HIV epidemic White et al. (1995) compared 57 studies which assessed the performances of HIV-1 seropositive asymptomatic and HIV-1 seronegative individuals in order to determine if it is possible to have cognitive deficits in asymptomatic people (White et al. 1995). The authors reported a median impairment rate of approximately 35% in asymptomatic HIV-infected patients and also pointed out that relatively comprehensive batteries (14 or more measures) were more likely to uncover the cognitive effects of HIV. Two batteries developed for use in China (Cysique et al. 2007b; Heaton et al. 2008) and Uganda (Robertson et al. 2007a) offer somewhat of a compromise between extensive batteries and screening batteries. The batteries used in these studies balanced the need to assess multiple domains with the importance of culturally valid test instruments and relatively short administration time. For a comparison of neuropsychological test batteries used in both the US and internationally see Tables 1 and 2.

Table 1

Standard neuropsychological test batteries

Neuropsychelogical domain	NIMH core neuropsychology battery for assessment of AIDS-related changes	Early HNRC battery	HNRC focused battery	HIV dementia scale
Premorbid Intelligence
	WAIS-R Vocabulary		WRAT-4
	National Adult Reading Test (NART)
Attention/Working Memory
	WMS-R Digit Span	WAIS-R Digit Span	Trail Making Test A	Anti-sacaddic Eye Movements: 20 Commands
	WMS-R Visual Span	Digit Vigilance	WAIS-III Letter-Number Sequencing
	Paced Auditory Serial Addition Test (PASAT)	Paced Auditory Serial Addition Test (PASAT)	Paced Auditory Serial Addition Test (PASAT)
Learning and Memory
Verbal		Story Learning		Recall of 4 Words
	Californa Verbal Learning Test (CVL T)	Story memory delay free recall	Hopkins Verbal Learning Test Revised
Visual	Reproduction Test (WMS)	Figure Learning	Brief Visuospatial Memory Test Revised
		Figure memory delay free recall
Speed of Processing
	Sternebrg Search Task	WAIS-R Digit Symbol	WAIS-III Symbol Search	Alphabet Writing
	Simple and Choice Reaction Time (Go/No Go)	Digit Vigilance	WAIS-III Digit Symbol
	WAIS-R Digit Symbol	Trail Making Test A	Trail Making Test A
Executive Function
	Category Test	Category Test	Category Test
	Trails A and B	Trails B	Trails B
			Wisconsin Card Sorting Test
Language
	Verbal Fluency (letter and category)	Thurstone Word Fluency	Verbal Fluency (letter and category)
	Boston Naming Test	Boston Naming Test
		Controlled Auditory Word Association Test
Visuoperception/Visuomotor
	Embedded Figures Test	WAIS-R Block Design
	Money’s Standardized road-Map Test of Direction Sense
Contructional Abilities
	WAIS-R Block Design			Cube Copy
	Tactual Performance Test
Motor Abilities
Fine Motor	Grooved Pegboard	Grooved Pegboard	Grooved Pegboard	Alphabet Writing
	Finger Tapping	Finger Tapping
	Grip Strength	Grip Strength
Gross Motor
Psychiatric Assessment
	Diagnostic Interview Schedule (DIS)	Beck Depression Inventory (BDI)	Beck Depression Inventory (BDI)-II
	Hamilton Rating Scale for Depression (HRSD)	Profile of Mood States (POMS)	Profile of Mood States (POMS)
	Spielberger State-Trait Anxiety Inventory (STAI)	Structured Clinical Interview for DSM-IV	Composite International Diagnostic Interview (CIDI)
	Mini-Mental State Examination
Source	Butters et al. 1990	Heaton et al. 1995	Woods et al. 2006	Power et al. 1995

Table 2

International neuropsychological test batteries

Neuropsychological domain	WHO neuropsychological	HUMANS—Spanish neuropsych battery	HNRC international test battery	South India test battery	International HIV dementia scale	ENARC study (Ethiopia)	Uganda	ACTG 5199 international battery
Premorbid Intelligence
		WAIS-R vocabulary
Attention/Working Memory
	Trail Making Test A	WAIS-R Digit Span					Digit Span, Forward and Backward
			WMS-III Spatial Span
		Variable Interval Reaction Time Task	Paced Auditory Serial Addition Test (PASAT)
		Working Memory Tests (Primary/Secondary)		Verbal N-Back Task
Learning and Memory
Verbal		WMS-R Logical memory
	WHO/UCLA Auditory Verbal Learning	California verbal Learning Test (CVLT)	Hopkins Verbal Learning Test Revised	WHO-AVLT	Registration and Delayed Recall of 4 common objects	Registration and Delayed Recall of 4 common objects	WHO-UCLA AVLT
Visual	WHO/UCLA Picture Memory Interference	Visual Reproduction (WMS_R)	Brief Viuospatial Memory Test Revised
Speed of Processing
	WAIS Digit Symbol	Figural Visual Scanning Task	WAIS-III Symbol Search	Visual N-Black Task			Color Trails I
	Trail making Test A	Simple and choice Reaction Time (Go/No Go)	WAIS-III Digit Symbol				Symbol Digit Modalities
	Color Trails I	Posner Letter Matching Test	Trail Making Test A
			Stroop Color Naming
Executive Function
	Color Trails II		Category Test	Tower of London
		Trails A and B	Color Trails II				Color Trails II
		Wisconsin Card Sorting Test	Wisconsin Card Sorting Test-64
		Stroop Color-Word Test	Stroop Color-Word Test
Language
	Verbal Fluency (names, animals)	Verbal Fleuncy (letter and category)	Action Fluency	Phonemic Fluency		Animal Naming		Semantic Verbal Fluency
		Boston Naming Test	Category Fluency (Animal Naming)	Category Fluency (Animal Naming)
Visuoperception/Visuomotor
	WAIS Block Design
Contructional Abilities
		WAIS-R Block design
Motor Abilities
Fine Motor	Grooved Pegboard	Grooved Pegboard	Grooved Pegboard			Grooved Pegboard (non-dominant hand)	Grooved Pegboard	Grooved Pegboard
					Thum-to-First finger Tapping Speed	Thumb-to-First finger Tapping speed
					Timed Alternating Hand Squeeze	Timed Alternating Hand Squeeze
				Finger Tapping		Finger Tapping (dominant Hand)		Finger Tapping
Gross Motor	Time Gait
Psychiatric Assessment
		SCID Interview	Beck Depression Inventory (BDI)-II
		Hamilton Rating Scale for Deppression (HRSD)	Structured Clinical interview for DSM-IV
		Spielberg State Trait Anxiety Inventory (STAI)
		Mini-Mental State Examination
		Hamilton Anxiety Rating Scale (HARS)
		Profile of Mood States (POMS)
		Sickness Impact Profile (SIP)
		Cognitive Difficulties Scale
		Acculturation Scale
Source	Maj et al. 1994	Wilkie et al. 2004	Cysique et al. 2007b; Heaton et al. 2008	Gupta et al. 2007	Sacktor et al. 2005	Clifford et al. 2007	Robertson et al. 2007a	Robertson et al. 2008

Conclusions

In summary, there are many challenges to the assessment of neurocognitive effects of HIV in resource-limited settings, including an overwhelming disease burden (HIV as well TB, malaria, and other infections), lack of infrastructure (including paved roads, access to clinics, and modern medical technology), lack of neuropsychological and neurological expertise, differences in education, culture, and language across borders, and a lack of appropriate normative data for many patient populations. Regardless of the challenges to international HIV neuropsychological studies, when studies are conducted with a standardized approach to assess cognition and rate levels of impairment, and when appropriate normative or control data are available, meaningful results can be obtained. While it is certainly not fair to directly compare (without re-norming) neuropsychological test performance in resource-limited settings with data from populations in Western cultures, existing neuropsychological test instruments have demonstrated sensitivity to HIV-associated neurocognitive effects in international studies and therefore can still be useful. Since tests and normative data developed in the US have demonstrated validity in Europe and Australia, identifying which factors (cultural, educational, linguistic, economic, etc) affect the generalizability of test results and which populations in various resource-limited settings share these characteristics will be important for future studies attempting to compare performance across international populations.