HIV in Japan: Epidemiologic puzzles and ethnographic explanations.

Japan is widely perceived to have a low level of HIV occurrence; however, its HIV epidemics also have been the subject of considerable misunderstanding globally. I used a ground truthing conceptual framework to meet two aims: first, to determine how accurately official surveillance data represented Japan's two largest epidemics (urban Kansai and Tokyo) as understood and experienced on the ground; and second, to identify explanations for why the HIV epidemics were unfolding as officially reported. I used primarily ethnographic methods while drawing upon epidemiology, and compared government surveillance data to observations at community and institutional sites (459 pages of field notes; 175 persons observed), qualitative interviews with stakeholders in local HIV epidemics (n = 32), and document research (n = 116). This revealed seven epidemiologic puzzles involving officially reported trends and conspicuously missing information. Ethnographically grounded explanations are presented for each. These included factors driving the epidemics, which ranged from waning government and public attention to HIV, to gaps in sex education and disruptive leadership changes in public institutions approximately every two years. Factors constraining the epidemics also contributed to explanations. These ranged from subsidized medical treatment for most people living with HIV, to strong partnerships between government and a well-developed, non-governmental sector of HIV interventionists, and protective norms and built environments in the sex industry. Local and regional HIV epidemics were experienced and understood as worse than government reports indicated, and ground-level data often contradicted official knowledge. Results thus call into question epidemiologic trends, including recent stabilization of the national epidemic, and suggest the need for revisions to the surveillance system and strategies that address factors driving and constraining the epidemics. Based upon its utility in the current study, ground truthing has value as a conceptual framework for research and shows promise for future theoretical development.

Introduction

Japan is widely perceived as a country with a low level of HIV occurrence (Suguimoto et al., 2014, Koerner and Ichikawa, 2011, Kihara and Komatsu, 2003, Wada et al., 2013). Surveillance data present in Japanese government reports support this perception (Ministry of Health Labour and Welfare, 2001, Ministry of Health Labour and Welfare, 2014, Ministry of Health Labour and Welfare, 2015); however, close scrutiny of the reports also reveals that key information is not included. This represents an opaqueness that has long hindered an accurate portrayal of HIV in Japan and which has been the source of considerable misunderstanding in the global health community.

Available epidemiologic information, based almost entirely upon government surveillance data, thus could indicate a very low HIV burden in Japan that is real, including remarkably low risk in the general population; or it could mean that a significantly larger HIV epidemic is being obscured. Due partly to a dearth of social science and qualitative research, little is known about how well available surveillance data capture the reality of HIV in Japan, and there is much to learn about why recent trends have been occurring. Accordingly, I conducted a study that had two main aims. The first was to determine how accurately official surveillance data represented the HIV epidemics in two metropolitan areas (Kansai and Tokyo) as understood and experienced on the ground in 2013. The second aim was to identify explanations for why the HIV epidemics were unfolding as officially reported. In meeting these aims, examination of surveillance data uncovered conspicuous epidemiologic puzzles. Ethnographic research revealed explanations for these puzzles, including factors that drove and constrained the epidemics. As an initial step in articulating the epidemiologic puzzles, it is first necessary to outline what is known of the epidemiology.

Epidemiology of HIV in Japan

From 1985, when the first diagnoses were reported, to the end of 2013, there were a cumulative 23,015 reported cases of HIV, of which 7203 (31.3%) were AIDS cases (Ministry of Health, Labour and Welfare, 2014). Notably, Japan's surveillance system defines HIV and AIDS cases permanently based upon the stage of infection at the time of initial diagnosis (Yoshikura, 2014). Additionally, these numbers do not include 1432 cumulative cases of HIV and AIDS among hemophilia patients iatrogenically infected through blood products, who are tracked separately from the Ministry of Health, Labour and Welfare (MHLW) surveillance system (Japan Foundation for AIDS Prevention, 2013a).

Regarding incidence, there were 1590 new HIV and AIDS cases in 2013 – the highest number of combined annual cases ever recorded in the country (Fig. 1). Of these, 1106 (69.6%) were new HIV cases, which was the second highest number on record. Yet, if only HIV is considered, Japan's national epidemic actually has been showing signs of stabilization in recent years (Ministry of Health, Labour and Welfare, 2014). New cases peaked in 2008 and have leveled off at approximately 1000 each year since then. This apparent plateau is a relatively late development, considering that new HIV cases reached their highest number globally from 1996 to 1999 and have been declining steadily since 2000 (UNAIDS, 2014a). However, if Japan's recent, putative trend continues, it could mark a crucial, positive shift in the epidemic.

Fig. 1

New HIV cases and HIV tests in Japan, 1985 – 2013 (Source: own construction, data from Ministry of Health, Labour and Welfare) (Ministry of Health Labour and Welfare, 2001, Ministry of Health Labour and Welfare, 2014). aDisaggregated annual HIV testing data reported by MHLW start in 1989. The aggregate total prior to1989 is 47,470 tests.

The year 2008 also marked the peak in the number of reported HIV tests in Japan (177,156 tests). After this, tests dropped for two consecutive years and were basically flat from 2010 to 2013 (Ministry of Health, Labour and Welfare, 2001). Conversely, of the total new HIV and AIDS cases in 2013, 484 (30.4%) were new AIDS cases, which was the highest number ever reported (Fig. 2). Though there had been a drop in AIDS cases in 2012, the first since 2005, an upturn again in 2013 continued an overall increasing trend since 1985 and maintained a proportion exceeding 30% (Ministry of Health, Labour and Welfare, 2014). This is significant in a high-income country with an advanced medical system and wide use of effective antiretroviral therapy (Suguimoto et al., 2014, UNAIDS, 2014b, WHO, 2011, Hashimoto et al., 2004).

Fig. 2

New AIDS cases and AIDS-related deaths, 1985–2013 (Source: own construction, data from Ministry of Health, Labour and Welfare) (Ministry of Health, Labour and Welfare, 2014).

Trends in reported route of infection

The first large-scale outbreak of HIV in Japan occurred in the first half of the 1980 s among the hemophiliacs described previously, and they accounted for the majority HIV cases until the mid-1990s (Hanabusa, 2008, Kihara et al., 2003). However, since 1985, when blood products began to be heat-treated, sex has been the main way HIV is transmitted. From 2000 onward, sexual contact among men who have sex with men (MSM) has outpaced all other routes. Correspondingly, in 2013, 70.5% of new HIV cases were via homosexual contact, almost all among men, followed by heterosexual contact (17.5%), unknown (high at 9.6%), other (2.1%), injection drug use (0.2%), and mother-to-child transmission (0.1%). A similar pattern was reported for new AIDS cases: 56.4% from homosexual contact, again almost all among men, followed by heterosexual contact (24.0%), unknown (very high at 16.1%), other (2.9%), and injection drug use (0.6%) (Ministry of Health, Labour and Welfare, 2014).

The Japanese epidemic is overwhelmingly male. Women comprised only 4.2% of HIV cases and 3.7% of AIDS cases in 2013, acquired mostly via heterosexual contact. Men outnumbering women in an HIV epidemic is unsurprising in itself. However, among HIV cases reported via heterosexual transmission in Japan, there also have been many more men compared to women for at least the past 10 years (Ministry of Health Labour and Welfare, 2014, Ministry of Health Labour and Welfare, 2015). Because women tend to be biologically and socially more vulnerable to HIV than men in many heterosexual contexts, this trend is unusual compared to other countries (UNAIDS, 2014c, UNAIDS, 2014d, Centers for Disease Control and Prevention, 2015). Indeed, among heterosexual HIV cases reported by foreign nationals in Japan, there have been more women compared to men, reflecting the more typical trend internationally. Also of note, among all Japanese men who acquired HIV sexually since the beginning of the epidemic, the older they were at the time of diagnosis, the more likely they were to report heterosexual infection (Ministry of Health, Labour and Welfare, 2014).

Gaps and international context

Three important surveillance measures are not reported by MHLW and are not known with confidence: national prevalence in the general population, including all persons currently living with HIV regardless of stage; the national HIV incidence rate (e.g., per 100,000 persons) for the general population; and the AIDS-related death rate. Recent national prevalence estimates by the Japan Foundation for AIDS Prevention (JFAP) and the National Center for Global Health and Medicine (NCGM) for 2013 and 2014 were 0.018% and 0.02%, respectively (Japan Foundation for AIDS Prevention, 2013b, Nishijima et al., 2015). These are probably erroneous estimates, mainly because one was calculated using incomplete MHLW AIDS mortality data, and the other did not subtract out deaths. Though Japan's Infectious Diseases Control Law requires physicians to report AIDS-related deaths within seven days (Ministry of Health Labour, and Welfare, 2015a), the country's National Institute of Infectious Disease characterized such reports as voluntary in practice and concluded that mortality estimates were therefore far below actual numbers (National Institute of Infectious Diseases, 2010). Between 1989 – the first year MHLW reported mortality data – and 2013, only 922 deaths of persons with AIDS were reported (Fig. 2) (Ministry of Health, Labour and Welfare, 2014). Just two other HIV prevalence estimates are available within the past several years. The first, in a Joint United Nations Programme on HIV/AIDS (UNAIDS) annual report, was 7900 persons for 2011 (UNAIDS, 2012e). Using Japan's population for that year (Ministry of Internal Affairs and Communications, 2015), this would yield a prevalence of 0.006%, much lower than the JFAP and NCGM estimates. The other, printed in a major newspaper in 2010, was considerably higher at 0.115% (Nagaoka, 2010), though the source was not cited. These estimates differ by a magnitude of greater than 19 times.

Although MHLW does not report the national HIV incidence rate for the general population, rates are reported by prefecture and age group, and among blood donors. Incidence in 2013 was highest for Tokyo, followed by Osaka Prefecture; and highest among 25–29-year-olds, followed by 30–34-year-olds. Each of these rates was below 3.0 per 100,000 (Ministry of Health, Labour and Welfare, 2014). The rate among blood donors, 1.21 per 100,000 (Ministry of Health, Labour and Welfare, 2014b), is the closest proxy to general population incidence available in the surveillance reports. As another proxy, a simple calculation can be used to estimate a crude national incidence rate. Dividing the number of new HIV/AIDS cases in 2013 by the population estimate (Ministry of Internal Affairs and Communications, 2015) yields an unadjusted rate of 1.25 per 100,000.

Irrespective of source – government surveillance report or otherwise – these are very small numbers when compared internationally. The HIV prevalence estimates, proxy estimates for incidence in the general population, and cumulative case numbers in Japan are all lower than 75–83% of recent, comparable estimates in neighboring countries in East Asia, the United States, and 117 nations reported by UNAIDS (UNAIDS, 2014a, Centers for Disease Control and Prevention, 2015, UNAIDS, 2014f, Korea Centers for Disease Control and Prevention, 2012, Hong Kong Special Administrative Region Department of Health, 2014, Centers for Disease Control R.O.C. (Taiwan), 2014, Ministry of Health of the People's Republic of China, 2012, UNAIDS, 2014g, Yin et al., 2014).

A dearth of social science and qualitative research

The English-language and Japanese literatures on HIV in Japan are dominated by clinical and epidemiologic studies using almost exclusively quantitative and biomedical data. A focus of the comparatively smaller body of social science research, including both qualitative and quantitative investigations, has been the experiences of a very specific population: the victims and families affected by the “Yakugai AIDS Scandal,” in which hemophilia patients were infected through blood products imported mainly from the U.S.(Cullinane, 2005; Mizota, Ozawa, Yamazaki & Inoue, 2006; Seki, Kakinuma, Kuchii & Ohira, 2015; Seki, Yamazaki, Mizota & Inoue, 2009). The scandal and its aftermath have been an important part of Japan's unique HIV history; however, hemophiliac HIV cases have comprised only 5.9% of cumulative HIV/AIDS cases in Japan since the beginning of the epidemic (Ministry of Health Labour and Welfare, 2014, Japan Foundation for AIDS Prevention, 2013a, DiStefano, 2016). Other studies, published in virtual isolation mainly by anthropologists, sociologists, and some economists, have examined HIV and sexual politics (Miller, 2002); a clinical phobia of HIV among uninfected persons (Miller, 1998); sexual activities and social relationships among persons living with HIV (Inoue, Yamazaki, Seki, Wakabayashi & Kihara, 2004); sex partner-seeking strategies among MSM;(Ishi, 2012) motivators and deterrents of HIV testing (Takahashi, Yamazaki & Kawata, 1999); and the economic impact of antiretroviral therapies (Kimura, 2002). Though these studies have provided insight into a few disparate aspects of HIV in Japan, more social science and qualitative research – particularly that which contextualizes a broad view of the country's epidemic – is needed to fill sizeable gaps in understanding. I sought to address this need, in part, by meeting the aims of the current study.

Conceptual framework and theoretical positioning

I used a ground truthing framework to guide study design, domains of inquiry, and interpretation of findings. As a conceptual approach to research, ground truthing's origins are quite outside the social and health sciences. It was first developed in the early 1980s as a way to calibrate satellite measurements of Earth with measurements taken on the ground (Moeser & Raschke, 1983). This approach has been applied to a limited degree in public health by using observations on the ground to validate aerial imagery or remote sensing data on health-related geographic factors, such as community access to food sources (Sharkey & Horel, 2008). More recently, a broader conceptualization of the approach, not including remote sensing, has been applied to verify government data (e.g., on neighborhood environmental hazards) and supplement these with local data from communities that had not been captured in official databases (Sadd et al., 2014). Used in this way, ground truthing can uncover gaps in government agency data, raise important issues at the local level, and provide evidence and rationale for policy change (Sadd et al., 2014).

It was this last conceptualization of ground truthing that I adapted as the framework for the current study. I compared HIV surveillance data and reports by MHLW to data that I collected ethnographically on the ground in Japan. Discovery of puzzles in the surveillance data led to asking why the puzzles occurred and what the explanations were.

Ground truthing has not been treated in the literature as a formal theory. However, in the manner that I applied it as a conceptual framework in the current study, its assumptions occupied a middle ground between two commonly delineated theoretical and epistemological orientations in the social sciences: positivist and postpositivist. That there is a “truth” that can be discovered at all through ground truthing is a positivist notion. This resonates with empiricism, naturalism, structuralism, and functionalism (Barnard, 2000; Hume, 1888; Rosenberg, 2008; Durkheim, 1895; Radcliffe-Brown, 1952; Malinowski, 1944; Levi-Strauss, 1963; Moses and Knutsen, 2012; Durkheim, 1897). Yet, in a postpositivist direction, ground truthing also assumes that multiple perspectives are needed to discover the truth, and that “official” knowledge is not sufficient (Sadd et al., 2014). This aligns more closely with constructivism, symbolic interactionism, poststructuralism, postmodernism, and their applications in the sociology of health and medical anthropology (Scheper-Hughes, 1993, Crotty, 1998, Blumer, 1969, Foucault, 1970, Bourdieu, 1977, Geertz, 1973, Marcus and Fischer, 1986, Moeran, 1990).

Methods

Methods and results in this article are reported according to criteria described in Tong, Sainsbury and Craig (2007) guidelines for the journal (Social Science & Medicine, 2010). To meet study aims within a ground truthing framework, I used primarily ethnographic methods while also drawing upon epidemiology – a combination of perspectives and techniques informed by Bourgois and colleagues' social science of medicine (Bourgois et al., 2006). This approach allowed for a flexible adoption of qualitative strategies from medical anthropology and sociology, while enabling a role for epidemiology in informing the ethnographic procedures, analyses, and interpretation.

For seven months in 2013 (January–August), I engaged in fieldwork in four cities in Japan's Kansai region: Osaka, Kyoto, Kobe, and Nara; and in the Tokyo Metropolitan Area – mainly Tokyo, but also in two neighboring prefectures: Chiba and Kanagawa. I chose these two urban areas because they were home to Japan's two largest regional HIV epidemics. I decided to base myself primarily in Kansai because of the trends in AIDS, specifically. Although the number of new AIDS diagnoses in Tokyo had started flattening in 2002, the Kansai region continued an upward trend in AIDS through 2009 (Fig. 2); and according to 2011 surveillance data, which were the latest available when I started the study, Kansai had the highest number of AIDS cases in Japan (Ministry of Health, Labour and Welfare, 2014). Data collection comprised observations, formal qualitative interviews, and document research, supplemented with emails and telephone calls. The institutional review board at my university in the U.S. and the university institute in Japan where I was affiliated approved study protocols.

Observations

Observation was a full-time method and comprised both unobtrusive and participant observation. To capture naturally occurring phenomena and reduce reactivity based on knowledge of being observed, unobtrusive observation involved no direct interaction with persons observed and were conducted covertly. Participant observation was overt and included verbal interaction that frequently took the form of informal, conversational interviews in the natural environments of persons observed. I approached and acquired oral informed consent from persons observed during participant observation. No one dropped out of the study after they provided informed consent to be observed. No informed consent was acquired from persons observed during unobtrusive observation.

After one month of fieldwork, I had completed a close study of MHLW surveillance reports and unprocessed surveillance data publicly available from MHLW, which pointed to specific population groups at increased risk of HIV in Japan. At the same time, I conducted initial coding of my ethnographic data, which comprised field notes from observations and document research, and I continued consulting the relevant scientific literature. My data confirmed and expanded upon the groups at increased HIV risk noted in the MHLW surveillance data, and additionally indicated that there were other key stakeholders in local HIV epidemics who could provide data critical to meet my study aims. The scientific literature confirmed the potential value of data from these groups. Considered together, this information allowed me to identify seven categories of stakeholders as appropriate groups on whom to focus subsequently: (1) interventionists (health care, mental health, and social service providers; community organizers) working directly on the HIV problem; (2) MSM and transgender women; (3) interventionists working with MSM or transgender women; (4) persons living with HIV; (5) persons diagnosed with a sexually transmitted infection (STI), or presenting for HIV or STI testing or counseling; (6) persons who had experienced or perpetrated violence, abuse, suicidality, or non-suicidal self-harm; and (7) interventionists working with persons affected by violence, abuse, suicidality, or non-suicidal self-harm. I used ethnographic mapping to locate areas in local communities where people from these stakeholder groups worked, socialized, sought health care and other services, or otherwise congregated in clusters. Observations of the target groups often led also to observations of persons who fell outside these groups when it was clear that relevant data could be captured (Table 1). In total, this yielded 459 pages of field notes derived from observations of 175 people.

Table 1

Observation sites and persons observed.

Observation sites	Persons observed
Infectious disease ward, outpatient clinic, pharmacy, other areas at a public city hospital; comprehensive, private medical clinic with a focus on HIV and STIs	Physicians; nurses; a pharmacist; a medical intern; front office and other staff; patients seeking HIV/STI testing, counseling, treatment
Public health centers; health and welfare bureaus in city halls and ward offices	Managers; program coordinators; public health nurses; community health educators; citizens utilizing HIV testing and counseling, and other health and social services
Community-based organizations and community centers focused on HIV	Directors; program coordinators; HIV case managers; testing counselors; social workers; other paid staff and volunteers; clients utilizing HIV testing, counseling, health education for HIV prevention, and programs for HIV-positive persons
Private corporation producing HIV and STI testing equipment	Company employees
Street-based HIV outreach events; MSM and transgender community events	Community and event organizers, attendees
Streets of sexual and gender minority (SGM) neighborhoods; a gay sex club; SGM bars, restaurants, and other businesses	MSM, transgender women, other SGM community members; business owners, employees, customers
Heterosexual brothels, love hotels, red light districts, and bars	Sex workers and their clients; other sex industry employees; bar owners, employees, customers
Intimate partner/domestic violence shelter	Shelter case managers, hotline workers
Homeless encampments; areas with high rates of poverty	Homeless and marginally housed persons; unemployed and underemployed laborers
Election campaign events; political demonstrations	Politicians; candidates; event attendees
Other common urban locations: government buildings; museums; hotels; apartment complexes; shops; train stations and trains; airports; cafes; universities	Immigrants; youth; drug and heavy alcohol users; perpetrators, witnesses, and victims of violence, vandalism, and other crimes; police officers; security guards; civilian neighborhood crime patrols; a mental health counselor; academics and students

Formal qualitative interviews

I conducted formal qualitative interviews in Japanese or a combination of Japanese and English, according to participant preference, with 32 of the persons observed in the study (Table 2). Formal interviewees provided written informed consent. I used purposive sampling to recruit from the seven target categories described previously. This yielded interview data from multiple perspectives needed for a sufficiently broad and contextualized understanding of the HIV epidemics under examination. Valid results emerge gradually in ethnographic fieldwork, and there are gaps in understanding as the iterative cycle of qualitative data collection and preliminary analysis unfolds (Charmaz, 2006, Ryan and Bernard, 2000). Thus, concurrently with purposive sampling, I also used theoretical sampling to fill gaps in the logic of results that emerged while in the field. This involved my recruiting additional participants with whom I further explored tentative findings, and challenged those findings with negative cases, until data saturation was exceeded.

Table 2

Sociodemographic characteristics of formal interview participants (n = 32).

Characteristics	No. (%)
Age, years (M = 40.9; SD = 9.9)
25-34	10 (31.2)
35-44	11 (34.4)
45-61	11 (34.4)
Gender
Male	16 (50.0)
Female	13 (40.6)
Transgender women (MTF)	3 (9.4)
Ethnicity
Japanese	26 (81.2)
Japanese and ≥1 additional ethnicity	3 (9.4)
Other	3 (9.4)
HIV serostatus
Negative	23 (71.9)
Positive	2 (6.2)
Unknown	1 (3.1)
No answer	6 (18.8)
Sexual orientation
Heterosexual/straight	15 (46.9)
Gay or lesbian/homosexual	11 (34.4)
Bisexual	2 (6.2)
Other	4 (12.5)
Employment status
Full-time	21 (65.6)
Part-time	8 (25.0)
Unemployed	3 (9.4)
Mean individual income: ¥4,429,938a (Mdn = ¥3,000,000)b
Mean household income: ¥6,396,774 c (Mdn = ¥5,000,000) d
Marital status
Unmarried	23 (71.9)
Married	9 (28.1)
Education
High school to some postsecondary	4 (12.5)
Trade school or junior college	4 (12.5)
Bachelor's degree	13 (40.6)
Graduate degree	11 (34.4)

$43,236 based on average 2013 exchange rates.

$29,280.

$62,432; M = 1.9 (Mdn = 1.0) persons in fiscal household.

$48,800.

Saturation was achieved when variations in newly collected data from participants were negligible based on initial coding in the field. I reached data saturation with the 26th interview. As a conservative measure to assure that I had captured the full range of knowledge and experiences across the seven target participant categories, and that I had challenged emerging consensuses in the data with potentially dissenting voices, I interviewed an additional six people. Conducting interviews 27 through 32 allowed for verification, correction, and a more nuanced understanding of findings from earlier interviews, but no significantly novel findings emerged. This method was consistent with recommended sampling standards in qualitative research, which indicate that cut points in recruitment should coincide with diminishing returns on the discovery of novel results in the data (NIH Office of Behavioral and Social Science Research, 1999).

After locating community and institutional observation sites (Table 1), I recruited formal interviewees in two ways. First, I invited persons whom I was observing, who were at least 20 years old – the age of majority (i.e., adulthood, voting) in Japan – and who met inclusion criteria in one of the seven participant categories. Second, a research assistant and I emailed and called observation sites; sent an information sheet listing my name, credentials, and objectives, the study's purpose, data collection formats, participant categories, and inclusion criteria; and asked for potential participants. Interventionists (categories 1, 3, and 7) must have provided direct prevention, case management, treatment, or other health care, mental health, or social services to clients or patients related to HIV, or to persons at increased risk of HIV (i.e., MSM, transgender women, persons affected by violence, abuse, suicidality, or non-suicidal self-harm) within the past year in Japan; or provided leadership, engaged in community organizing, or led other activities that promoted the physical health, mental health, rights, or social welfare of persons living with or at increased risk of HIV. For category 2, participants had to be a man or transgender woman who had voluntarily engaged in sexual activity with a man at least once in the past year. For category 4, participants had to be HIV-positive. For category 5, persons must have been diagnosed with an STI in the past year or had to be currently seeking HIV or STI testing or counseling. For category 6, interviewees must have ever experienced or perpetrated violence, abuse, suicidality, or non-suicidal self-harm. At the end of each interview, I provided interviewees with the study information sheet and asked them to refer qualified people to the study. These strategies enabled both purposive and theoretical sampling.

In total, I invited 50 people; 32 were interviewed one time for a recruitment rate of 64%. Participants who did not join the study as interviewees either did not respond to emails and phone calls, or they stated that they did not have sufficient knowledge about the topics in the study. Most persons who refused participation offered the name and contact information of someone else whom they believed was more qualified. All interviewees were from at least one of the target categories, and 59% represented two or more categories.

The term “formal” is used here to distinguish these interviews from the informal, conversational interviews that were part of participant observation. The formal interviews were scheduled in advance, had a mean duration of 2.3 h (range: 48 min–3.3 h), and were conducted at participants' workplaces (75%) or in a café or restaurant (25%) according to participants' preference. Thus, most interviews were private while others were conducted in public places with non-participants as close as the next table. Interviewees completed a brief sociodemographic questionnaire at the time of the interview (Table 2).

For each interview, I used one of seven semistructured interview guides with open-ended questions and probes, which I developed from a review of relevant literature. Each interview guide was tailored to one of the seven participant categories, though most of the content was the same. I asked all participants questions pertaining to their work, volunteer, or student activities, then asked them to describe the current state of the HIV epidemics in their local geographic area (i.e., city) and regionally, based on their personal experiences and observations. I broke up these broad questions about HIV epidemics into smaller questions that asked about HIV occurrence; testing; risky behaviors; behavioral, biomedical, and structural prevention; treatment; and knowledge, attitudes, and beliefs about HIV. I brought the most recent government surveillance reports to each interview, and next I summarized the reports verbally while the interviewees read through them. I then asked questions about how the epidemiologic data compared to their experience of local and regional HIV epidemics; what factors explained the epidemics as they were officially reported; what the reasons were for Japan's apparent successes in managing its HIV epidemics; and what other countries could learn from Japan's experience.

In addition to these questions, I asked interventionists who worked in HIV and with populations at risk (i.e., participant categories 1, 3, and 7) more questions about their job (e.g., What kind of education, training, and/or license is required to do your job? If there are any other background requirements for your job, what are they?), and more questions about the client, patient, or other citizen population(s) whom they served (e.g., What group or groups do you provide your services to most often? Please tell me about your work and activities with this group of people. Compared to your own professional or volunteer experience and observations, how accurately do these MHLW surveillance reports represent the HIV epidemic among the people you serve in your local area? What forces are increasing risk and pushing the number of HIV infections higher among the people you serve? What forces are decreasing risk and pushing the number of HIV infections lower among the people you serve?).

I used probes conservatively to avoid introducing additional bias to interviewee responses. Moreover, when interviewees described their understanding of their local and regional HIV epidemics, I confirmed with each of them that this was based on their own experiences or their observations of the people they served, and not based on knowledge of research on this topic or their desire to produce fruitful results for this study. Interviews were audio recorded and transcribed verbatim.

I also wrote and brought to the interviews a three-page community resource guide with local and national contact information for HIV and other STI prevention and treatment; general medical and mental health care; acute crisis counseling and suicide prevention; and community based organizations serving sexual minorities. I gave the resource guide to all participants, some of whom had shared personal information indicating a potential need for such services, while several others had described someone they knew who could benefit.

Document research

I collected 116 documents with HIV content or other content relevant to local and regional HIV epidemics that targeted persons living with HIV, their partners, family, friends, and employers; key populations at higher risk of HIV; health care and social service providers; blood donors; schools; and the general population. The documents were produced by government institutions, nongovernmental organizations (NGOs), community-based organizations (CBOs), other civil society groups, and businesses (Table 3). None of the documents included in the analysis were from the published scientific literature.

Table 3

Documents included in analysis (n = 116).

Documents and content	No. (%)
Exclusively or predominantly HIV content
Information on HIV programs and organizations with prevention, treatment, or case management services	27 (23.3)
Health education materials on HIV	23 (19.8)
Information for HIV-positive persons on community building, living, and working with HIV in society	6 (5.2)
Forms given to clients and patients in HIV and STI clinical settings	5 (4.3)
HIV medication handbooks and treatment guidelines	4 (3.4)
HIV statistical data other than current, government surveillance reports	3 (2.6)
Citizens’ blog posts about government surveillance	3 (2.6)
Government HIV/AIDS case report form for physicians	1 (0.9)
Other content relevant to HIV epidemics
STI health education materials and information on services	9 (7.8)
Sexual minority health and mental health materials; information on community organizing, events, and venues	14 (12.1)
Violence, suicidality, and trauma educational materials and information on services	4 (3.4)
Information on drug, alcohol, and other addiction services and support	2 (1.7)
Health education materials, information, and applications for services for other health problems	6 (5.2)
Blood donation regulations, consent forms	2 (1.7)
Sex education guidelines for public schools	3 (2.6)
Other documents reflecting relevant physical and sociocultural environments	4 (3.5)

Supplemental data and member checking

While in Japan in 2013, I collected additional data from a public health nurse through a series of emails. As a key strategy to strengthen validity and reliability in a single-researcher study, I also used in-person member checking with study participants throughout the project to confirm, challenge, and alter my evolving understanding of the data (Morse, 2015). I included such questions in the interview guide and used them more frequently later in the study during formal interviews and participant observation. After returning from Japan, I collected final member checking data (through December 2014) via email and telephone with an HIV surveillance specialist from MHLW. Because I completed member checking with questions in person or via email, and to maintain an appropriate balance of analytic independence, this strategy did not include returning interview transcripts to participants for comment or correction.

Analyses

As a check on linguistic accuracy, research assistants transcribed and translated Japanese interview recordings into English, extracted HIV content and other relevant information from collected documents, and translated these excerpts. I then completed three stages of coding, adapted from a grounded theory sequence established by Glaser (1978) and Charmaz (2006), across the triangulated data with ATLAS.ti 7. This included field notes from observations, and both the original Japanese and translated interview transcripts and document excerpts.

The first stage, initial coding, began during data collection, continued afterward, and generated a preliminary set of hierarchical codes that was applied to the entire qualitative dataset. It included constant comparison of data within and across data sources and participants to verify important points or settle questions arising from inconsistencies. Initial coding also involved abductive inference, in which I shifted between induction and deduction – considering possible explanations for data, forming preliminary hypotheses, then, using theoretical sampling, testing the hypotheses empirically by returning to the field.

In the second stage, I engaged in “expanded coding,” which is different from the data reduction of “focused coding” that is typically the second step in Glaserian and Charmazian grounded theory. In expanded coding, I added new codes as novel categories became evident, generating a maximum of 193 codes. These were organized at two hierarchical levels: major codes, assigned to broad categories of meaning in the data; and minor codes, subordinate to the major codes and denoting more specific subcategories of meaning.

The third stage was theoretical coding. Despite its name, the purpose was not to produce formal, grand theory. Rather, it was an integrative process of data reduction that gave clearer shape to the data, created a coherent analytic story, and developed concrete results – i.e., what Glaser advocated as middle-range theory, grounded in and associated exclusively with current study data. During this process, I combined codes that were sufficiently related to generate main findings. For example, 36 codes and the data attached to them were combined and reduced into the results on one major topic: local stakeholders’ assessments of the accuracy of MHLW surveillance reports. These included major codes such as “ACCURACY OF EPI REPORTS,” “EXPLAIN PUZZLES IN EPI DATA,” and “GEOGRAPHIC AREAS,” and minor codes such as “ACCURACY OF EPI REPORTS_PROBLEMS WITH MISSING DATA,” “EXPLAIN PUZZLES IN EPI DATA_HIV'S LOW NUMBERS,” and “GEOGRAPHIC AREAS_KANSAI.” As another example, 32 codes and the data attached to them were reduced to identify the major factors constraining the HIV epidemics under study. My ethnographic findings reported in the results section below thus comprise the theoretical product of the three-stage coding process in this sense. The discussion section considers these results in the context of other research and explores their implications.

Regarding position and reflexivity, I am a North American male academic (Ph.D.) with a background in public health and anthropology and a history of living, working, and conducting research periodically in Japan over 22 years. I speak, read, and write Japanese. Throughout analyses, I remained reflective on the potential influence of my position on the results.

Results

Seven main puzzles arose from my examination of the official surveillance data. Explanations for these puzzles, including factors driving and constraining the epidemics, emerged from my analysis of the ethnographic data collected via observations, formal interviews, and document research.

Puzzle 1: why were the reported numbers measuring overall HIV occurrence so low?

Instead of providing reasons for a low level of HIV burden, participant narratives typically challenged the notion that occurrence was as low as stated in government surveillance reports. Most participants were pessimistic at recent developments in their local, regional, and national epidemics, and there was substantial mistrust of the official numbers. This was mainly because data in the reports were so closely linked to trends in testing, which were not thoroughly captured in the current surveillance system. For example, referring to an MHLW report, a public health center administrator stated, “The reality … I’d guess there are 1.5 times as many people who are positive as what is reported in this statistic.” A counselor at an HIV NGO was more pointed:

How accurate do I consider it? I think that 1–2% of it is accurate. That's low, right? … We cannot say that this is the current situation in Japan…. Most people in Japan cannot access testing, and this is not at all shown in the report…. This is based on my work experience.

A few participants did speculate about geographic and cultural reasons for low occurrence, assuming it was a real phenomenon. For example, an HIV prevention consultant suggested that a temporal lag from when HIV first appeared in the U.S. and Western Europe (1981) to when it first appeared in Japan (1985) gave the Japanese time to prepare and learn about effective prevention methods. Another reason, advanced by an HIV program coordinator, concerned a prevailing culture of sanitation in Japan that had discouraged needle sharing among drug injectors and helped to prevent transmission in that population. However, there was no consensus around such explanations. Rather, when I asked what lessons other countries could learn from Japan's apparent successes in controlling their epidemic, participants generally balked; they did not feel that Japan's case was a success story regarding HIV. For example, an HIV case manager expressed doubt: “Have we succeeded? I wonder …. I don't think anybody can learn from Japanese public health strategy.” Similarly, an HIV counselor at another NGO stated:

To be honest, I don’t know whether Japan has succeeded in prevention. I think Japan conceals more … that you might be infected … In Japan, you have to meet people working at the health center if you go there for testing. And you don’t want to do that. You prefer not doing anything…. People don’t get tested…. And that's why it is not showing up in the numbers …. why the numbers are small …. I don’t think it is a success.

Puzzle 2: why was the recent trend in the number of new HIV cases a flattening plateau?

Similar to the issue in Puzzle 1, most participants questioned this portrayal of Japan's HIV situation. Some qualified that while such a trend might be the case nationally, it was not accurate in their local or regional epidemics. They typically perceived a worsening HIV situation, based upon their own lived experiences and their observations of clients and patients at risk of or living with HIV. This frustrated them. There was also no belief that there had been dramatic improvements in HIV prevention to account for the significant change in the epidemic curve presented in official reports. For example, an employee of a love hotel, where couples came to have sex and sex workers frequently brought clients, discussed her interpretation of MHLW statistics:

These are reported numbers, right? … We have to know that these are not the actual numbers of current Japan …. The number [of people with HIV] is, in a word, not going down …. Because I work at a so-called love hotel, my work is actually very much related to this…. About half of the guests are female sex workers and their male clients…. They still have unsafe sex – because I go into the room after they leave, and the rate of condom use is evident …. We put a condom on the bed as a service, but … there are cases that I see it unused… So I wonder where their sense of ease comes from… Like heterosexual men … when I talk to them, they believe that they are all right … I don’t know where their weird confidence comes from.

This narrative was indicative of a broader pattern of obstacles to safer sex in the sex industry. Participants reported that unprotected vaginal and anal sex were common among non-brothel-based (“delivery herusu”) sex workers, such as those who frequented the love hotel.

Interviews and observations indicated that there were also protective features of the sex industry that reduced HIV risk, although these were offset by limitations. One example was the built environment in MSM sex clubs. Many had box-style designs in which customers occupied tiny, individual, adjoining box rooms. These were set up for mainly non-anal sex, as one participant – a sex club employee – explained: “Basically, oral sex. Well, we don’t have a shower. So, well, there are people who have anal sex, but many people don’t do it.” Yet, there was reduced access to free condoms and lubricant compared to the past, especially in MSM sex clubs. This did not favor condom use when people did have anal sex in these settings. There were also protective norms in heterosexual brothels. For example, according to interview data, much of the sex in two major types of brothels (herusu and “pink salons”) was neither vaginal nor anal; it was often only oral or manual. In one type of brothel where vaginal and anal sex did occur (“soaplands”), condom use was the norm; but there was no universal policy in heterosexual brothels overall that required condom use from all sex workers and their clients during all sexual encounters. Many sex workers at brothels tested for HIV and STIs monthly, but unfortunately, this was at their own cost.

Negative perceptions of the HIV situation were also present outside the sex industry. For instance, a local elected official representing a city ward (i.e., district) had access, via his position, to some unofficial numbers that he asked me not to report. He was pessimistic when comparing his district, where he stated there was an increasing number of HIV-positive persons, to MHLW data:

This might be the case when you look at Japan nationally; but if you zero in on my ward, I do not think [HIV] is low in any way…. I do not think the awareness-raising projects for prevention have succeeded …. There are [number redacted] who have disability certificates indicating they have HIV. This can never be considered a low rate. And my ward is not the only one.

A program coordinator at an HIV CBO compared what she had observed in her clients with MHLW's reports for 2011 and 2012, the most recent data at the time:

According to the report, the number of newly infected is decreasing … two years in a row. I cannot think that it is really decreasing … because people's behaviors have not changed, and the number of tests is decreasing.

Study data indicated the presence of strong partnerships between the government and a well-developed, non-governmental sector of HIV interventionists. MHLW provided support for NGOs and CBOs working mainly with four priority populations for prevention (MSM, sex workers, immigrants, and youth); there were generally good relationships and cooperation among NGOs and CBOs, comprised significantly of volunteers; and at least one HIV prevention coalition existed across sectors during the study period. To the extent that the recent flattening trend might represent an actual stabilization of Japan's epidemics, this could have played a significant role as a constraining factor. However, when I offered this hypothesis to participants, most were not convinced. Instead, some advanced another explanation that implicated the government. It was this: leadership changes approximately every two years in public institutions were a common practice, particularly in public health centers; this stifled the development of innovative HIV prevention and testing programs and the sustainability of existing programs; and as a result, the HIV situation continued to worsen. An excerpt from my field notes summarizes a conversation with “A-san,” who had worked on prevention projects with one public health center (“PHC 1”):

A-san and I also talked about working with staff at PHC 1 …. She said it is difficult because … the director of a section with whom you make initial contact and plans for a project will be transferred out in two years. That, combined with what she described as a typical … “public employee” [koumuin] mindset in Japan, made it difficult to start and sustain projects … They don’t want to commit to new projects that will 1) add to their workload; and 2) have difficulty continuing when they transfer out. They don’t want to be responsible for passing the buck, or feel responsible if the project falls apart. She said they come into a position and try to maintain status quo, make no waves, not mess anything up, and pass it along to the next person in their position the way they found it.

Puzzle 3: why had AIDS continued an increasing trend for so many years with a dip in 2012?

Although the decrease in the number of AIDS cases nationally in 2012 was notable because it had been the first since 2005, participants focused more on what they discerned to be an almost unrelenting worsening of the AIDS situation over three decades. The consensus was that this was due to people testing for HIV late or not at all, and that late testing continued to worsen. Participants observed that most new AIDS diagnoses were occurring when people sought medical care for illness that was actually undiagnosed, advanced HIV disease. This was exacerbated by stigma and secrecy, which discouraged many living with HIV from disclosing to sex partners and others. A public health nurse working in disease prevention at a public health center discussed her catchment area: “Certainly, the number of AIDS patients is increasing …. There are more people whose HIV is detected after it has developed into AIDS … They do not go to the hospital until they have symptoms.” This was confirmed by an outpatient nurse at a hospital: “In this hospital, it reached 30% last year …. We think it is a problem …. The number of such patients has been increasing … for about the past three years.” The situation at a private medical clinic was more acute. The physician director discussed:

After 2009, more than 50% of patients who were diagnosed with HIV at my clinic did not at first think of HIV …. They visited me because of high fever, significant fatigue, prolonged diarrhea, lesions of unknown reason, and so on. When I recommended an HIV test, all of them were surprised…. And most of them concealed their HIV infection, so people could not learn about HIV from HIV patients. Of course, I do not condemn them …. I want to point out that the most important problem is stigma in Japanese society.

An HIV test counselor spoke of the testing issue:

I think it was two years ago, when we offered testing, there were as many people as the limit we had; and we had to reject some of them. But last year, in 2012, the number of people coming to test did not reach the limit. So it is indeed decreasing.

Inconsistent knowledge about HIV contributed to people testing late or never. This started with gaps and deficiencies in sex education curricula in schools, particularly in terms of limited content on HIV and sexual and gender minorities. Though there had been improvements in recent years, these were heterogeneous in geographic coverage, school policy, and implementation by individual teachers. Across sources, the ethnographic data indicated that the legacy of decades of inadequate curricular attention, especially during critical years of child and adolescent development, contributed to homophobia, transphobia, and HIV-related stigma; and it continued to adversely affect perceived risk and testing behaviors in at-risk populations. The elected ward official discussed the lack of HIV content in schools:

It is necessary to work more broadly for prevention for the young. The city government does not tell the teachers whom it employs in the municipal schools to teach children about HIV. They teach about infection through blood or mother-to-child transmission, but not sexual infection…. I think this is big…. It is a very big deal that this is not done in schools.

A staff member at an HIV CBO discussed the problem broadly, including the variability in content that I also found in document analysis of sex education guidelines:

When I was doing the test-counseling services, I was frequently talking to younger people at the site, and they were not using condoms. That's why they came to test. They didn’t get appropriate education on condom use …. They are only allowed to talk about pregnancy and maybe some of the STIs – not deeply about safer sex …. Depending on the teacher … the school, school principal … they are really different …. Usually they don’t talk about sexual orientation, gender issues …. “You can be OK if you are not heterosexual.” You know, nobody will tell you that in Japan, basically. So, if you are questioning your sexual orientation, it's very hard …. I can’t even imagine.

A “mostly-retired” sex worker, who also provided HIV and STI awareness trainings, stated:

Most Japanese people don’t have knowledge about sexual health … When I was a student, they taught us only the mysterious aspects … things that we cannot use … I sometimes go to universities to give talks, and the students don’t understand at all… I feel disappointed.

Inconsistencies in awareness extended also to providers and other interventionists and stakeholders in HIV epidemics, among whom significant discrepancies in current and accurate knowledge about HIV were evident. For example, a line from my field notes following observations at a hospital stated, “B-san, a nurse who works with HIV patients, seems to be confused on the difference between HIV and AIDS.” One topic about which there was a general lack of awareness among interventionists and community members alike was biomedical HIV prevention technologies, such as pre-exposure prophylaxis (PrEP) and non-occupational post-exposure prophylaxis (PEP). This was related to the government's not having approved these prevention strategies and was indicative of Japan's overall reliance upon health education and behavioral intervention models. Other technologies, such as nucleic acid tests (NAT) for HIV (also called polymerase chain reaction [PCR] tests), which enable detection of infections as recent as 10 days prior to screening, were available in some clinical settings, but people typically were not aware of this. Another excerpt from my field notes illustrates:

C-san, who is male-to-female transgender, has never heard of PEP or PrEP, NAT or PCR testing, or the concept of “antibody” testing …. She believes having no symptoms means there is no need for STI tests. She has no perceived HIV risk … though, based on our conversation, she seems at considerable risk.

Participants generally had not noticed decreases in new AIDS cases in 2012 in their local areas, but they still offered explanations for the putative national dip as reported by MHLW. Mainly, they credited an advanced medical system and effective HIV treatment and care. Triangulated data across sources aligned with this view and indicated that if and when people did receive a positive test result, the HIV care continuum (i.e., linkage to medical care, maintenance of ongoing treatment, viral suppression to prevent progression to AIDS) typically worked well in Japan – at least in the urban areas of the study. This included good access to care for most people, subsidies for antiretroviral medications, and as a result, a typically low direct cost of treatment to patients. However, the system worked less well for people in rural areas, who often traveled long distances to Tokyo and major cities in Kansai to receive quality HIV care. A pharmacist described the benefits offered at the public hospital where he worked, which took patients regardless of ability to pay:

People can go to the hospital and have treatments even if they don’t have money, and I think that is a good point. They can get medicines …. They can be saved…. Even homeless people can get treatment here when their HIV is detected.

An HIV counselor discussed both sides of the issue. HIV care was not free for most people or in most hospitals, but he gave credit to subsidized medical expenses, which were widely available: “They really work well. I think the patients pay 10,000 or 20,000 yen (approximately $100–$200) per month…. The system has done a good job.” He later added:

People living in rural areas … cannot get good medical treatment like people in Tokyo or other big cities … It's possible that people have only one hospital where they can go. And when the level of care is low at that hospital, the situation is really bad. Doctors who treat HIV patients are not infectious diseases specialists …. In many cases … they use old medications that are not used anymore.

Puzzle 4: why had HIV testing decreased then stagnated after a peak in 2008?

Several factors combined in 2009 and afterward to curtail testing. First, the H1N1 swine influenza pandemic of 2009 became the primary concern in Japan's public health centers, which had the effect of diverting staff, resources, and activities away from HIV counseling and testing. People were therefore discouraged from seeking these services from public health centers, which were perceived as the primary sites of HIV testing in the country. Additionally, to limit their potential exposures to H1N1, citizens generally avoided nonessential visits to crowded places, particularly facilities associated with health care like public health centers and private clinics. The effect was also apparent in HIV testing venues used by NGOs, some of which were temporarily closed in 2009 for this reason. An infectious disease physician who worked regularly with an HIV CBO stated, “We were affected. Our testing site actually was closed from May to June.”

Second, in the period 2005–2008, there had been many advertisements and social marketing campaigns for HIV testing, in addition to ample government funding and attention. The campaigns included collaborations between JFAP and Advertising Council Japan, involved celebrities, and were wide credited with maximizing testing and awareness in the country – but only temporarily. Starting in 2009, there was a noticeable reduction in both the volume and visibility of these campaigns. According to participants, national and local government HIV budgets also were reduced, affecting institutions like public health centers and NGOs/CBOs engaged in HIV testing and counseling. Even where HIV budgets were not reduced, there were reports of funds being redirected from prevention activities to treatment and care. In an excerpt from my field notes, I summarize part of a conversation with the 64-year-old veteran owner of multiple gay bars and clubs:

He blamed most of the problem on … decreased media coverage, especially TV news; fewer advertisements for HIV, testing, and prevention; fewer ads from [name of local HIV CBO focused on MSM] compared to the past – which was all linked to less financial support from MHLW.

A hospital-based nurse commented about her local area:

I think the government stopped supporting testing. There were three places where people could test for free, but now there is only one… It increased the number of patients who come to the hospital after already developing AIDS. I think it's a problem that it is not easy to get tested.

Third, starting in 2009, there was a decreased sense of alarm and generally not much public interest in HIV. There was particularly broad consensus on this finding, and it was a source of significant concern, particularly among those working professionally in HIV prevention and medical care. The clinic physician was not optimistic: “Stigma and discrimination bring a lack of interest in HIV …. The lack of interest in society makes HIV risk higher …. This lack of interest will remain for the time being.”

An interviewee working in the private sector linked the second and third factors:

As a priority, we have to sell HIV home testing kits, so I remember that it had an effect on us, too. It's about the level of social concern… It had its peak in 2008, and in that year, there were many TV commercials … various things like big events, concerts. Usually, events like that were led by MHLW, and in 2009 … the ads for HIV tests gradually disappeared… I think the effect was quite large.

Fourth, there were several barriers to accessible testing options. Most sites that offered free HIV testing had very limited service hours. For example, public health centers in Kansai cities typically offered testing for only 90–120 min between 9:00 a.m. and 3:30 p.m., sometimes Monday through Friday, often only one weekday each week, and as infrequently as once every two months. Few offered evening or weekend hours; and if they did, it was only twice per month. Tokyo was similar, with most public health centers offering testing only one weekday for 60-120 min, once or twice per month.

Some NGOs and private clinics offered tests at more convenient times. For example, one HIV NGO offered testing on Sundays, but with a limit of only 40 people. I observed people lining up for this service, trying to obtain priority wristbands for testing and demonstrating demand for expanded hours. Rapid tests with same-day results were available but mainly limited to a few NGOs and private clinics. Like all HIV tests in Japan, these were blood tests. Other biological specimens (i.e., oral solution, urine) were not an option for testers. Nucleic acid tests (NAT), which enable detection of infections as recent as 10 days prior to screening, were even harder to access. One private clinic where I conducted repeated observations offered rapid antibody testing and NAT five days each week, including Saturdays, for seven hours each day; however, these services came at a cost: about $40 for a rapid test and $150 for a NAT.

Puzzle 5: why were there more men than women among reported heterosexual infections?

Two main reasons emerged. The first was that pervasive homophobia, even in urban centers, pressured men to report HIV infection via heterosexual transmission, whether that was true or not. It was likely that most of the male infections reported as heterosexual were actually cases of sexual transmission among MSM. An HIV physician explained:

The proportion of infection from sexual interaction between men is higher that it appears …. When they are first asked about the possible route of infection…. many of them say they don’t know, or they lie and say they went to a “soapland” [type of heterosexual brothel]. If they answer like this, we mark “heterosexual interaction….” There are many “unknowns”… It is possible that they do not provide true information, depending on their application for health insurance coverage – which is based on statistics when they are diagnosed with HIV: their age, route of infection…. I think that's why the ratio of heterosexual women is low. There should be more women if there are people infected through heterosexual sex.

The second explanation was that heterosexual women in Japan used condoms as their main form of birth control, which had the added benefit of protecting them from HIV infection by male partners. Several participants agreed that this was a contributing factor; however, interview data also indicated that this adjuvant protection might be eroding as other contraceptive methods, particularly the birth control pill, were recently becoming more popular after decades of little use. The mostly-retired sex worker discussed this:

Japanese women don’t take the pill very often…. Sex workers do a bit more often than other women …. It is not advertised. Women generally don’t have a positive perception of them….We don’t know who takes them and who doesn’t. There is an air that makes us feel it's difficult to talk about … between women, and also between men and women. For many, condom use is more common.

An HIV case manager was more critical of this hypothesis. Using member checking, I asked her to comment on what I had heard from others about women and condoms:

Until a few years ago, that could be one reason. But so many women take pills nowadays …. It's been increasing …. That's one question we always have: why women's rates are so low in Japan. That's one thing that we can’t figure out.

Puzzle 6: why, with increased age, were Japanese men with sexually-acquired HIV more likely to be reported as heterosexual infections?

The same underlying homophobia that largely explained Puzzle 5 was also the main reason for this trend. The double stigma of being HIV-positive and having acquired the virus through sex between men was felt more acutely with advanced age; therefore, most of the men in the higher age groups who reported heterosexual transmission probably acquired HIV homosexually and were intentionally misreporting. As the same HIV physician continued to discuss:

I think it is because of the ‘wisdom of age’ …. The younger generation tends to admit their sexuality. Most of my patients in their 50s or people who’ve already developed AIDS don’t say they are gay …. They say they don’t know how they got it. For example, they, “went to Thailand to have sex with commercial sex workers” when they were younger …. But really, the older they get, the less they talk about their sexuality. Even when his partner is beside him, he says that person is his friend. Not his sexual partner. Just a friend.

MHLW surveillance reports have highlighted that the number of AIDS cases among persons aged 60+ has increased in recent years, and that men who reported heterosexual infection were more likely than any other group to discover they had AIDS when they first learned they were HIV-positive. These reports, together with HIV experts’ observations of ubiquitous unprotected sex and a lower level of HIV awareness among older MSM, have prompted the first prevention programming aimed at this population in Japan. For example, one HIV NGO was publishing a quarterly newspaper for middle-age and older MSM. The organization's director described these efforts:

We started publishing it last year. Well, the middle-aged gay men don’t read this [indicating another publication they produce for younger MSM] because it's written in small print – especially if they are over 60… This is the reason on the surface. But there is a deeper reason. It's that they don’t want to know about AIDS. There has been a culture like that …. In the area where people who are over 60 or 70 get together … there are only a few bars where you can find this paper, about eight …. And when we brought this to the bars … they didn’t want to get involved in AIDS awareness. There is a culture like this in [name of area]. We created this paper to change this …. The contents are quite different, too [compares side-by-side with other publication for younger MSM]. The style is different: it doesn’t look very gay. At first glance, it looks more medical, but inside there are articles about culture and gay anime stories set in the Edo period. It's an exploration of history and culture.

Puzzle 7: why did official surveillance reports not include national-level incidence rates, mortality rates, or prevalence for the general population?

Study data point to two possible explanations. The first is that Japan did not have a particularly strong HIV surveillance system. For instance, an HIV prevention consultant, who had witnessed the entire history of the epidemic in Japan and had been involved in the government's response over the decades, reported that the country's sentinel surveillance was underdeveloped; and this precluded solid estimation of incidence in the general population:

I think this is the system, OK? You should get incidence from sentinel surveillance, like new military conscripts, pregnant women, or the rate of some selected population … closer to the general population. But in Japan, that is not done …. There is some surveillance on people, but we cannot say how representative it is of the population …. there are problems of accuracy.

According to a current MHLW surveillance specialist, the Ministry simply did not have rates for new HIV and AIDS cases in the national population. His explanation was that Japan, “does not have enough long-term results yet.”

There was obvious confusion among study participants about the original sources of surveillance numbers, even among interviewees who worked at or in collaboration with MHLW. For example, the surveillance specialist explained that the numbers were derived primarily from HIV tests at public health centers, and that MHLW typically did not receive testing data from private clinics or hospitals. However, document analysis indicated that all physicians were mandated by law to report new cases to their prefectural governor, ward mayor, or local public health center within seven days of diagnosis. In theory, then, MHLW should have been receiving data from private clinics and hospitals via prefectural and local government institutions. In practice, the system did not appear to be operating in such a straightforward manner; and it was only through data triangulation from multiple sources that this latent process was revealed.

The surveillance figures did not include data from HIV home test kits, although these were growing in use mainly because access to venue-based testing was limited and HIV remained highly stigmatized. At public health centers, HIV testing remained anonymous, and the centers’ procedures prevented MHLW from distinguishing between single and multiple tests by the same person. Donated blood was regularly tested using NAT, and HIV incidence among blood donors was included in official surveillance reports. However, no other NAT data were included in the numbers, so NAT results from personal HIV screening at private clinics were not being counted.

All of this indicated that MHLW potentially was missing a large amount of data. Posts from Japanese citizens on blogs, HIV organizations’ web pages, and other websites underscored this point. They questioned MHLW data on AIDS-related deaths, criticized the absence of information on the number of people living with HIV, and expressed a lack of faith in MHLW's surveillance system, overall. For example, on one website, a person wrote, “I heard that MHLW doesn't report what it knows to be the number of HIV-infected people to the mass media for fear of causing a panic. Is that true????” Another site, which posts people's text messages, included 1199 entries, mostly supportive, on the thread entitled “Don't trust MHLW's AIDS statistics!” (exclamation part of title).

The second possible explanation for this puzzle is that Japan's surveillance system was not the deciding factor, but instead that MHLW's practices regarding what information is made publicly available were unconventional. The strongest evidence in favor of this explanation is the information that the surveillance reports do include, such as HIV and AIDS incidence rates by region; HIV (only) incidence rates by age group and among blood donors; cumulative HIV and AIDS case reports; and some mortality data in appendix tables. If MHLW had enough confidence in these estimates to report them, it stands to reason that they could calculate and report estimates for the national population.

Discussion

Contextualization of main findings

Using a ground truthing conceptual framework, I found that participants representing various stakeholder groups were pessimistic about the HIV epidemics not only in their local and regional areas, but also nationally; that these epidemics were experienced and understood as worse than MHLW reports indicated; and that there were significant problems with available surveillance data. In short, what I found on the ground often contradicted official knowledge, sometimes quite sharply.

One prominent example was the trend in the number of new HIV cases. Official surveillance reports painted an optimistic picture: a flattening plateau nationally; and when disaggregated by region, an even more favorable outlook on the areas investigated. Although the slopes for new cases since 2008 in most regions were flat or upward, the slopes were actually downward in the two largest epidemics: slightly down in the Kansai region and more steeply down in the Tokyo metro area (Fig. 3) (Ministry of Health, Labour and Welfare, 2014). In marked contrast, study participants typically perceived a worsening HIV situation characterized by more uncounted cases in Kansai and Tokyo compared to the past.

Fig. 3

New HIV cases by region, 1985–2013 (Source: own construction, data from Ministry of Health, Labour and Welfare (2014).

Such pessimism has been rarely expressed in the literature on Japan. Notable exceptions include Nishijima et al., who determined that the country's prevention efforts had been unsuccessful (Nishijima et al., 2015); and Kihara and Komatsu, who suggested that Japan's prevention capacities were underdeveloped due to a very small national epidemic (Kihara & Komatsu, 2003). Both papers based these conclusions upon official surveillance data. To my knowledge, the current study is the first to find a broader sense of pessimism regarding local and regional Japanese epidemics based largely upon the firsthand experiences of local stakeholders.

Another, more complex example was the trend in new AIDS cases. Participants reported having experienced or observed a virtually unabated exacerbation of the AIDS situation over decades. This view differed conspicuously with regional surveillance data, which indicated that Tokyo's new AIDS cases started flattening out in 2002, and Kansai's AIDS slope had been flat since 2009 (Ministry of Health, Labour and Welfare, 2014). However, the local knowledge on this point more or less aligned with MHLW's national-level data (see Fig. 2).

Even where there was not direct contradiction, information acquired at the ground level provided context and supplemented the government data. For example, surveillance reports had documented the persistent problem of at least 30% of new HIV cases already having advanced to AIDS at time of diagnosis (Ministry of Health, Labour and Welfare, 2014). The ethnographic data provided explanations for this, including a predominance of late testing at the local level; surprisingly inconsistent knowledge about HIV; and a legacy of inadequate attention to HIV and sexual and gender minorities in sex education curricula.

Although most of the results were novel in Japan, my findings on waning public interest and inconsistent knowledge about HIV, barriers to testing and late testing, gaps and deficiencies in sex education, and obstacles to safer sex complement research in Japan by other authors (Kihara et al., 2003, Kihara et al., 2002, Iwamuro, 2007, Nosaka and Chigusa, 2008, Japan International Cooperation Agency, 2010, Horino et al., 2016, Ishimaru et al., 2016). My data on how the swine flu pandemic of 2009 was a factor in curtailing HIV testing were also supported by Japan's National Institute of Infectious Diseases, which speculated that a reduction by 16% in HIV consultations and testing might have occurred for this reason (National Institute of Infectious Diseases, 2010). Koerner and Ichikawa (2011) cautioned that if true, this event exposed a weakness in the HIV testing and reporting system.

Data from the present study indicated that most male infections reported as heterosexual were probably cases of sexual transmission among MSM; that this was due to pervasive homophobia; and that this effect was more apparent in older men. Support for this finding is found in research by Ichikawa (2009), who stated that many HIV cases officially classified in the unknown category in Japan might be MSM infections; Inoue, Takaku, Yajima and Ikushima (2015), who found that among HIV patients who originally reported heterosexual contact as the route of HIV transmission at time of diagnosis, 35.7% had given a false reason and 28.6% actually had had same-sex contact ; and Eguchi, Wada & Smith (2014), who reported that older age was associated with increased prejudice toward persons with HIV.

A related explanation for Puzzle 5 was that heterosexual women tended to use condoms for birth control, which also protected them from HIV; but that this protection was eroding with recently increased use of the birth control pill. Research as recent as 2010 confirmed that condom use was the most common conceptive method in Japan, and that the pill had not been approved for use until 1999 (Japan International Cooperation Agency, 2010). Within two years of its approval, another study found that use of the pill by commercial sex workers in Japan was associated with irregular or no condom use (Kimoto, 2001).

I also found that HIV occurred in a highly stigmatizing environment. My findings related to stigma were grounded in an especially large amount of data and related to a larger literature on HIV syndemics. This merited detailed treatment in a separate paper (DiStefano, 2016).

The surveillance issue

Most participants were skeptical of official numbers generated by MHLW. Triangulated study data indicated that, at best, the government agency had some unorthodox reporting practices. At worst, Japan's surveillance system probably was not capturing a significant amount of HIV testing data. Over a decade ago, other authors identified a few of the same problems that I found. These included high proportions of HIV and AIDS cases in the unknown route of infection category, and processes that made it impossible to eliminate duplicate cases (Kihara et al., 2003, Nemoto, 2004). Kihara et al. (2003) concluded, “In view of its essential importance … Japan's HIV/AIDS surveillance system should be strengthened and updated without delay”.

It appears that MHLW has been aware of at least some of these issues for many years. In 2000, the Ministry issued national guidelines that emphasized the importance of improving HIV surveillance (Kihara et al., 2003). As recently as 2013, the Institute for Infectious Diseases, which is organized under MHLW, stated that the surveillance system was missing many HIV-positive persons during earlier stages of infection, which was a major reason for Japan's high proportion of initial AIDS cases (National Institute of Infectious Diseases, 2013). However, it is unclear exactly what MHLW has been doing to address these issues in a concrete way. Data in the present study suggest that the Ministry's efforts have not brought significant improvements to the surveillance system.

At the time of this writing, the most recent surveillance data signal continuation of HIV's stabilizing trend. Final data from 2014 and preliminary data from 2015 indicate that new HIV cases are holding steady at approximately 1000 per year, having decreased by 15 cases in 2014 and by 86 cases in the first three quarters of 2015. The number of new AIDS cases also have decreased in the past two years: reduced by 29 cases in 2014 and by 7 cases in the first three quarters of 2015 (Ministry of Health Labour and Welfare, 2014, Ministry of Health Labour and Welfare, 2015). Thus, after three years of oscillating rises and falls, following by two consecutive years of decreases, it is appropriate to speculate whether Japan's AIDS epidemic, too, is finally plateauing.

Although MHLW data continue to provide reason for optimism, the results of the present study call into question widely held beliefs about low HIV occurrence in Japan and decrease confidence in officially reported epidemiologic trends, including current stabilization of the national epidemic. Positioned in a ground truthing framework, however, this dissonance can also signal positive opportunities (Sadd et al., 2014). For example, the ethnographic findings highlight local and regional HIV concerns that have received little attention in the national HIV discourse in Japan; and they provide novel empirical evidence from a different – that is, not purely epidemiological – perspective that can be leveraged as rationale for policy and other systemic changes related to HIV.

Recommendations and directions for future research

Changes might begin with the surveillance system. Data collection and reporting processes should be revised. The system should broaden sources of HIV testing data and establish a reliable method of estimating incidence and prevalence. To achieve this, MHLW should strongly consider moving from anonymous HIV testing to a confidential, names-based system; and both sentinel surveillance and reporting of AIDS-related mortality should be strengthened.

These recommendations also imply a need for additional study and improved evaluation of the surveillance system. Kuhn(1996) emphasized the need to study the community structure of science – that to understand scientific knowledge, one needs to understand “the special characteristics of the groups that create and use it”. A future ethnographic study of MHLW's HIV units is thus warranted. Process evaluations, structured according population health surveillance theory (El Allaki, Bigras-Poulin, Michel & Ravel, 2012), should accompany solid mechanisms for outcome evaluation – both of which might require building new capacity.

Several other recommendations beyond those related to surveillance are indicated. First, Japan should scale up strategies that incorporate factors constraining their HIV epidemics, as identified in this study. As one example, the country's generally strong HIV medical care emerged as a constraining factor. Indeed, UNAIDS recently reported that 96% of HIV-positive persons in Japan who had initiated antiretroviral therapy were still on treatment 12 months later, an important measure of the HIV care continuum (UNAIDS, 2012e). However, the present study also showed that access to that care was significantly diminished for those living in rural areas. Expansion of equitable HIV medical access to rural populations should be a priority.

HIV testing should be made more accessible and convenient. Increased access to venue-based testing would likely increase testing rates, and similar to the U.S. and other countries, expansion of NAT availability for screening is recommended, which could result in decreases of HIV transmission in the highly infectious acute stage. Data from a recent study commissioned by MHLW show that the number of HIV tests completed using home test kits has increased steadily since 2001 and comprised 35.1% of total tests in 2013 (Kato, 2014). Additional research is needed to gauge the effect this is having on HIV epidemics in Japan.

Both regional and national social marketing campaigns for HIV testing are needed again and should be sustainable. The 2005–2008 campaigns were mentioned by many participants as a major reason for the 2008 peaks in HIV testing and reported new cases, though testing appears to have been increasing sharply since 2002. There was also another time when this occurred, and more dramatically: in 1992, there was a dramatic spike in testing and cases (Fig. 1). Study narratives for this time period centered on the fact that it was the AIDS Panic era and a time when foreigners, particularly female sex workers from Asian countries, were a focus of the panic. Only two interviewees mentioned an increase in HIV cases around this time, and no one brought up the spike in HIV testing. Other authors have discussed this epidemiologic event (Nemoto, 2004, Kihara et al., 1997), but few explanations are offered for why it occurred. Historical research on the 1992 peak is thus indicated to inform the design of new social marketing campaigns and other strategies to increase testing.

Condoms and lubricant should be made more freely available, especially in MSM sex clubs. The cost can be added into customer fees, or new government subsidies and partnerships with condom manufacturers might be sought. In heterosexual brothels, a100% condom use policy should be implemented; and infractions should be costly to brothel owners, not to sex workers (Pisani, 2008). Thailand's policy for commercial sex venues provides a successful model (Rojanapithayakorn & Hanenberg, 1996). Similarly, monthly HIV and STI testing for sex workers should be required and the cost shifted to their employers, consistent with an occupational health perspective (Grudzen & Kerndt, 2007). Multiple trials have demonstrated that PrEP and nonoccupational PEP are effective in preventing HIV infection (WHO, 2015, WHO, 2014). The Japanese government should approve both and consider policy changes necessary for health insurance to cover them. Additionally, Japanese public agencies should seriously consider how the system of rotating employees every two years affects HIV prevention and testing programs.

Limitations

Results were derived from data collected in only two regions in Japan, and I used nonprobability sampling to recruit participants and collect documents. This limits generalizability of the results. Interview data were susceptible to reporting bias in two ways. First, they involved participants’ self-report of sometimes stigmatized topics. Second, when I asked what lessons other countries could learn from Japan's apparent HIV successes, cultural humility could have played a role in participants' downplaying Japan's merits and insisting that it was not a role model. Limitations to observational data included possible reactivity and social desirability bias. My positionality as a public health and social scientist who teaches both ethnographic methods and epidemiology could have introduced bias into the results, mainly derived from questions I chose to ask, questions I did not ask, and my interpretation of the data. I took rigorous and multiple measures to minimize all such effects as described in the methods section.

Conclusions

The ground truth in Japan's two largest HIV epidemics raises significant doubts about official knowledge in the form of government surveillance data. This uncertainty, considered with factors that I identified as driving HIV in Japan, could be masking a national epidemic that is much larger than official numbers indicate. At the same time, even if it is indeed worse that the numbers make it appear, other factors that I identified as constraining HIV make it possible that “worse than it appears” for Japan might still equate to a truly lower HIV burden compared to most countries. Perhaps before lessons for other countries can be determined, it first will be necessary to obtain a more accurate epidemiologic account at the national, regional, and local levels. As in other countries, epidemiology tends to drive HIV policy in Japan; so it is important to get it right. It is my hope that the results of this paper help to more clearly define these issues, clarify the mechanisms by which they are operating, and identify potential ways to move toward solutions. Finally, based upon its high level of utility in shaping this study's design, the questions asked, and interpretation of results, ground truthing has demonstrated value as a conceptual framework for social science research on health and shows promise for future theoretical development.