Disorders specifically associated with stress: A case-controlled field study for ICD-11 mental and behavioural disorders.

As part of the development of the Eleventh Revision of International Classification of Diseases and Related Health Problems (ICD-11), the World Health Organization Department of Mental Health and Substance Abuse is conducting a series of case-controlled field studies using a new and powerful method to test the application by clinicians of the proposed ICD-11 diagnostic guidelines for mental and behavioural disorders. This article describes the case-controlled field study for Disorders Specifically Associated with Stress. Using a vignette-based experimental methodology, 1,738 international mental health professionals diagnosed standardized cases designed to test key differences between the proposed diagnostic guidelines for ICD-11 and corresponding guidelines for ICD-10. Across eight comparisons, several proposed changes for ICD-11, including the addition of Complex PTSD and Prolonged Grief Disorder, produced significantly improved diagnostic decisions and clearer application of diagnostic guidelines compared to ICD-10. However, several key areas were also identified, such as the description of the diagnostic requirement of re-experiencing in PTSD, in which the guidelines were not consistently applied as intended. These results informed specific revisions to improve the clarity of the proposed ICD-11 diagnostic guidelines. The next step will be to further test these guidelines in clinic-based studies using real patients in relevant settings.

The World Health Organization (WHO) is currently revising the International Classification of Diseases, with its Eleventh Revision (ICD-11) expected to be approved by the World Health Assembly in 2018. Disorders Specifically Associated with Stress is a new grouping proposed for the ICD-11 chapter on Mental and Behavioural Disorders that is intended to capture disorders that in part reflect maladaptive reactions to stressful or traumatic events; that is, a history of a stressful or traumatic event is necessary, but not sufficient, to produce the psychopathology represented by the disorder (Maercker et al., 2013a, Maercker et al., 2013b). The proposed ICD-11 grouping of Disorders Specifically Associated with Stress includes some diagnoses that exist in various disorder groupings in the ICD-10, including Post-Traumatic Stress Disorder (PTSD), Adjustment Disorder, and Acute Stress Reaction, as well as two new diagnoses: Complex PTSD and Prolonged Grief Disorder (see Maercker et al., 2013a, Maercker et al., 2013b)*.

Mental disorders specifically associated with stress are relative newcomers to psychiatric classification. The ICD-8, approved by the World Health Assembly in 1965, introduced a category of “Transient situational disturbance”, which included adjustment problems, severe stress reactions, and combat neurosis. The ICD-9, approved in 1975, specified two separate disorders: Acute stress reaction and Adjustment reaction. The ICD-10, approved in 1990, included two additional disorders as independent diagnoses: PTSD and Enduring Personality Change after Catastrophic Experience. The latter diagnosis was described as sometimes following long-term exposure to extreme stress (e.g., concentration camp imprisonment or torture) and as sometimes being preceded by PTSD.

The ICD-11 Working Group on Disorders Specifically Associated with Stress was established by the WHO Department of Mental Health and Substance Abuse in 2011 to: (a) review available scientific evidence, (b) assess clinical and policy information on the use of these disorders, including perspectives from specialist and primary health care settings throughout the world; (c) review proposals for DSM-5 and consider their suitability for global applications; (d) assemble and prepare specific proposals, including the placement and organization of relevant categories; and (e) develop drafts of the content (e.g., definitions, descriptions, diagnostic guidelines) with a particular focus on improving clinical utility of relevant diagnostic categories (see First, Reed, Hyman, & Saxena, 2015).

Previous WHO surveys conducted in collaboration with partnering international psychiatric and psychological organizations showed that Disorders Specifically Associated with Stress categories rank high in clinical use in daily practice. For example, PTSD was the 7th most commonly used category among psychiatrists (Reed, Correia, Esparza, Saxena, & Maj, 2011) and 8th among psychologists (Evans et al., 2013). These surveys also asked participants to suggest diagnostic categories they felt should be added to the classification. Complex PTSD was the most frequently recommended category to be added, and Prolonged Grief Disorder (or some other form of pathological bereavement) was the 11th most commonly recommended category (Robles et al., 2014). The Working Group took these results into consideration when reviewing available scientific evidence on the validity of eligible categories as well as clinical and policy information on their clinical utility.

Field trial process and rationale

WHO has emphasized the importance of clinical utility in the revision process, and has defined clinical utility in relation to a range of implementation characteristics related to a particular category (International Advisory Group, 2011, Reed, 2010). One stated goal of the ICD-11 revision process is to develop a classification that is as useful as possible for service providers who are in the best position to identify and offer mental health services to individuals in need. Thus, field studies of the proposed classification should investigate the reliability—and to the extent possible, the validity—of its diagnostic guidelines as well as how clinicians understand and utilize those guidelines. The Field Studies Coordination Group (FSCG), which reports to the International Advisory Group for the Revision of ICD-10 Mental and Behavioural Disorders, is responsible for overseeing the ICD-11 field studies and for providing recommendations to the relevant Working Groups to improve their proposals based upon field study findings. The FSCG acts as an independent entity in the oversight of the field studies to counterbalance possible biases Working Groups might have in reviewing their own material or interpreting the results of related studies. The WHO Department of Mental Health and Substance Abuse, in collaboration with the FSCG, has designed a series of specific studies to test proposed ICD-11 diagnostic concepts and guidelines (see Keeley et al., in press, for a rationale and more thorough description of the field study strategy).

The first phase of field testing consists of case-controlled field studies administered over the internet among members of the Global Clinical Practice Network (www.globalclinicalpractice.net), which as of October 2015 consists of over 12,000 mental health and primary care professionals from more than 140 countries, representing diverse geographical, disciplinary and lingual backgrounds, who have volunteered to participate in field testing related to ICD-11 Mental and Behavioural Disorders (Reed et al., 2015). These studies use case vignettes as a stimulus for investigating diagnostic decision-making by clinicians, thereby controlling the variability associated with the case material in order to isolate specific factors associated with diagnostic decisions (Evans et al., 2015). For this reason, they are referred to as “case-controlled” field studies. A second phase of field testing will consist of clinic-based “ecological implementation” studies, which involve application of the guidelines to patients in relevant practice settings, where the clinical picture of the case will be less controlled and generalizability to real clinical settings is higher (see Keeley et al., in press).

As a part of a systematic program of field studies, the case-controlled field studies, administered over the internet, offer several advantages over past methods of testing diagnostic classification systems. First, the use of a standardized, written vignette as the diagnostic stimulus provides a much greater degree of experimental control (Evans et al., 2015). In clinic-based trials, the patients to be diagnosed are an unknown factor and it is generally not possible to distinguish variability related to the diagnostic guidelines from variability associated with the case presentation (Keeley et al., in press). Thus, a case-controlled field trial offers a unique ability to investigate how practitioners implement a diagnostic system, including those characteristics most central to WHO's definition of clinical utility. Second, online administration ensures a wider range of professionals can participate in the trial than would be feasible for in-person studies, in keeping with WHO's goal of including a broad range of practitioners from all areas of the world who might use the ICD-11. Third, these studies are cost-effective, as a large amount of information can be gathered at much less expense and much more quickly than would be possible in studies based in clinical settings. These case-controlled field studies are intended to provide specific feedback on how practicing mental health professionals use the diagnostic system so that diagnostic guidelines can be revised and improved before the final release of ICD-11.

This article describes the case-controlled field study for Disorders Specifically Associated with Stress, the first of the series of case-controlled field studies. This study was designed to assess how clinicians interpreted the proposed diagnostic guidelines, to investigate what factors influenced their diagnostic judgments, and to compare the features of the proposed diagnostic guidelines for ICD-11 with those of ICD-10. On the basis of the results of this study, the Working Group was asked to suggest revisions to the diagnostic guidelines in areas where study results indicated that they were not functioning as intended. The case-controlled field study for Disorders Specifically Associated with Stress focused on the specific changes proposed by the Working Group for ICD-11 (outlined in the next section). Individual experimental manipulations (see Table 1) systematically tested clinicians’ diagnostic decision making under prescribed conditions to determine whether proposed diagnostic guidelines for ICD-11 offered superior performance to those of ICD-10.

Table 1

Research questions for the case-controlled field study of disorders specifically associated with stress.

Comparison 1	Do clinicians appropriately recognize the required symptom of re-experiencing and apply it correctly as a diagnostic requirement for the ICD-11 diagnosis of PTSD?
Comparison 2	Do clinicians appropriately recognize functional impairment and apply it correctly as a diagnostic requirement for the ICD-11 diagnosis of PTSD?
Comparison 3	Do clinicians appropriately base the ICD-11 diagnosis of PTSD on the presence of the required core symptoms, or do they tend to over-diagnose PTSD based on a history of specific types of stressors?
Comparison 4	Can clinicians differentiate the proposed ICD-11 diagnostic requirements of Complex PTSD from those of PTSD? Further, does the ICD-11 diagnosis of Complex PTSD provide a better fit than the ICD-10 diagnosis of Enduring Personality Change after Catastrophic Experience?
Comparison 5	Do clinicians inappropriately diagnose Complex PTSD based on a history of a severe and long-lasting stressor rather based on the required symptoms?
Comparison 6	Can clinicians differentiate Prolonged Grief Disorder from a normal grief response based on the proposed ICD-11 diagnostic guidelines?
Comparison 7	Do clinicians appropriately differentiate PTSD from Adjustment Disorder based on the required symptoms, or do they tend to inappropriately base this distinction on the nature of the stressor?
Comparison 8	Do clinicians appropriately exclude diagnoses of Adjustment Disorder that do not evidence preoccupation and functional impairment as required by proposed ICD-11 guidelines?

Proposals for ICD-11 diagnostic guidelines and related questions

This section describes the specific proposals developed by the Working Group for Disorders Specifically Associated with Stress in ICD-11, highlighting important differences with ICD-10. (See Maercker, Brewin, Bryant, Cloitre, & van Ommeren et al., 2013, for a more complete description, rationale, and supporting evidence.) Changes from ICD-10 to ICD-11 provided the basis for the specific research questions assessed in this field study, as described below.

PTSD. The first essential feature of PTSD proposed by the Working Group is exposure to an event or situation (either short- or long-lasting) of an extremely threatening or horrific nature. The proposed diagnostic guidelines provide examples of events that are commonly associated with PTSD but do not specify required characteristics or provide a definitive list of qualifying events. In addition, the required symptomatic features of PTSD proposed for ICD-11 were substantially simplified, consisting of three core elements: (a) re-experiencing the traumatic event in the present, typically occurring as vivid intrusive images, memories, or flashbacks; (b) deliberate avoidance of reminders of the traumatic event, which may take the form of internal avoidance of relevant thoughts and memories, or external avoidance of people, conversations, activities, or situations reminiscent of the event; and (c) persistent perceptions of heighted current threat, for example as indicated by hypervigilance or an enhanced startle reaction. An additional requirement not explicit in ICD-10 is that the disturbance cause significant impairment in personal, family, social, educational, occupational or other important areas of functioning. (See First et al., 2015 for complete proposed diagnostic guidelines for PTSD). Empirical evidence for the specified essential features of PTSD was primarily drawn from studies investigating the symptom structure of previous PTSD definitions (e.g., Elklit and Shevlin, 2007, Gootzeit and Markon, 2011, Yufik and Simms, 2010).

The present study focused on three key diagnostic differences between ICD-11 and ICD-10 in the description of PTSD (see Table 1). The proposed requirement of re-experiencing in the present as an essential feature of PTSD in ICD-11 represents a substantive change from ICD-10. Thus, the first research question (Comparison 1) addressed whether clinicians could clearly distinguish case descriptions that included re-experiencing as a diagnostic requirement for PTSD from those that did not. The second question (Comparison 2) concerned whether clinicians would distinguish cases involving functional impairment from those without, and if that information would influence their diagnostic decision. The third question (Comparison 3) assessed whether clinicians would base the diagnosis of PTSD on the essential symptomatic features rather than on the characteristics of the external event. The Working Group's proposal recognizes that individuals may develop PTSD in response to events that are experienced as extremely threatening or horrific even if they are not stereotypically associated with PTSD. Conversely, individuals may be exposed to events that are commonly associated with PTSD and in its aftermath develop a depressive disorder, an anxiety disorder, an adjustment disorder, or no disorder.

Complex PTSD. Complex PTSD as proposed for inclusion in ICD-11 is not an entirely new disorder but rather represents a comprehensive reformulation of ICD-10's Enduring Personality Change after Catastrophic Experience. Complex PTSD may develop following exposure to an event or series of events of an extreme and prolonged or repetitive nature that are experienced as extremely threatening or horrific and from which escape is difficult or impossible (e.g., torture, concentration camps, prolonged domestic violence, repeated childhood sexual or physical abuse). The proposed essential features of the diagnosis include the three core symptomatic features of PTSD in addition to characteristic disturbances in three domains: (a) severe and pervasive problems in affect regulation; (b) persistent beliefs about oneself as diminished, defeated or worthless, accompanied by deep and pervasive feelings of shame, guilt or failure related to the stressor; and (c) persistent difficulties in sustaining relationships or in feeling close to others. The description of Complex PTSD includes the symptoms most frequently reported by participants in the DSM-IV PTSD field trials (van der Kolk, Roth, Pelcovitz, Sunday, & Spinazzola, 2005) as well as from studies of chronically traumatized adults (de Jong et al., 2005, Morina and Ford, 2008). Disturbances in the three additional domains specified for Complex PTSD were found to occur frequently in the most severe 20% of cases in all three studies. More recent studies (Cloitre et al., 2013, Elklit et al., 2014, Hansen et al., 2015, Knefel and Lueger-Schuster, 2013, Perkonigg et al., 2015) used latent profile and factor analysis to validate the Complex PTSD symptom structure according to the ICD-11 model in different samples. These studies found separate classes of PTSD and Complex PTSD patients and thus supported the ICD-11 Complex PTSD proposal.

In this study, a key research question (Comparison 4) was whether clinicians could differentiate Complex PTSD from PTSD based on the presence or absence of symptoms in the three additional domains. Further, given that Complex PTSD is formulated as a replacement for ICD-10 Personality change after Catastrophic Experience, it should provide a better fit to cases of this type and be diagnosed more reliably. Additionally, while the diagnosis of Complex PTSD is associated with the experience of a stressor with particular characteristics, a history of exposure to such a stressor is not sufficient for the diagnosis if the required symptoms are not manifested. Therefore, in parallel to PTSD, Comparison 5 examined whether clinicians would apply the symptomatic requirements in making the diagnosis of Complex PTSD, or if their diagnostic choices would be unduly influenced by the nature of the stressor the individual experienced.

Prolonged Grief Disorder. Prolonged Grief Disorder, a new diagnostic category proposed for ICD-11, is defined as a disturbance in which, following a bereavement event, there is persistent and pervasive grief response characterized by longing for the deceased or persistent preoccupation with the deceased, accompanied by intense emotional pain that has persisted for an atypically long period of time following the loss (more than 6 months at a minimum). The grief response clearly exceeds expected social or religious norms for the individual's culture and context. The empirical basis for the Prolonged Grief Disorder proposal was provided by a series of clinical research studies that described the symptom profile of the disorder (e.g., Burnett et al., 1997, Horowitz et al., 1997, Prigerson et al., 2009, Shear et al., 2011). A number of studies have shown that people who experience the symptoms of Prolonged Grief Disorder experience serious psychosocial and health problems, including other mental health difficulties such as suicidality, harmful health behaviours, substance abuse, or physical disorders such as high blood pressure and elevated rates of cardiovascular disorder (Fujisawa et al., 2010, Kersting et al., 2011, Maercker et al., 2008, Prigerson et al., 2009). This study examined the critical question (Comparison 6) of whether clinicians can make a clear distinction between Prolonged Grief Disorder and normal bereavement reactions based on the diagnostic guidelines proposed by the Working Group.

Adjustment Disorder. The diagnostic category of Adjustment Disorder has been criticized for its wide variety of presenting symptoms and relative absence of distinctive features (Casey and Bailey, 2011, Strain and Diefenbacher, 2008). At the same time, it is one of the most frequently used diagnoses in daily practice by psychiatrists and psychologists around the world (Evans et al., 2013, Reed et al., 2011). Despite its frequent current use as a residual category, the Working Group proposed diagnostic guidelines for Adjustment Disorder in ICD-11 based on specific positive symptoms, a change favored by experts in the field (Baumeister et al., 2009, Casey and Doherty, 2012, Semprini et al., 2010). The Working Group's proposal for ICD-11 defines Adjustment Disorder as a maladaptive reaction to identifiable psychosocial stressors or life changes characterized by: (a) preoccupation with the stressor or its consequences as manifested in affective and cognitive symptoms; (b) failure to adapt to the stressor; and (c) significant functional impairment. The current study examined whether clinicians would distinguish Adjustment Disorder from PTSD based on the symptomatic presentation, or whether they would be influenced by the nature of the stressor in making this distinction (Comparison 7). A second question (Comparison 8) was how clinicians would apply the symptomatic requirements in determining whether to assign a diagnosis of Adjustment Disorder to case vignettes, particularly in cases where preoccupation and functional impairment are absent.

Method

Participants

Participants were drawn from the Global Clinical Practice Network (GCPN), a worldwide network of mental health professionals established for the purpose of the ICD-11 case-controlled field trials. Mental health professionals were invited to join the GCPN through national and regional professional associations; international and national conferences in psychology, psychiatry, and related disciplines; professional listservs; and professional word-of-mouth. For more information on the history and development of the GCPN, see Reed et al. (2015). The study was implemented in three languages: English, Spanish, and Japanese†. At the time the study was launched, 3,669 GCPN members met eligibility requirements: (a) self-rated advanced proficiency or fluency in one of the three study languages, and (b) currently providing clinical services to patients or engaged in direct clinical supervision. Eligible GCPN members were sent a personalized initial invitation email, followed by reminder emails two and four weeks later. Data collection lasted two months for each language. Following planned data inspection procedures, forty-three responses were dropped from the database, either for completing the study twice (e.g., in multiple languages) or otherwise being ineligible for the study (e.g., being a member of a group with a formal role in aspects of ICD-11 revision directly related to the study). Of the eligible participants, 2,320 (64%) responded to the survey link and began the study. Of the 2,320, 1,738 (74.9%; 47.37% of total) completed enough of the study for data analysis.

Thus, the final sample consisted of 1,738 mental health professionals representing 76 different nationalities. The regional distribution of participants is shown in Table 2, along with their gender, profession, mean age, and years of experience. Participants who completed the study were no different than those who were invited but did not participate in terms of age or years of experience. The participants who completed the study were slightly more likely to be male and to come from the Asian region (virtually all Japanese-speaking participants came from this region).

Table 2

Demographic information.

	English	Spanish	Japanese
	f (%)	f (%)	f (%)
WHO Global Region
AFRO	53 (6.0)	0	0
AMRO-North	231 (26.0)	0	0
AMRO-South	55 (6.2)	279 (68.2)	0
EMRO	48 (5.4)	0	0
EURO	377 (42.4)	127 (31.1)	0
SEARO	94 (10.6)	0	0
WPRO-Asia	9 (1.0)	0	437 (99.3)
WPRO-Oceania	21 (2.4)	0	0
Other	1 (0.1)	3 (0.7)	3 (0.7)
Gender
Male	478 (53.8)	214 (52.3)	359 (81.6)
Female	409 (46.0)	195 (47.7)	81 (18.4)
Profession	f (%)	f (%)	f (%)
Counseling	87 (9.8)	4 (1.0)	1 (0.2)
Medicine	311 (35.0)	177 (43.3)	410 (93.2)
Nursing	10 (1.1)	0	4 (0.9)
Psychology	443 (49.8)	199 (48.7)	17 (3.9)
Social work	11 (1.2)	5 (1.2)	1 (0.2)
Sex Therapy	2 (0.2)	0	0
Other	25 (2.8)	24 (5.9)	7 (1.6)
Age	M (SD)	M (SD)	M (SD)
	46.78 (10.73)	46.16 (11.76)	46.64 (10.79)
Years of Experience	M (SD)	M (SD)	M (SD)
	15.35 (10.10)	16.56 (10.62)	15.09 (10.40)
Total N	889	409	440

Note: AFRO = African region; AMRO-North = North American region (U.S. and Canada); AMRO-South = South American region (Latin America); EMRO = Middle Eastern region; EURO = European region; SEARO = South Eastern Asian region; WPRO-Asia = Asian part of Western Pacific region; WPRO-Oceania = Australia and New Zealand. WHO Global Regions AMRO and WPRO were divided into two parts to distinguish high-income, predominantly English-speaking parts of those regions from other countries.

Materials

The study was administered through Qualtrics, a web-based survey program. The materials in the study included proposed diagnostic guidelines for Disorders Specifically Associated with Stress for ICD-11 (PTSD, Complex PTSD, Prolonged Grief Disorder, Adjustment Disorder, Acute Stress Reaction, and Other Disorder Specifically Associated with Stress) and ICD-10 (PTSD, Enduring Personality Change after Catastrophic Experience, Adjustment Disorder, Acute Stress Reaction, and Other Reaction to Severe Stress), a series of 11 case vignettes specifically developed for diagnostic comparisons, and diagnostic and clinical utility questions regarding the vignettes.

Specifically, participants provided a diagnosis (or indicated that no diagnosis was warranted) for each vignette with which they were presented and then answered specific questions about the presence or absence of each essential feature of the diagnosis as specified in the diagnostic guidelines, using wording adapted from the ICD-10 or ICD-11 diagnostic guidelines. Participants also rated the ease of use of the diagnostic guidelines on a four point scale (Not at all easy to use to Extremely easy to use), the goodness of fit of the diagnostic guidelines for describing the vignette on a four point scale (Not at all accurate to Extremely accurate), how frequently they encountered patients with that diagnosis on a five point scale (Never to Very frequently: multiple times per week), how similar patients in their regular practice were to the vignette on a four point scale (Not at all similar to Extremely similar), the level of functional impairment of the person in the vignette on a five point scale (No impairment to Complete impairment), and the severity of the symptoms in the vignette on a five point scale (No symptoms to Extremely severe symptoms).

A total of 11 vignettes were generated for the study (see Table 3). Because of the specific content required for each vignette, and a desire to create case examples that matched both expert definition of the symptoms and actual presentation of those symptoms in real clinical situations, content experts (members of the Working Group for Disorders Specifically Associated with Stress) drafted initial versions of the vignettes based on explicit instructions. One common criticism of vignette studies is the artificiality of cases (see Evans et al., 2015); vignette writers were therefore instructed to draw upon their clinical experience and to base the vignettes on actual cases (obscuring personal details for confidentiality). The vignettes covered a range of ages and included both men and women, but did not include child cases. Although written by clinicians from diverse countries, the vignettes did not explicitly identify specific social or cultural groups.

Table 3

Correct diagnoses for vignettes according to ICD-11 and ICD-10.

Comparison	Vignettes	Correct ICD-11 Diagnosis	Correct ICD-10 Diagnosis
1	1A	PTSD	PTSD
	1B	Other Disorder Specifically Associated with Stress	PTSD
2	1A	PTSD	PTSD
	1 C	Other Disorder Specifically Associated with Stress	PTSD
3	1A	PTSD	PTSD
	1D	PTSD	PTSD
4	1A	PTSD	PTSD
	2A	Complex PTSD	PTSD or Enduring Personality Change after Catastrophic Experience
5	2A	Complex PTSD	PTSD or Enduring Personality Change after Catastrophic Experience
	2B	PTSD	PTSD
6	3	No diagnosis	No diagnosis
	4	Prolonged Grief Disorder	Adjustment Disorder
7	5A	Adjustment Disorder	Adjustment Disorder
	5B	Adjustment Disorder	Adjustment Disorder
8	5A	Adjustment Disorder	Adjustment Disorder
	6	No diagnosis	No diagnosis

Vignettes were explicitly developed to exemplify certain characteristics, such as a case that displayed all PTSD symptoms except for clear functional impairment (Comparison 2; see Table 1, Table 3). These experimental manipulations of the case descriptions were designed to test specific changes or additions to ICD-11. Vignettes were pre-tested by content experts (a) to determine whether the desired characteristics were indeed present, (b) to confirm the presence or absence of required features, and (c) to confirm expert consensus about the correct diagnosis for the case. Based on the results of the pretest, some vignettes were modified to clarify content or ensure that specific symptoms were recognizable.

All materials were developed in English and then translated into Spanish and Japanese. Translators followed a rigorous forward and backward translation procedure utilizing bilingual content experts. Occasionally, translation efforts suggested wording clarifications to materials in English in order to maintain conceptual equivalence.

Procedure

Participants received an email invitation to participate in the study through Qualtrics. Upon entry to the study, participants were randomly assigned to view either the ICD-10 or ICD-11 diagnostic guidelines for the disorders included in the study. Participants were blind to their assigned condition. Once participants had reviewed the guidelines, they were randomly assigned to one of eight comparisons (see Table 1). Within each comparison, participants viewed two vignettes, paired to highlight a specific change from ICD-10 to ICD-11 (see Table 3). The order of presentation of the two vignettes was counterbalanced across participants. After reading a vignette, participants were asked to provide a diagnosis from a preset list (including the relevant disorders from ICD-10 or ICD-11, as assigned), write in a different diagnosis, or indicate that no diagnosis was warranted. Participants could refer to the assigned diagnostic guidelines while making their decision. They then rated the specific presence or absence of each of the essential features as specified in the ICD-10 or ICD-11 diagnostic guidelines (as assigned) for the diagnosis they had selected. After reviewing the specific diagnostic requirements, clinicians were given the option of changing their final diagnosis. Participants then rated the diagnostic and clinical utility questions described above. If the selected final diagnosis was incorrect, participants received an additional differential diagnosis question, inquiring why the person had assigned the selected diagnosis rather than the correct diagnosis (without explicitly identifying the selected diagnosis as incorrect). Participants then completed the sequence again for a second vignette (i.e., the vignette comparison was a within-participant variable). Participants were unable to go back and change their responses as the study progressed. The detailed description of the specific comparisons follows.

Comparison 1 (Vignettes 1A and 1B). The first comparison examined the new ICD-11 diagnostic requirement for PTSD that the event must be re-experienced as though it were occurring in the present, rather than simply remembered. Vignette 1A included a description of re-experiencing in the present, whereas vignette 1B included only memories of the stressful event. Otherwise, the two vignettes expressed similar symptoms of PTSD. Expert pretesting confirmed that no re-experiencing (as defined by ICD-11) occurred in vignette 1B. Under ICD-10 diagnostic guidelines, both would fit within the diagnosis of PTSD; however, under ICD-11 diagnostic guidelines only 1A should be labeled PTSD with 1B receiving another diagnosis (e.g., Other Disorder Specifically Associated with Stress).

Comparison 2 (Vignettes 1A and 1 C). The second comparison tested the inclusion of functional impairment in the diagnostic requirements for PTSD under ICD-11. Vignette 1A included associated functional impairment where vignette 1 C did not. All other symptoms of PTSD were equivalent between the two vignettes. Expert pretesting confirmed that functional impairment was not present in vignette 1 C. Under ICD-10 diagnostic guidelines, both cases should receive a diagnosis of PTSD, while only 1A should be diagnosed as PTSD under the proposed ICD-11 diagnostic guidelines.

Comparison 3 (Vignettes 1A and 1D). Comparison 3 targeted the possible misapplication of the PTSD category by participants who assigned the diagnosis based on the nature of the stressor rather than on the nature of the symptoms. Vignette 1A described a traumatic event that is commonly associated with PTSD (a severe car accident), where 1D had identical symptoms but a less prototypic event (having part of a ceiling fall on him while conducting home repairs). Expert pretesting confirmed that the individual experienced a traumatic event as defined by ICD-11. Under ICD-11, both cases should be given a diagnosis of PTSD; under ICD-10 it would be justifiable not to assign a PTSD diagnosis to vignette 1D.

Comparison 4 (Vignettes 1A and 2A). Comparison 4 examined the differentiation of PTSD and Complex PTSD. Vignette 1A described classically defined PTSD (meeting all ICD-11 requirements) where vignette 2A met all ICD-11 requirements for Complex PTSD. A correct classification under ICD-10 would be a diagnosis of PTSD for vignette 1A and either PTSD or Enduring Personality Change after Catastrophic Experience for vignette 2A; under ICD-11 the correct response would be a diagnosis of PTSD for vignette 1A and Complex PTSD for vignette 2A. Evidence for the efficacy of the new category would be a higher proportion of participants selecting the correct diagnosis for vignette 2A using ICD-11 than ICD-10.

Comparison 5 (Vignettes 2A and 2B). Comparison 5 examined whether clinicians could discriminate between the symptoms required for a diagnosis of Complex PTSD and the nature of the stressor. Specifically, vignette 2B described an individual exposed to a chronic and extreme stressor from which escape was difficult or impossible that would commonly be associated with Complex PTSD. However, the person in vignette 2B displayed standard PTSD symptoms without any of the additional three symptoms required for Complex PTSD. Expert pretesting confirmed that the three additional Complex PTSD symptoms were not present in vignette 2B. If clinicians provided a diagnosis of Complex PTSD for vignette 2A and PTSD for vignette 2B under the ICD-11 system, they would be utilizing the system as intended.

Comparison 6 (Vignettes 3 and 4). Comparison of vignettes 3 and 4 examined the addition of the new Prolonged Grief Disorder category for ICD-11. Vignette 3 described a culturally normative grief reaction, where vignette 4 described a reaction that outlasted cultural expectations and fulfilled the diagnostic description for Prolonged Grief Disorder. It was predicted that under ICD-10 vignettes 3 and 4 would be more likely to receive no diagnosis, and those diagnoses provided for vignette 4 would have little consistency (e.g., Depressive Episode, Adjustment Disorder, Other Reaction to Severe Stress). Under ICD-11, vignette 3 should receive no diagnosis and vignette 4 should receive a diagnosis of Prolonged Grief Disorder. This comparison tested whether clinicians could differentiate the diagnosis of Prolonged Grief Disorder from normal grief and whether the addition of the diagnosis improves the ICD-11's coverage of psychopathological conditions representing appropriate targets of treatment.

Comparison 7 (Vignettes 5A and 5B). Comparison 7 examined whether clinicians could discriminate between the symptoms required for a diagnosis of Adjustment Disorder and the nature of the stressor. Vignette 5A fulfilled the diagnostic guidelines for ICD-11 Adjustment Disorder, evidencing both preoccupation with the stressor and functional impairment. Vignette 5B described an individual with the same symptomatic presentation of Adjustment Disorder, over the same time span, but who had experienced a type of stressor that is more commonly associated with PTSD. If clinicians provided a diagnosis of Adjustment Disorder for both vignette 5A and vignette 5B under both the ICD-10 and ICD-11 systems, they would be utilizing the systems appropriately. If, however, the nature of the stressor overrode the symptomatic presentation in the minds of clinicians, they would assign a diagnosis of PTSD for vignette 5B. Because the diagnostic guidelines proposed for ICD-11 for both Adjustment Disorder and for PTSD were more explicit than those in ICD-10, it was expected that this effect would be reduced for ICD-11.

Comparison 8 (Vignettes 5A and 6). Comparison of vignettes 5A and 6 addressed the changes for Adjustment Disorder in ICD-11, specifically the requirement of preoccupation with the stressor and the explicit inclusion of functional impairment. As noted, vignette 5A fulfilled the diagnostic guidelines for Adjustment Disorder, evidencing both preoccupation with the stressor and functional impairment. Vignette 6 evidenced a similar level of symptoms and the same time span as vignette 5A, but without preoccupation with the stressor or impairment in functioning. Under the ICD-10 diagnostic guidelines, both vignettes 5A and 6 should receive Adjustment Disorder diagnoses. Under the ICD-11 diagnostic guidelines, only vignette 5A should receive a diagnosis of Adjustment Disorder, while vignette 6 should receive no diagnosis.

Analysis

Participants’ diagnostic decisions were compared across diagnostic systems for a single vignette or across vignettes within a single system using χ2 tests. When diagnoses were compared across diagnostic systems and vignettes (i.e., a three-way interaction), we used the G2 statistic, which is a log-linear transformation of the χ2 distribution (Rao & Scott, 1984).

Results

Comparison 1–Re-experiencing in the present in PTSD

This comparison was designed to examine whether clinicians recognized the proposed requirement that trauma be re-experienced in the present for PTSD in ICD-11. PTSD diagnoses dropped from 1A to 1B using ICD-11 (consistent with the changes in the diagnostic guidelines), but an equivalent drop occurred for ICD-10, χ2(1) = 0.79, ns (see Figure 1). Thus, using ICD-10 as a baseline, there was not a significant reduction in the use of PTSD as a diagnosis for vignette 1B in ICD-11. The lack of re-experiencing in the present seemed to make clinicians less likely to give a diagnosis of PTSD under the ICD-10 diagnostic guidelines as well. Further, PTSD remained the most common diagnosis for 1B under ICD-11.

Figure 1

Percentages of diagnostic choices for Comparison 1: Do clinicians appropriately recognize the required symptom of re-experiencing and apply it correctly as a diagnostic requirement for the ICD-11 diagnosis of PTSD?

Note: Correct diagnoses are in Table 3.

Given that clinicians did not follow the pattern expected for ICD-11 diagnoses of vignette 1B, it was instructive to see why they gave the diagnosis of PTSD and whether they endorsed the presence of re-experiencing in the present. As can be seen in Table 4, among the participants who gave a PTSD diagnosis for vignette 1B using ICD-11 guidelines (n = 75), there was more uncertainty about re-experiencing than for any other element of the PTSD diagnostic guidelines. Many participants affirmed the presence of re-experiencing in contradiction to expert evaluations.

Table 4

Endorsement of diagnostic guidelines for PTSD diagnoses of vignette 1B using ICD-11.

	Yes	No	Not Sure
Exposed to trauma	100%	0%	0%
Re-experiencing in present	59%	17%	24%
Avoidance	100%	0%	0%
Arousal/hypervigilance	90%	7%	3%
Symptoms develop after trauma	91%	3%	3%
Functional impairment	99%	0%	1%
Symptoms last several weeks	100%	0%	0%
Complex PTSD	10%	84%	6%

Comparison 2–Functional impairment in PTSD

Comparison 2 examined the proposed ICD-11 requirement of functional impairment for PTSD. Specifically, vignette 1A evidenced functional impairment whereby vignette 1 C showed the same symptoms but was still working and had intact relationships. As with comparison 1, there was not the expected shift for individuals using ICD-11; they gave diagnoses of PTSD at the same rate for both vignettes, χ2(1) = 0.02, ns (see Figure 2). Of those who assigned a PTSD diagnosis to vignette 1 C using ICD-11 (see Table 5), most endorsed the presence of functional impairment, but more indicated uncertainty than for any other element of the PTSD diagnostic guidelines.

Figure 2

Percentages of diagnostic choices for Comparison 2: Do clinicians appropriately recognize functional impairment and apply it correctly as a diagnostic requirement for the ICD-11 diagnosis of PTSD?

Note: Correct diagnoses are in Table 3.

Table 5

Endorsement of diagnostic guidelines for PTSD diagnoses of vignette 1 C using ICD-11.

	Yes	No	Not Sure
Exposed to trauma	100%	0%	0%
Re-experiencing	100%	0%	0%
Avoidance	92%	8%	0%
Arousal/hypervigilance	93%	2%	4%
Symptoms developed after trauma	97%	1%	2%
Functional impairment	67%	11%	22%
Symptoms last several weeks	99%	0	1%
Complex PTSD	16%	75%	9%

Comparison 3–Severity of the trauma in PTSD

Comparison 3 examined how clinicians would respond to an event that is subjectively experienced as frightening or horrific but that is less commonly associated with PTSD (vignette 1D) versus a traumatic event that is more prototypic for PTSD (vignette 1A). The diagnosis of PTSD should not change from vignette 1A to 1D in ICD-11. Under ICD-10, other diagnoses would technically be correct, but there was nothing explicitly excluding vignette 1D from receiving a diagnosis of PTSD. Treating any diagnosis other than PTSD as incorrect, there was not a significant difference across vignettes using the ICD-11 or ICD-10, G2(4) = 5.74, ns (see Figure 3). In other words, clinicians were equally likely to diagnose both cases as PTSD under both diagnostic systems. Subjectively defined, non-prototypic traumas were considered by clinicians using either system to be legitimate precursors for the diagnosis of PTSD.

Figure 3

Percentages of diagnostic choices for Comparison 3: Do clinicians appropriately base the ICD-11 diagnosis of PTSD on the presence of the required core symptoms, or do they tend to over-diagnose PTSD based on a history of specific types of stressors?

Note: Correct diagnoses are in Table 3.

Comparison 4–Differentiating PTSD and Complex PTSD

The proposed addition of Complex PTSD to ICD-11 necessitates an assessment of clinicians’ ability to differentiate it from regular PTSD. If Complex PTSD is an improvement over Enduring Personality Change after Catastrophic Experience in ICD-10, the diagnostic preference for vignette 2A (Complex PTSD) should be clearer for ICD-11 than for ICD-10. Clinicians were able to successfully distinguish cases of PTSD and Complex PTSD using ICD-11, χ2(2) = 112.70, p < .0001. Further, the three-way interaction indicated that the change in correctness between vignettes shifted significantly from ICD-10 to ICD-11 in the expected direction, G2(4) = 110.90, p < .0001 (see Figure 4), indicating that Complex PTSD was a cleaner solution than using the concept of Enduring Personality Change. In other words, the ICD-11 concept of Complex PTSD appeared to clarify the diagnostic decision for vignette 2A relative to the ICD-10 system.

Figure 4

Percentages of diagnostic choices for Comparison 4: Can clinicians differentiate the proposed ICD-11 diagnostic requirements of Complex PTSD from those of PTSD?

Note: Correct diagnoses are in Table 3.

Comparison 5–Stressor severity in PTSD and Complex PTSD

Comparison 5 examined the effect the nature of the stressor had in assigning a diagnosis of Complex PTSD. The analyses below focus only on those who gave diagnoses of PTSD, Complex PTSD, or Enduring Personality Change after Catastrophic Experience, ignoring other incorrect diagnoses as these are not relevant to differentiating PTSD and Complex PTSD (see Figure 5). An interesting split occurred. Participants diagnosing vignette 2A under ICD-11 did not significantly differ from those using ICD-10, χ2(1) = 2.19, ns However, for diagnoses of vignette 2B, participants using the ICD-10 were more likely to diagnose the case with PTSD, χ2(1) = 17.92, p < .001. The three-way interaction indicates that the shift between diagnostic systems was not equal, G2(4) = 68.14, p < .0001. Participants using the ICD-11 were more likely to give a Complex PTSD diagnosis to vignette 2B on the basis of the nature of the traumatic event, while this option was less enticing for Enduring Personality Change after Catastrophic Experience in ICD-10.

Figure 5

Percentages of diagnostic choices for Comparison 5: Do clinicians inappropriately diagnose Complex PTSD based on a history of a severe and long-lasting stressor rather based on the required symptoms?

Note: Correct diagnoses are in Table 3.

An examination of those participants using ICD-11 who gave either diagnoses of PTSD or Complex PTSD for vignette 2B follows. For those clinicians (n = 50) who selected Complex PTSD, all but one endorsed that the trauma was extreme (see Table 6). However, many endorsed the presence of persistent changes in affect regulation, a sense of worthlessness accompanied by guilt, and sustained problems maintaining intimate relationships. Nonetheless, a number of individuals indicated that those symptoms were not present, or they were unsure. Of those clinicians who gave a diagnosis of PTSD (n = 46), 16 recognized that a diagnosis of Complex PTSD was possible. The nature of the trauma swayed clinicians’ decisions, indicating that the diagnostic guidelines needed further clarification.

Table 6

Endorsement of diagnostic guidelines for Complex PTSD diagnoses of vignette 2B using ICD-11.

	Yes	No	Not Sure
Exposed to extreme trauma	98%	2%	0%
Re-experiencing	96%	4%	0%
Avoidance	94%	2%	4%
Arousal/hypervigilance	90%	8%	2%
Symptoms developed after trauma	80%	6%	14%
Persistent problem with affect regulation	74%	18%	8%
Persistent change in worthlessness and guilt	28%	56%	16%
Persistent difficulties sustaining relationships	58%	26%	16%
Functional impairment	92%	6%	2%

Comparison 6–Prolonged Grief Disorder

The addition of a diagnosis of Prolonged Grief Disorder assumes clinicians’ ability to differentiate it from a normal grief process. First, vignette 4 was nearly always recognized as Prolonged Grief Disorder under ICD-11, but a wide variety of diagnoses were offered under ICD-10 (see Table 7). Thus, the addition of Prolonged Grief Disorder provides a clearer mechanism for capturing this sort of case. Second, for ICD-11, clinicians distinguished clearly between the cases, such that vignette 4 received a different pattern of diagnoses than vignette 3, χ2(2) = 116.65, p < .0001. However, clinicians under either system did not do particularly well in recognizing that vignette 3 should not receive any diagnosis. Treating any diagnosis as incorrect, clinicians using the ICD-11 system (50% correct, 50% incorrect) were marginally better those using ICD-10 (37% correct, 63% incorrect), χ2(1) = 3.38, p = .06. Of the 20 individuals who gave a Prolonged Grief Disorder diagnosis to vignette 3, there was disagreement as to whether the grief reaction had lasted for an abnormally long period of time or had been a culturally normal response (50% Yes, 35% No, 15% Not Sure).

Table 7

Diagnoses of vignette 4.

Diagnosis	Percent
ICD-11
Prolonged Grief Disorder	92%
PTSD	3%
Complex PTSD	1%
Adjustment Disorder	2%
Other Diagnosis	1%
No Diagnosis	1%
ICD-10
PTSD	6%
Enduring Personality Change After Catastrophic Experience	20%
Adjustment Disorder	37%
Other Reaction to Severe Stress	11%
Other Diagnosis	20%
No diagnosis	6%

Comparison 7–Nature of the stressor for Adjustment Disorder and PTSD

This comparison examined whether clinicians could discriminate between symptoms required for Adjustment Disorder and the nature of the stressor. The majority of clinicians identified vignette 5A as a case of Adjustment Disorder across both ICD-11 and ICD-10. Examining just those who gave diagnoses of PTSD and Adjustment Disorder while lumping all other diagnoses into an “other” group, a higher percentage of Adjustment Disorder diagnoses occurred under ICD-11 than ICD-10, G2(7) = 135.14, p < .0001, even though both groups of clinicians had difficulty with the distinction (see Figure 6). Thus, the distinction between PTSD and Adjustment Disorder diagnoses was clearer using ICD-11 versus ICD-10 diagnostic guidelines.

Figure 6

Percentages of diagnostic choices for Comparison 7: Do clinicians appropriately differentiate PTSD from Adjustment Disorder based on the required symptoms, or do they tend to inappropriately base this distinction on the nature of the stressor?

Note: Correct diagnoses are in Table 3.

For those who gave a diagnosis of PTSD for vignette 5B, the diagnostic system made a difference in participants’ level of certainty about the presence of particular guidelines. Specifically, under ICD-11 (n = 28), there was little agreement about the presence of all three core symptoms of PTSD, with the majority indicating no presence of avoidance or hyperarousal (see Table 8). Under ICD-10 (n = 51), that disagreement only occurred for symptoms of avoidance or emotional numbing.

Table 8

Endorsement of diagnostic guidelines for PTSD diagnoses of vignette 5B.

	Yes	No	Not Sure
ICD-11
Exposed to trauma	100%	0%	0%
Re-experiencing	64%	18%	18%
Avoidance	32%	46%	21%
Arousal/hypervigilance	18%	46%	36%
Symptoms developed after trauma	79%	18%	3%
Functional impairment	100%	0%	0%
Symptoms last several weeks	93%	7%	0%
Complex PTSD	14%	86%	0%
ICD-10
Exposed to trauma	100%	0%	0%
Symptoms developed after trauma	94%	2%	4%
Re-experiencing	94%	2%	4%
Avoidance	62%	24%	14%
Arousal/hypervigilance	94%	2%	4%
Enduring personality change	16%	76%	8%

Comparison 8–Functional impairment and preoccupation in Adjustment Disorder

Two important changes were proposed in the ICD-11 diagnostic guidelines for Adjustment Disorder: (a) functional impairment is necessary and (b) the person must exhibit preoccupation with the stressor. For vignette 5A, for both ICD-11 and ICD-10 the majority of diagnoses were Adjustment Disorder, and the relative rates of giving an Adjustment Disorder diagnosis versus another diagnosis versus no diagnosis were equal across systems, χ2(2) = 4.33, ns. For vignette 6, the rate of assigning an Adjustment Disorder versus another disorder versus no disorder diagnosis did not change by diagnostic manual, χ2(2) = 1.88, ns. Nonetheless, no diagnosis was more likely for vignette 6 when controlling for the lack of a difference between manuals, G2(4) = 80.28, p < .0001 (see Figure 7). Thus, the ICD-11 Adjustment Disorder diagnostic guidelines did not provide a clearer basis for identifying cases that should not receive the diagnosis.

Figure 7

Percentages of diagnostic choices for Comparison 8: Do clinicians appropriately exclude diagnoses of Adjustment Disorder that do not evidence preoccupation and functional impairment as required by proposed ICD-11 guidelines?

Note: Correct diagnoses are in Table 3.

Discussion

This study is the first of a series of case-controlled internet-based field studies developed by WHO to address questions regarding the proposed changes to the ICD-11 Mental and Behavioural Disorders Chapter. Its intended purpose was to provide direct evidence related to the clinical utility of proposed changes and the ability of practicing clinicians to implement the proposed diagnostic guidelines. The results indicated areas that appear to be improvements to the classification as well as areas that need to be further clarified before further studies in clinical settings. The mechanism for making these evidence-based changes is a formal feedback process between the Field Studies Coordination Group (FSCG) and the Working Group. The FSCG developed recommendations for the Working Group based upon the results of this study. The Working Group then proposed further changes through an iterative process of communications between the two groups. Changes to the “Essential Features” section of the diagnostic guidelines for the Disorders Specifically Associated with Stress made as a result of this process are depicted in Table 9, incorporating changes to the guidelines designed to address areas where the guidelines were not functioning as intended according to the results of this study. It is important to note that the Essential Features section represents only a small portion of the proposed ICD-11 diagnostic guidelines. Other sections included in the ICD-11 diagnostic guidelines that are not provided in Table 9 include: boundary with normality, boundary with other conditions (differential diagnosis), course features, issues related to culture and gender, and developmental presentations. (See First et al., 2015 for a description and the complete proposed diagnostic guideline for PTSD.) These revised diagnostic guidelines will be further tested and revised based on results obtained in clinical settings with real patients prior to being finalized by WHO.

Table 9

Proposed ICD-11 diagnostic guidelines for Disorders Specifically Associated with Stress after revisions based on study results: Essential Features only.

PTSD Essential Features

• Exposure to an event or situation (either short- or long-lasting) of an extremely threatening or horrific nature. Such events include, but are not limited to, natural or human-made disasters; combat; serious accidents; torture; sexual violence; terrorism; assault; acute life-threatening illness (such as a heart attack); witnessing the threatened or actual injury or death of others in a sudden, unexpected, or violent manner; and experiencing the sudden, unexpected or violent death of a loved one.

• Following the traumatic event or situation (generally within 1 month but nearly always within several months), the development of a characteristic syndrome that lasts for at least several weeks, consisting of three core elements:

1. Re-experiencing the traumatic event in the present, in which the event(s) is not just remembered but is experienced as occurring again in the here and now. This typically occurs in the form of vivid intrusive images or memories; flashbacks, which can vary from mild (there is a transient sense of the event occurring again in the present) to severe (there is a complete loss of awareness of present surroundings); or repetitive dreams or nightmares that are thematically related to the traumatic event(s). Re-experiencing is typically accompanied by strong or overwhelming emotions, such as fear or horror, and strong physical sensations. Re-experiencing in the present can also involve being overwhelmed or immersed in the same intense emotions that were experienced during the traumatic event, which may occur in response to reminders of the event. Reflecting back or ruminating about the event(s) and remembering the feelings that one experienced at that time do not constitute re-experiencing.

2. Deliberate avoidance of reminders likely to produce re-experiencing of the traumatic event(s). This may take the form either of active internal avoidance of relevant thoughts and memories, or external avoidance of people, conversations, activities, or situations reminiscent of the event(s). In extreme cases the person may change his or her environment (e.g., move to a different city or change jobs) to avoid reminders.

3. Persistent perceptions of heightened current threat, for example as indicated by hypervigilance or an enhanced startle reaction to events such as unexpected noises. Hypervigilant persons constantly guard themselves against danger and feel themselves or others close to them to be under immediate threat either in specific situations or more generally. They may adopt new behaviours designed to ensure safety (e.g., only sit in certain places on trains, repeatedly check in vehicles’ rear-view mirror).

• The disturbance causes significant impairment in personal, family, social, educational, occupational or other important areas of functioning. If functioning is maintained only through significant additional effort, or is significantly impaired compared to the individual's prior functioning or what would be expected, then the individual would be considered impaired due to the disturbance.

Complex PTSD Essential Features

• History of exposure to a stressor of an extreme and prolonged or repetitive nature from which escape is difficult or impossible. Such events include, but are not limited to, torture, concentration camps, slavery, genocide campaigns and other forms of organized violence, prolonged domestic violence, and repeated childhood sexual or physical abuse.

• History of the following three core elements of PTSD that developed during or after the traumatic event (generally within one month but nearly always within several months) and lasting for at least several weeks:

1. Re-experiencing the traumatic event after the traumatic event has occurred, in which the event(s) is not just remembered but is experienced as occurring again in the here and now. This typically occurs in the form of vivid intrusive images or memories; flashbacks, which can vary from mild (there is a transient sense of the event occurring again in the present) to severe (there is a complete loss of awareness of present surroundings); or repetitive dreams or nightmares that are thematically related to the traumatic event(s). Re-experiencing is typically accompanied by strong or overwhelming emotions, such as fear or horror, and strong physical sensations.

2. Deliberate avoidance of reminders likely to produce re-experiencing of the traumatic event(s). This may take the form either of active internal avoidance of relevant thoughts and memories, or external avoidance of people, conversations, activities, or situations reminiscent of the event(s). In extreme cases the person may change his or her environment (e.g., move house or change jobs) to avoid reminders.

3. Persistent perceptions of heightened current threat, for example as indicated by hypervigilance or an enhanced startle reaction to events such as unexpected noises. Hypervigilant persons constantly guard themselves against danger and feel themselves or others close to them to be under immediate threat either in specific situations or more generally. They may adopt new behaviours designed to ensure safety (e.g., only sit in certain places on trains, repeatedly check in vehicles’ rear-view mirror). In Complex PTSD, unlike in PTSD, the startle reaction may in some cases be diminished rather than enhanced.

• Severe and pervasive problems in affect regulation. Examples include heightened emotional reactivity to minor stressors, violent outbursts, reckless or self-destructive behavior, dissociative symptoms when under stress, and emotional numbing, particularly the inability to experience pleasure or positive emotions.

• Persistent beliefs about oneself as diminished, defeated or worthless, accompanied by deep and pervasive feelings of shame, guilt or failure related to the stressor. For example, the individual may feel guilty about not having escaped from or succumbing to the adverse circumstance, or not having been able to prevent the suffering of others.

• Persistent difficulties in sustaining relationships and in feeling close to others. The person may consistently avoid, deride or have little interest in relationships and social engagement more generally. Alternatively, there may be occasional intense relationships, but the person has difficulty sustaining them.

• The disturbance causes significant impairment in personal, family, social, educational, occupational or other important areas of functioning. If functioning is maintained only through significant additional effort, or is significantly impaired compared to the individual's prior functioning or what would be expected, then the individual would be considered impaired due to the disturbance.

Prolonged Grief Disorder Essential Features

• History of bereavement following the death of a partner, parent, child, or close other person.

• A persistent and pervasive grief response characterized by longing for the deceased or persistent preoccupation with the deceased accompanied by intense emotional pain (e.g. sadness, guilt, anger, denial, blame, difficulty accepting the death, feeling one has lost a part of one's self, an inability to experience positive mood, emotional numbness, difficulty in engaging with social or other activities).

• The grief response has persisted for an abnormally long period of time following the loss, clearly exceeding expected social, cultural or religious norms for the individual's culture and context. Grief responses within 6 months of the bereavement, and for longer periods in some cultural contexts, should not be considered to meet this requirement.

• The disturbance causes significant impairment in personal, family, social, educational, occupational or other important areas of functioning. If functioning is maintained only through significant additional effort, or is significantly impaired compared to the individual's prior functioning or what would be expected, then the individual would be considered impaired due to the disturbance.

Adjustment Disorder Essential Features

• A maladaptive reaction to an identifiable psychosocial stressor or multiple stressors (e.g., single stressful event, ongoing psychosocial difficulty or a combination of stressful life situations) that usually emerges within a month of the stressor. Examples include divorce, illness or disability, socio-economic problems and conflicts at home or work.

• The reaction to the stressor is characterized by preoccupation with the stressor or its consequences, including excessive worry, recurrent and distressing thoughts about the stressor, or constant rumination about its implications.

• Failure to adapt to the stressor produces noticeable impairment in personal, social or occupational functioning, e.g., relationship conflict, performance problems at work or school, reduced ability to respond appropriately to normal stressors.

• The disturbance causes significant impairment in personal, family, social, educational, occupational or other important areas of functioning. If functioning is maintained only through significant additional effort, or is significantly impaired compared to the individual's prior functioning or what would be expected, then the individual would be considered impaired due to the disturbance.

Acute Stress Reaction Essential Features (not considered a mental disorder but rather a normal reaction to severe stress that still may require treatment)

• Exposure to an event or situation (either short- or long-lasting) of an extremely threatening or horrific nature. Such events include, but are not limited to, natural or human-made disasters; combat; serious accidents; torture; sexual violence; terrorism; assault; acute life-threatening illness (such as a heart attack); witnessing the threatened or actual injury or death of others in a sudden, unexpected, or violent manner; and experiencing the sudden, unexpected or violent death of a loved one.

• The development of a response to the stressor that is considered to be normal given the severity of the stressor. The response to the stressor may include transient emotional, somatic, cognitive, or behavioural symptoms, such as being in a daze, confusion, sadness, anxiety, anger, despair, overactivity, inactivity, social withdrawal, or stupor. Autonomic signs of anxiety (e.g., tachycardia, sweating, flushing) are commonly present and may be the presenting feature.

• Symptoms typically appear within hours to days following the stressful event, and usually begin to subside within a few days after the event or following removal from the threatening situation, when this is possible. In cases where the stressor continues or removal is not possible, symptoms may persist but are usually greatly reduced within approximately 1 month as the person adapts to the changed situation.

Other Disorder Specifically Associated with Stress Essential Features

A diagnosis of Other Disorder Specifically Associated with Stress should be used only in cases in which:

• The clinical presentation does not satisfy the definitional requirements of any of the other disorders in this section or of Acute Stress Reaction;

• The symptoms are not better explained by another Mental or Behavioural Disorder specified elsewhere in ICD (e.g., a Depressive Disorder or an Anxiety Disorder);

• The clinical presentation is judged to be a Mental or Behavioural Disorder occurring in specific association with an identifiable stressor;

• The symptoms cause distress or functional impairment in personal, family, social, educational, occupational or other important areas of functioning. If functioning is maintained only through significant additional effort, or is significantly impaired compared to the individual's prior functioning or what would be expected, then the he or she would be considered impaired due to the disturbance.

Overall, proposed ICD-11 diagnostic guidelines for Disorders Specifically Associated with Stress fared well when compared with the ICD-10. The results of the study suggest that the additions of both Complex PTSD and Prolonged Grief Disorder (Comparisons 4 and 6) represent significant clarifications of the diagnostic landscape relative to ICD-10. Clinicians were able to differentiate Complex PTSD and Prolonged Grief Disorder from similar conditions and normality. Similarly, the ability to distinguish between PTSD and Adjustment Disorder improved relative to ICD-10. As such, mental health professionals were able to apply the diagnostic guidelines for these disorders as intended in response to standardized case material. At this stage of guideline development, it is important to know whether the guidelines fit clear cases as intended; if not, then the diagnostic guidelines are not sufficiently useful. This information is required prior to applying the guidelines in field studies in clinical settings, where lack of diagnostic reliability may be due to variations in case presentation rather than to unclear guidelines.

However, clinicians exhibited difficulty with some other proposed changes. Specifically, they did not make the proposed distinction between symptoms of re-experiencing in the present and memories in PTSD (Comparison 1). Indeed, there appeared to be substantial confusion as to whether symptoms of re-experiencing in the present were represented in the vignettes, despite expert confirmation of the absence of such symptoms. The FSCG therefore recommended that the Working Group revise and clarify the definition of re-experiencing in the proposed guidelines. The results of this study suggested that clinicians were interpreting re-experiencing more broadly than had been intended by the initial guidelines. Upon examination, the Working Group recognized that their definition of re-experiencing included only cognitive descriptions and did not account sufficiently for affective re-experiencing (which was likely what clinicians were responding to in the study). The Working Group therefore suggested specific revisions to this aspect of the guideline to be tested in field studies in clinical settings (see Table 9).

A second difficulty concerned the inclusion of functional impairment in PTSD and Adjustment Disorder (Comparisons 2 and 8). Clinicians tended to assign these diagnoses even when impairment was not explicitly present. Perhaps this finding is an artifact of the methodology. Vignettes are inherently an incomplete story, and so when functional impairment was not described explicitly in conjunction with the symptoms of PTSD or Adjustment Disorder, clinicians may have inferred its presence from other features of the vignette. Alternatively, clinicians may have noted the absence of functional impairment, but concluded that a diagnosis of PTSD or Adjustment Disorder was the best fit in any case, because no other mental disorder better described the pattern of symptoms and they judged the individual described to be in need of treatment. Functionally, it is rare to encounter an individual with substantial mental health symptoms who is not experiencing some level of impairment as a result, and so clinicians commonly emphasize symptomatology and may not examine carefully the requirement of functional impairment in their diagnostic consideration. Another possibility is that the demand characteristics of the study made it more likely that participants would assign a diagnosis to all cases. Many clinicians may work in environments where it is unusual not to give some sort of diagnosis. But another possibility is that the definition of functional impairment in the guidelines may have been misleading. The wording of the guideline only referenced observable instances of impairment, but did not allow for circumstances under which the person did not exhibit disruption in gross external markers of impairment such as employment or disrupted relationships but continued to function with greater difficulty and suffering because of the interfering nature of the symptoms. Of the explanations offered, the last is most consistent with the findings from the study. Participants mostly confirmed the presence of functional impairment in cases where Working Group members had rated it as absent, likely because the clinicians associated severe symptoms with difficulty in maintaining role responsibilities by their very nature (i.e., a tendency to avoid makes maintaining work responsibilities difficult and re-experiencing symptoms make those activities distressing). In response, the Working Group altered their definition of functional impairment to include cases where an individual may continue to function in gross terms, but has to exert exceptional effort to do so because of interference from symptoms (see Table 9).

Another consistent finding of this study is that clinicians were tempted to diagnose a case based upon the occurrence of a potentially traumatic event and the nature of the event rather than on the nature of the symptoms (Comparisons 5 and 7). Technically, each diagnosis in the Disorders Specifically Associated with Stress grouping is defined and differentiated by the nature of the symptomatic reaction to the stressor and not by the stressor itself. However, the severity of the stressor is an imperfect correlate of the disorder type, moving from Adjustment Disorder to PTSD to Complex PTSD as stressor severity and chronicity increase. The Working Group revised the description of these disorders to de-emphasize the nature of the stressor and focus upon the symptomatic profile of each condition (see Table 9). They also added additional information to the “Boundary with Other Disorders and Normality” section of the guidelines, helping to differentiate when clinicians should diagnose each of the three conditions. These guidelines clarify that stressful events are common, and many people do not develop any lasting symptoms in their aftermath, and that individuals may develop a range of disorders following exposure to traumatic circumstances, including depressive and anxiety disorders in addition to Disorders Specifically Associated with Stress, thereby emphasizing that the symptomatic picture is the key determining feature of the diagnosis.

A major concern about the addition of Prolonged Grief Disorder has been that clinicians would over-apply the concept to cases of normal grieving (Comparison 6). While participants’ responses in this study generally support the addition of Prolonged Grief Disorder to ICD-11 given that there was little disagreement about positive cases, the results of this study also raise some concerns about the differentiation of Prolonged Grief Disorder from normal grief. In response, the Working Group added language to the guidelines stipulating that a diagnosis should never be made within 6 months of the bereavement event, and also greater clarification about how best to determine a normative reference within the culture of the bereaved person (see Table 9).

Limitations

The results of this study should be interpreted in light of its limitations. No vignette methodology can address all questions relevant to the development of ICD-11 diagnostic guidelines. Specifically, clinic-based field studies and epidemiological work are necessary to address the fit of the guidelines to actual clinical cases. Second, the GCPN is not intended to be a representative sample of all mental health professionals; the participants in the GCPN are specifically interested in contributing to the improvement of mental health diagnosis or they would not have volunteered to participate in the network. Thus, the results may not generalize to some clinicians, situations, contexts, or languages. However, the GCPN represents a broad sample of mental health professionals and the study received greater participation than most other studies of diagnoses. Third, the vignettes described adult cases and the clinical utility of the proposed changes for children and adolescents remains to be established. The key changes to PTSD and Adjustment Disorder are of particular importance in this regard because the greater specificity of the guidelines may lead to missed cases if child-specific expression of re-experiencing and preoccupation is not taken into account (Scheeringa, Myers, Putnam, & Zeanah, 2012). Fourth, we chose to collapse the results across all three languages in which the study was conducted. Small linguistic differences did occur, but none impacted the overall pattern of results presented here. It is worth noting that the Japanese sample was more homogenous, but this is also reflective of diagnostic practice in Japan. In Japan, only physicians may assign a diagnosis and the large majority of psychiatrists are male. Nonetheless, differences in culture and language are important in the interpretation and implementation of diagnostic guidelines, and deserve further attention.

Finally, there is an important question regarding whether diagnosis is best viewed as a process of systematic hypothesis-testing or as matching information to well-learned pre-existing templates or exemplars (Elstein & Schwartz, 2002). Investigating the recognition of changes to existing diagnoses, particularly relatively small changes, favors the hypothesis-testing approach and its application of specific rules. Clinicians using a pattern-matching approach may be slow to adjust their templates, such that performance levels only improve after repeated exposure to and practice with the diagnostic changes. Thus the vignette methodology may be best at identifying problems with the initial application of the new rules and have less to say about levels of accuracy that can be achieved over the longer term.

Conclusion

As the first of a series of case-controlled studies on different areas of the proposed diagnostic guidelines for ICD-11 Mental and Behavioural Disorders, this study demonstrates that this method is viable for testing major elements of the implementation of the proposed diagnostic guidelines by clinicians. The results of this study were successful in identifying aspects of the proposed ICD-11 diagnostic guidelines for Disorders Specifically Associated with Stress that could be accurately and consistently implemented by clinicians as well as aspects of the proposed ICD-11 diagnostic guidelines that successfully clarified the diagnostic landscape in comparison to ICD-10.

The results also indicated areas in which clinicians had greater difficulty in applying the guidelines, providing an opportunity for clarifying the diagnostic guidelines prior to testing them in clinical settings. The findings of this study also highlight the fact that some aspects of the changes introduced in the ICD-11 may be difficult for practicing clinicians, and that changes in the diagnostic guidelines may not be sufficient to alter their diagnostic behavior. For example, the positive symptom requirements for Adjustment Disorder appeared to be difficult for clinicians to distinguish accurately, and clinicians may still experience a need for a residual mental disorder category for individuals who they judge to need treatment but who do not meet the diagnostic requirements of other disorders. This study helps to identify areas in which additional education related to new conceptualizations in ICD-11 is likely to be required. Based on this evidence of specific difficulties clinicians may encounter in implementing the ICD-11 diagnostic guidelines, WHO can develop targeted educational programs and make special efforts to highlight those areas when the ICD-11 is formally adopted.