When does grief become pathological? Evaluation of the ICD-11 diagnostic proposal for prolonged grief in a treatment-seeking sample.

Background: Prolonged grief disorder (PGD) will be newly included in the ICD-11, while a clinically similar diagnosis, persistent complex bereavement disorder (PCBD), has already been added to the DSM-5. Only few studies have evaluated these criteria-sets for prolonged grief. Objective: The aim of this study was to evaluate the ICD-11 accessory symptom threshold and compare the diagnostic performance of the two criteria-sets in treatment-seeking bereaved persons. Method: 113 grief treatment-seeking bereaved persons completed the Interview for Prolonged Grief-13. We used receiver operator characteristic analysis to determine an optimum ICD-11 accessory symptom threshold. We calculated diagnostic rates for PGD and PCBD and examined associations of PGD and PCBD caseness with concurrently assessed psychopathology and prolonged grief symptoms assessed one month later.
Results: An ICD-11 threshold of six accessory symptoms distinguished optimally between interview-diagnosed participants with and without prolonged grief. The prevalence of PGD (69%) was significantly higher than that of PCBD (48%) and of PGD with a 6-symptom threshold (47%). PGD caseness was associated with the relation to the deceased, 6-symptom threshold PGD and PCBD caseness with the time since loss. All criteria-sets were linked to concurrent prolonged grief, depression, and general mental distress. PCBD and 6-symptom threshold PGD but not PGD were associated with prolonged grief severity one month later. Conclusions: The results support the validity of PGD and PCBD but, at the same time, they provide further support for differing prevalence rates. Using an empirically determined ICD-11 accessory symptom threshold could prevent the pathologisation of grief reactions.

A significant minority of bereaved people fail to adjust to the loss of a loved one and develop persistent and disabling grief. Such prolonged grief is characterized by intense longing and yearning, avoidance of reminders, difficulty accepting the death, bitterness, numbness, and social/identity disruption (Prigerson et al., 2009). Although prolonged grief shares certain features with post-traumatic stress disorder and depression (e.g. intrusions, feeling that life is unfulfilling or meaningless) and is often comorbid (Simon et al., 2007), research has shown that prolonged grief can be discriminated from these disorders (e.g. Boelen & van den Bout, 2005; Dillen, Fontaine, & Verhofstadt-Denève, 2009). A meta-analysis has found that one in ten bereaved persons following non-violent death suffer from prolonged grief (Lundorff, Holmgren, Zachariae, Farver-Vestergaard, & O’Connor, 2017). Research suggested several risk factors for prolonged grief, including female gender, lower education, a close relationship with the deceased, violent, unexpected, and multiple loss, and comorbid psychopathology (Heeke, Kampisiou, Niemeyer, & Knaevelsrud, 2019; Lobb et al., 2010).

However, the interpretation of research findings is hampered by the reliance on self-report measures and a multiplicity of different criteria-sets for prolonged grief (American Psychiatric Association [APA], 2013; Prigerson et al., 2009; Shear et al., 2011; WHO, 2018). Prigerson et al. (2009) proposed criteria for prolonged grief disorder (PGD), while Shear et al. (2011) suggested criteria for complicated grief. Previous studies have evaluated and compared these criteria-sets (e.g. Cozza et al., 2019; Maciejewski, Maercker, Boelen, & Prigerson, 2016). The present study was focused on criteria now included in the two dominant classification systems, ICD-11 (WHO, 2018) and DSM-5 (APA, 2013). The DSM-5 lists persistent complex bereavement disorder (PCBD) as a condition requiring further research (APA, 2013). It seems to be a combination of Prigerson et al. (2009)’s criteria for PGD and Shear et al. (2011)’s complicated grief concept. PCBD diagnosis requires the loss of a loved one, one separation distress symptom, and six additional symptoms to the point of functional impairment for at least 12 months following the loss (APA, 2013). PGD has now been included as a diagnostic entity in the ICD-11 (WHO, 2018). In a beta version of ICD-11 PGD, seven diagnostic features were introduced (beta version PGD; Maercker et al., 2013). The current version of ICD-11 PGD includes twelve diagnostic features but no specification of the accessory symptoms threshold (WHO, 2018). Although all ICD-11 versions are based on Prigerson et al. (2009)’s criteria, findings on Prigerson et al. (2009)’s PGD may not be generalized to ICD-11 PGD as the number of accessory symptoms needed to meet the criterion has not been defined (Eisma & Lenferink, 2018; but see Killikelly & Maercker, 2017). A diagnosis of ICD-11 PGD requires having experienced the loss of a significant other, one of two separation distress symptoms, and at least one accessory symptom to a functionally impairing degree for at least six months after the loss (WHO, 2018). In sum, ICD-11 features and DSM-5 criteria for prolonged grief share certain characteristics (e.g. separation distress as core symptom). However, they differ in important aspects: (a) time criterion, (b) additional symptoms threshold, and (c) content of additional symptoms (see Tables 1 and 2).

Table 1.

ICD-11 features for PGD and item match.

Symptoms of ICD-11 PGD	Item match (instrument, item number)
1. A persistent and pervasive longing for the deceased or	How often have you felt yourself longing or yearning for the person you lost (PG13 + 9, 1)
2. A persistent and pervasive preoccupation with the deceased	Do you think so much about her/him that it is hard for you to do the things you normally do (PG13 + 9, 14)
3. Accompanied by intense emotional pain, e.g.: Sadness	How often have you had intense feelings of emotional pain, sorrow, or pangs of grief related to the lost relationship (PG13 + 9, 2)
4. Guilt	Do you feel guilty in relation to loss (PG13 + 9, 15)
5. Anger	Do you feel angry about her/his death (PG13 + 9, 18)
6. Denial	Is it hard for you to believe that she/he is really dead (PG13 + 9, 19)
7. Blame	I feel that it is unfair that I should live when this person died (ICG, 16)
8. Difficulty accepting the death	Have you had trouble accepting the loss (PG13 + 9, 7)
9. Feeling one has lost a part of one’s self	Do you feel confused about your role in life or feel like you don’t know who you are (i.e. feeling that a part of yourself has died) (PG13 + 9, 6)
10. An inability to experience positive mood	Have you had trouble experiencing joy, contentment, or happiness since the loss (PG13 + 9, 17)
11. Emotional numbness	Do you feel emotionally numb since the loss (PG13 + 9, 11)
12. Difficulty engaging with social or other activities	Do you feel that moving on (e.g. making new friends, pursuing new interests) would be difficult for you now (PG13 + 9, 10)
13. Functional impairment	Have you experienced a significant reduction in social, occupational, or other important areas of functioning (e.g. domestic responsibilities) (PG13 + 9, 13)

PGD = prolonged grief disorder. PG13 + 9 = extended version of the Interview for Prolonged Grief-13. ICG = Inventory of Complicated Grief.

Table 2.

DSM-5 criteria for PCBD and item match.

Symptoms of DSM-5 PCBD	Item match (instrument, item number)
1. Persistent yearning/longing for the deceased	How often have you felt yourself longing or yearning for the person you lost (PG13 + 9, 1)
2. Intense sorrow and emotional pain	How often have you had intense feelings of emotional pain, sorrow, or pangs of grief related to the lost relationship (PG13 + 9, 2)
3. Preoccupation with the deceased person	Do you think so much about her/him that it is hard for you to do the things you normally do (PG13 + 9, 14)
4. Preoccupation with circumstances of the death	Memories of the deceased upset me (ICG, 2)
5. Difficulty accepting the death	Have you had trouble accepting the loss (PG13 + 9, 7)
6. Disbelief or numbness	Do you feel emotionally numb since the loss (PG13 + 9, 11)
7. Difficulty positive reminiscing about deceased	Have you had trouble with positive reminiscing about the deceased (PG13 + 9, 16)
8. Bitterness or anger	Do you feel bitter over the loss (PG13 + 9, 9)
9. Maladaptive appraisals about the self associated with the loss (e.g. self-blame)	I feel that it is unfair that I should live when this person died (ICG, 16)
10. Excessive avoidance of stimuli	How often have you tried to avoid reminders that the person you lost is gone (PG13 + 9, 4)
11. A desire to die to be with the deceased	Do you feel a desire to die in order to be with the deceased (PG13 + 9, 21)
12. Difficulty trusting other people	Has it been hard for you to trust others since the loss (PG13 + 9, 8)
13. Feeling alone or detached from other persons	Do you feel alone or detached from other individuals since the loss (PG13 + 9, 20)
14. Feeling that life is empty or meaningless orone is unable to function without the deceased	Do you feel that life is unfulfilling, empty, or meaningless since the loss (PG13 + 9, 12)
15. Confusion about one’s role and diminished identity (e.g. feeling that part of self died)	Do you feel confused about your role in life or feel like you don’t know who you are (i.e. feeling that a part of yourself has died) (PG13 + 9, 6)
16. Difficulties to pursue interests or plan for the future	Do you feel that moving on (e.g. making new friends, pursuing new interests) would be difficult for you now (PG13 + 9, 10)
17. Functional impairment	Have you experienced a significant reduction in social, occupational, or other important areas of functioning (e.g. domestic responsibilities) (PG13 + 9, 13)

PCBD = persistent complex bereavement disorder. PG13 + 9 = extended version of the Interview for Prolonged Grief-13. ICG = Inventory of Complicated Grief.

Several recent studies have compared the diagnostic performance of the ICD-11 and DSM-5 proposals. In a sample of bereaved military family members (Cozza et al., 2019), ICD-11 PGD (82%) outperformed PCBD (47%) in case identification (i.e. a score of ≥ 30 on the self-report measure Inventory of Complicated Grief, ICG; Prigerson et al., 1995). They concluded that case identification improved for PCBD when the associated symptom numbers were lowered to one or two. In a community sample (Maciejewski et al., 2016), interview-based PCBD and beta version PGD with a higher accessory symptoms threshold were similar in terms of predictive validity and prevalence rates (11–14%). Likewise, in a conjugally bereaved elderly sample (O’Connor et al., 2019), PCBD and PGD yielded similar prevalence rates (6–9%). In two bereaved community samples from the Netherlands (Boelen, Lenferink, Nickerson, & Smid, 2018; Boelen et al., 2018), PCBD and PGD resulted in substantially different prevalence rates (6–8% vs. 18–19%). After gradually raising the accessory symptoms threshold for PGD, diagnostic agreement with PCBD improved. Overall, ICD-11 features and DSM-5 criteria for prolonged grief seem to differ substantially in terms of prevalence rates.

It is important to further evaluate these diagnostic proposals in more populations (e.g. grief treatment-seeking persons) as research is still limited by sample characteristics and the reliance on self-report measures. Only one study had assessed symptoms of prolonged grief based on clinical interviews (Maciejewski et al., 2016) despite the fact that self-reported symptoms may lead to an overestimation of diagnostic rates in contrast to interview-based assessments (e.g. Steel et al., 2009). The Interview for Prolonged Grief-13 (PG-13; Prigerson et al., 2009) seems to be the most widely used structured interview (Heeke et al., 2019; Lundorff et al., 2017) but it has not been employed for criteria evaluation up to now. The ICD-11 accessory symptoms threshold is particularly problematic. It has not been clearly defined which means that at least one additional symptom is needed (WHO, 2018). Receiver operating characteristic (ROC) curves are the most informative way of empirically determining an optimum symptom threshold by depicting the trade-offs between sensitivity and specificity for a diagnostic criterion (Swets, 1988). To our knowledge, no prior study has empirically evaluated the additional symptoms threshold for PGD as per ICD-11 (WHO, 2018).

The aim of this study was to examine the diagnostic performance of ICD-11 features and DSM-5 criteria for prolonged grief in treatment-seeking bereaved persons. First, we aimed to determine an optimum accessory symptom threshold for ICD-11 PGD that could differentiate between interview-diagnosed individuals with and without prolonged grief. Second, we compared prevalence rates between ICD-11 PGD and PCBD and examined diagnostic agreement. Third, we investigated loss-related correlates of PGD and PCBD caseness. Finally, to evaluate concurrent and predictive validity, we compared cases and non-cases of PGD and PCBD in terms of concurrently assessed psychopathology as well as prolonged grief symptoms assessed one month later.

Method

Participants and procedure

Participants were recruited for a multicentre RCT to evaluate grief-specific integrative cognitive behavioural therapy in comparison to an active but non-specific control condition, present-centred therapy, at four university outpatient mental health clinics in Germany from May 2017 to December 2018 (Rosner, Rimane, Vogel, Rau, & Hagl, 2018; trial registration: DRKS00012317). The study protocol has been approved by the Institutional Review Board of the Catholic University Eichstaett-Ingolstadt (2016/21). In this study, we used data from persons who sought treatment in the pilot trial (i.e. each therapist had to treat a pilot case under supervision before entering the main trial, n = 70) and who were the first persons recruited for the main trail (n = 43). Treatment-seeking persons, aged 18 to 75 years, whose losses had occurred at least 6 months previously were invited to a first clinical interview. If the person scored 20 points or more on the German version of the ICG (Lumbeck, Brandstätter, & Geissner, 2013), a subsequent baseline assessment was conducted. Informed consent was obtained from all participants. During the baseline assessment, a trained interviewer conducted the PG13 to assess prolonged grief diagnosis and the German version of the Structured Clinical Interview for DSM-IV Axis-I Disorders (SCID-I; Wittchen, Zaudig, & Fydrich, 1997) for comorbidity. Self-report measures were given to the participants for completion at home. During the third appointment, eligible participants were informed about the randomization result. Individuals who did not meet the eligibility criteria were informed about treatment alternatives. In this study, we used data from all participants who had completed the baseline assessment, irrespective of their eligibility for the trial. The total sample of 113 treatment-seeking bereaved persons consisted of 54 persons who were randomized in the pilot or main trial and 59 individuals who had been excluded.

Measures

Sociodemographic and loss-related information was obtained, including age, gender, education, employment, family status, time since loss, relationship to the deceased, and circumstances of the death. Psychiatric comorbidity assessment was undertaken using the German version of the SCID-I (Wittchen et al., 1997).

The extended German version of the PG-13 (PG13 + 9; Vogel, Pfoh, & Rosner, 2017) was used to assess prolonged grief. The criteria for a prolonged grief diagnosis corresponded to Prigerson’s PGD proposal (Prigerson et al., 2009). The criteria were met if the participant had, for six months or longer, after the loss (a) experienced at least one separation distress symptom at least once a day (≥ 4 on a 5-point scale: 1 = never/not at all, 5 = several times a day/extremely), (b) reported at least five out of nine cognitive, emotional, and behavioural symptoms (each symptom rated as ≥ 4), and (c) showed significant impairment in social, occupational or other important domains. Nine new items were included in the PG13 + 9 in order to cover most of the features in the ICD-11 proposal (Killikelly & Maercker, 2017; Maercker et al., 2013). The full PG13 + 9 has been published elsewhere (Rosner et al., 2018).

Prolonged grief symptoms were also measured using the German version of the ICG (Lumbeck et al., 2013). Participants were asked to rate the extent to which they had experienced 19 grief symptoms during the previous month on a 5-point scale ranging from 0 = never to 4 = all the time. A prolonged grief score was computed (range: 0–76), Cronbach’s alpha was .83

Symptoms of depression were measured using the German version of the Beck Depression Inventory II (BDI-II; Hautzinger, Keller, & Kühner, 2009). The 21 items refer to depressive symptoms during the previous week and are rated on a 4-point scale, resulting in scores ranging from 0–63. A total depression score was computed, Cronbach’s alpha in this study was .82.

Acute suicidality was assessed using the Columbia-Suicide Severity Rating Scale (C-SSRS; Posner et al., 2011). The five items in the intensity of suicide ideation subscale were used to rate the intensity of current suicide ideation on a 5-point scale (score range: 1–25).

The Screening for Somatoform Disorders (SOMS-7D; Rief & Hiller, 2008) was used to assess somatoform symptoms. Participants were asked to rate the extent to which they had suffered from 53 somatoform symptoms during the previous seven days on a 5-point scale (0 = not at all, 4 = very much). A somatization severity score was calculated. Cronbach’s alpha was .93 in this study.

General mental distress was measured using the Global Severity Index (GSI) from the German version of the Brief Symptom Inventory (BSI; Franke, 2000). The BSI is a widely used 53-item measure of subjective distress caused by psychological and somatic symptoms over the previous seven days. Responses are scored on a 5-point scale (0 = not at all, 4 = extremely). The GSI is calculated by adding up the nine subscales of the BSI (score range: 0–4). In this study, Cronbach’s alpha of the GSI was .95.

Data analysis

All symptoms of ICD-11 features and DSM-5 criteria were mapped on items from the PG13 + 9 and ICG. Eleven of the 13 PGD symptoms were extracted from the PG13 + 9 and one symptom from the ICG (Table 1). The 16 PCBD symptoms were represented by 14 PG13 + 9 items and 2 ICG items (Table 2).

We used ROC analysis to determine an optimum cut-off score for ICD-11 accessory symptoms that best distinguished between interview-diagnosed prolonged grief cases and non-cases (n = 60 and 53). The area under the curve (AUC) was calculated to establish the probability of a randomly selected participant being correctly classified in the appropriate group. Youden’s index was employed to determine an optimum threshold as the maximization of sensitivity and specificity was considered equally important in this study.

To investigate the prevalence rates of PGD and PCBD, we counted the number of participants who fulfilled the respective criteria-set. Each symptom score was dichotomized in absent (0) and present (1). PG-13 + 9 items were treated as present if they scored ≥ 4 on the 1-to-5-point scale. ICG items were rated as present if they scored ≥ 3 on the 0-to-4 scale. PGD caseness (unspecified version of PGD, PGD unspec) was determined by following the ICD-11 diagnostic rule (WHO, 2018), which requires the endorsement of at least one separation distress symptom (symptoms 1–2, Table 1), at least one accessory symptom (symptoms 3–12, Table 1), and the functional impairment item (symptom 13) for at least six months since the loss (time since loss variable, Table 3). In addition, we followed the same diagnostic rule but used the result of the ROC analysis to determine ICD-11 PGD caseness with an empirically evaluated symptom threshold (i.e. empirical version of PGD, PGD emp). PCBD diagnostic status was determined as follows (APA, 2013): endorsement of one B criterion item (yearning or preoccupation, symptoms 1–4, Table 2), at least six C criterion items (reactive distress or social/identity disruption; symptoms 5–15, Table 2), and the D criterion item (impairment; symptom 16, Table 2) for at least 12 months following the loss (time since loss variable, Table 3). Pairwise agreement between the diagnostic tests was evaluated using kappa statistics.

Table 3.

Sociodemographic and loss-related characteristics.

Characteristic	Total sample (N = 113)
Female, % (n)	81.4 (92)
Age in years, M (SD)	51.68 (13.85)
Education, % (n)
≤ 12 years	53.9 (61)
> 12 years	46.0 (52)
Employment, % (n)
Employed	61.0 (69)
Retired	25.7 (29)
Other	13.3 (15)
Marital status, % (n)
Married/in a relationship	36.2 (41)
Divorced/single	31.9 (36)
Widowed	31.9 (36)
Relation to the deceased, % (n)
Child	22.1 (25)
Partner	40.7 (46)
Parent	23.9 (27)
Other	13.3 (15)
Cause of death, % (n)
Natural	76.9 (87)
Unnatural	23.1 (26)
Expectation of the death, % (n)
Expected	33.6 (38)
Unexpected	66.4 (75)
Time since loss in months, M (SD)	46.14 (51.84)
Interview-diagnosed prolonged grief, % (n)	53.1 (60)

Prolonged grief diagnosis was assessed by trained raters using the Interview for Prolonged Grief-13.

Differences between cases and non-cases of PGD unspec, PGD emp, and PCBD in terms of loss-related variables (i.e. time since loss, relation to the deceased, expectation of the death, cause of death) were computed using χ2 tests and t-tests. T-tests were used to contrast cases and non-cases of the respective criteria-set in concurrently assessed psychopathology (i.e. disturbed grief, depressive symptoms, suicidality, somatoform symptoms, general mental distress, and number of comorbidities; Bonferroni-corrected α = .008). With regard to predictive validity, we compared cases and non-cases of each criteria-set that were included in treatment with available ICG data at baseline and one month later (T2, before the start of treatment, N = 44). A regression analysis was also performed for each criteria-set to investigate whether dummy-coded caseness could predict prolonged grief symptoms at T2 after controlling for baseline grief symptoms. All tests were two-tailed with α = .05. Data analyses were carried out using Stata version 14.0 (StataCorp, USA).

Results

Sample characteristics

Sociodemographic and loss-related characteristics are shown in Table 3. Participants were on average 51.6 years old (range: 18–81). Most participants were women (81%) and currently employed (61%). Sixty-one participants had completed secondary education only (≤ 12 years), while 52 had been to college (> 12 years). The majority of participants had lost a partner (46%), parent (27%), or child (25%). The deaths were mostly natural (87%) but unexpected (75%). The mean time since loss was 46.1 months (range: 6–292).

Evaluation of the ICD-11 diagnostic threshold

We conducted a ROC analysis to determine an optimum accessory symptom threshold for the ICD-11 features. As summarized in Table 4, a threshold of at least six symptoms optimally classified participants as meeting or not meeting criteria for PGD. The AUC of .93 was excellent (SE = .02, 95% CI: .86 – .97). This 6-symptom threshold was associated with a sensitivity of 82% and a specificity of 92%, leading to 87% of all participants being classified correctly. A threshold of one accessory symptom, as suggested for the ICD-11 (WHO, 2018), had a sensitivity of 100% and a specificity of 6%, classifying 55% of all participants correctly.

Table 4.

Discriminant values of ICD-11 accessory symptom numbers for identifying prolonged grief cases.

ICD-11 accessory symptoms	Sensitivity	Specificity	Younden’s J
1	1.00	0.06	0.06
2	1.00	0.09	0.09
3	1.00	0.30	0.30
4	0.98	0.56	0.54
5	0.88	0.75	0.63
6	0.82	0.92	0.74
7	0.65	0.96	0.61
8	0.37	1.00	0.37
9	0.15	1.00	0.15
10	0.07	1.00	0.07

Prolonged grief diagnosis was assessed by trained raters using the Interview for Prolonged Grief-13. Sixty participants suffered from prolonged grief, 53 participant did not. Values in bold refer to the optimal cut-point for accessory symptoms.

Prevalence rates and diagnostic agreement

The diagnostic rate of ICD-11 PGD unspec (69%) was significantly higher than that of PCBD (48%; χ21 = 41.02, p = .001) and PGD emp (i.e. at least six accessory symptoms; 47%; χ21 = 34.54, p = .001). There were 18 unique PGD unspec cases (i.e. meeting PGD unspec but not PCBD or PGD emp criteria), 1 unique PCBD case (i.e. meeting PCDB but not PGD unspec or PGD emp criteria), and two unique PGD emp cases (i.e. meeting PGD emp but not PGD unspec or PCBD criteria). The pairwise agreement between PGD unspec and PCBD (Kappa = 0.51, 95% CI: 0.36–0.67) and PGD unspec and PGD emp (Kappa = 0.54, 95% CI: 0.39–0.69) was moderate, while the agreement between PGD emp and PCBD was substantial (Kappa = 0.77, 95% CI: 0.65–0.88).

Loss-related correlates

We compared participants meeting versus not meeting ICD-11 and DSM-5 criteria-sets in terms of loss-related characteristics (i.e. time since loss, relation to the deceased, expectation of death, cause of death). As shown in Table 5, PGD unspec cases had lost a child more often (31%) than non-cases (3%). Participants meeting versus not meeting PGD unspec features did not differ in any other characteristic (all ps ≥ .611). In contrast to non-cases, PGD emp cases had been bereaved more recently. Likewise, participants meeting PCBD criteria had been bereaved more recently than participants not meeting the criteria. PCBD and PGD emp cases and non-cases did not differ in any other variable (all ps ≥ .133 and .072, respectively).

Table 5.

Differences in loss-related correlates between participants meeting vs. not meeting ICD-11 features and DSM-5 criteria for prolonged grief.

	DSM-5 PCBD			ICD-11 PGD unspec			ICD-11 PGD emp
Variable	Cases(n = 54)	Non-cases(n = 59)	t/χ2	Cases(n = 78)	Non-cases(n = 35)	t/χ2	Cases(n = 53)	Non-cases(n = 60)	t/χ2
Relation to the deceased, % (n)			5.59			13.96**			7.01
Child	25.9 (14)	18.6 (11)		30.8 (24)	2.9 (1)		28.3 (15)	16.7 (10)
Partner	48.1 (26)	33.9 (20)		41.0 (32)	40.0 (14)		45.3 (24)	36.7 (22)
Parent	16.7 (9)	30.5 (18)		17.9 (14)	37.1 (13)		13.2 (7)	33.3 (20)
Other	9.3 (5)	17.0 (10)		10.3 (8)	20.0 (7)		13.2 (7)	13.3 (8)
Cause of death, % (n)			0.84			0.26			1.58
Natural	77.8 (42)	76.3 (45)		75.6 (59)	80.0 (28)		71.7 (38)	81.7 (49)
Unnatural	22.2 (12)	23.7 (14)		24.4 (19)	20.0 (7)		28.3 (15)	18.3 (11)
Expectation of the death, % (n)			0.74			0.11			1.27
Expected	29.6 (16)	37.3 (22)		34.6 (27)	31.4 (11)		28.3 (15)	38.3 (23)
Unexpected	70.4 (38)	62.7 (37)		65.4 (51)	68.6 (24)		71.7 (38)	61.7 (37)
Time since loss in months, M (SD)	34.89 (26.4)	56.44 (65.77)	–2.25*	44.91 (48.01)	48.89 (60.17)	–0.38	34.50 (24.40)	56.42 (65.94)	–2.28*

PCBD = DSM-5 criteria for persistent complex bereavement disorder. ICD-11 PGD unspec = ICD-11 guidelines for prolonged grief disorder (PGD). ICD-11 PGD emp = ICD-11 guidelines for PGD with an accessory symptom threshold of six symptoms. *p < .05, **p < .01.

Concurrent and predictive validity

Participants meeting versus not meeting ICD-11 and DSM-5 criteria-sets were contrasted with regard to psychopathology (Table 6). PGD unspec cases showed higher levels of concurrently assessed prolonged grief symptoms, depressive symptoms, and general mental distress as well as more psychiatric comorbidities than non-cases. Relative to non-cases, cases of both PGD emp and PCBD displayed more psychiatric comorbidities, higher levels of prolonged grief, depression, suicidality, and general mental distress. One month later, at T2, participants meeting versus not meeting PGD unspec features did not differ in prolonged grief severity, whereas PGD emp and PCBD cases reported higher levels of prolonged grief than non-cases (see Table 6). After adjusting for baseline prolonged grief symptoms, caseness was no longer associated with grief symptoms at T2, irrespective of the criteria-set (PGD unspec: β = – .08, p = .612; PGD emp: β = .10, p = .659; PCBD: β = .14, p = .427).

Table 6.

Differences in psychopathology between participants meeting vs. not meeting ICD-11 features and DSM-5 criteria for prolonged grief.

	DSM-5 PCBD			ICD-11 PGD unspec			ICD-11 PGD emp
Variable	Cases(n = 54)	Non-cases(n = 59)	t	Cases(n = 78)	Non-cases(n = 35)	T	Cases(n = 53)	Non-cases(n = 60)	t
Prolonged grief (ICG)	47.15 (10.31)	33.06 (8.71)	7.83***	43.96 (11.10)	30.82 (7.64)	6.34***	47.88 (9.65)	33.14 (8.95)	8.37***
Depression (BDI-II)	28.86 (9.03)	21.54 (10.22)	3.19**	28.40 (8.97)	16.93 (9.20)	4.38***	30.31 (8.66)	20.93 (9.41)	4.39***
Suicidality(C-SSRS)	5.52 (3.98)	2.21 (4.38)	3.29**	5.11 (5.74)	1.92 (3.85)	1.91	6.82 (5.87)	2.34 (4.29)	3.11**
Somatization (SOMS-7D)	30.61 (23.05)	27.58 (21.64)	0.55	32.37 (22.65)	18.60 (18.20)	2.16	33.50 (24.13)	24.87 (19.68)	1.62
General mental distress (BSI)	1.32 (0.45)	0.99 (0.48)	3.74***	1.33 (0.54)	0.62 (0.47)	4.49***	1.34 (0.42)	1.00 (0.66)	2.93**
Number of comorbidities (SCID-I)	0.55 (0.69)	0.22 (0.49)	2.98**	0.50 (0.67)	0.03 (0.16)	4.04***	0.56 (0.70)	0.18 (0.46)	3.56**
Prolonged grief (ICG) – 1 month later a	45.04 (9.82)	38.31 (11.33)	2.10*	42.81 (11.12)	39.12 (9.99)	0.86	45.29 (10.28)	38.35 (10.65)	2.19*

PCBD = DSM-5 criteria for persistent complex bereavement disorder. ICD-11 PGD unspec = ICD-11 guidelines for prolonged grief disorder (PGD). ICD-11 PGD emp = ICD-11 guidelines for PGD with an accessory symptom threshold of six symptoms. ICG = Inventory of Complicated Grief. BDI-II = Beck Depression Inventory. C-SSRS = Columbia-Suicide Severity Rating Scale. SOMS-7D = Screening for Somatoform Disorders. BSI = Brief Symptom Inventory. SCID-I = Structured Clinical Interview for DSM-IV Axis-I Disorders. an = 44. *p < .05, **p < .008, ***p < .001.

Discussion

This study evaluated the diagnostic performance of ICD-11 features and DSM-5 criteria for prolonged grief in grief treatment-seeking bereaved persons who had completed the PG-13 interview. To evaluate the ICD-11 accessory symptom threshold, we performed a ROC analysis to determine an optimum symptom threshold that could best distinguish between interview-diagnosed individuals with and without prolonged grief. The results indicate that the ICD-11 threshold of at least one accessory symptom had high sensitivity but insufficient specificity. Only 55% of all participants could be correctly identified with this threshold, pointing to many false positives in classifying prolonged grief. Prior research has suggested that raising the ICD-11 symptom threshold might harmonize agreement between ICD-11 and DSM-5 criteria-sets (Maciejewski et al., 2016; Boelen et al., 2018; but see Cozza et al., 2019). At the same time, this might be one way of reducing the risk of pathologising grief reactions. In support of this, we found that a 6-symptom threshold had excellent predictive accuracy in differentiating between participants with and without interview-diagnosed prolonged grief, classifying 87% of all participants correctly. Based on this result, we highly recommend reconsidering the accessory symptom threshold for the current ICD-11 PGD guidelines (WHO, 2018). Some authors have cautioned against an under-inclusiveness of too restrictive criteria for prolonged grief (e.g. Cozza et al., 2019) as this may lead to overlooking cases who suffer from disabling grief and could benefit from grief-specific treatments (Mauro et al., 2018). However, an international field study of the ICD-11 guidelines has shown that clinicians are poor in differentiating normal grief reactions from mental disorders (Keeley et al., 2016), suggesting that unspecified criteria for prolonged grief could lead to many misdiagnoses.

The results of this study indicate a higher prevalence of PGD unspec than PCBD in treatment-seeking bereaved persons. The diagnostic rate of PGD unspec was 69%, while the rate of PCBD was 48%. The diagnostic agreement was only moderate (Landis & Koch, 1977), and this is substantially lower than would be expected for two criteria-sets representing the same diagnostic entity (Boelen et al., 2018). After determining PGD status with a 6-symptom threshold, the prevalence of PGD emp decreased (47%) and the agreement with PCBD improved, resulting in substantial agreement (Landis & Koch, 1977). This is in line with studies on bereaved community samples that showed that successively increasing the number of accessory symptoms needed to meet a ICD-11 PGD diagnosis (Boelen et al., 2018; Boelen, Lenferink, & Smid, 2019; Maciejewski et al., 2016) or lowering the DSM-5 PCBD additional symptoms threshold (Cozza et al., 2019) substantially increased agreement between the two criteria-sets. Former comparative studies reported lower prevalence rates for both criteria-sets (6–19%; Boelen et al., 2018; Maciejewski et al., 2016; O’Connor et al., 2019). One prior study used beta version ICD-11 PGD (Maercker et al., 2013) with a higher additional symptoms threshold (Maciejewski et al., 2016) that seems to have been more conservative than the final version of the ICD-11 PGD (WHO, 2018) used in this study. Moreover, we sampled exclusively treatment-seeking bereaved persons who were likely more distressed with regard to overall grief, which could also explain the higher rates in this study.

PGD unspec caseness was associated with the relation to the deceased and PGD emp and PCBD caseness was related to time since loss, corroborating previous findings on the correlates of the ICD-11 and DSM-5 criteria-sets (Boelen et al., 2019). This is also consistent with research on risk factors of prolonged grief demonstrating that grief levels tend to decrease as time goes by (e.g. Schaal, Jacob, Dusingizemungu, & Elbert, 2010), even though associations are mostly weak and heterogeneous (Heeke et al., 2019). Likewise, previous research has shown that a close relationship to the deceased (e.g. loss of a child) is a strong risk factor for prolonged grief (Heeke et al., 2019; Lobb et al., 2010).

In support of concurrent validity, PGD unspec, PGD emp, and PCBD caseness was linked to higher levels of concurrently assessed prolonged grief, depression, and general mental distress. For all criteria-sets, respective cases suffered from more psychiatric comorbidities than non-cases. These results correspond to the relations of PGD and PCBD with concurrent mental distress (e.g. Boelen et al., 2018; Comtesse & Rosner, 2019; Maciejewski et al., 2016). With regard to predictive validity, we found that PGD emp and PCBD cases reported greater prolonged grief severity than non-cases one month after baseline. However, meeting the features for PGD unspec was not associated with grief symptoms over time. This finding is mostly consistent with previous studies showing predictive validity for beta version PGD (Maercker et al., 2013) and PCBD (Boelen et al., 2018; Maciejewski et al., 2016) but not for PGD as per ICD-11 (Boelen et al., 2018). Yet, after adjusting for baseline prolonged grief symptoms, no criteria-set was associated any longer with grief symptoms assessed one month later. This indicates a high dependence on baseline grief levels.

This study has several strengths. First, the assessment of prolonged grief was based on clinical interviews carried out by trained raters. This not only allowed us to conduct a ROC analysis with a solid criterion standard (i.e. prolonged grief diagnosis) but it may also have reduced the risk of overestimating grief rates which may occur by relying solely on self-reported symptoms (e.g. Steel et al., 2009). Second, we investigated a treatment-seeking sample with many subthreshold and clinical cases that was diverse in terms of educational and economic background and relation to the deceased. This underpins the ecological validity of this study.

The limitations of this study need to be acknowledged. First, the majority of items used to determine PGD and PCBD diagnostic status were taken from the PG13 + 9 interview, although three items were also taken from a self-report measure, the ICG. Neither of these two instruments was specifically designed to assess symptoms according to the ICD-11 features or DSM-5 criteria. Future replications with specifically designed instruments are therefore needed. Second, women were overrepresented in our sample and there was a large variability with regard to the mean time since loss, both aspects may limit the generalizability of the results. Third, the evaluation of predictive validity was restricted to self-rated prolonged grief symptoms and a four-week time period. Future work should evaluate this aspect more broadly in terms of comorbid psychopathology and impairment over a longer period of time.

These findings may have implications for clinical practice and research. They suggest that heightening the ICD-11 accessory symptom threshold might be a promising way of preventing the pathologisation of grief reactions and, at the same time, harmonizing agreement between ICD-11 features and DSM-5 criteria for prolonged grief. The results show that PGD emp is more similar to PCBD in terms of diagnostic rates, agreement, and predictive validity and thus more likely to identify the same groups of bereaved persons. Research findings on risk factors and treatment based on PCBD might be more generalizable to PGD emp and vice versa. Clearly, more research across different populations is required to determine the extent to which ICD-11 PGD with a higher accessory symptoms threshold is similar to PCBD and previously validated proposals such as Prigerson et al. (2009)’s PGD in terms of risk factors, prevalence rates and criterion validity.