Validity and reliability of chronic tic disorder and obsessive-compulsive disorder diagnoses in the Swedish National Patient Register.

OBJECTIVES: The usefulness of cases diagnosed in administrative registers for research purposes is dependent on diagnostic validity. This study aimed to investigate the validity and inter-rater reliability of recorded diagnoses of tic disorders and obsessive-compulsive disorder (OCD) in the Swedish National Patient Register (NPR).
DESIGN: Chart review of randomly selected register cases and controls.
METHOD: 100 tic disorder cases and 100 OCD cases were randomly selected from the NPR based on codes from the International Classification of Diseases (ICD) 8th, 9th and 10th editions, together with 50 epilepsy and 50 depression control cases. The obtained psychiatric records were blindly assessed by 2 senior psychiatrists according to the criteria of the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR) and ICD-10. PRIMARY OUTCOME MEASUREMENT: Positive predictive value (PPV; cases diagnosed correctly divided by the sum of true positives and false positives).
RESULTS: Between 1969 and 2009, the NPR included 7286 tic disorder and 24,757 OCD cases. The vast majority (91.3% of tic cases and 80.1% of OCD cases) are coded with the most recent ICD version (ICD-10). For tic disorders, the PPV was high across all ICD versions (PPV=89% in ICD-8, 86% in ICD-9 and 97% in ICD-10). For OCD, only ICD-10 codes had high validity (PPV=91-96%). None of the epilepsy or depression control cases were wrongly diagnosed as having tic disorders or OCD, respectively. Inter-rater reliability was outstanding for both tic disorders (κ=1) and OCD (κ=0.98).
CONCLUSIONS: The validity and reliability of ICD codes for tic disorders and OCD in the Swedish NPR is generally high. We propose simple algorithms to further increase the confidence in the validity of these codes for epidemiological research. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Randomly drawn sample of International Classification of Diseases (ICD) 8, ICD-9 and ICD-10 cases.

Meticulous review of the medical records performed blindly by two psychiatrists.

Excellent inter-rater reliability.

Limited number of cases.

Introduction

Several countries maintain nationwide longitudinal hospital discharge registers with mandatory individual-level reporting of diagnoses. In addition to facilitating health surveillance, such registers provide fertile soil for epidemiological research. In the 1960’s, the Swedish National Board of Health and Welfare started collecting information regarding inpatients at public hospitals, the National Patient Register (NPR).1 The NPR and other Swedish health registers use the 10-digit personal identification number (PIN), introduced in Sweden in 1947, which has full coverage of residents in Sweden.2 The PIN is essential for linkage between registers and for data collection throughout the healthcare system. Initially, the NPR contained information about all inpatient psychiatric care, but somatic care had limited coverage and geographic coverage was incomplete. The NPR includes all inpatient care in all counties of Sweden from 1973 for psychiatry and from 1987 for somatic clinics. From 2001, the register also started collecting information on outpatient appointments, including psychiatry, from private as well as public caregivers. Primary care is not covered in the NPR.

Obsessive-compulsive disorder (OCD) is a mental disorder characterised by intrusive thoughts, urges or images (ie, obsessions such as ‘did I really lock that door?’), followed by repetitive behaviours aimed to reduce the associated anxiety (ie, compulsions such as repeatedly checking that the door is locked). It is a prevalent disorder affecting ∼2% of the population.3 Tics are recurring, involuntary, rapid and sudden non-rhythmic motor movements or vocalisations. In Tourette's syndrome (TS), the person experiences multiple motor and one or more vocal tics for more than 1 year. In chronic tic disorders (CTDs), motor or vocal tics (but not both) are present for more than 1 year. In the Diagnostic and Statistical Manual of Mental Disorders system (DSM),4 the onset must be before the age of 18 years for both TS and CTD, whereas in the ordinary non-research version of the International Classification of Diseases (ICD) no such requirement exists. The prevalence of TS is around 0.5–0.9% in paediatric populations.5

The NPR is one of several Swedish registers that have been successfully used for research in many disorders and linking data from several registers greatly increases the potential. A large variety of epidemiological and genetic studies of psychiatric disorders, including OCD, have been based on the NPR.6–12 Strengths of such register-based studies include the very large, unselected sample sizes, resulting in high statistical power and minimising the risk of ascertainment biases, the possibility of examining risk factors prospectively and the possibility of reconstructing large family pedigrees.

The usefulness of the cases in the NPR for further research is dependent on their diagnostic validity. The most common way to assess the validity of diagnoses in the NPR has been to assess randomly drawn cases through chart review, that is, the assessment of medical records to confirm diagnosis.1 The general picture is that the positive predictive value (PPV) is high in many diagnoses.1 2 In psychiatry, the validity of some diagnostic codes, such as schizophrenia13 14 and bipolar disorder,15 has been well established, while the validity of other diagnoses such as tic disorders, including TS and OCD, have yet to be studied. A recent Finnish study has reported good validity of the register-based diagnosis of TS,16 but we could not find other validation efforts in the Nordic countries.

Aim

To facilitate further genetic epidemiological research, this study aimed to validate the diagnoses of tic disorders, including TS and OCD, in the Swedish NPR.

Methods

The Swedish NPR

All diagnoses were recorded according to the ICD-8 (1969–1986), ICD-9 (1987–1996)17 and ICD-10 (1997–2009).18

Cases were identified in the NPR between 1969 and 2009 using the following ICD codes: for tic disorders, ICD-8: 306,2, ICD-9: 307C and ICD-10: F95.X. The latter included the following subcodes: F95.0 (transient tic disorder), F95.1 (chronic motor or vocal tic disorder), F95.2 (TS), F95.8 (other tic disorders), F95.9 (unspecified tic disorder). For OCD, the codes were: ICD-8: 300,3, ICD-9: 300D and ICD-10: F42.X. Diagnosis at one point in time sufficed for inclusion.

Procedures

After approval of the Swedish central ethics board (Dnr Ö 27–2012), we requested PINs from a random sample of cases and controls out of the NPR from the Swedish National Board of Health and Welfare. The individuals were not asked for consent, as this would introduce selection biases. The PINs were used to obtain the medical records for a chart review to verify the register diagnoses.

We requested PINs of 100 random tic disorder cases and 100 OCD cases. In addition, for every second tic disorder case, we drew one random control case with a diagnosis of epilepsy (G40.X). Similarly, for every second patient with OCD, we drew one random control with a registered diagnosis of major depression (F32.X). The purpose of the control files was to prevent having the reviewing psychiatrist guess the diagnosis by an overly limited number of possible diagnoses. Cases and controls were drawn from three different counties: Stockholm, Södermanland and Gotland, selected for convenience.

Retrieval of medical records

Once the random cases and controls had been identified, we sent written requests to the appropriate archives or clinics, based on the hospital and medical specialty codes associated to the cases. We requested the full medical records. Several waves of reminders were sent out.

Chart review

Two psychiatrists conducted a chart review of the medical records (blind to the actual register diagnosis) and a diagnosis was established independently by each of the two raters, based on all available information in the medical records at hand. While all diagnoses are added into the register as ICD codes, we assessed the cases according to both ICD-10 and Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR). We reasoned that the results would be more meaningful and generalisable since DSM system has become the global standard in psychiatric research. Furthermore, neither the Swedish nor English versions of the ICD-10 contain operational diagnostic criteria, but, instead, they contain narrative descriptions of the disorders, further justifying the additional validation against DSM-IV-TR criteria4 (which are very similar to the ICD-10 research criteria). The size of the medical records ranged from 1 to over 1000 pages. Some cases did not contain enough information to allow arriving at a diagnosis; an example could be someone going through knee surgery having an additional diagnosis of a psychiatric condition that was not described in the medical records.

Statistical analyses

We calculated the PPV, that is, those cases diagnosed correctly divided by the sum of the true positives and the false positives and their corresponding 95% CIs. Inter-rater reliability was calculated using the Cohen's κ statistic.19 SPSS Statistics V.19 was used for the calculations.

Results

Annual incidence of tic disorders and OCD in the Swedish NPR

Figures 1 and 2 show the annual incidence of the studied disorders in the NPR. The incidence drastically increased from 2001, as outpatient care was included in the register. Consistently, the vast majority of cases in the NPR are coded with ICD-10 (table 1).

Figure 1

Annual incidence of tic disorders in the Swedish National Patient Register. Black bars: number of new cases diagnosed from 1969 to 2000 (inpatients only). White bars: number of new cases diagnosed from 2001 to 2009 (inpatients and outpatients).

Figure 2

Annual incidence of obsessive-compulsive disorder in the Swedish National Patient Register. Black bars: number of new cases diagnosed from 1969 to 2000 (inpatients only). White bars: number of new cases diagnosed from 2001 to 2009 (inpatients and outpatients).

Table 1

Incidence of tic disorder and OCD cases by ICD version

	Tic disorders		OCD
ICD version	Frequency (%)	Cumulative frequency (%)	Frequency (%)	Cumulative frequency (%)
ICD-8	325 (5)	325 (5)	3230 (13)	3230 (13)
ICD-9	310 (4)	635 (9)	1688 (7)	4918 (20)
ICD-10	6651 (91)	7286 (100)	19 839 (80)	24 757 (100)

ICD, International Classification of Diseases; OCD, obsessive-compulsive disorder.

Table 2 displays the median age and sex distribution of the entire sample of requested files as well as the confirmed cases.

Table 2

Age and sex of the sample

	Tic disorders		OCD
	Requested files (N=100)	Confirmed diagnoses (N=59)	Requested files (N=100)	Confirmed diagnoses (N=48 (rater 1)-49 (rater 2))
Median age	13.5	13.0	33.0	31.0
Female/male	29/71	19/40	53/47	26–27/22

OCD, obsessive-compulsive disorder.

Validity and reliability of tic disorder codes

After several reminders were sent out, we received 73 of the 100 requested tic disorder files (table 3). Of the received files, nine did not contain sufficient information and were excluded from analyses. Among the remaining 64 patients identified in the NPR as having an ICD-8/ICD-9/ICD-10 tic disorder, 59 were true-positive cases. There was 100% of agreement between the two raters regarding the presence or absence of a tic disorder (κ=1). The remaining five cases were false positive cases, that is, they were not confirmed on chart review. This corresponds to an overall PPV for tic disorders of 0.92 (95% CI 0.82 to 0.97); all versions of ICD had high validity (table 4).

Table 3

Number of requested, received and final patient files included in the analyses, by ICD version and county

	Tic disorders				OCD
	Number of files				Number of files
	RequestedN	ReceivedN (%)	ExcludedN (%)	FinalN (%)	RequestedN	ReceivedN (%)	ExcludedN (%)	FinalN (%)
Total	100	74 (74)	10 (13)	64 (87)	100	75 (75)	7 (9)	68 (91)
ICD version
ICD-8	25	20 (80)	2 (10)	18 (90)	33	22 (67)	2 (9)	20 (91)
ICD-9	25	18 (72)	4 (22)	14 (78)	33	26 (79)	1 (4)	25 (96)
ICD-10	50	36 (72)	4 (11)	32 (89)	34	27 (79)	4 (15)	23 (85)
County
Stockholm	51	40 (78)	7 (17)	33 (83)	51	36 (71)	5 (14)	31 (86)
Södermanland	31	19 (61)	3 (16)	16 (84)	33	24 (73)	0 (0)	24 (100)
Gotland	18	15 (83)	0 (0)	15 (100)	16	15 (94)	2 (13)	13 (87)

ICD, International Classification of Diseases; OCD, obsessive-compulsive disorder.

Table 4

Positive predictive values according to the two raters (senior psychiatrists) who assessed the NPR cases, by ICD version

	Tic disorders		OCD
	Rater 1	Rater 2	Rater 1	Rater 2
Total	0.92 (0.82–0.97)	0.92 (0.82–0.97)	0.72 (0.60–0.82)	0.71 (0.58–0.81)
ICD version
ICD-8	0.89 (0.64–0.98)	0.89 (0.64–0.98)	0.55 (0.32–0.76)	0.55 (0.32–0.76)
ICD-9	0.86 (0.56–0.97)	0.86 (0.56–0.97)	0.64 (0.43–0.81)	0.64 (0.43–0.81)
ICD-10	0.97 (0.82–0.99)	0.97 (0.82–0.99)	0.96 (0.76–0.99)	0.91 (0.70–0.98)

CIs in brackets.

ICD, ICD, International Classification of Diseases; NPR, National Patient Register.

Further examination of individual ICD-10 subcodes for tic disorders (table 5) revealed that three of four patients who received a F95.1 (CTD) code in the register were diagnosed as having TS (F95.2) by the raters (motor and vocal tics could both be identified in the clinical histories). Unspecified tic disorder (F95.9) cases were diagnosed by the raters as either TS (F95.2), CTD (F95.1), unspecified tic disorder (F95.9) or transient tics (F95.0).

Table 5

ICD-10 tic disorder cases

ICD-10 code in the NPR	Rater 1	Rater 2
F95.0—transient	F95.9—unspecified	F95.2—TS
F95.0—transient	F95.2—TS	F95.2—TS
F95.1—chronic	No tics	No tics
F95.1—chronic	F95.2—TS	F95.2—TS
F95.1—chronic	F95.2—TS	F95.2—TS
F95.1—chronic	F95.2—TS	F95.2—TS
F95.9—unspecified	F95.0—transient	F95.0—transient
F95.9—unspecified	F95.1—chronic	F95.1—chronic
F95.9—unspecified	F95.1—chronic	F95.1—chronic
F95.9—unspecified	F95.2—TS	F95.2—TS
F95.9—unspecified	F95.0—transient	F95.0—transient
F95.9—unspecified	F95.9—unspecified	F95.9– unspecified
F95.9—unspecified	F95.9—unspecified	F95.1—chronic
F95.9—unspecified	F95.9—unspecified	F95.1—chronic
F95.9—unspecified	F95.9—unspecified	F95.0—transient
F95.9—unspecified	F95.9—unspecified	F95.9– unspecified
F95.2—TS	F95.1—chronic	F95.2—TS
F95.2—TS	F95.2—TS	F95.2—TS
F95.2—TS	F95.2—TS	F95.2—TS
F95.2—TS	F95.2—TS	F95.2—TS
F95.2—TS	F95.2—TS	F95.2—TS
F95.2—TS	F95.2—TS	F95.1—chronic
F95.2—TS	F95.2—TS	F95.2—TS
F95.2—TS	F95.2—TS	F95.2—TS
F95.2—TS	F95.2—TS	F95.2—TS
F95.2—TS	F95.2—TS	F95.2—TS
F95.2—TS	F95.2—TS	F95.2—TS
F95.2—TS	F95.2—TS	F95.1—chronic
F95.2—TS	F95.2—TS	F95.2—TS
F95.2—TS	F95.2—TS	F95.2—TS
F95.2—TS	F95.2—TS	F95.2—TS
F95.2—TS	F95.2—TS	F95.2—TS

ICD, ICD, International Classification of Diseases; NPR, National Patient Register; TS, Tourette's syndrome.

When using the DSM-IV-TR criteria instead of ICD-10, there were three cases who were given another third position code in DSM-IV-TR based on an age of onset above 18 years of age. Two cases were assessed as ICD-10 F95.0 (transient) but would be F95.9 according to DSM-IV-TR, whereas one case of ICD-10 CTD would be F95.9 according to DSM-IV-TR.

Validity and reliability of OCD codes

We were able to obtain 75 of the requested 100 OCD files. Of the received files, seven did not contain sufficient information and were excluded from analyses. The final number of OCD cases included in the analyses was 68 (details in table 3). Among 68 patients identified in the NPR as having ICD-8/ICD-9/ICD-10 OCD, 48 (according to rater 1) or 49 (according to rater 2) were true-positive cases, indicating outstanding inter-rater reliability (κ=0.98). The remaining 20 (rater 1) or 19 (rater 2) were false positive cases, that is, they were not confirmed on chart review. This corresponded to an overall PPV for OCD of 0.72 (95% CI 0.60 to 0.82) for rater 1 and 0.71 (95% CI 0.58 to 0.81) for rater 2 (table 4). There were marked differences between ICD versions; only ICD-10 codes had the desired level of validity (91–96%). Table 6 lists the patients who were deemed to not fulfil the DSM-IV or ICD-10 criteria for OCD and the reasons for this.

Table 6

OCD false positive cases

NPR diagnosis	ICD	Correct diagnosis/problem according to raters
OCD	8	Depressive ruminations
OCD	8	Psychosis
OCD	8	Ruminations
OCD	8	Dentist phobia
OCD	8	Eating disorder
OCD	8	Dentist phobia
OCD	8	Adjustment disorder
OCD	8	Eating disorder
OCD	8	Asperger syndrome
OCD	9	Kleptomania
OCD	9	Depressive ruminations
OCD	9	Depressive ruminations
OCD	9	Bizarre behaviour not related to OCD
OCD	9	Substance misuse
OCD	9	Paedophilia
OCD	9	OCPD, possibly psychotic
OCD	9	Health anxiety
OCD	9	Adjustment disorder
OCD	10	Asperger syndrome
OCD	10	Drug related

ICD, International Classification of Diseases; NPR, National Patient Register; OCD, obsessive-compulsive disorder; OCPD, obsessive-compulsive personality disorder (Anankastic Personality Disorder in ICD).

False negatives

We received 43 of the 50 requested epilepsy files and 44 of the 50 requested depression files, which we randomly mixed with the tic disorder and OCD files, respectively. None of the epilepsy control files were diagnosed as having a tic disorder, and none of the depression control files were diagnosed as having OCD, indicating that there were no false negative cases in these randomly selected files.

Discussion

National hospital discharge registries and other patient registers are important cornerstones for medical epidemiology research. The Swedish NPR is one the world's largest and most comprehensive registers, and establishing the diagnostic validity and reliability of the disorders under study is a fundamental step. This study is the first to report on the diagnostic validity of tic disorders and OCD in the Swedish NPR. With an approximate prevalence of 1% and 2%, respectively, CTDs and OCD are major sources of disability and reduced quality of life worldwide; their causes are largely unknown. Overall, the validity and inter-rater validity of the ICD codes in the Swedish NPR were satisfactory, paving the way for a wide range of studies, for example, genetic epidemiology and pharmacoepidemiology, in these understudied psychiatric conditions. A similar study conducted in Finland, which included telephone interviews in addition to chart reviews, found that the validity of ICD-10 tic disorder cases was also over 90%.16

While the tic disorder cases showed very good PPV when assessed as one broad diagnosis (eg, ICD-10 code F95), there was less agreement regarding the third position of the ICD codes (such as F95.2). For example, clinicians seemed to diagnose CTD (F95.1) even when there was evidence of lifetime presence of motor and vocal tics. Unspecified tic disorder (F95.9) cases were diagnosed by the raters as either TS (F95.2), CTD (F95.1), unspecified tic disorder (F95.9) or transient tics (F95.0), suggesting that F95.9 is used more freely by clinicians or perhaps as a default code in many cases. On the basis of these results and the current DSM criteria (eg, regarding duration), we propose an algorithm to ensure that only patients who have chronic tics are included in future studies (box 1). Similar algorithms have been used successfully in other disorders when using the NPR for research.15

1. All cases with at least one diagnosis of tic disorder in International Classification of Diseases eighth revision (ICD-8) or ICD-9.

2. All ICD-10 cases that have at least one diagnosis of chronic tic disorder (F95.1) or Tourette syndrome (F95.2), unless they receive a final diagnosis of transient tic disorder (F95.0) within the same year as the initial diagnosis. The latter would indicate that the clinician has adjusted the diagnosis based on the insufficient duration of the symptoms.

3. If other (F95.8) or unspecified (F95.9) tics are coded, at least one additional tic disorder diagnosis (F95.1, F95.2, F958 or F95.9) is required, unless they receive a final diagnosis of transient tic disorder (F95.0) within the same year as the initial diagnosis.

The validity for the ICD-10 OCD codes was excellent (PPV>90%). This is reassuring because over 80% of all OCD codes in the NPR come from ICD-10. By contrast, the ICD-8 and ICD-9 OCD codes in our study yielded an unacceptably high rate of false positives. There may be a number of reasons for the marked difference across ICD revisions. In general, the diagnostic precision of psychiatric diagnoses has increased over time as DSM has made its mark on psychiatry, and diagnostic criteria were introduced in ICD. The ICD 8–9 versions did not include the narrative descriptions that were introduced in ICD-10 and may therefore have been more prone to interpretations and misdiagnosis, resulting in weakened diagnostic validity. It is also likely that the increased use of structured diagnostic procedures has contributed to this increased precision. Tic disorders are, arguably, easier to recognise and diagnose, and therefore seem less susceptible to changes in the diagnostic systems. Our recommendation is that only ICD-10 OCD codes be used in future research.

Strengths and limitations

The main strengths of our study include a randomly drawn sample of cases and controls, and the meticulous blind review of the medical records by two independent psychiatrists, who had outstanding inter-rater reliability. The main limitation is the relatively small number of studied cases, which may have yielded some random findings. While the small number of epilepsy and depression controls did not allow us to meaningfully calculate negative predictive values, it was reassuring that none of these files contained disorders of interest, hinting at a low risk of false negatives. Another inherent limitation of any chart-review study is the lack of in-person observations, which are particularly important in the diagnosis of tic disorders.

Conclusions

The proportion of ‘true’ tic disorder (ICD-8, ICD-9 and ICD-10) and OCD (ICD-10 only) cases in the Swedish NPR is high, paving the way for a wide range of epidemiological studies in these understudied conditions.