Validity of nosological classification.

The term "nosological classification" is often used in connection with medical classification systems, and the tendency is to equate it with "diagnosis" and "validity." However, particularly in the case of psychiatry this is far from always being the case. From a scientific point of view, the two most up-to-date classification systems in use today - the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV), and the International Classification of Diseases, 10th Revision (ICD-10) - may be considered as the theoretical basis of current psychiatric nosology. In this paper we show that the instrumentally generated DSM-IV or ICD-10 diagnoses of schizophrenia have relatively low validity in comparison with clinician expert diagnoses. If medical classification is to be realistic, simple to use, and reliable, nosological systems must be based not only on established facts, but also on theoretical assumptions regarding the nature of disease.

Since their official introduction, the International Classification of Diseases, 10th Revision (ICD-10),[1] and the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV),[2] operational classification systems have largely become an integral part of the body of knowledge of psychiatrists throughout the world and instruments they constantly refer to. In this article I look at some of the questions that have been raised in connection with these classifications, both as a result of the growing number of critical analyses and of my own experience. This short contribution does not claim to provide exhaustive answers, but merely to stimulate further discussion.

Psychiatrists probably all started adopting operational diagnostic classification systems, such as the ICD and DSM classifications, on the assumption that the reliability of the diagnoses therein defined was unequivocally demonstrated to be very high across the centers and even countries of evaluation, without realizing that the general consensus was based on the lowest level of validity conceivable, since it resulted from the mutual agreement of experts rather than on any proven facts concerning the etiology of mental disorders. This means that in the absence of biological markers for most psychopathological disorders, diagnostic features were based on clinical descriptions, resulting in “official” nosological groupings. One of the main objections raised by clinical psychiatrists was that in many instances diagnoses were based on the numbers of certain symptoms.[3] Nevertheless, in spite of initial warnings of oversimplification, the two most widely used official classifications - DSM and ICD - came to be largely regarded as nosologically valid by medical doctors, official institutions, and even the public at large. The interesting, but logical, paradox is that those least satisfied with these so universally acclaimed classifications are probably the psychiatrists. In this article, I would like to briefly discuss two frequently asked questions: (i) what is the validity of the current diagnostic process? and (ii) what are the weak points of the DSM and ICD classifications?

What is the validity of the current diagnostic process?

Clinical psychiatric practice is mainly based on unstructured interviews. This approach yields excellent results in terms of diagnosis, provided it is carried out by experienced clinicians; unfortunately it is the least objective, reproducible, and reliable one.[4]

The answer to this problem would appear to be validated rating scales, administered by trained examiners. However, although such scales prove very reliable in terms of interrater and intertest results and validity, this applies only to symptoms and syndromes and not to diagnoses.

Structured interviews have relatively high reliability yet lower validity because this type of interview does not provide a framework that makes it possible to follow all the leads that a patient may offer. Previous psychiatric history, information from the entourage, previous response to medication, as well as difficult-to-define features related to “clinical impression” are usually omitted from operational definitions. There is nearly no room for clinical hunches or intuition on the part of the doctor using the DSM-IV or ICD-10 classifications.

Karl Popper is noted for stating that the ultimate test for the validity of a theory is to try to disprove it. If the theory stands the test, we may keep it, but if it fails, then it should be replaced by another theory.[5] With this in mind, I would like to discuss the findings of a study I carried out at the Mental Health Clinical Research Center (MHCRC) of the University of Iowa College of Medicine on the reproducibility and validity of the ICD-10 and DSM-IV clinical and operational diagnoses of schizophrenia, which clearly showed the limitations of structured diagnostic interviews for schizophrenia. This study compared clinical diagnoses made by clinicians using unstructured interviews and operational diagnoses generated from a computer algorithm derived from the Comprehensive Assessment of Symptoms and History (CASH).[6]

Background

The DSM-IV nosological concept of schizophrenia has been strongly contested by many researchers, such as, for example, Maj in 1998.[7] Schizophrenia, as defined by DSM-IV, does not follow any “classic” paradigm. It is a diagnosis by exclusion. The symptomatological, chronological, and functional criteria, taken together, arc not sufficient to characterize schizophrenia as a syndrome, so that exclusion criteria are decisive for the diagnosis. What we currently call schizophrenia is merely a heterogeneous group of nonaffectivc psychotic syndromes whose etiology is unknown. Does the schizophrenic syndrome have a special character that cannot be translated into operational terms? Does the diagnosis of the trained psychiatrist rely on a holistic impression of the subject, which operational criteria are unable to communicate? Do DSM-IV criteria fail to catch one or more clinical aspects that are essential for the diagnosis? If all essential elements of the schizophrenic syndrome are present in the DSM-IV definition, are they described in insufficient detail? Or is the clustering of symptoms not appropriately defined?

Most databases for biological research in psychiatry are now produced with the help of structured diagnostic interviews. Structured interviews represent the mainstay of diagnostic instruments in psychiatry, particularly those which allow some freedom to follow individual leads that may emerge. They can also be programmed for computerized scoring. For example, the Schedule for Clinical Assessment in Neuropsychiatry (SCAN)[8] and Comprehensive Assessment of Symptoms and History (CASH)[9] are excellent structured interviews and recording instruments for documenting the signs, symptoms, and history of subjects evaluated in research studies on the major psychoses and affective disorders. Nevertheless, structured interviews have substantial limitations that restrict their diagnostic validity. Any diagnosis that relies on the subjective interpretation of patient reports or laboratory tests, as well as on instrumental assessment, carries some risk of error. This error may be due to the equipment used (faulty equipment, poor calibration), to human error on the part of the assessors (poor training, carelessness, mislabeled samples or reports), or to the patients (misreporting or inconsistency in what patients say or do). Almost all diagnostic procedures include one or other of these elements. Medical diagnosticians are not infallible, and probably will never be so.[9]

Structured interviews provide broad descriptive coverage in order to enable investigators to make diagnoses using a variety of criteria, but they cannot provide an appropriate instrument for making a differential diagnosis. The validity of arbitrarily constructed diagnoses can be temporary only. When a disorder becomes better understood, the symptoms held to be the most reliable may well prove to lose their importance as indicators of the condition. In time, phenomenologically (arbitrarily) constructed diagnoses and clinician “gold standard” diagnoses should logically diverge. The poorer the correlation between the construct and the clinician diagnosis, the greater the probability that the construct does not reflect contemporary knowledge and should be corrected or replaced.

Aim of the study

The aim of the study was to answer the following questions: (i) Is there a satisfactory correlation between computer-processed (ie, algorithmic) ICD-10 diagnoses and clinician (“gold standard”) diagnoses of schizophrenia? (ii) Is there satisfactory correlation between computer-processed (ie, algorithmic) DSM-IV diagnoses and clinician (“gold standard”) diagnoses of schizophrenia? (iii) In which way does the degree of correlation affect the diagnostic validity of ICD-10 and DSM-IV schizophrenia?

Hypothesis

Assuming the expert clinician diagnosis (“holistic approach”) is valid, observation of a low correlation between clinician and algorithmic diagnoses reflects the low validity of the algorithmic diagnosis.

The medical records of 43 subjects used in the DSM-IV Field Trial Iowa Site were analyzed. DSM-IV diagnoses as well as ICD-10 diagnoses were made, using unstructured interviews (clinical expert diagnoses), and the structured, operational diagnostic (CASH) method, which records the relevant signs and symptoms (algorithmic diagnoses). To enhance the validity of the results of the unstructured psychiatric examinations, we controlled all 43 medical records with regard to the consistency of the objective medical and subjective patient data. The symptoms and syndromes listed in CASH were carefully evaluated by welltrained MHCRC specialists.

The diagnostic algorithm was applied directly to the CASH diagnoses.

Diagnostic algorithms were prepared for, and applied to, the DSM-IV and ICD-10 diagnoses of schizophrenia.

Algorithmic diagnoses and expert clinician diagnoses were correlated by calculating the kappa coefficient (Table I).

Possible explanations for the observed diagnostic discordance were proposed.

Results

As can be seen in Table I, only a marginal correlation between expert clinician and algorithmic DSM-IV and ICD-10 diagnoses of schizophrenia was found. Assuming the expert clinician diagnoses of schizophrenia (made by the “holistic approach”) were indeed valid (the “gold standard”), the implication is that the validity of algorithmic diagnoses was relatively low.

Four main limitations of the arbitrarily made diagnoses of DSM-IV and ICD-10 schizophrenia were found, relating to: (i) symptom severity thresholds; (ii) evaluation of the mood syndrome; (iii) specification of psychotic/mood duration ratio; and (iv) ICD-10/DSM-IV differences in the specification of hallucinations.

Discussion

The results of the study show that instrumcntally generated DSM-IV or ICD-10 diagnoses of schizophrenia had relatively low validity when compared with clinician expert diagnoses. These findings are in agreement with the views expressed by Maj in his editorial,[6] and lead to the following questions:

Is it possible to determine whether the operational approach is disclosing the intrinsic weakness of the concept of schizophrenia or the intrinsic limitations of the operational approach?

Is there, perhaps, beyond the individual phenomena, a “psychological whole” that the operational approach fails to grasp, or is such a “psychological whole” simply an illusion that the operational approach unveils?

Is there a possibility that the potential of the operational approach has not been fully tapped? For example, some important “classic” features such as autism were omitted in the operational criteria of schizophrenia.

Does the form and content of the subjective experiences of individuals who arc diagnosed as having schizophrenia require more in-depth investigation and characterization, reversing the recent process of reduction of psychotic phenomena to their lowest common denominator?

What are the weak points of the DSM and ICD classifications?

After years of experience with the DSM-IV and ICD-10 classifications, some more or less anticipated weak points of these classifications have become evident. Many critical analyses have been published, eg, the recently published article by Tucker.[10] The current DSM and ICD process gives the image of precision and exactness. Indeed, we as psychiatrists have come to believe that we are dealing with clear and discrete disorders rather than arbitrary symptom clusters. We are now being taken at our own word by managed care companies that stipulate that if a patient's symptoms fulfill current criteria for schizophrenia or recurrent depressive disorder, drug treatment must be given strictly according to the textbook. In fact, to quote Gary J. Tucker “at best, we are between Scylla and Charybdis - we no longer want to say that each patient is a unique individual, nor can we honestly say that every case clearly fits diagnostic criteria.”[10] All of this apparent precision overlooks the fact that, as yet, we have no identified etiological agents for psychiatric disorders. In psychiatry, no matter how scientifically and precisely we use scales to evaluate the patient's pathological symptoms, all we are really doing is simply pattern recognition. We are still only making an empirical diagnoses and not etiological ones based on disruptions of structure of function.

After these considerations I would like to briefly consider some more optimistic perspectives that I believe could positively influence psychiatric classification and nosology in the near future. New, exciting concepts and paradigms are looming on the horizon of psychiatric classification. New intellectual frameworks for psychiatry have been introduced, for example by Kandel,[11] who proposes that the genes expressed in the brain encode proteins that play important roles at specific stages of the development, maintenance, and regulation of the neural circuits that underlie behavior. Modern cognitive psychology is exploring language, perception, memory, motivation, and skilled movements in ways that are proving to be stimulating, insightful, and rigorous. The recent merger of cognitive psychology with neural science, to give birth to cognitive neuroscience, is proving to be one of the most exciting areas in biology.

Through these and others hypotheses, psychiatry is searching for a new identity and a new nosological approach. ICD-10 and DSM-IV have offered psychiatrists worldwide consensual and more or less valid diagnostic hypotheses. But now, after years of extensive use, the time has come for a critical appraisal of both classifications. A renewed involvement of psychiatry with biology and neurology is not only scientifically important, but also epitomizes the scientific competence that should be the basis for the clinical specialty of psychiatry in the near future.

As for clinical assessment, I fully agree with Tucker that the time has come to merge the empirical psychiatry of today's classification systems with the story and actual observation of the patient. Accurate observation of symptoms and the story of the patient must be included in our diagnostic processes.[9] Perhaps multiaxial classification will prove to be one of the ways out of oversimplification.

A renaissance of psychopathological research should be encouraged. Several excellent and very sophisticated tools like SCAN or CASH have already been developed, but unfortunately their interpretation and even their terminology is not identical. We should work carefully on achieving a broad international consensus on the assessment and terminology of psychological signs and symptoms, in the same way that we worked on the whole system of psychiatric classification some years ago.

I would like to conclude with a quotation from my wonderful host and coworker from Iowa, the excellent clinician and researcher Nancy Andreasen, and propose an answer to one of the questions posed by the recently deceased distinguished Danish psychiatric taxonomist and great friend of mine from Ârhus, Eric Strömgren. Nancy Andreasen wrote in a very recent article[12].

“While evidence-based decision making is a core value of medicine, and while DSM has done a valuable service in standardizing diagnostic practices, we as physicians must also devote a part of our time and energy to understanding how our patients feel and think and change subjectively. This is central to our role as doctors - if we are going to help them as healers, and if we are going to develop innovative insights about disease processes to test in research paradigms.”

Eric Stromgren asked in 1992[4]: “We are carried on by a huge taxonomic wave. Returning to classification, to taxonomy, we must ask the question: Are we just now in what could be called a 'taxonomorphic' age?”

It seems to me that the right answer to Strômgreifs question today is: “Yes, we are.”

Table I.

Correlation between DSM-VI / ICD-10 diagnoses and expert clinical diagnoses

• DSM-IV algorithm
	Expert clinical diagnoses
Kappa	0.34
• ICD-10 algorithm
	Expert clinical diagnoses
Kappa	0.37
Kappa>0.75	excellent correlation
0.4<kappa<0.74	good correlation
kappa<0.4	marginal correlation