Preliminary development of a deep learning-based automated primary headache diagnosis model using Japanese natural language processing of medical questionnaire.

BACKGROUND: Primary headaches are widespread and costly public health problems. However, there are insufficient medical resources for their treatment in Japan due to two reasons. First, the numbers of headache specialists and clinics remain insufficient. Second, neurologists and neurosurgeons mainly treat headaches in Japan. However, they mainly work as general stroke neurologists, so they cannot focus on primary headache treatment. To solve these problems, we preliminarily developed a deep learning (DL)-based automated diagnosis model from patients' Japanese unstructured sentences in the medical questionnaire using a DL framework. We hypothesized that the model would reduce the time and burden on both doctors and patients and improve their quality of life.
METHODS: We retrospectively investigated our primary headache database and developed a diagnosis model using the DL framework (Prediction One, Sony Network Communications Inc., Japan). We used age, sex, date, and embedding layer made by the medical questionnaire's natural language processing (NLP).
RESULTS: Eight hundred and forty-eight primary headache patients (495 women and 353 men) are included. The median (interquartile range) age was 59 (40-74). Migraine accounted for 46%, tension-type headache for 47%, trigeminal autonomic cephalalgias for 5%, and other primary headache disorders for 2%. The accuracy, mean precision, mean recall, and mean F value of the developed diagnosis model were 0.7759, 0.8537, 0.6086, and 0.6353, which were satisfactory.
CONCLUSION: The DL-based diagnosis model for primary headaches using the raw medical questionnaire's Japanese NLP would be useful in performing efficient medical practice after ruling out the secondary headaches. Copyright:

INTRODUCTION

Headache is a widespread and costly public health problem.[4] Migraine and tension-type headache (TTH) are included as primary headaches in the International Classification of Headache Disorders (ICHDs), and they are the main types of primary headaches. In Japan, the overall prevalence of migraine is 8.4%, and 74.2% of them complain that migraine headache impairs their daily activity significantly.[2,13] Furthermore, about 15–20% of Japanese people have TTH, and 22.4–29.2% complained that TTH disturbed their performances.[2,11,18] However, only 2.7% of the migraine sufferers consult a medical facility regularly,[13] and 59.4% of the primary headache patients had never consulted a physician about their headaches.[18] Therefore, most of the headache patients presumably manage the pains by taking over-the-counter medications.[2] Besides, if the headache patients consult doctors, only neuroimaging is conducted to exclude emergent or organic diseases, and the diagnosis for detail primary headache and its treatment are inadequate. Even when the doctors diagnosed primary headaches, their knowledge of treatment is unsatisfactory, leading to patient dissatisfaction.[2]

We think that these insufficient medical resources for headache treatment in Japan are due to two reasons. First, the numbers of headache specialists and headache clinics remain insufficient.[2] Second, in Japan, mainly neurologists and neurosurgeons treat headaches, but almost all of those in the acute care hospital work as general stroke neurologists at the normal outpatient, emergency room, operating room, catheterization room, stroke care unit, intensive care unit, and rehabilitation room.[14] Although a simple headache diagnosis flow chart is available,[2] they cannot afford to focus on primary headache treatment because of the busyness.

Furthermore, the World Health Organization declared the COVID-19 pandemic in March 2020, which led to challenges in health-care systems and societies worldwide. This pandemic has been producing a rapid shift in favor of telemedicine instead of an in-person consultation. Kristoffersen reported that 86% of the neurological department changed their headache practice during the lockdown into telephone consultations or video consultations in Denmark and Norway.[6] Gatti and López-Bravo reported a similar shift toward telemedicine in Italy[3] and Spain.[9] In Japan, similar to the other countries, online medical care has been officially performed from April 2020. There are increasing 4830 hospitals or clinics with online systems in June 2020, and telemedicine for headaches started.[19] However, even under this shift toward telemedicine not only equipment but also more medical staff would be needed, and it is feared that telemedicine will increase staff ’s burden.[9]

Therefore, an automated diagnosis system is still expected, in addition to the telemedicine system. Different from the common diseases, headaches are diagnosed mainly based on the personal interview,[8] not on the neurophysiological tests, neuroimaging, nor blood tests. Although a simple headache diagnosis flow chart[2] and systematic classification are available,[12] automated diagnosis specialized for primary headache has not been established. Kwon reported that the questionnaire with subdivided 75 items could be used for the artificial intelligence (AI)-based automated diagnosis for primary headache,[8] but AI-based automated diagnosis system using “raw” sentences in the medical questionnaire has not been reported.

Therefore, ahead of the world, we preliminarily tried to develop a deep learning (DL)-based diagnosis model from patients’ Japanese unstructured sentences in the medical questionnaire at outpatient using DL framework, Prediction One (Sony Network Communications Inc., Tokyo, Japan)[16] with Japanese natural language processing (NLP). The model is for the automated diagnosis of the primary headache based on our database. If the model can be established, it will be helpful for both routine outpatient and telemedicine during this COVID-19 era and reduce the time doctors and nurses spend interviewing and diagnosing. This is the first attempt to develop a DL-based diagnosis model with Japanese NLP of the “raw” medical questionnaire regarding primary headache.

MATERIALS AND METHODS

The study was approved by our hospital’s research ethics committee (KCHE-2020-5) on October 30. We gained written informed consent for this study from all of the patients, the legally authorized representative of the patients, or the deceased patients’ next of kin. All methods were carried out following relevant guidelines and regulations (Declaration of Helsinki). All personal patient information were deleted from the database for this study to protect patient privacy.

Study population

This retrospective study included 848 consecutive new patients aged over 15 years of age at our neurosurgical outpatient who were revealed as a primary headache after neurological examination, personal interview, laboratory tests, head computed tomography, and magnetic resonance angiography between 2015 and 2019. The primary headaches were defined in the ICHD 3rd edition (beta version) (ICHD-3 beta),[12] and we diagnosed the primary headaches according to it. We classified the patients into three categories; migraine (Part I, 1. in ICHD-3 beta), TTH (Part I, 2. in ICHD-3 beta), and others, including trigeminal autonomic cephalalgias (TACs, Part I, 3. in ICHD-3 beta) and other primary headache disorders (OPHDs, Part I, 4. in ICHD-3 beta).

Making the diagnosis model by Prediction One

We used Prediction One framework to make an automated diagnosis model. We used four variables for making the model by Prediction One; age, sex, date of consultation, and the patients’ Japanese unstructured description in the medical questionnaire. Prediction One read the dataset and automatically divided them into almost half as internal training and cross-validation datasets. It automatically adjusted and optimized the numerical variables in a way that is easy to process statistically and mathematically. The unstructured description Japanese sentences were analyzed through NLP using an original Japanese morphological analysis library made by Sony Network Communications Inc. Predefined unimportant words or those that appeared too infrequently were treated as “stop words,” excluded from the analysis. After making the embedding layer from the sentences, using the embedding layer, age, sex, and date, Prediction One selected an appropriate algorithm with ensemble learning and made the best diagnosis model by an artificial neural network (ANN) with internal cross-validation. The details of ANN are trade secrets and could not be provided [Figure 1]. We obtained a confusion matrix of the model from the dataset and calculated accuracy, precision, recall, and F value, which are performance indicators of the AI-based prediction models.

Figure 1:

Chart of making deep learning-based automated diagnosis model. The Japanese unstructured description in the medical questionnaire was processed using Japanese natural language processing with Japanese morphological analysis, and the embedding layer was produced. Using the embedding layer, age, sex, and date, a deep learning framework (Prediction One) made an automated diagnosis model. DL: Deep learning, EOS: End of sentences, SW: Stop word, NLP: Natural language processing, TTH: Tension-type headache.

Statistical analysis

Results are shown as median (interquartile range) because the variables were not with a normal distribution. Mann–Whitney U-test and Chi-square test were used. These statistical analyses were performed with the SPSS software version 24.0.0. (IBM, New York, USA). A two-tailed P < 0.05 was considered statistically significant.

RESULTS

Clinical characteristics

Clinical characteristics of the 848 patients (495 women and 353 men) with primary headaches are summarized in [Table 1]. The median (interquartile range) age was 59 (40–74). Migraine accounted for 46%, TTH for 47%, TACs for 5%, and OPHDs for 2%. The median ages of the migraine patients were significantly younger than those of TTH and TACs (both P < 0.001 by Mann–Whitney U-test). The ratio of women was not significantly higher in the migraine group compared to other groups (P > 0.05 by Chi-square test) [Table 1].

Table 1:

Patients characteristics.

Characteristics of the Japanese unstructured description in the medical questionnaire

The median (interquartile range) number of Japanese words was 105 (77–155). Predefined stop words were 188 types of word, but those new were not provided by Prediction One framework. We could not obtain the NLP embedding layer nor identify the specific word contributing to the model accuracy.

DL-based automated diagnosis model

Prediction One produced the DL-based automated diagnosis model using our primary headache patients’ database. [Table 2] is the confusion matrix derived from the model. The accuracy, mean precision, mean recall, and mean F value were 0.7759, 0.8537, 0.6086, and 0.6353, respectively.

Table 2:

Confusion matrix of the predicted and actual diagnosis.

DISCUSSION

We herein report 848 primary headache patients who newly came to our hospitals. We preliminarily made the DL-based automated diagnosis model for primary headaches using Japanese NLP. Our model had satisfactory performance based on only four variables. This suggested the possibility of creating a DL-based automated diagnosis model for primary headaches with Japanese NLP analysis after ruling out secondary headaches.

DL and Japanese NLP

Recently, AI is attracting, and it is a transitional period regarding AI from machine learning to DL.[5] DL can analyze not only numerical variables but also images and sentences, and NLP can convert sentences into the embedding layer for DL analysis. In other languages, NLP to text analysis of electronic medical records could achieve the early diagnosis of conditions.[1,20] However, Japanese lags behind other languages in adapting to NLP because the Japanese language does not have spaces between the words, and it has omission of subjects and a variety of verb conjugations. With the recent advance in the technologies, Japanese NLP has been used in medical situations for predicting inpatient falls,[10] distinguishing types of falls from the nursing records and incident reports in electronic medical records,[21] and application in neurolinguistics analysis.[7] However, there are no reports on the association between primary headache and Japanese NLP as far as we know.

We think that the automated diagnosis model for primary headaches is suitable for Japanese NLP analysis because patients often write their headache characteristics in detail in the medical questionnaire compared with the answers to the medical staff ’s closed questions. If we can know the diagnosis almost only from the patients’ medical questionnaire, we could save the times and burdens of doctors, nurses, and patients without an in-person consultation during this COVID-19 and telemedicine era. Furthermore, many Asian languages, including Japanese, have onomatopoeia.[15] For example, zuki-zuki, gan-gan, and doku-doku mean severe pulsatile headache; sara-sara, piripiri, and toku-toku mean mild pulsatile headache; zuki-n or ga-n means severe thunderclap headache; and ji-n means continuous mild headache. The onomatopoeia can help to make diagnoses and to decide medications. Furthermore, different people have different things they want to appeal to, so the word and sentence orders and their numbers can change accordingly. Therefore, we tried to make a diagnosis model using Japanese NLP. Our preliminary model had satisfactory performance based on only four variables. Compared to the previous similar report on the AI-based diagnosis model using 75 detailed subdivided questionnaires,[8] our model suggested that “raw” Japanese sentences could be useful as variables for DL-based model development.

Despite the easiness, advantages, and future potential of DL, the majority of medical staff cannot treat DL software.[17] However, as simple DL software like Prediction One is being developed, there is a need for an active interest in using it to benefit medical staff and patients.[5] Our study is just one example and suggested the utility of DL software. We are just doctors and not so familiar with DL, but we could make such a prediction model. We believe that DL-based efficient medicine would be performed worldwide, and our stress and workload would be reduced in the future, especially during this COVID-19 and telemedicine era.

Limitation of this study

First, the number of the sample was small, and this study performed in a single hospital. Other home doctors or internal medicine hospitals also treat headaches, so not all headache patients in our medical area could necessarily be studied. Second, we did not separate chronic migraine and TTH (15 or more days per month) and those nonchronic (less than 15 days per month). Its mechanism is supposed to be different in each type.[2] Therefore, we should have investigated the frequency per month and used detailed information to make the diagnosis model. Besides, we should have investigated the second digits of the primary headache diagnosis based on the ICHD-3 beta. Third, we performed internal cross-validation but did not perform external validation, and the diagnosis model made from our hospital’s dataset cannot be applied to other hospitals.

CONCLUSION

We preliminarily made a DL-based automated diagnosis model for primary headaches. The accuracy, mean precision, mean recall, and mean F value were 0.7759, 0.8537, 0.6086, and 0.6353, respectively. In the future, after ruling out the secondary headaches, the DL-based automated diagnosis model using Japanese NLP of unstructured sentences in the medical questionnaire may lead to perform efficient medical practice, reduce the burden on both doctors and patients, and improve their quality of life.