Automatic illness prediction system through speech.

Due to the COVID-19 epidemic and the curfew caused by it, many people have sought to find an ADPS on the internet in the last few years. This hints to a new age of medical treatment, all the more so if the number of internet users continues to expand. As a result, automatic illness prediction online applications have attracted the interest of a large number of researchers worldwide. This work aims to develop and implement an automated illness prediction system based on speech. The system will be designed to forecast the sort of ailment a patient is suffering from based on his voice, but this was not feasible during the trial, therefore the diseases were divided into three categories (painful, light pain and psychological pain), and then the diagnose process were implemented accordingly. The medical dataset named "speech, transcription, and intent" served as the baseline for this study. The smoothness, MFCC, and SCV properties were used in this work, which demonstrated their high representation to human being medical situations. The noise reduction forward-backward filter was used to eliminate noise from wave files captured online in order to account for the high level of noise seen in the deployed dataset. For this study, a hybrid feature selection method was created and built that combined the output of a genetic algorithm (GA) with the inputs of a NN algorithm. Classification was performed using SVM, neural network, and GMM. The greatest results obtained were 94.55% illness classification accuracy in terms of SVM. The results showed that diagnosing illness through speech is a difficult process, especially when diagnosing each type of illness separately, but when grouping the different illness types into groups, depending on the amount of pain and the psychological situation of the patient, the results were much higher.

Introduction

Researchers and companies have been attracted to the automatic disease prediction system (ADPS) in recent years because it enables rapid access to medical treatment, avoids traffic and long-distance travel, and is especially beneficial to senior citizens, regardless of the new era the world is entering with the spread of COVID-19 and the periodic curfews implemented by many countries to prevent the virus from spreading.

It is critical to understand the human speech production process in order to deal with ADPSs professionally. Phonetics is the study of the sounds produced by human speech. Speech is produced by pushing air from the lungs to the larynx (respiration), where the vocal cords may be open to allow for air flow or may vibrate, producing sound (phonation). The articulators in the mouth and nose, who are responsible for articulation, will affect the lungs' airflow.

Acoustic and articulatory phonetics are two perspectives of spoken sounds. Acoustic phonetics is a subfield of linguistics, but it is also a branch of physics, focusing on the acoustic and physical aspects of sound waves. Articulatory phonetics is the method through which human bodies are employed to produce spoken sounds [1]. Three processes are required to generate speech. To begin, an energy source is required. Anything that makes sound requires an energy source, and in the case of human speaking sounds, the energy source is the air coming from the lungs. The second mechanism is the sound source, which is the vocal cords of the larynx (or vocal folds). The articulators are the third mechanism, and they are used to filter or shape the sound. The oral cavity (mouth space), the nasal cavity (space within and behind the nose), the jaws, teeth, lips, and, most importantly, the tongues are all articulators [2].

Acoustic voiced sounds are produced when air from the lungs passes through the vocal cords, causing the vocal cords to vibrate and produce periodic signals. Due to the unique physical qualities of each human being's vocal cords, distinct frequencies are produced during voice production, which reflects the unique attributes of each human being's acoustics [3]. The epiglottis is the second component of the human speech production system. Each individual's epiglottis has a unique bulk and flexibility [4,5]. This also results in the addition of new elements to speech, regardless of the tong position or the mouth cavity, which both result in various pronunciations for the same words when spoken by different persons. The varied properties of the epiglottis also vary across genders, with female formants being more frequent than male formants and the spectrum of voiced sounds often decreasing in loudness with increasing frequency. All of these acoustic phenomena are a result of speech production. As a result, it is feasible to identify gender-specific elements in acoustic speech signals [6].

According to the source of creation, speech characteristics can be classified as acoustic or articulation features. If a human being is weary or in pain, his articulators will reflect his medical condition, and his sound will be created differently, since speaking requires a great deal of energy, and the first thing that will be impacted in his voice is his fatigue or discomfort [7], [8], [9]. The amount of verbal and gestural communication generated is influenced by the severity of acute pain. When pain intensity rises, so does spontaneous verbal communication [10]. Furthermore, research demonstrates that when pain is more acute, not only the use of nonverbal pain behaviors, but also the usage of co-speech gestures, rises in the realm of visible physical communication [11]. These gestures are rich in semantic content and offer crucial information about a sufferer's pain experience. In contrast to pain behaviors, demonstrating that when pain is more acute, people use multimodal communication resources to give greater information about their suffering. [12], [13], [14]. This is regarded evidence that any human health condition may be translated through his voice, but the most critical aspect is to select the most pertinent characteristics that will translate the sickness, infection, discomfort, or any other health issue.

Other factors that increase the challenge of health detection in general are the following:

Certain psychological situations among different individuals, under certain circumstances, can affect the patient's judgment of his health situation. Sickness can be faked.

To discern health automatically from vocal expressions, we must delete superfluous and dangerous information such as organ noise and human reactions that might result in abnormal voices.

This work has made several contributions, that overcome the limitations of the state-of-art researches, which are listed below:

Gender and speaker independency, a new features selection method enabled the system to function independently with respect to gender and speaker, and this is a huge contribution to health detection.

Accuracy compared to literature survey; high classification accuracy results are achieved through the recognition strength of the features extracted from the feature extraction method deployed. Notably, when applied to the medical speech, transcription, and intent dataset, the method achieves 94.55% classification accuracy, and this is the highest accuracy result gained compared to the state-of-art works.

Generality, the same system setup was used to detect health for all languages, genders and ages represented in the dataset, and no tweaking was done for each category of people. This demonstrates that the characteristics derived in this work are highly capable of detecting illness and that they function across languages, recording frequencies, genders, and ages and are speaker independent.

Age and language independent, all the results given in this work were gained by recognizing all limits of ages, all languages, and all genders in the datasets utilized without removing any category from the dataset or merging multiple categories into one group to avoid misclassification. This was a good challenge because all three datasets contained conflicting emotions, including happy and anger, neutral and calm, and disgust and surprise, which are difficult to distinguish.

The related works will be described in Section 2 of this article. The statistics and features of the benchmark datasets used in this research, as well as the frame-work for the proposed technique, will be discussed in depth in Section 3. Section 4 will present and debate the work's highest categorization findings. Section 5 will summarize the findings from this effort.

Related Work

Numerous studies have established the presence of distinct acoustic and physiological characteristics in human voices that can be utilized to identify patient symptoms, but they have not yet achieved the needed classification accuracy. Several scholars have investigated ADPSs using a variety of various methodologies and processes.

One of the processes upon which ADPS's rely is on clinical reports to identify the patient's highest expected condition [15]. Another mechanism by which healthcare ADPSs operate is through the question-and-answer approach, which relies on user text input (guided by users). The algorithm will then provide a list of the most likely illnesses [16], [17], [18], [19]. Numerous researchers have applied machine learning techniques to the diagnosis of cardiac disease [20]. Another technique is to utilize natural language processing and deep learning to extract text that may be used to identify certain symptoms for the purpose of illness prediction [21], [22], [23] and several additional studies. According to a 2014 study conducted by Maree Johson, nearly 20,000 state-of-art papers in the field of voice to text ADPS were published in 2014. These works utilized computational linguistics, natural language processing, human language technology, or text mining [24]. According to the literature study undertaken for this work, the suggested work belongs to the field of ADPS since it does not include linguistics, natural language processing, or speech conversion for the purpose of text mining. This paper developed an ADPS that estimates the degree of pain a patient is experiencing based just on his speech, without requiring any additional language processing. Numerous difficulties were encountered, one of which was the psychological state of the human person and its influence on voice. When a person is psychologically unstable, his vowels are stated differently than when he is stable [25,26]. The other difficulty was dealing with the similarity of children's voices between the ages of three and seven years; hence, children were excluded from the studies used to validate the approach described in this study, and since children rely on adults to speak out for them. Noise and distortion were a challenge in many of the recordings in the used dataset, as they will be in real life. The suggested effort has several limits that must be established, such as its interaction with youngsters and its lack of familiarity with other languages. We were unable to identify the voice signal for each condition in this work, but we were able to differentiate between diseases that have psychological effects and those that do not, as well as diagnose diseases based on the intensity of pain, strong and mild pain.

Dataset Selection

The dataset used in this study is version 1 of the medical voice, transcription, and intent dataset. The collection is comprised of spoken descriptions in (wav) audio format that detail the patient's medical problems. Each vocal description is accompanied with a transcription in the (csv) text format, and then each is tagged with a specific type of illness. This is a single-language dataset comprised of North American English slang. It enumerates 25 symptoms, all of which are back pain, acne, blurry vision, body feeling weakened, cough, ear ache, emotional pain, feeling cold, feeling dizzy, foot ache, hair falling out, difficulty breathing, head ache, heart hurts, infected wound, injury from spots, internal pain, joint pain, knee pain, muscle pain, neck pain, open wound, shoulder pain, skin issue, and stomach ache. Each symptom has a unique number of recordings, which required us to disregard some of the recordings for certain symptoms in order to get an equal number of recordings for each symptom, which is 225 records for each symptom. This work has a total of 5625 recordings [27]. We found a lot of distortion and noise in some of the wave recordings throughout our study, which we attribute to the low quality of the microphones used to record those samples and the loud setting in which they were recorded. This necessitated the use of a forward-backward digital filter, dubbed the zero-phase filter, in all recordings. The justification for adopting zero-phase filtering is because it retains filtered features exactly where they appeared in the unfiltered signal, which has no effect on classification performance. These filters conduct zero-phase digital filtering in both directions, forward and backward. After filtering the data in the forward direction, the filter sequence will be reversed, re-filtering the signal [28].

As we have addressed most of the main concepts of health detection systems, we will now explain our framework proposed in this research in detail.

Framework

The suggested technique is depicted in Fig. 1 using a block diagram. The dataset and the suggested method's major phases will be illustrated in depth in the following four sections.

Fig. 1

The block diagram of the method proposed in this work.

Preprocessing Phase

Numerous processes were applied during this stage. To begin, there is the noise reduction step. The noise reduction forward-backward filter was used to eliminate noise from wave files recorded online in order to account for the significant level of noise observed in several samples from the medical speech, transcription, and intent dataset utilized in this study. Second, the process of sampling and filtering. Each of the 25 illnesses included in the collection had at least 225 samples, and several had significantly more. Thus, we picked 225 samples for each illness and eliminated samples with excessive noise based on our listening to all samples, which was a lengthy procedure. Trimming was the third procedure. The majority of the recordings were made under various time constraints. As a result, all samples were shortened to three seconds. Segmentation was the fourth procedure. All samples were segmented at a ratio of (0.05%) to the original signal, and the overlap ratio used in this study was (0,025%), which results in a (50%) overlap, and the total number of segments created will be (2n-1), where n is the number of original segments.

Feature Extraction

A feature is a quantifiable quality derived from the substance under observation [29]. The most critical component of feature extraction is identifying the aspects that are most relevant to the issue statement. In this work, the smoothness, mel frequency cepstral coefficient (MFCC) with 12 degrees, and spectral centroid variability (SCV) features were extracted. These features are strongly related to human beings experiencing pain, and were chosen based on experiments conducted on 15 different types of speech features. The mean and standard deviation in multiple degrees were calculated for each feature, therefore, each feature generated 8 features, representing the (mean, and seven different degrees for the standard deviation. As a results, 112 features were generated from the feature extraction phase, all were utilized through experiment (8 smoothness features, 96 MFCC features, and 8 SCV features).

If smoothness is defined as the transaction of speech via air, then smooth speech results in a delayed transaction, whereas rough speech results in a faster transaction [30]. Smoothness was determined in two domains: the temporal domain and the spectral domain. Smoothness is determined using Eq. (1) and (2) [31]. Where var and var denote the temporal and spectral domain variances of the spectral feature, P denotes the feature's dimension, and N denotes the feature's time domain length

Feature Selection

The evolutionary feature selection method (genetic method) was used in this study was to filter feature groups before giving them to the neural Network (NN), in order to choose the best feature groups according to a given criterion, and then to pass the best selected feature group to the NN.

The idea of the genetic algorithm was to weight each feature group (each two features) with a fitness function, that shows the classification power of that certain group, and the weight will be used to combine the groups to form new ones. Weighted groups will be checked, and merged, until the criterion is fulfilled, at which point the best group will be handed to the NN. The neural network is used for classification or as an integrated feature selection technique, and it is also utilized in data mining and machine learning [32,33]. Three considerations motivated the employment of a filter ranking algorithm in this work. To begin, filter out irrelevant variables. Second, to take advantage of the variable selection criteria based on the order of the variable ranking approaches. Finally, and maybe most importantly, their simplicity and success as reported by online apps. Each variable is scored using a ranking criterion, and then a threshold is determined through experimentation and utilized to exclude variables that fall below that threshold [29].

Filter feature selection techniques are those that are applied before to classification; hence, ranking methods are considered filter methods. The fundamental premise behind feature selection methods is to pick unique features that convey important information about various classes in the dataset by leveraging a feature's basic attribute. This quality is referred to as feature relevance, and it quantifies a feature's ability to categorize distinct classes [29,34,35].

Neural Network

In this study, the support vector machine (SVM), backpropagation NN, and Gaussian mixture model (GMM) classifiers were used to predict the patient's discomfort. The structure of the neural network was two hidden neural network layers, the first layer with 80 hidden nodes, and the second layer with 40 hidden nodes. The input were 25 diseases and the output were 3 groups of diseases. The fitness function of the NN, was a NN that evaluated the classification accuracy of each sub group. Three kernel functions used with the SVM, were the polynomial kernel of degree, calculated through Eq. (3):

The radial basis function kernel, calculated through Eq. (4):and the sigmoid function, calculated through Eq. (5):

The radial kernel function achieved the highest accuracies.

Hardware and Software Platform

All tests were conducted in MATLAB R2019a on a computer equipped with an Intel Core i7-8750H Central Processing Unit (CPU) running at 2.2 GHz and 32 GB of Random Access Memory (RAM) (RAM). The MATLAB instructions are executed on the Windows 10 platform.

System Setting and Tuning

All of the findings described will be derived using the same system parameters and will not be changed for each sickness category contained in the dataset. The objective was to design a system capable of addressing a wide variety of languages and accents using the same parameters in order to achieve generalizability. The variables that were fixed during the system's setup were tweaked until the highest classification accuracy was achieved, but then they were fixed. Those most important variables are the following:

The length of the window dividing the signal specifies the length of the segment that will be extracted from the original signal in order to extract the desired characteristics. The window length was calculated through Eq. (6):

The length of the step sets the overlap between the current and previous windows in order to capture the majority of the possible divisions of the original signal. The step length was calculated through Eq. (7):

The standard deviation degree sets the number of degrees needs for the standard deviation to achieve the highest accuracy.

There are other variables that were fixed, but we have discussed the most important ones.

Experimental Results

Many experiments were conducted through our work, two of the most important experiments will be discussed that achieved the highest results.

Experiment number 1 (Exp1)

The purpose of this experiment was to anticipate the disease represented in each wave recording and to classify each recording according to one of the 25 diseases represented in the dataset. The confusion matrix of the best accuracy classification results obtained using the three classifiers SVM, NN, and GMM is shown in Figs. 2 , 3 and 4 . As shown in the three confusion matrices, the classification accuracies produced by the SVM, NN, and GMM were 50.8 %, 48.7 %, and 31.5 %, respectively.

Fig. 2

The Confusion Matrix of the best classification accuracy gained from predicting 25 diseases in Exp 1 using SVM.

Fig. 3

The Confusion Matrix of the best classification accuracy gained from predicting 25 diseases in Exp 1 using NN.

Fig. 4

The Confusion Matrix of the best classification accuracy gained from predicting 25 diseases in Exp 1 using GMM.

The three classifiers used produced poor results that are unacceptable in any ADPS. Through analyzing the results some conclusions were gained, such as:

The classification accuracy of each individual disease lye near the mean of the overall classification accuracy, which shows the difficulty of gaining high classification accuracy from this approach.

It has been noticed that some diseases have been misclassified with a certain group of diseases and not with all diseases which led to the idea of clustering the diseases into three different groups. For example, in Fig. 2, column (1), ear ache, foot ache, head ache, Infected wound, injury from spots, internal pain, joint pain, knee pain, muscle pain, neck pain, open wound, shoulder pain, and stomach ache wave samples have been misclassified with back pain by (9, 14, 11, 8, 7, 9, 11, 8, 9, 12, 23, 10, and 7) wave samples. Whereas, the all-other diseases were not misclassified classified with back pain in that high number of wave samples, the misclassifications with other diseases were (1, 1, 0, 0, 1, 1, 0, 1, 4, 2, and 1), which shows that a certain number of diseases share the same effect on speech, different than other diseases.

The highest classification accuracy gained from the SVM classifier was 70.2% with the heart hurts disease, which make sense, because any pain in the heart affects the breath of the human being, and as results, will affect his speech.

There are some characteristics that are shared among similar recordings of the same condition, as seen by the findings of Exp1. Regardless of the modest findings obtained, the 50.8 % classification accuracy indicates that there are some common characteristics among the recordings, but these characteristics need to be increased.

We discovered that illnesses are classified into three groups based on the misclassification distribution of samples in the confusion matrices. Each illness within the same group is categorized incorrectly as another disease within the same category. After analyzing the disorders, we discovered that they may be classified into three categories based on the level of discomfort they produce.

There are certain diseases that affect the human voice, primarily due to the amount of pain they cause. These diseases include (painful diseases, back pain, internal pain, joint pain, knee pain, muscle pain, neck pain, open wound, shoulder pain, stomach ache, injury from spots, infected wound, head ache, and ear ache. Foot discomfort) (which will be referred to as Group 1 in Exp2), and these disorders have an effect on the phonetics of articulation. There are several disorders that impact the acoustic phonetics of human speech, which may be classified into two categories. To begin, psychological ailments such as acne, blurred eyesight, physical weakness, hair loss, and skin problems (which will be called Group 2 in Exp2). Second, disorders associated with frequency such as (cough, emotional discomfort, feeling chilly, difficulty breathing, heart ache, and dizziness) (which will be called Group 3 in Exp2).

Following the conclusion of our trials, we observed that several disorders had been clustered together. Each category of illnesses is misclassified within itself and not in relation to other groupings. When we classified those diseases into categories, as indicated in Table 1 , we discovered that some behaviors were shared by diseases within the same category. Certain disorders are accompanied by pain, while others produce no discomfort but impair the human being's emotional state, and yet others have a direct effect on the vocal folds, affecting the frequency of speech. As a result, Experiment 2 was conducted.

Table 1

The groups of diseases generated after analyzing the results of Exp1.

No.	Acoustic phonetic feature related disease		Articulator phonetics feature related disease
	Frequency Related	Psychological Related	Painful diseases
1	cough	acne	back pain
2	emotional pain	blurry vision	internal pain
3	feeling cold	body feels weak	joint pain
4	heard to breath	hair falling out	knee pain
5	heart hurts	skin issue	muscle pain
6	feeling dizzy		neck pain
7			open wound
8			shoulder pain
9			stomach ache
10			injury from spots
11			infected wound
12			head ache
13			ear ache
14			Foot ache

Experiment number 2 (Exp2)

The purpose of this experiment is to classify illnesses into three categories. Where each set of disorders has a similar pattern of behavior or effect on the human body or mind and speech. Following the implementation of this experiment, some conclusions were reached, such as:

High classification accuracy was achieved using all three classifiers used in this study, although the SVM classifier achieved the highest classification accuracy, as seen by the confusion matrix in Fig. 5 .

Fig. 5

The Confusion Matrix of the best classification accuracy gained from predicting 3 groups of diseases using SVM.

Figs. 6 and 7 illustrate the classification results of the NN and GMM classifiers, respectively.

Fig. 6

The Confusion Matrix of the best classification accuracy gained from predicting 3 groups of diseases using NN.

Fig. 7

The Confusion Matrix of the best classification accuracy gained from predicting 3 groups of diseases using GMM.

Figs. 5, 6, and 7, still show that there is a partial misclassification between the three prevented groups of disease, and this can be justified by the poor representation of the illness in the wave recordings, or there might be some intersection in the features extracted from the three different groups of diseases, that caused this misclassification.

The least classification accuracy was gained through the GMM, through classifying group 2, that was misclassified with group 1 via 164 wave samples.

No uniform misclassification was found in Exp 2, all three groups are misclassified with each other. the only uniform found was, group 1, achieved the highest classification accuracy with respect to all three classifiers, 96.83%, 94.35%, and 94.92%, with SVM, NN, and GMM, respectively.

Group 2 have achieved the lowest classification accuracies with NN and GMM, it might be justified by its poor wave samples, but as long as the same groups have achieved the second place in classification with SVM, so that justification is not totally true, but the right justification is, the three group of diseases, still have common features, that are leading to this misclassification.

The limitation of the proposed work, was the time needed by the genetic algorithm to select the best group of features. Adding to this limitation, through experiment, it wasn't applicable to diagnose the 25 diseases represented in the dataset, therefore, it was a must to group identical diseases in three different groups. Those groups were fixed through try and error during experiment. Diseases that are related to acoustic phonetic feature were determined, and diseases related to articulator phonetics feature were also determined. Then the first was divided to frequency related diseases and psychological related diseases.

Conclusion

Diagnosing illnesses via speech is a difficult process that was previously unattainable, but with the advent of machine learning, everything is conceivable. Our objective was to diagnose illnesses directly from human speech using a machine learning approach, however we were unable to get the desired results, despite the 50.1 percent classification accuracy achieved using the SVM classifier, but on the other hand, high accuracy result of (94.55%) was gained when divided the diseases into three related groups.

This investigation revealed that illnesses may be classified into several categories, including severe and light discomfort, psychological or physical, emotional or non-emotional, and others. Each of the above groups exhibits similar behavior when it comes to communication. This reflection leaves distinct characteristics associated with each form of sickness.

The advantage of the proposed work is the capability of illness diagnoses through speech, which is considered a new approach in this field, and is extremely required in certain applications were speech is the only source of information available to diagnose the illness of a patient.

Through experiment, the proposed work faced a couple of limitation. First, the time, the needed by the genetic algorithm was high. Second, the failure of the proposed work to diagnose each of the 25 diseases represented in the dataset utilized, which means the failure is extracting features that can diagnose each of the 25 diseases separately.

Future Work

The findings of Exp1 can be increased by using more feature types or machine learning methods. Although the genetic technique utilized in Exp2 produced positive findings, it was a time-consuming procedure. Thus, developing a more efficient feature selection approach can accelerate the ADPS.

As the challenges and the extreme number of random factors of this field have been discussed in the introduction section, the response service methodology (RSM) program can be used to predict the perfect setting to the system proposed, by suggesting the experiments with the highest probability of achieving the best classification results.

Searching for features that are more disease related, that can diagnose each disease separately, will be more applicable, because when patients use ADPS's, they usually are looking for exclusive answers, not broad answers.

The input of this work is only speech, but adding other data such as human hand or arm movement as an input, and using both types of data as combination, and predicting the disease accordingly, might increase the classification accuracy, since human bodies translate pain through moving parts of it, such as moving their arms, hand, shoulders, toughing their foreheads or legs.

Declaration of Competing Interest

None.