Development of Decision Support System to Predict Neurofeedback Response in ADHD: an Artificial Neural Network Approach.

INTRODUCTION: Clinical decision support system (CDSS) is an analytical tool that converts raw data into useful information to help clinicians make better decisions for patients. AIM: The purpose of this study was to investigate the efficacy of neurofeedback (NF), in Attention Deficit Hyperactivity Disorder (ADHD) by the development of CDSS based on artificial neural network (ANN).
METHODS: This study analyzed 122 patients with ADHD who underwent NF in the Parand-Human Potential Empowerment Institute in Tehran. The patients were divided into two groups according to the effects of NF: effective and non-effective groups. The patients' record information was mined by data mining techniques to identify effective features. Based on unsaturated condition of data and imbalanced classes between the patient groups (patients with successful NF response and those without it), the SMOTE technique was applied on dataset. Using MATLAB 2014a, a modular program was designed to test both multiple architectures of neural networks and their performance. Selected architecture of the neural networks was then applied in the procedure.
RESULTS: Eleven features from 28 features of the initial dataset were selected as effective features. Using the SMOTE technique, number of the samples rose to around 300 samples. Based on the multiple neural networks architecture testing, a network by 11-20-16-2 neurons was selected (specify>00.91%, sensivity=100%) and applied in the software.
CONCLUSION: The ANN used in this study has led to good results in sensivity, specificity, and AUC. The ANN and other intelligent techniques can be used as supportive tools for decision making by healthcare providers.

INTRODUCTION

Attention deficit/hyperactivity disorder (ADHD) is a chronic disorder characterized by resistant neurotic symptoms associated with (or without) hyperactivity and impulsivity. The prevalence of this disorder has been reported between 2% and 29% in different countries (1, 2). Without effective treatment, children and adolescents with ADHD are at risk of behavioral and educational problems (3), mood and anxiety disorders (4), physical injuries and harms (5), and drug abuse disorders (6).

There is no definitive treatment for ADHD, but there are treatments that can improve the symptoms. In this regard, two types of treatments are most often considered which include drug treatment and non-drug treatment. One type of non-drug treatments is neurofeedback (7). Neurofeedback (NF), formerly called electroencephalography (EEG) biofeedback, and occasionally referred to as neurotherapy, is an intervention for ADHD based on data showing that many individuals with ADHD have more slow-wave (especially theta) power in their EEG than those without ADHD, and conversely, less beta power (8).

Before and during this type of treatment, attentional and behavioral variables were evaluated through the Integrated Visual and Auditory Continuous Performance Test (IVA/CPT or IVA + PLUS). This tool evaluates attention and response control to auditory and visual stimuli. This test can be administered to children (ages 6 and older), adolescents, and adults (9). Also, this test produces information on the condition of the disease and also, this information can be used to determine the effectiveness of treatment (10, 11).

Although NF is an effective intervention to reduce ADHD symptoms (12), but it imposes high cost to the society. The patients are usually treated for 25 to 50 sessions over the months. The cost of these sessions ranges from several hundred to several thousand dollars and health insurances don’t cover this treatment (13, 14). For example, the annual cost of this disorder in the United States is estimated at between $ 36 billion and $ 52 billion. This is because of the economic cost and the duration of treatment. Parents and other people involved in the treatment process should be more careful about choosing NF (3, 15).

On the other side, according to studies, clinicians using intelligent systems such as clinical decision support system (CDSS) can achieve significant achievements such as reducing medical and medication errors, compliance with standard guidelines for treatment and prescribing, reducing costs, and ultimately improving the quality of health care provided (16).

AIM

In this study, for optimizing the use of NF in ADHD, a neural network (NN) based model was proposed to predict the effectiveness of the NF for the patients with ADHD.

METHODS

This developmental research has been done in 6 steps as follow:

The data were collected from patients with ADHD who were referred to the Parand-Human Potential Empowerment Institute in Tehran. The study population was all patients with IVA + PLUS tests at their first visit to the hospital and their information was stored in an electronic format.

One hundred and twenty-two patients were participated in this study. Participants were divided into two groups, with 75 in the positive response to NF group, and 47 in a group in which the NF was not effective. The diagnosis of the efficacy of the treatment was the responsibility of the Parand Institute’s psychologists.

Of the 122 electronic medical records (EMR), 10 EMRs were incomplete and excluded from the dataset. Finally, the dataset with 112 EMRs containing 28 items were selected for data mining. But, given the dimensions of the data collected in this study, this dataset was not considered as saturated information.

The information was evaluated using IBM SPSS Modeler 14.2 and the important attributes were identified. In this data mining software, choosing important attributes, reducing the dimensions of the data, and considering the overall volume of information led to better classification of the performance because it displays information in a more efficient and optimized dimension. At the end, a model is created for selecting the effective attributes to determine the output of the dataset.

The following relation is required for each dataset in any data mining analysis associated with valid and accurate classifications. This relationship indicates that the number of samples must be at least 30 times the number of attributes for proper analysis in the subject matter of the class. The output of the previous step was to determine the most important attributes of the 28 items. Because of failure to apply the relation1 in the dataset, a method for normalizing the data dimensions was required. So, the number of samples increased to about 300.

Samples Number >= 30 * attributes number

Relation 1: The ratio of the number of samples to the number of attributes.

This method was faced with two problems: data insufficiency and imbalanced dataset. A dataset is imbalanced if the classes are not approximately equally represented; here, two-thirds of the data was in the class +1 (patients treated with NF) and the rest was in class -1 (Patients who did not benefit from the NF). Synthetic Minority Over-Sampling Technique (SMOTE) was applied to solve these problems. SMOTE is an approach to the construction of classifiers from imbalanced datasets (2). SMOTE was performed by using Data Mining with R (DMwR) library in the R programming package to create balance in dataset and increase the number of samples.

ANN parameters have a great influence on the training and accuracy of the network. Back propagation algorithm is the best method for training multi-layer perceptron networks. In this study, Levenberg-Marquardt was used, which is characterized with high computing speeds.

Also, the “tansig” function was used as a function of NN activation. This function is equivalent to the hyperbolic tangent function (which is the activator functions for the construction of all kinds of NNs), and also this function can be quickly run in MATLAB, although the result may be slightly different.

This function creates a good balance in networks in which the run speed is very important but, the exact form of the activation function is not important (17). The number of training generations was set at 1000 and the limit of training time was infinite. Because of the application of the Levenberg-Marquardt learning function due to the high speed of its run, the use of this ultimate time is not a problem.

In order to achieve maximum accuracy, the data are evaluated by the principal component analysis (PCA). PCA is a multivariate technique that analyzes a data table in which observations are described by several inter-correlated quantitative dependent variables (9). Using this technique, in addition to overcoming the curvature of the dimension implicitly, increases the accuracy of the NN in data analysis. The main advantage of this technique is to reduce data dimensions and compression of information without data missing (18).

“MATLAB 2014-a” was used to design a program for patient information processing. This program was applied to find the best NN architecture. The modular programming capability of the MATLAB was used to simplify the design process and the program was split into a set of functions (Figure 1). The purpose of this program was to test the different modes of network architecture with the aim of discovering the best available architecture to predict the usefulness of NT based on data of IVA + PLUS test at the first session (Figure 2).

Figure 1.

The model used to select the effective attributes

Figure 2.

Dependencies of program functions in different network modes

Thus, within the two nested loops, the number of neurons in each of the network layers was allocated and the new network is trained according to the selected architecture. After training the network with 70% of the data, 30% of the remaining data was used in two groups of validation (10%) and testing (20%). The purpose of the network test was to investigate the predictive power of the generated NN based on criteria such as sensitivity, specificity, accuracy, and the receiver operating characteristic (ROC) space. Finally, the program was designed to automatically create all modes (different neuron combinations in the first and second hidden layers) and report the results of their testing.

RESULTS

The output of selecting important attributes was the eleven important and effective attributes as provided in Figure 3.

Output from the testing of various structures of the NN is given through a table shown in Figure 4. This table contains a variety of architectures (various combinations of the first and second hidden layers with a variety of neurons) and criteria.

Figure 4.

Different structures of the neural network

The area under the curve (AUC) is an accepted traditional performance metric for a ROC curve. Based on the AUC, as shown in Figure 5, a model with a structure of 11-20-16-2 was selected. In this model there are 20 neurons in the first hidden layer and 16 neurons in the second hidden layer.

Figure 5.

The network with the optimal structure

The reduction of network prediction error, by increasing the number of generations, is also shown in Figure 6.

Figure 6.

Error-Correction learning in NN

Figure 7 showed confusion matrix of the network in relation to a variety of training, validation, experimental, and total data. The performance of machine learning algorithms is typically evaluated by a confusion matrix.

Figure 7.

Confusion matrix of the network

Figure 8 showed a ROC curve of the NN with the selected structure.

Figure 8.

ROC curve for preferred structure

Based on the selected architecture, the CDSS was designed as shown in Figure 9 in which the user, by entering information extracted from the IVA + PLUS test, could obtain the result of a network analysis about the usefulness of the NF for that particular patient. .

Figure 9.

A view of CDSS based on ANN for predicting the effects of NF for patients with ADHD

DISCUSSION

The results of the study showed that the attribute value was greater than -0.91% and the sensitivity was equal to 100% indicating a good performance of the system. Focusing on important factors when presenting NF and targeting the choice of type of therapy (games or films used) is one of the expected applications of this system. This intelligent system can be used to support psychologists to make right decision for patients by gathering the information extracted from IVA + PLUS test at the first visit. Choosing the right treatment path using games or films tailored to selected factors can also result in better treatment outcomes. E-games are effective in disease prevention, diagnosis, treatment, and promotion of awareness about risk factors (19).

Artificial neural network has been used in healthcare fields. For example, the aim of Wang and et al. was to explore relevant factors of hemorrhagic contusions following decompressive craniotomy (DC) in traumatic brain injury (TBI), and create an artificial neural network (ANN) prediction model of the risk factors of hemorrhagic contusions. They concluded that the ANN prediction model has a high accuracy to forecast haemorrhage (20). In a study, Skoch and et al. applied ANN modelling to a consecutive cohort of pediatric aneurysmal subarachnoid haemorrhage cases to assess its ability to predict symptomatic cerebral vasospasm. This ANN model was able to accurately predict all 16 outcomes (21). Ramachandran et al. applied the neural network in diabetic retinal screening and revealed that deep neural networks can be integrated into community screening to successfully detect both diabetic retinopathy and diabetic macular edema (22).

Based on a study, CDSSs can improve clinical practice and patient outcomes in five application areas including disease process management, care and treatment, drug prescription, evaluation and prevention by performing a series of functions. The significant effects of these systems are improving the quality of care and increasing patient safety, increasing the cost-effectiveness, and promoting knowledge of clinicians through the accessibility of resources and useful information to optimize decision making (16).

Several researchers, separately, conducted a systematic review on chronic and acute disease management, practitioner performance and patient outcomes and concluded that the use of CDSS improves the care process, but has no effect on patient outcome (23-25). Nieuwlaat et al. and Hemens et al. argued that the use of CDSS improves drug prescribing, monitoring, and management processes but its effect on patient outcome is not clear (26, 27). Souza et al. in their study expressed that CDSS effects on patient outcomes, safety, costs of care, and provider satisfaction remain poorly supported (28).

The outcome of the system is related to the user interface for proper interactions between patients and clinicians directly (29, 30). Kawamoto et al. suggested that CDSS significantly improved clinical practice if some features are considered such as providing alerts, reminders, recommendations, and periodic performance feedback. (31). Roshanov et al. believed that to develop an effective CDSS, the factors such as system design, user interface, local context, implementation strategy, and evaluation of its impact on user satisfaction and workflow, costs, and unintended consequences should be taken into account (32).

Therefore, in the design of the system, one should consider the components which are enhancing the performance of the system. For example, patient registry, patient encounter scheduler, trial management, clinical decision support, progress note generator, workload and outcomes report generator and translation of written guidelines into actionable, and real-time clinical recommendations are the most important (33). However, since the implementation of such systems is expensive, factors such as organizational commitment and attention (34), extensive commitment of personnel (35), and the clinician team working as the main users of the system can all have a significant impact on the performance of these systems (36).

Also, the users should be aware of this fact that these systems can be used as tools for saving clinician time in order to divert their attention to the main issues and facilitate their access to references and educational materials such as online information (37, 38).

CONCLUSION

This CDSS can support psychologists to make right decision for patients based on the information extracted from IVA + PLUS test at the first visit. Certainly, align with progression in technology, comparative study and clinical feedbacks will lead to further improvement in this field.