Prediction of the performance of artificial neural networks in mapping sEMG to finger joint angles via signal pre-investigation techniques.

The inputs to the outputs of nonlinear systems can be modeled using machine and deep learning approaches, among which artificial neural networks (ANNs) are a promising option. However, noisy signals affect ANN modeling negatively; hence, it is important to investigate these signals prior to the modeling. Herein, two customized and simple approaches, visual inspection and absolute correlation, are proposed to examine the relationship between the inputs and outputs of a nonlinear system. The system under consideration uses biosignals from surface electromyography as inputs and human finger joint angles as outputs, acquired from eight intact participants performing movements and grasping tasks in dynamic conditions. Furthermore, the results of these approaches are tested using the standard mutual information measure. Hence, the system dimensionality is reduced, and the ANN learning (convergence) is accelerated, where the most informative inputs are selected for the next phase. Subsequently, four ANN types, i.e., feedforward, cascade-forward, radial basis function, and generalized regression ANNs, are used to perform the modeling. Finally, the performance of the ANNs is compared with findings from the signal analysis. Results indicate a high level of consistency among all the aforementioned signal pre-analysis techniques from one side, and they also indicate that these techniques match the ANN performances from the other side. As an example, for a certain movement set, the ANN models resulted in the rotation estimation accuracy of the joints in the following descending order: carpometacarpal, metacarpophalangeal, proximal interphalangeal, and distal interphalangeal. This information has been indicated in the signal pre-analysis step. Therefore, this step is crucial in input-output variable selections prior to machine-/deep-learning-based modeling approaches.

Introduction

Electromyography (EMG) is a technique used to record and analyze myoelectrical signals, called electromyograms, generated by skeletal muscles. An EMG signal detected noninvasively on the skin surface is called surface EMG (sEMG). sEMG exhibits stochastic and nonstationary features [1]. These biosignals represent a muscle activity during a movement and reflects a person's intention when he/she moves his/her limbs. In fact, sEMG is nonlinearly related to joint kinematics (angle, velocity, and acceleration), joint moments, and muscle forces. These signals have been investigated for many purposes, including the diagnosis of neuromusculoskeletal diseases, movement analysis, rehabilitation evaluation, and recently, position- and force-based control of human–machine interfaces, such as exoskeletons and prostheses.

Many approaches can be used for mapping between sEMG and joint kinematics and kinetics, such as i) integrated biomechanics and multibody dynamics [2] and ii) discrete classification of a person's intention (motion or force) based on sEMG activity using the signal analysis and machine learning (ML) methods [3, 4, 5, 6, 7, 8, 9, 10]. However, these methods have drawbacks that limit their usability. Alternatively, joint variables can be reconstructed from sEMG via continuous and proportional ML-based estimation approaches [11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23], including artificial neural networks (ANNs), as many of their variables are known as universal approximators [24, 25, 26]. However, training an ANN to model multi-input multi-output systems requires sufficient computational resources and is time consuming; additionally, ANNs are sensitive to noisy signals. Hence, studying the dependencies between the inputs and outputs of such complex biosystems is important and facilitates appropriate input–output selections prior to modeling; hence, optimal solutions with minimal resources and time can be achieved.

The selection of the most activated muscles contributing to limb motions (in terms of sEMG signals and their representing feature/feature sets) that reduces redundancies can be achieved by different methods, which can be classified into the following: i) dimensionality reduction (e.g., principal component analysis (PCA) [27, 28] and linear discriminant analysis [4]), ii) variables selection (e.g., mutual information (MI) [12], forward and backward selection [29]), and iii) hybrid approaches (e.g., integrated PCA and MI [5]). Some of these approaches suffer from input–output linear dependencies (e.g., PCA), long-time implementations as they are model-based selections (as in forward and backward selections), and failure to obtain optimal input variable sets (as in the forward selection). However, although these methods are powerful, simpler and customized approaches are preferred in the current study owing to multiple participants involved and many modeling algorithms used with an intensive parametric study applied to each one.

This study, which proposes two simple visual and numerical techniques, has three main objectives: 1) to investigate the provided dataset quality (as it is assumed to be a worldwide benchmark); 2) to investigate the dependencies between sEMG channels or inputs and kinematics degree-of-freedom (DOF) readings or outputs, such that the appropriate input variables can be selected, thereby reducing the ANN learning times; and 3) to estimate the performance of four types of ANNs prior to the training phase, in proportional and continuous mappings of sEMG to hand joint angles using those simple, low-computation techniques. Hence, data of eight individuals performing two sets of movements were obtained from a publicly available dataset [9].

The remainder of the paper is organized as follows: Section 2 presents dataset acquisition and processing. Section 3 explains signal pre-investigation techniques and the ANN models used. Section 4 provides results of the data pre-investigation phase and ANN modeling. Results discussion and study limitations are presented in Section 5. Finally, the conclusions and future studies are presented in Section 6.

Material and methods

Dataset description

The dataset used in this project was obtained from the second version of the Non-Invasive Adaptive Prosthetics (Ninapro) dataset. It is publicly available and provides researchers (especially of the biorobotics field) with a benchmark, where different studies can be performed and compared among the same datasets to enhance upper-limb myoelectric control approaches [9, 10]. Owing to long training times required to achieve the optimal performance of feedforward and cascade-forward ANNs, extensive parametric studies on each of the four ANN types, and computing resource limitations, eight intact participants were randomly selected from this dataset in the current study (six males; two females; eight right handed; age 26.75 ± 2.76 years). This number is within the average population of many other related studies [12, 13, 14, 18, 19, 20, 23, 28]. sEMG signals of the arm were recorded via 12 Trigno wireless double-differential electrodes from Delsys, Inc. (www.delsys.com). Two approaches were adopted for electrode placement (the electrode placement and acquisition protocol are shown in [9]). The first is dense sampling, where eight of the electrodes are equally spaced around the forearm beneath the elbow at a fixed distance from the radio-humeral joint. The second is precise anatomical positioning, where four electrodes are placed on the main activity spots of the flexor digitorum superficialis, extensor digitorum superficialis, biceps brachii, and triceps brachii. Synchronously, hand kinematics were measured using the CyberGlove II dataglove (www.cyberglovesystems.com), which comprises of 22 sensors that provide readings proportional to the joint angles of the hand (fingers, palm, and wrist). It is noteworthy that the glove requires special calibration procedures to accurately represent the angles, which was not performed by the dataset team. The sampling rate of the sEMG and kinematics data were 2 kHz and about 25 Hz, respectively. Each participant was asked to sit on an adjustable chair with his/her arm comfortably resting on a desk. A laptop placed in front of the participants acted as a visual stimulus, where they were instructed to imitate the shown postural, movement, or force pattern movie with their right hands. Each movement was repeated six times serially, interrupted by a rest posture. From all the movement sets performed, Table 1 shows nine wrist and nine grasping movements required in implementing typical activities of daily living that are considered in the current study. Readers may refer to [9] for more details.

Table 1

The selected movement sets considered in this study. Readers may refer to [9] for more details.

Wrist movements
Wrist supination about the middle finger	Wrist pronation about the little finger	Wrist radial deviation
Wrist pronation about the middle finger	Wrist flexion	Wrist ulnar deviation
Wrist supination about the little finger	Wrist extension	Wrist extension with closed hand
Grasping types
Power sphere grasp	Tripod grasp	Quadpod grasp
Three-finger sphere grasp	Prismatic pinch grasp	Lateral grasp
Precision sphere grasp	Tip pinch grasp	Parallel extension grasp

Digital signal processing (DSP)

Data processing

In beginning of the DSP phase, the occupied bandwidths of the signals were obtained to determine the appropriate cutoff frequencies of the digital filters by finding the power spectral density (PSD) and the single-sided amplitude spectrum (AS) of each data type (Figure 1). It is well known that kinematics data have much lower frequency components than sEMG. It is typical to smooth sEMG and kinematics data with band pass (BP) and low pass (LP) filters, respectively. In such input–output mapping problems (i.e., estimation or regression), it is important that the original input and output data are synchronized after the filtration process. Because a delay associated with the filtration exists, a filter with a 0-phase was applied to each data type by filtering it forward and backward to eliminate the delay. This delay is specifically inherent in infinite impulse response (IIR) filters, which is the type employed in this study, and it depends on the frequency [31]. Generally, IIR filters required a much lower-order parameter, resulting in much smaller delays compared to finite impulse response filters to satisfy a set of design constraints. According to these observations and plots, referring to the well-known frequency characteristics of these signals and to be consistent with related sEMG-based kinematics prediction studies [32,33], the digital filters were designed as summarized in Table 2. The Butterworth filter is one of the most frequently used IIR filters, especially for sEMG problems. Its features include the following: flat passband and stopband regions, monotonicity in the entire region, no ripples, a nonlinear phase, and a small rolloff steepness [31]. Data processing was implemented via MATLAB's Signal Processing Toolbox, and the following data investigation techniques were performed via the MATLAB package.

Figure 1

(a) PSD and (b) single-sided AS of sEMG channel #1 for repetition #2 of the wrist flexion movement for a participant.

Table 2

Characteristics of the filters applied to raw data. f: cutoff frequency. BP/LP: Band pass/low pass. IIR: infinite impulse response.

Data type	Pass type	fco (Hz)	Design category	Design method	Phase	Order
sEMG	BP	[10 400]	IIR	Butterworth	0	4th
Dataglove	LP	10	IIR	Butterworth	0	4th

From preliminary investigations of the dataglove readings, it was discovered that these data were extremely noisy and the range of motion (ROM) varied significantly from one DOF to another. This was expected as they were never calibrated. Owing to the specific features of sEMG signals, it is important to normalize them before further analysis. sEMG signals can be normalized by different methods, from which maximum voluntary contraction techniques are standard methods used for this purpose and typically performed at the time of data acquisition [34]. However, this step was not performed for the Ninapro dataset. Alternatively, one of the normalization methods frequently used in studying EMG-kinematics/kinetics relations is the zero-score (z-score, aka standard score) (e.g., [32, 35, 36]), where new signals have zero mean and unit standard deviation. It can be computed using the following formula:where is the z-scored value of data point , is the sample mean, and is the sample standard deviation. This standardization was applied for both the sEMG and the kinematics readings.

Feature extraction

Owing to its characteristics, sEMG does not provide much valuable information to represent muscle activities. Therefore, a suitable feature or feature set is extracted from these signals to represent informative observations from sEMG more efficiently. Additionally, feature extraction reduces the number of samples used in modeling, resulting in less computation time with improved performance.

First, most of the sEMG features used previously, from the domains of time, frequency, and time–frequency, were investigated in terms of their representations of muscle activity and computation complexity. Owing to its popularity in utilization (e.g., [12, 13, 32, 33, 37]), computation simplicity, and suitability for real-time applications, as well as after preliminary studies on ANN modeling with different features and feature sets, the root-mean squared (RMS), as shown in Eq. (2), was selected to represent the sEMG feature:where is the sEMG value of the muscle at the sample, and is the total number of samples in the processing window (length). Similarly, the mean value (mean) within the processing window was selected to represent the dataglove readings (Figure 2). A sliding window of length ms with a 50% overlap was selected. These values are consistent with those in related studies ([32, 33, 38]). It is noteworthy that obtaining the best feature(s) from different domains is out of the scope of the current study.

Figure 2

Visualization of features extracted during the wrist flexion movement for a participant: (a) and (b) represent the RMS of the sEMG channel #1 and the mean of I-MCP, respectively. The values shown were filtered and z-scored.

Data pre-investigation and ANN modeling

When investigating filtered, z-scored dataglove readings, in many cases, some problems in their quality are unexpectedly exhibited. As an example, the low repeatability and high jitteriness of the dataglove values are shown in Figure 3. This motivated us to investigate those processed data carefully before the modeling. It is noteworthy that to reduce the computational time required in ANN modeling, 15 DOFs (as shown in Table 3) was included, whereas the others were excluded because those 15 DOFs contributed to the majority of hand motions. The system considered in this study has 12 sEMG inputs and 15 DOF outputs. It is well known that ANNs are sensitive to noisy signals. Hence, it is important to exclude the noisiest signals, which consequently reduces the system size, reduces the ANN training times, and accelerates its convergence. The goal of the studies described in Subsections 3.1 and 3.2 are to propose simple techniques to i) obtain the most activated muscles via sEMG readings, thereby contributing to the considered motions; and ii) investigate the quality of the selected DOF. In other words, it was hypothesized that the aforementioned techniques could predict the ANN modeling accuracy early.

Figure 3

Low repeatability and high jitteriness in dataglove readings during the six repetitions of (a) and (b) wrist flexion movement; (c) and (d) prismatic pinch grasp. The data of (a)/(c) and (b)/(d) represent T-IP and M-MCP, respectively.

Table 3

DOF considered for each finger. CMC: Carpometacarpal, MCP: Metacarpophalangeal, PIP: Proximal Interphalangeal, DIP: Distal Interphalangeal, and IP: Interphalangeal.

Thumb (T)	Index (I)	Middle (M)	Ring (R)	little (L)
CMC	MCP	MCP	MCP	MCP
MCP	PIP	PIP	PIP	PIP
IP	DIP	DIP	DIP	DIP

Visual inspection

From the experimental protocol, it is assumed that the motion of each joint during each task is repeated six times, interrupted with the rest posture. Therefore, the behaviors of sEMG channel(s) and DOF during a single movement and the full set of movements are visualized in stacked plots herein. For example, all sEMG channels with a single DOF for a single movement are plotted together and shown in Figure 4, whereas Figure 5 presents a single sEMG channel plotted throughout an entire movement set (wrist or grasp). This type of figures indicates muscles that are more active (contributing) during the movements. Meanwhile, Figures 6 and 7 show the behavior of a certain DOF during all considered tasks. These figures capture the quality of a DOF during an entire movement set. It should be noted that, in such figures, six distinguished repetitions should appear during a movement for almost all the sEMG and DOF readings. Otherwise, the readings can be considered of low quality. Figures 4, 5, 6, and 7 present the z-scored, filtered reading for both the inputs (sEMG) and outputs (hand DOF).

Figure 4

Behavior of all sEMG channels and a DOF through (a) wrist pronation about the middle and (b) three-finger sphere grasp movements; y-axes represent the RMS for sEMG and the mean for dataglove readings. ch: channel.

Figure 5

Behavior of sEMG channel #12 through (a) the wrist movement set for participant #2 and (b) the grasp movement set for participant #8. y-axis represents RMS of the sEMG signals. mov: movement.

Figure 6

Behavior of I-PIP through (a) wrist and (b) grasp movement sets. y-axis represents the mean of the I-PIP readings.

Figure 7

Behavior of I-DIP through (a) wrist and (b) grasp movement sets. y-axis represents the mean of the I-DIP readings.

Numerical inspection

Although the visual inspection of the data can indicate their quality and relations in some cases, it can be misleading in others as distinct criteria do not exist. Therefore, a numerical measure is necessitated to perform this objective. One such measure, absolute correlation (AC), is proposed and compared with the MI.

Absolute correlation

In this study, the Pearson correlation coefficient (R), defined by Eq. (3) [39], was computed to examine the association between i) an sEMG channel and all DOFs, and ii) a DOF and all sEMG channels.where is the Pearson correlation coefficient between random variables and which has a range of [-1 +1]; is the covariance between X and Y; and are the standard deviations of and , respectively.

In the first case, the absolute value of R between each sEMG channel (in terms of RMS value) and all DOFs (mean value) was obtained separately for each participant and each movement. Subsequently, the average values were computed among the wrist, grasp, and both movement sets over all participants, as shown in Figure 8 (a). This indicates the contribution of each sEMG channel in the rotations of all DOFs. By contrast, Figure 8 (b) presents the absolute value of R between a single DOF and all sEMG channels, which facilitates in distinguishing the DOF that has the highest relation to the overall sEMG channels.

Figure 8

Numerical inspection over wrist (W), grasp (G), and both sets: (a)/(c) Mean AC/MI between RMS value of an sEMG channel and all DOF; and (b)/(d) Mean AC/MI between a DOF and all sEMG channels. The values were averaged among all participants.

MI

The methodology presented in Subsection 3.2.1. primarily assumes a linear dependency between the inputs (sEMG) and outputs (kinematics), which is not the case and may result in inaccurate inferences. Alternatively, information theory (Shannon entropy [40])-based measures, such as MI, can quantify nonlinear or chaotic dependencies. It can be calculated based on the following equation [41]:where is the MI between input variable and output variable ; is the joint probability density function between and ; and and are the marginal probability density functions of and , respectively. The MI was calculated for the two cases described in Subsection 3.2.1, as shown in Figures. 8 (c) and (d).

ANN modeling

An ANN is a circuit containing highly interconnected processing elements (neurons) through connections of certain strengths (weights) with certain arrangements (architecture), which can learn from its inputs and outputs to perform a certain function. These neurons operate in parallel. The network's architecture, neurons, and learning algorithm specify the functionality of the network [42]. In the current study, four ANN types used in continuous input–output mapping were trained/designed and tested to proportionally estimate the hand kinematics (outputs) from sEMG (inputs), i.e., feedforward [24], cascade-forward [43], radial basis function [25], and generalized regression [26] ANNs. For the feedforward and cascade-forward ANNs, single and two hidden layers were used with hyperbolic tangent sigmoid neurons, whereas the output layer was linear. Their training parameters (weights, and biases) were adjusted using the Levenberg–Marquardt-based back propagation algorithm. The number of hidden neurons ranged from 5 to 50. The radial basis function ANN comprises a radial basis hidden layer with Gaussian neurons and a linear output layer. It can be trained by the orthogonal least squares, as described by [44]. The optimal number of Gaussian neurons and the spread, which relates to the hidden neuron bias, were the parameters investigated in this study. Similarly, the generalized regression ANN has a radial basis hidden layer and a normalized linear output layer; however, it can be designed by only a one-pass iteration depending on the input/target values. It was subjected to a spread parametric study to obtain the optimal performance. For all ANN types, the accuracy measure was based on the R values, as shown in Eq. (3), among the test data subset. The details of the ANN modeling and testing are presented in [45]. MATLAB's Neural Network Toolbox was used to conduct the ANN modeling and evaluation.

Results

Data visualization can facilitate in determining the dataset quality and some input–output relations. It is noteworthy that the plots shown in Figures 4, 5, 6, and 7 present the z-scored, filtered readings for both the sEMG and DOF. As shown in Figure 4, as the sEMG values changed, the DOF generally changed either proportionally or inversely, i.e., concurrent relation. The exceptions were clear for sEMG channels 11 and 12 in many cases, where the change was much lower. This was observed in plots such as those presented in Figure 5, where the behavior of each sEMG channel was tracked throughout all movements. In addition, plots such as those in Figure 5 indicate fair to good distinguished repetitions of most sEMG channels and poor repetitions for channels 11 and 12 during the motions. However, for most cases, the kinematics readings suffered from jitteriness and low repeatability, as shown in Figures 6 and 7, where the pattern of six distinct repetitions were assumed to appear for each wrist motion or grasp type, which did not occur for many DOFs. Furthermore, the repeatability of the DIP joints was the worst among all joints, and it was better in the wrist tasks than in the grasping ones for all DOFs. It is noteworthy that these visual investigation techniques can be misleading in some cases as no distinct and numerical threshold exists to evaluate the signal quality, which necessitates more appropriate numerical measures.

Figure 8 summarizes the average results over all participants and each movement set of the numerical-based approaches: i) AC, as shown in plots (a) and (b); and ii) MI, plots (c) and (d). The following are reflected from these plots:

Figures. 8 (a) and (c) show that the mean AC and the MI values, between an RMS value of a sEMG channel and all DOFs, are minimum for channels 11 and 12. This indicates that the muscles under channels 11 and 12, biceps brachii and triceps brachii, were less activated than the others during the hand motions. Hence, the contributions of sEMG channels 11 and 12 in the rotations of all DOFs were significantly less than those of the others.

The patterns of AC and MI between a DOF and all sEMG channels, as shown in Figures. 8 (b) and (d), are almost consistent. For example, the values of AC between a DOF and all sEMG signals, for the wrist set, were ordered in a descending manner as follows: PIP, CMC, MCP, and DIP; and for the MI, they were CMC, PIP, MCP, and DIP. For more clarity, Table 4 introduces the descending patterns of the AC and MI (and that of the ANN which will be explained later) for the wrist and grasp tasks.

Table 4

Descending accuracy patterns of results achieved using numerical inspection techniques, AC and MI, calculated between a DOF and all sEMG channels, and the ANN (RNN). Order 1 means the highest accuracy and 4 means the lowest. The T's IP joint was averaged with the DIP joints as it was similar to the DIP joints in the fingers.

Measure	Order
	1	2	3	4
Wrist set
AC	PIP	CMC	MCP	DIP
MI	CMC	PIP	MCP	DIP
RNN	PIP	CMC	MCP	DIP
Grasp set
AC	CMC	MCP	PIP	DIP
MI	CMC	MCP	DIP	PIP
RNN	CMC	MCP	PIP	DIP

In most of the cases, the DIP joints exhibited the least sensitivity to the muscle activities, as shown in Figures. 8 (b) and (d). This may indicate 1) that the selected muscles/muscle regions in the experimental setup exhibited weak responsibility in moving the DIP joints; 2) the dependency of DIP rotations on the PIP within the same finger; 3) the less accuracy associated with the sensors measuring these joints; and/or 4) the less ROMs required in performing the aforementioned tasks.

Therefore, a significant agreement occurred between the findings of the visual and numerical investigation techniques. This motivated us to exclude sEMG channels 11 and 12 from the next phase in the ANN modeling. This reduced the system dimensionality and would facilitate the ANN in achieving the optimal performance, where only the most informative inputs would be selected for the next phase. Hence, the system under consideration became one comprising 10 inputs (sEMG) and 15 outputs (dataglove DOF).

The ANN modeling performances, as well as the AC and MI findings, are summarized in Tables 5 and 6 for the wrist and grasp sets, respectively. (Note: therein, the performances of the four ANN types were averaged for simplicity because they were similar, with similar finding trends). Remarkably, the findings of the ANN matched the pattern of AC (see Figure 8 (b)) and agreed in most cases with the MI results (see Figure 8 (d)). For example, for the grasp set, the values of AC between a DOF and all sEMG signals were ordered in a descending manner as follows: CMC, MCP, PIP, and DIP. This pattern was also observed from the ANN modeling, as expected. The reader can refer to Table 4 for more examples. The ANN performance was much better for the wrist case compared with the grasp case, as predicted by both the visual and numerical inspection approaches. This may be owing to the nature of the neutral positions in which the joints were bent, and that the grasping motions were more complicated, compared with those of the wrist, resulting in more motion artifact noise in the readings. The ANN accuracies for the CMC, MCP, PIP, and DIP of the wrist/grasp sets were as follows: 0.8746/0.7628, 0.7752/0.6427, 0.8873/0.5373, and 0.6925/0.4966, respectively.

Table 5

Mean numerical investigations (AC and MI) between a DOF and all sEMG channels, and the mean performance over all ANNs (RNN) in estimating the DOF throughout the wrist dataset.

Table 6

Mean numerical investigations (AC and MI) between a DOF and all sEMG channels, and the mean performance over all ANNs (RNN) in estimating the DOF throughout the grasp dataset.

Interestingly, the techniques conducted, both visually and numerically, matched the ANN findings significantly. Hence, those techniques are important for estimating the ANN performance prior to spending long periods in training those networks with the possibility of poor performance.

Discussion

sEMG signals are nonlinearly related to joint angles of the human limbs. The traditional mapping approaches require comprehensive knowledge of the human anatomy and physiology, and they often yield results that deviate significantly from the correct mapping. However, black-box systems, such as ANNs, can capture this relationship using only data pairs of inputs and outputs. Meanwhile, ANN and other ML algorithms are sensitive to noisy or irrelevant observations, where signal pre-investigation techniques should be conducted prior to the training phase.

Initially, the sEMG and kinematics readings were visualized via different types of plots. It was discovered that both data types, even after filtration, z-scoring, and feature extraction, presented some problems in terms of large irregular behaviors and jitteriness along repetitions, and for the kinematics signals, small ROMs were produced (before z-scoring). These findings, and others described in Section 4, were observed from the proposed AC technique as well (Subsection 3.2.1). Both the visual inspection and the AC techniques agreed with the standard MI approach (Subsection 3.2.2). Furthermore, these techniques successfully predicted the ANN performance before the models were trained and tested. This enabled a reduction in the long periods required for training ML and deep learning (DL) models on a dataset, which have similar drawbacks by deciding whether to or not to use it or at least select the most appropriate signals it involves. Additionally, such signal pre-investigation steps can enhance the ML/DL modeling accuracy when applied to classification and regression problems. It should be noted that although the AC and MI techniques predicted the ANN performance well, they cannot be used to create sEMG-kinematics mappings. The reasons for the problems in sEMG readings may be the absence of induced forces during the tasks, as the participants were instructed to imitate the movements without exerting large forces, in addition to the dynamic environment associated with the acquisition protocol. The effects of the missed calibration procedure for the dataglove DOF appear to be the main reason for the issues concerning the data. After contacting Cyber Glove Systems LLC and referring to the Cyberglove II manual [46], it was discovered that the readings for each participant must be calibrated individually for each DOF (total of 22) at the time of acquisition via a special software. The team of the Ninapro dataset did not perform this step as it would be time consuming (a large number of participants and DOFs), and they thought that it would not pose a problem as most ML approaches employed in EMG-kinematics mappings are insensitive to the linear scaling of data [9]. By contrast, it is believed in this study that the missed calibration procedure is crucial owing to the following reasons: I) Based on the CyberGlove II manual, the calibration formula is not pure scaling, i.e., it has an offset factor, which is expressed as Eq. (5):where is the actual DOF angle (o); is the measured DOF value proportional to the DOF angle (V); and are the gain and offset coefficients, (o/V) and (V), respectively. Therefore, the values provided in the dataset repository are in units of V. Consequently, this renders it difficult for an ML model (or ANN) to adapt to 30 factors, as in in our case (15 DOFs2 factor/DOF), as an additional function would be added for the model to search for the optimal calibrated joint angles in addition to its original complex nonlinear mapping. II) Owing to its fabrication nature, CyberGlove II suffers from cross coupling, in which neighboring MCP flexion/extension and abduction/adduction motions are affected by each other [47]. In fact, the calibration of the dataglove, specifically of CyberGlove types, has been investigated in many studies [47, 48, 49], which aim to simplify and improve this process. Whether an accurate calibration is required depends on where the glove would be applied. For example, for studies concerning joint angle predictions from EMG signals to control exoskeletons and prostheses worn by disabled people, it is clear that calibration is crucial. Meanwhile, employing a dataglove for virtual-reality-based rehabilitation purposes does not require extremely precise calibrations. Hence, higher-quality, calibrated datasets are recommended for studies concerning the mapping of EMG to human joint kinematics and kinetics, as well as muscle forces.

It is noteworthy that repetition reproducibility within a certain movement cannot be tested numerically either for sEMG or kinematics signals owing to their different repetition lengths, i.e., each movement could not be partitioned to six equal periods, one per each repetition. However, if this was possible, then another testing approach would be required where, for each movement, a good-quality signal must exhibit a high AC/MI between periods, and this applies to both types of signals.

The good performances of the ANN and the pre-inspection techniques in predicting the CMC, MCP, and PIP rotations compared with the DIP may be owing to: i) the joint's neutral positions, where these joints can be bent to larger ROMs in performing the required motions compared with the DIP joints; and ii) the additional noise in the DIP measurements. Estimating the wrist movements was more accurate than estimating the grasp; this could be attributed to the noisier DOF readings during the grasp tasks in addition to the nature of the neutral positions in which the joints were bent.

Limitations regarding this study involve the exclusion of the least-informative sEMG channels (inputs) based on the pure values of the numerical measures, i.e., the AC and MI between a DOF and all sEMG, not the normalized ones. It was speculated that normalizing these measures by dividing them by their maximum values, as an example, and then discarding those that have values lower than the appropriate thresholds would be more accurate for excluding the most irrelevant sEMG channels. For our case, Table 7 shows the normalized values of AC and MI for the wrist and grasp sets. If the threshold for the MI was set to 0.5, then sEMG channel 11 would be included in the ANN modeling whereas channel 12 would not. Another issue was that the AC and MI did not consider redundancy and complementariness among the input variable sets [50]. For example, conducting a few FFNN trials while maintaining all the 12 sEMG channels occasionally resulted in slightly better performances than excluding channels 11 and 12; meanwhile, in other trials, lower performances were yielded, each with the same hidden and output neurons. Nevertheless, other approaches considered issues of input (or feature) redundancy and/or relevancy, such as joint mutual information [51], conditional mutual-information-based feature selection [52], and more recently, conditional mutual-information-based feature selection considering interactions [53], and MI combined with the measurement of kernel canonical correlation analysis [54].

Table 7

Normalized values for the AC and MI measures between a sEMG channel and all DOF. W: wrist set; G: grasp set.

sEMGch #	Normalized AC			Normalized MI
	W	G	Mean ofW and G	W	G	Mean ofW and G
1	0.9656	1.0000	0.9919	0.8315	0.8740	0.8940
2	0.8654	0.9415	0.9074	0.5813	1.0000	0.7501
3	0.9542	0.7876	0.8980	1.0000	0.8164	1.0000
4	0.8837	0.8352	0.8750	0.8276	0.9075	0.9019
5	0.9180	0.8719	0.9107	0.8020	0.7041	0.8179
6	0.9070	0.9159	0.9221	0.5667	0.6493	0.6265
7	0.9261	0.9607	0.9520	0.7423	0.9082	0.8394
8	1.0000	0.9692	1.0000	0.9690	0.8116	0.9754
9	0.8906	0.9070	0.9085	0.8658	0.9055	0.9296
10	0.8697	0.9577	0.9167	0.7119	0.9425	0.8275
11	0.6294	0.6539	0.6474	0.4056	0.6363	0.5037
12	0.5122	0.5736	0.5438	0.1208	0.1116	0.1249

Conclusion

In this study, a dataset, including sEMG and hand kinematics signals, was carefully investigated using different visual and numerical methods from different aspects. The findings of these methods were consistent with those of MI. Moreover, all of them successfully predicted the ANN performance. This indicated the importance of signal pre-investigation techniques when selecting the inputs or features used in training ANN and ML models. In the future, more advanced input variable ranking techniques will be applied on different ML algorithms in datasets whose kinematics readings are well calibrated, less noisy, and have clearer movement periodicities. Additionally, they should be tested on various features that encompass domains of time, frequency, and time–frequency. Finally, based on observations of this study, “if–then”-based intelligent models, such as fuzzy Logic, should be used for modeling purposes.

Declarations

Author contribution statement

Wafa Batayneh: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data.

Enas Abdulhay: Contributed reagents, materials, analysis tools or data.

Mohammad Alothman: Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding statement

This work is funded by the Deanship of Research at Jordan University of Science and Technology, Grant number 20180035.

Competing interest statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.