Validation of the Perception Neuron system for full-body motion capture.

Recent advancements in Inertial Measurement Units (IMUs) offers the possibility of its use as a cost effective and portable alternative to traditional optoelectronic motion capture systems in analyzing biomechanical performance. One such commercially available IMU is the Perception Neuron motion capture system (PNS). The accuracy of the PNS had been tested and was reported to be a valid method for assessing the upper body range of motion to within 5° RMSE. However, testing of the PNS was limited to upper body motion involving functional movement within a single plane. Therefore, the purpose of this study is to further validate the Perception Neuron system with reference to a conventional optoelectronic motion capture system (VICON) through the use of dynamic movements (e.g., walking, jogging and a multi-articular sports movement with object manipulation) and to determine its feasibility through full-body kinematic analysis. Validation was evaluated using Pearson's R correlation, RMSE and Bland-Altman estimates. Present findings suggest that the PNS performed well against the VICON motion analysis system with most joint angles reporting a RMSE of < 4° and strong average Pearson's R correlation of 0.85, with the exception of the shoulder abduction/adduction where RMSE was larger and Pearson's R correlation at a moderate level. Bland-Altman analysis revealed that most joint angles across the different movements had a mean bias of less than 10°, except for the shoulder abduction/adduction and elbow flexion/extension measurements. It was concluded that the PNS may not be the best substitute for traditional motion analysis technology if there is a need to replicate raw joint angles. However, there was adequate sensitivity to measure changes in joint angles and would be suitable when normalized joint angles are compared and the focus of analysis is to identify changes in movement patterns.

Introduction

Traditionally, motion capture has been collected through the use of high-speed optical systems which allows for dynamic movements to be captured and analysed [1, 2]. These systems serve as a reliable and suitable methods to measure complex movements and are often considered as the gold standard in motion capture, providing an estimated accuracy (RMS error) of less than 1.00° and 1.50° for static and dynamic measurements respectively [2-4]. An example of a high-speed optical system is the VICON motion analysis system (Oxford Metrics Group Ltd., Oxford, UK). For such systems, passive reflective markers are placed on specific body landmarks which reflect light back into the sensors of the cameras and the system computes the 3D positions of the makers within the environment using the triangulation method [1, 2, 4].

3D optoelectronic motion capture systems have improved significantly over the years due to higher camera resolution, better calibration techniques, and improvements in manufacturer tracking software which allows for more accurate marker tracking and greater post processing capabilities [1, 5]. With these changes in technology, consistency in accuracy by such systems have showed a nearly three-fold improvements in agreement over the last 20 years with no systems exceeding an error of 1.0mm [5]. Due to their validity and reliability as compared to other motion capture systems [2], optoelectronic motion capture systems have been extensively used in a variety of field, ranging from functional movement tasks [6, 7] to sports performance [8, 9] and injury prevention [10, 11], thus contributing meaningful information and advancements to the field of biomechanics over the years [1].

Although optical based systems have been commonly used for motion analysis, they are still expensive, have somewhat limited portability and typically require movement to be recorded within the confines of a laboratory environment [4, 12, 13]. Set-up and calibration also involves a lengthy preparation process [14]. Furthermore during data collection, markers have to be in line-of-sight of the cameras as any occlusion or alterations to camera positions will affect the reliability of the data [2]. Thus even though optical based systems are considered as the gold standard with consistently accurate measurements, the desire by researchers to collect accurate and reliable kinematic in applied situations for research and for the routine analysis of movements within normal training environments without being obstructed by environmental restrictions and lengthy processing time has increasingly driven researchers to search for alternatives [1, 15].

One such alternative is the inertial measurement unit (IMU). IMUs are devices that comprise of a gyroscope, an accelerometer and magnetometer and measures the motion of the user through embedded data fusion, human body dynamics and physical engine algorithms which allows for the estimation of kinematic data of body segments within a 3D space to be obtained [3, 16]. These IMUs allow users to capture and record real-time motion with a fast set-up process and minimal space restrictions thus increasing the portability and allowing movement capture to be conducted within real-life environments [16]. Hence, IMUs could potentially serve as a solution to the limitations that optical based motion capture systems face.

Importantly, recent advancements in IMUs offers the possibility of its use as a cost effective, portable alternative to analyzing biomechanical performance in an efficient way [3, 17, 18]. In general a combination of higher sampling rate, better sensor fusion algorithms and lower drift have resulted in increased accuracy of the data collected by these sensors [19]. As such IMUs have been used in various applications in sports, such as technique identification and performance, tracking athletes’ performance and for injury screening [2, 14, 20]. One example of technique identification and performance is a study conducted by Bosch, Shoaib [21], where researchers were able to distinguish between experienced and novice rowers by simply placing sensors on the lower leg, the lower back and upper back and analyzing consistency in postural angles and timing of the strokes. With respect to tracking athletes’ performance, Shepherd, Giblin [22] used IMUs effectively to monitor shooting kinematics and coordination patterns of netball athletes. This provided coaches and athletes with actionable insights to develop practices to achieve more consistent forearm angle at ball release and increase their chances of scoring during a game. IMUs have also been used for injury screening and prevention by evaluating movements and specifically to identify the presence of abnormal movements during performance [23]. They allow scientists to quantitatively monitor and detect movement kinematics or volumes of training which has been linked to overuse injuries and hence allow them to issue preventive warnings [23-25]. As seen from the studies above, information obtained from these IMUs could be utilized by sports scientists and professional coaches in a myriad of ways and within ecological contexts [17, 18, 26]. Furthermore with a combination of lower costs, greater ease of use, increased portability for use in the field, it could eventually drive the use of these IMUs in sports not only exclusively for elite athletes but for a much wider target audience [14, 27]. However, accuracy of the IMUs depends on the specific sensors used, the software algorithms of the tested IMU, and the measured joint and task that is being performed [19]. Therefore it is necessary that specific validation of each new IMU system is completed before any IMU can be used routinely in assessments [19].

One such commercially available IMU is the Perception Neuron motion capture system (PNS) (Perception Neuron, Noitom, Miami, FL, USA). It is an adaptable and affordable motion capture system offering user-friendly technology that was developed to analyse movements for various industries from film makers to game developers, biomechanics researchers and sports and medical analysts. Some of the benefits of the PNS includes its ease of use since the sensors are interchangeable and also the portability of the system due to the ability to rely on a portable battery for its energy needs. In addition, the PNS has the capacity to record the data locally on a memory card or transfer the data through WiFi instead of a wired connection.

Currently, validation of the PNS is still limited. One such validation study was conducted by Sers, Forrester [12], where the accuracy of joint angles measurements acquired by the PNS was referenced against a gold standard optoelectronic systems (VICON). Movements analysed included neck flexion/extension, neck lateral flexion, neck rotation, torso flexion/extension, torso later flexion, torso rotation and shoulder abduction. Participants were tasked to perform each movement twice and at self-selected fast and slow speeds. Results suggests that the PNS is a valid method to assess the majority of the upper body range of motion during static movements with systematic and random bias for majority of the range of motion differences being below 4.5°, with an exception for the knee extension where values were 6.1–9.0°. However Sers, Forrester [12] only analysed the validity of the PNS using only postural angular kinematics of the upper body.

Therefore, the purpose of the study was to further validate the PNS with reference to a conventional optoelectronic motion capture system for full-body human movement during more dynamic movements and to determine its feasibility in a comprehensive full body kinematic analysis. Specifically, performance of the PNS was assessed using kinematic measures (e.g., joint angles) in comparison to the VICON motion analysis system using a series of full-body human movements. Walking, jogging, and floorball wrist shots were used as a proposed task vehicle as these activities are predicated on its feature as a discrete multi-articular task that involves the coordination of multiple body parts, joints and limb segments with object manipulation. The wrist shot was specifically chosen out of the many shots available within floorball as it is the most common shot used by players to score a goal and it is executed at a fast pace. The inclusion of these movements allows the testing of the PNS’ ability to measure kinematic data during dynamic and fast paced tasks, which was previously not examined in the study undertaken by Sers, Forrester [12]. In addition, results from this study could potentially provide evidence for researchers that the PNS could be a viable tool to collect motion capture data in a more representative environment outside of a lab setting. This is especially so when it is not feasible to set up an optical based system. For example, in situations where there is a need to analyse movement over a larger area, if an optical based system is used, a greater number of cameras would be required to cover this increase in volume. In this case, the PNS could be used to overcome the limitations of the lab space and the number of cameras needed whilst still providing valid, reliable and objective data.

It is hypothesized that measurements from the PNS would be comparable to the measurements from a traditional optoelectronic motion capture analysis system (VICON), thus validating the use of the PNS as an alternative for full-body human motion capture. By establishing the validity of the PNS, it ensures that the system can adequately measure full-body movement. The PNS could then potentially be used as an alternative to the current optical motion systems during sports biomechanical applications. This could help mitigate the restrictions of the traditional 3D optical motion system which are expensive and better allow collection of data within real-world settings outside the restricted confines of a laboratory.

Methods

Participants

Ten participants (7 males and 3 females) with a mean age of 26 ± 2 years, mean height of 1.70 ± 0.09 m and mean weight of 60.48 ± 11.91 kg participated in this study. This study was approved by the University’s Institutional Review Board (IRB-2019-03-005) and written consent was obtained prior to the start of any data collection. Exclusion criterion was musculoskeletal injury, illness or disease for the past 6 months prior to the time of testing, mainly to ensure safety for the participants.

Instrumentation

Movement data was collected simultaneously with the PNS and an eight-camera VICON MX40 optical motion capture system recording at 120Hz each. Participants’ movement were also captured and recorded via two digital video cameras (CASIO EXILIM EX100, Tokyo, Japan) from two planes of observation (frontal and sagittal). The VICON optoelectronic motion analysis system was chosen as it is a widely used system and is considered as the current laboratory gold-standard with a high accuracy during dynamic trials [12, 19, 28].

Thirty-three reflective markers (19-mm) were placed on anatomical landmarks based on the Full-body modeling with Plug-in Gait (Vicon Peak, Oxford, UK). There was one adjustment made to the marker set, this was to the T10 marker. The T10 marker was shifted to the same level as the sternum for easier location purposes. The PNS consisted of 17 IMUs placed at specific landmarks in accordance to PNS guidelines as shown in Fig 1. The PNS suit was connected to a computer through a USB connection and the Axis Neuron software (AXIS Neuron, Noitom, Miami, Florida, USA) was used to view the data in real-time.

Fig 1

Perception Neuron IMU sensor configuration.

Calibration of both systems was also completed prior to each movement recording. The PNS was calibrated according to the manufacturer’s instructions using a four-step calibration process (see Table 1) and was zeroed after five repetitions of each task to remove any errors. The VICON motion analysis system was calibrated using the T-pose.

Table 1

Perception Neuron 4-step calibration.

Posture calibration	Details
1. Steady pose	Participant to be seated and to remain as still as possible
2. A-pose	Participant to place palms down on the side of the thighs and keep feet parallel
3. T-pose	Participants to have shoulders abducted by 90° with the palms facing the floor
4. S-pose	Participants to bend knees approximately 45° and place arms in front and position them parallel to the floor

Protocol

All participants underwent one test session. Anthropometric measurements including height, weight, leg length, knee width, ankle width, elbow width and hand thickness was recorded from every participant at the start of the session. Participants were then suited up in the PNS 17-neuron full body suit and the 39 19-mm retroreflective markers for the VICON system were placed on anatomical landmarks according to the Plug-in-Gait full-body model (VICON Motion Systems, Oxford Metrics Group Ltd) and held in place using medical grade double-sided adhesive tape as seen in Fig 2.

Fig 2

Participant set-up with both PNS and VICON markers front and back.

The red squares indicates the PNS sensor positions.

Participants were required to perform a series of tasks. Description of these can be found in Table 2. Participants performed 10 repetitions for each task. During the performance of the tasks, care was taken to ensure that participants’ movement were not obstructed in any way as a result of the reflective markers and the PNS. Compression garments (2XU Compression Shorts, Melbourne, Australia) were used during the data collection to minimize the marker movements whilst still allowing participants to be covered. Compression garments were chosen as it had been previously reported that markers placed over compression garments demonstrate low variance in movement compared to the markers placed on the skin [29].

Table 2

Description of tasks to be performed.

Task	Description
Stationary walk	Participants to swing their arms, bring their knees up and feet back down to the same point where they took off from at a self-determined walking pace to mimic the walking movement. Participants performed the task whilst remaining within a 90 x 90 cm demarcated spot on the lab floor
Distance walk	Participants to walk along a 3m line on the lab floor at a self-determined walking pace
Stationary jog	Participants to swing their arms, bring their knees up and feet back down to the same point where they took off from at a self-determined jogging pace to mimic the jogging movement. Participants performed the task whilst remaining within a 90 x 90 cm demarcated spot on the lab floor
Distance jog	Participants to walk along a 3m line on the lab floor at a self-determined jogging pace
Stationary floorball wrist shot	Participants to stand 4m in front of an open goal and take shots towards the goal
Moving wrist shot	Participants to stand 7m in front of an open goal, dribble the ball along a 3m line and take a shot towards goal once they are 4m in front of the goal

Data analysis

Data collected from the PNS were exported as a processed bvh file. Displacement of the VICON markers in 3D space were reconstructed and manually labelled for all trials using VICON Nexus 2.9.0. Joint angle data from both systems were calculated from the 3D coordinates using a custom script programmed using MATLAB (R2018b, Mathworks Inc., Natick, MA USA) which computes both data on the same basis in order to ensure that data from both systems are comparable. Elbow flexion/extension, shoulder flexion/extension, shoulder abduction/adduction, hip flexion/extension, hip abduction/adduction, knee flexion/extension and ankle flexion/extension angles were calculated from both systems and comparisons between measurements obtained from the two systems were made. These flexion angles were adapted based on the study of Lee, Chow [30] and Lazzaeri, Kayser [8]. Subsequently, joint angles from both devices were smoothed using the “sgolayfilt” function with a polynomial function of degree 3 in MATLAB [31]. The Savitzky-Golay filter is based on the principle of local least squares fitting of a polynomial approximation [32, 33]. This filter is effective at preserving the width and height of the waveform [32, 33]. In addition to using the same recording speed, all trials were also time normalized to 100 points to allow for simultaneous comparisons.

To compare the joint angles from the VICON and PNS, Pearson’s correlation and RMSE were evaluated for all movements on the raw joint angles. Pearson correlation was used to test the linear relationship between the joint angle measurement from PNS and VICON system. Correlation values were interpreted as such (in either direction positive or negative direction): 0–0.3 represents a weak relationship, 0.3–0.7 represents a moderate relationship and 0.7–1.0 represents a strong relationship [34]. Correlations were calculated between the VICON and PNS per joint, per activity and per subject. The standard deviation, mean and p-value reported were based on the average of these correlations. All statistical analyses were completed using MATLAB.

RMSE was used to compare the two methods as a measure of fit [35],

Where X(t) is the value from the PNS suit, Y(t) is the observed value from VICON and n is the total number of time points.

Group averages and SD were presented for Pearson correlation and RMSE.

The construction of Bland-Altman plots [36, 37] was another way to determine the relationship between the two measurement systems. Correlation exploration between the error and the mean value was included to examine for the existence of heteroscedasticity [38]. Mean bias and limits of agreement (LOA) were calculated according to Bland and Altman [37] for comparisons on raw joint angles to check for systematic errors. LOA was calculated as such, where refers to the mean difference and SD refers to the standard deviation.

LOA (%) was also used to check for the agreement between joint angle calculations from the two measurement systems. LOA (%) was defined as the percentage of readings that lie within the limits of agreement. This was calculated for both raw and normalized joint angles. Normalization of joint angle data were conducted using z-score transformation as such, where x refers to the data point value, μ refers to the mean and σ represents the standard deviation.

Results

Table 3 contains the Pearson’s R correlation between the PNS and VICON joint angle calculations. Most correlation values were statistically significant, with the exception of ankle flexion/extension during stationary jog and stationary wrist shot. Most values showed strong positive correlation with the exception of hip abduction/adduction during stationary walk at 0.63 ± 0.40, shoulder flexion/extension during distance walk with a value of 0.59 ± 0.33, shoulder abduction/adduction during stationary wrist shot, distance wrist shot and during distance walk with a correlation of 0.58 ± 0.50, 0.50 ± 0.46 and 0.66 ± 0.43 respectively, and ankle flexion/extension during stationary walk, stationary jog and stationary wrist shot with a correlation of 0.59 ± 0.53, 0.45 ± 0.54 and 0.55 ± 0.50 respectively. Correlation was strong for knee flexion/extension across all movements with correlation ranging between 0.80 and 0.99.

Table 3

Pearson’s correlation comparison between PNS and VICON.

Joint Angle	Stationary walk	Distance walk	Stationary jog	Distance jog	Stationary wrist shot	Distance wrist shot
Elbow (flex/ext)	0.96	0.91	0.85	0.94	0.84	0.85
SD	0.06	0.17	0.31	0.08	0.25	0.17
p-value	< 0.001	0.0023	0.026	< 0.001	0.014	< 0.001
Shoulder (flex/ext)	0.80	0.59	0.82	0.88	0.76	0.91
SD	0.20	0.33	0.29	0.18	0.33	0.15
p-value	0.0038	0.047	0.040	0.004	0.032	0.0053
Shoulder (abd/add)	0.77	0.66	0.79	0.87	0.58	0.50
SD	0.21	0.43	0.29	0.14	0.50	0.46
p-value	< 0.001	0.03	0.04	< 0.001	0.02	0.04
Hip (flex/ext)	0.95	0.82	0.73	0.90	0.84	0.82
SD	0.06	0.22	0.35	0.08	0.28	0.25
p-value	< 0.001	0.0016	0.039	< 0.001	0.0077	0.016
Hip (abd/add)	0.63	0.84	0.71	0.74	0.93	0.89
SD	0.40	0.17	0.42	0.26	0.13	0.17
p-value	0.0325	0.0039	0.037	0.012	0.0072	0.0024
Knee (flex/ext)	0.99	0.99	0.88	0.99	0.80	0.96
SD	0.00	0.02	0.30	0.01	0.37	0.06
p-value	< 0.001	< 0.001	0.037	< 0.001	0.011	< 0.001
Ankle (flex/ext)	0.59	0.82	0.45	0.73	0.55	0.78
SD	0.53	0.22	0.54	0.36	0.50	0.19
p-value	0.0082	0.018	0.15	0.024	0.056	0.0034

Table 4 displays the RMSE comparisons between the PNS and VICON joint angle calculations on the raw values. RMSE values for raw values were generally below 4° for all joint angles and across all movements except for shoulder abduction/adduction for all movements and for ankle flexion/extension during stationary jog and distance jog.

Table 4

RMSE comparison between PNS and VICON for raw values for all movements.

Joint Angle	Stationary walk	Distance walk	Stationary jog	Distance jog	Stationary wrist shot	Distance wrist shot
Elbow (flex/ext)	3.40	2.04	3.89	1.92	2.81	3.20
SD	2.15	1.48	2.96	1.00	2.18	1.75
Shoulder (flex/ext)	1.90	1.12	1.94	1.78	2.23	1.99
SD	0.80	0.65	1.53	1.16	1.97	1.12
Shoulder (abd/add)	7.14	5.36	5.97	5.70	11.85	15.15
SD	2.97	3.16	3.80	2.57	10.24	9.32
Hip (flex/ext)	2.78	2.63	2.91	2.76	1.83	3.58
SD	1.61	1.19	1.51	1.23	1.29	2.68
Hip (abd/add)	1.50	1.47	1.32	1.80	1.00	1.67
SD	0.95	1.29	0.85	0.95	0.62	0.86
Knee (flex/ext)	1.48	2.56	5.26	2.19	1.12	3.35
SD	0.56	1.16	9.13	1.27	0.81	2.23
Ankle (flex/ext)	2.77	3.11	4.33	4.47	1.21	3.83
SD	1.20	1.37	3.07	1.94	1.34	1.65

Tables 5 and 6 includes the mean bias and limits of agreement (LOA) for each joint angle and across all movements. The ankle flexion/extension showed the largest mean bias of 22.74° during stationary walk. Whereas the hip flexion/extension had the least differences as compared to the other joint angles with a mean bias of 0.56° during stationary walk. There was a trend for joint angles to be overestimated using the PNS as compared to the VICON across movements. Only the angles for shoulder flexion/extension and shoulder abduction/adduction was underestimated. Existence of heteroscedasticity was found for most joint angles and across all movements. An example of a Bland-Altman plot for hip flexion/extension during distance wrist shot can be found in Fig 3. All the Bland-Altman plots for all movements and joint angles can be found in S1 Appendix.

Table 5

Limits of agreement and mean bias for all stationary movements.

	Stationary walk			Stationary jog			Stationary wrist shot
Joint Angle	Lower limit	Mean bias	Upper limit	Lower limit	Mean bias	Upper limit	Lower limit	Mean bias	Upper limit
Elbow (flex/ext)	-42.55	-16.20	10.15	-68.01	-15.06	37.90	-40.53	-20.34	-0.15
correlation (heteroscedasticity)	p < 0.001        r = -0.62			p < 0.001        r = -0.52			p < 0.001        r = -0.17
coefficient of determination	r2 = 0.38			r2 = 0.27			r2 = 0.029
Shoulder (flex/ext)	-10.42	4.04	18.50	-11.87	2.61	17.09	-23.74	1.92	27.58
correlation (heteroscedasticity)	p < 0.001        r = -0.53			p < 0.001        r = -0.39			p < 0.001        r = -0.39
coefficient of determination	r2 = 0.28			r2 = 0.15			r2 = 0.16
Shoulder (abd/add)	-20.59	9.07	38.73	-15.39	12.16	39.72	-69.77	-7.52	54.72
correlation (heteroscedasticity)	p < 0.001        r = -0.17			p < 0.001        r = -0.24			p < 0.001        r = -0.24
coefficient of determination	r2 = 0.028			r2 = 0.057			r2 = 0.058
Hip (flex/ext)	-41.00	-0.56	39.88	-56.02	-5.21	45.59	-29.08	-4.11	20.85
correlation (heteroscedasticity)	p < 0.001        r = -0.59			p < 0.001        r = -0.51			p < 0.001        r = -0.29
coefficient of determination	r2 = 0.34			r2 = 0.26			r2 = 0.086
Hip (abd/add)	-13.56	-1.35	10.86	-12.62	-3.43	5.75	-19.26	-3.91	11.44
correlation (heteroscedasticity)	p < 0.001        r = -0.40			p < 0.001        r = -0.31			p < 0.001        r = -0.60
coefficient of determination	r2 = 0.16			r2 = 0.096			r2 = 0.36
Knee (flex/ext)	-19.88	-8.63	2.62	-43.36	-9.34	24.68	-16.49	-5.37	5.76
correlation (heteroscedasticity)	p < 0.001        r = -0.50			p < 0.001        r = -0.16			p < 0.001        r = -0.16
coefficient of determination	r2 = 0.25			r2 = 0.025			r2 = 0.025
Ankle (flex/ext)	-36.23	-22.74	-9.25	-46.11	-17.15	11.81	-29.63	-17.14	-4.65
correlation (heteroscedasticity)	p < .001        r = -.31			p < .001        r = -.61			p < .001        r = -.37
coefficient of determination	r2 = .097			r2 = .37			r2 = .14

Table 6

Limits of agreement and mean bias for all movements with distance.

	Distance walk			Distance jog			Distance wrist shot
Joint Angle	Lower limit	Mean bias	Upper limit	Lower limit	Mean bias	Upper limit	Lower limit	Mean bias	Upper limit
Elbow (flex/ext)	-35.47	-21.17	-6.88	-28.02	-12.22	3.57	-41.82	-20.62	0.57
correlation (heteroscedasticity)	p < 0.001        r = -0.23			p < 0.001        r = -0.43			p = 0.044        r = -0.014
coefficient of determination	r2 = 0.056			r2 = 0.19			r2 = 0.00019
Shoulder (flex/ext)	-3.49	7.26	18.01	-8.31	3.54	15.39	-26.81	-1.42	23.98
correlation (heteroscedasticity)	p < 0.001        r = -0.30			p < 0.001        r = -0.24			p < 0.001        r = -0.44
coefficient of determination	r2 = 0.090			r2 = 0.056			r2 = 0.19
Shoulder (abd/add)	-22.63	11.10	44.82	-22.24	9.80	41.85	-61.27	2.88	67.03
correlation (heteroscedasticity)	p < 0.001        r = -0.44			p < 0.001        r = -0.31			p < 0.001        r = 0.31
coefficient of determination	r2 = 0.19			r2 = 0.099			r2 = 0.10
Hip (flex/ext)	-55.40	-11.18	33.05	-40.31	-5.24	29.82	-50.33	-7.42	35.49
correlation (heteroscedasticity)	p < 0.001        r = -0.69			p < 0.001        r = -0.70			p < 0.001        r = -0.56
coefficient of determination	r2 = 0.47			r2 = 0.49			r2 = 0.32
Hip (abd/add)	-12.88	-1.79	9.29	-12.54	-1.71	9.12	-16.17	-1.49	13.19
correlation (heteroscedasticity)	p < 0.001        r = -0.059			p < 0.001        r = -0.039			p < 0.001        r = -0.61
coefficent of determination	r2 = 0.0034			r2 = 0.15			r2 = 0.37
Knee (flex/ext)	-22.64	-7.72	7.19	-18.96	-6.64	5.69	-20.74	-7.53	5.67
correlation (heteroscedasticity)	p = 0.029        r = -0.14			p < 0.001        r = -0.29			p < 0.001        r = -0.14
coefficient of determination	r2 = 0.018			r2 = 0.087			r2 = 0.019
Ankle (flex/ext)	-34.05	-19.85	-5.65	-40.74	-17.10	6.71	-36.98	-20.13	-3.28
correlation (heteroscedasticity)	p < .001        r = -.38			p < .001        r = -.47			p < .001        r = -.35
coefficient of determination	r2 = .14			r2 = .22			r2 = .12

Fig 3

Bland-Altman plot of agreement for hip flexion/extension for distance wrist shot using (a) raw joint angles (b) normalized joint angles for all participants. Solid horizontal lines represent the mean difference and the dashed horizontal lines represents the 95% limits of agreement (± 1.96 SD).

Table 7 shows the LOA (%) between the PNS and VICON joint angle calculations for both raw and normalized values. LOA (%) for raw values ranged from 92.61% to 97.48% across all movements while LOA (%) for normalized values was higher with almost all being above 95% with the exception of shoulder abduction/adduction during distance wrist shots at 94.47% and ankle flexion/extension during distance walk at 94.85%. Fig 3 shows the comparison between Bland-Altman plots using raw and normalized joint angles. In addition, it was observed that joint angle wave characteristics were more similar when using normalized rather than the raw joint angles. An example of this is depicted in Fig 4A and 4B.

Table 7

LOA (%) comparison between PNS and VICON for raw and normalized values for all movements.

	Stationary walk		Distance walk		Stationary jog		Distance jog		Stationary wrist shot		Distance wrist shot
Joint Angle	Raw	Norm	Raw	Norm	Raw	Norm	Raw	Norm	Raw	Norm	Raw	Norm
Elbow (flex/ext)	94.59	95.82	93.71	95.30	93.95	95.85	94.85	96.49	96.54	96.65	93.12	95.12
Shoulder (flex/ext)	95.27	95.91	96.23	96.80	94.50	96.15	96.22	96.46	96.57	96.77	92.68	95.16
Shoulder (abd/add)	95.41	95.36	95.26	95.48	93.70	95.85	95.59	96.70	94.81	95.48	92.61	94.47
Hip (flex/ext)	95.91	95.05	96.31	96.14	93.95	95.65	96.10	96.33	96.90	96.86	93.64	95.61
Hip (abd/add)	95.05	95.64	96.64	96.48	95.50	95.55	97.00	97.57	97.37	96.75	93.75	95.11
Knee (flex/ext)	95.00	95.10	94.38	95.45	94.70	95.05	94.86	96.26	97.48	96.89	94.40	95.83
Ankle (flex/ext)	96.32	96.18	95.14	94.85	96.60	95.95	96.39	96.32	97.70	97.26	95.76	95.66

Fig 4

Hip flexion/extension angles during a stationary wrist shot using raw joint angle values (a) and Hip flexion/extension angles during a stationary wrist shot using normalized joint angle values (b). Data shown is from one trial from one participant.

Table 8 includes the range of movement tested for all joint angles across the different movements.

Table 8

Range of joint angles tested for each movement.

	Stationary walk		Stationary jog		Stationary wrist shot		Distance walk		Distance jog		Distance wrist shot
Joint Angle	Lower	Upper	Lower	Upper	Lower	Upper	Lower	Upper	Lower	Upper	Lower	Upper
Elbow (flex/ext)	38	170	39	154	60	165	105	170	47	168	54	158
Shoulder (flex/ext)	17	67	20	65	19	93	12	73	22	77	14	101
Shoulder (abd/add)	33	179	41	179	41	179	106	183	99	181	21	179
Hip (flex/ext)	101	171	86	165	92	172	45	175	94	175	87	175
Hip (abd/add)	82	107	83	109	76	133	35	109	82	113	74	126
Knee (flex/ext)	61	172	49	169	108	175	41	177	49	172	64	174
Ankle (flex/ext)	80	111	70	120	76	127	74	122	73	125	72	124

Discussion

The purpose of the study was to validate the PNS during full-body human motion capture in comparison to an accepted optoelectronic motion capture systems (i.e., VICON motion analysis system). To establish the validity of the PNS, elbow flexion/extension, shoulder flexion/extension, shoulder abduction/adduction, hip flexion/extension, hip abduction/adduction, knee flexion/extension and ankle flexion/extension angles were calculated from both systems and compared. These joint angles were analysed across movements such as stationary walk, distance walk, stationary jog, distance jog, stationary floorball wrist shot and moving wrist shot. Statistical analyses included RMSE, Pearson’s R correlation and Bland-Altman analysis.

Overall, all joint angles across all movements had a RMSE of < 4°, except for ankle flexion/extension during stationary jog and distance jog which had a RMSE of 4.33 and 4.47 respectively and also for shoulder abduction/adduction which had a large RMSE above 11° for both stationary and distance wrist shots. Pearson’s R correlation also showed strong relationship for most joint angles except for shoulder flexion/extension during distance walk, hip abduction/adduction during stationary walk and ankle flexion/extension during stationary walk, stationary jog and stationary wrist shot.

Specifically, values for knee flexion/extension across all movements were acceptable, having an average RMSE of 2.66° and a strong relationship with the highest average Pearson’s R correlation value of 0.93 among all the other joints angles. There was overall good agreement between the two measurement systems for hip flexion/extension as well with an average Pearson’s correlation value of 0.84 across all movements and all movements had a RMSE of < 4°. RMSE for ankle flexion/extension was below 5° and Pearson’s correlation ranged from moderate to strong across all movements. This indicates that the overall waveform for knee flexion/extension and hip flexion/extension is similar between the two measurement systems. These results were in accordance to results from other studies comparing IMU measurements against VICON motion capture systems. Nüesch, Roos [39] compared the sagittal plane ankle, knee and hip kinematics collected from an inertial sensor system and the VICON motion capture system. Kinematic data was collected during walking and running trials and reported that RMSE was found to be below 5° for walking trials and below 8° for running trials across the different joint angles. In a study conducted by Bolink, Naisas [40], the use of an IMU was evaluated against the VICON motion capture system during three different activities of daily life: gait, sit-to-stand transfers and block step-up transfers. It was found that the IMU was a valid tool to measure dynamic pelvic angles with a strong Pearson’s correlation ranging between 0.85 and 0.94 and RMSE values between 2.7° to 4.5°. Seel, Raisch [41] compared knee flexion/extension measurements between IMU and VICON motion capture systems during gait trials and found RMSE ranging from 1.62–3.3°. Even though Nüesch, Roos [39] reported a higher measurement error as a result of the increase in the speed of movement from walking to running, this difference was not evident in the current study possibly because walking and running speeds were self-selected.

In general, results for upper body joint angles were not as good as compared to lower body joint angles. This is in line with systematic reviews [42, 43] which reported that the large variability observed in the upper body measurements could be due to how functional upper body tasks usually involves complex movements within two-three axes. Even with this, the magnitude of error for elbow flexion/extension and shoulder flexion/extension were still comparable to those reported in previous studies. Pearson’s R correlation (0.84 to 0.96) and RMSE (< 4°) for elbow flexion/extension measurements found in this study was similar to previous studies done [42, 44, 45]. Shoulder (flex/ext) also had a moderate to strong Pearson’s R correlation and an RMSE of < 3° across all movements which is within the range of error found in other studies [18, 42].

Results for shoulder abduction/adduction differed most from previous studies. Pearson’ R correlation was found to range between moderate to strong (0.58 to 0.87) with lower values during stationary and distance wrist shots. RMSE was also higher than what was reported for other joints, ranging from 5.36° to 15.15°. Similar to Pearson’s R correlation, RMSE values were also higher during stationary and distance wrist shots. Most studies have suggested that any RMSE up to 5° can be regarded as reasonable and are precise enough for a system to be considered as a valid method for motion analysis in clinical settings. However, when errors are above 5°, it should raise concerns as it is considered large enough to mislead clinical interpretation [12, 40, 43, 46, 47]. Therefore the use of raw shoulder abduction/adduction joint angles should be treated with caution, with the degree of acceptable limits directly dependent on the intended application [48].

Bland-Altman analysis showed that the mean systematic bias was below 10° for most joint angles with the exception of the elbow flexion/extension for all movements, shoulder abduction/adduction for distance walk, hip flexion/extension during distance walk and ankle flexion/extension for all movements (Tables 5 and 6). Bland-Altman plots for all raw and normalized joint angles can be found in Fig 3. Based on the recommendation of previous studies, a 10° lower or upper LOA was deemed acceptable for occupational biomechanics applications [18, 49, 50]. Therefore, joint angles obtained from the PNS seems satisfactory except for elbow flexion/extension for all movements, ankle flexion/extension for all movements, shoulder abduction/adduction and hip flexion/extension during distance walk for which data should be handled with discretion especially during dynamic movements when absolute measurement of joint angles is required.

Results showed significant correlation between the mean joint angle value and the difference between the two measurements system for almost all the joints and across movements. Therefore, there is a possibility of heteroscedasticity [38] where the amount of error from the PNS might increase as the measured joint angle value increases. Significant correlation was found even with a small coefficient of determination (r2) between the mean joint angles and difference between the two measurements. The observed significance despite a very small r2 is most probably due to the large sample size (above 19000 data points). However, validation is still confirmed for the range of movement that was investigated in this study (see Table 8), but caution should be applied when analyzing larger joint angles (when higher values are possible).

When using normalized and raw joint angles for comparison between the two systems, LOA (%) results were consistently higher for normalized values (range from 94.47% to 97.57%) as compared to raw joint angles (range from 92.49% to 97.52%). In addition, it was observed that joint angle wave characteristics were more similar when using normalized rather than the raw joint angles. The normalization process of the joint angles managed to preserve the waveform characteristics (e.g., peak joint moment) and reduce the variance between the two systems. As such, it is suggested that when using the PNS, the analysis of normalized joint angles is preferred instead of the raw joint angles as the PNS is better at detecting changes in movement patterns rather than to approximate the absolute joint angles.

Limitations

A limitation of the current study is the small collection time frame for each trial. It has been reported in previous studies that IMUs accuracy differed accordingly to the type of tasks being performed and is affected by the duration, complexity and speed of the task performed [18, 39, 42, 51–54]. Accuracy of IMUs is also affected by time of the recorded task. Its accuracy decreases over time as a result of drift which continually increases and the amount of error is compounded over time [54, 55]. Drift occurs due to the magnetic perturbations in the environment around the sensors and also the ability of the sensors to locate and track the same initial frames [56]. The amount of drift is dependent on the system and the direction of movement, with accuracy decreasing more for movements that occur within 3-axes over a sustained period of time [56]. Data included in the current study was collected within 20–30 seconds before the next calibration. Therefore, future studies could include longer trials to detect how much the PNS drifts over time and its resilience against magnetic perturbation in order to provide more information on its accuracy and if any correction is needed in the algorithms.

Previous studies have found that the accuracy of IMUs is negatively influenced by speed [57]. Larger RMSE was reported by Nüesch, Roos [39] during running as compared to walking on a treadmill, with errors of < 8° and < 5° respectively. Similarly, Cooper G., Sheret I. [51] found a RMSE of < 4° during running and < 1° of the knee during walking. However, the effect of speed on the accuracy of the PNS was not as evident as in other studies. Comparisons between walking and jogging tasks did not elicit a significant change in the results. One possible reason could be that the walking and jogging speeds were self-selected and not controlled, therefore the walking and jogging speeds might not have been significantly different enough to result in a difference. This result was similar to the study conducted by Sers, Forrester [12] where no obvious difference was detected with a change in movement speeds as speeds were also self-selected. Therefore, future studies could also analyse the accuracy of the PNS with movements at different controlled speeds. In the current study transverse plane data was not analysed, future studies could look into including comparisons between the two systems within the transverse plane.

Finally, in this study the impact on accuracy from the suit on the soft tissue artifact from the markers was not accounted for which could be a potential source of uncertainty. Therefore, future research could use rigid marker clusters that are fixed to the PNS to minimize the differences between the two systems associated with soft tissue artifact.

Conclusion and recommendations

The validity of the PNS was tested against the VICON motion analysis system by comparing joint angle measurements obtained by the two systems during a series of full-body human movements. In general, the PNS performed well against the VICON motion analysis system and most joint comparisons were similar to that of what was reported in previous studies, with the exception of elbow flexion/extension, shoulder abduction/adduction during the stationary and distance wrist shots, hip flexion/extension during distance walk and ankle flexion/extension.

With reference to Bland-Altman analysis, most of the joint angles were found to have been overestimated by the PNS with the exception of the shoulder flexion/extension and shoulder abduction/adduction. Mean bias for most joint angles were within acceptable recommendations from previous studies [18, 49, 50]. LOA range when using raw joint angles was larger than what is recommended for biomechanics applications [18, 49, 50]. Thus, caution should be applied when using raw joint angle measurements obtained from the PNS for analysis. In addition, validation of the PNS is confirmed for the range of movements that were investigated in this study and caution should also be applied when analyzing any joint angles beyond this range. When using normalized joint angles for comparisons, LOA (%) values were consistently better as compared to when raw joint angles were used. In conclusion, results suggests that the PNS could be more suited to analyse movement patterns through the use of normalized rather than raw joint angles and that researchers could consider the use of the PNS when doing so.

Bland-Altman plots.

(PDF)

Click here for additional data file.

26 Jan 2021

PONE-D-20-33330

Validation of the Perception Neuron system for full-body motion capture

PLOS ONE

Dear Dr. Choo,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by February 15th, 2021. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Tumay Tunur, Ph.D.

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtxQ

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This is a paper where the authors have tried to validate the PNS IMU-based system kinematic output against that of the VICON optical capture system. The need for their work is that validation of the PNS system is currently limited. The significance of their work is that, if the PNS system kinematic output is validated, then one could conduct motion analysis studies by using a less expensive system which is easier to transport, it does not require a lot of time to set up and can be used outside of a laboratory environment in a real-world setting. However, besides the cost-related and portability aspects of the authors’ arguments in favor of the PNS system, there are multiple examples of optical capture systems used in a real-world setting, either for studying sports performance or occupational factors. Furthermore, the claim of the authors that set up and calibration is very lengthy for optical capture systems, is not in agreement with our experience, assuming knowledge of optical capture principles. Our volume calibration procedure takes approximately 1 minute consistently, and within the calibration volume one can perform any activity. The marker-model calibration, which is specific to the computational model used to analyze motion, is done once, also done in less than 1 minute (6 seconds to be exact) rather than re-calibrating every time a new activity is to be performed, which seems to be the case with the PNS. However, the authors do claim to be calibrating both systems before each activity (page 8, line 167), which does not make sense with respect to the VICON system.

There are some major concerns before this manuscript is considered for publication. Some of them are:

1) It seems that the output from each system was filtered in the exact same way (page 10, lines 195-197). How did the authors determine that the frequency content to be filtered from both systems for each activity was identical and, therefore, presumably they used the same filtering process and parameters? Data need to be provided towards this end. If this comment is not correct, then how each signal output from each device for each activity was filtered and why? This information is directly related to the quality of the data output being compared.

2) The authors have used the Plug-In-Gait marker set to compute the kinematics for the activities they studied (page 7, lines 161-162). The Plug-In-Gait marker set, however, is tied to several assumptions tied to the Plug-In-Gait computational model. It appears, however, that the authors did not use the Plug-In-Gait model to compute the kinematics of interest (page 10, lines 187-191). Why did the authors use this marker set model? The marker model itself does not ensure 3D segment definition by 3 marker non-collinearity per segment. Therefore, what assumptions did the authors use for the purposes of 3D segment definition (especially for the femur and tibia/fibula) and corresponding 3D motion computation? This information is also directly related to the quality of the data output being compared.

3) Observing Figure 2, it appears that pelvis is defined by the 2 ASIS markers and the 2 PSIS markers. These markers appear to have been placed on the shorts/trunks of the subject, presumably directly over the specific anatomical landmarks (it is not clear, but it appears that a mid-thigh marker is also placed on the subject’s trunks). How did the authors made certain that all these markers on the trunks remained over the anatomical landmarks of interest throughout the duration of all activities after each calibration? This information is also directly related to the quality of the data output being compared.

4) Observing Figure 2, it appears that the mid-thoracic marker (typically placed on the 10th thoracic spinous process) is placed neither on the 10th thoracic spinous process, nor in line with the thoracic spinous processes. This would create a trunk orientation offset. How has such an offset been accounted for during the kinematic computation of the shoulder motion, which, presumably is measured relative to the trunk?

5) Transverse plane data are very important, especially when considering injury prevention (in sports and occupational biomechanics) and in the clinical decision-making process. Given the ambitions of the authors for the PNS outlined in the introduction, the authors need to provide transverse data output comparisons between the 2 systems.

6) Observing Figures 1 and 2, it appears that PNS can provide ankle-related kinematics. Given the importance of ankle kinematics, especially for all lower extremity activities (in walking gait approximately 80% of the ambulatory power generation is related to the ankle), and given the desire of the authors to involve functional activities in this study, ankle kinematic output comparisons need to be made, even if only for the sagittal plane and consistently with the study of Nuesch and Ross.

7) From Tables 3 and 4 it appears that for the same activity the r and RMSE increased and decreased, respectively, for most activities as a function of increasing speed. The authors suggest that this may be related to subjects performing these activities at self-selected speeds. Although performing an activity at a self-selected speed, indeed, has been found to decrease the within trial or within condition/speed variability, the mere increase in velocity results in higher kinematic variability in VICON even from a motion artifact point of view. Therefore, the authors’ argument of self-selected speed is ambiguous. On the contrary, this suggests that either PNS’s performance for activities that are static or quasi static is substandard or the earlier arguments regarding the filtering process need to be given serious consideration.

8) What is the normalization process implemented in the Bland altman plots?

Consequently, for all these major reasons I think this manuscript is not appropriate for publication in the current time and in the current form.

Reviewer #2: The goal of this study, the evaluation of an IMU system for the measurement of joint angles, is worthy and timely as both hardware and software/algorithms for the detection of human movement have improved. Additionally, this study looks at 3d motion in contrast to earlier work which has evaluated the system in 2d only. Likewise, multiple ranges of motion and activities were evaluated by this work.

The work shows some thoroughness in scope, although that is not always reflected in the presentation. In particular, certain key details are inconsistent between text and tables, and not all the information that is collected is presented. This is easily remedied, I think, as I note below. There are also some minor grammatical issues or issues of clarification that noted.

Introduction

Lines 52-53

Change to “reliable and suitable methods to measure complex movements and are often considered as the gold standard in motion capture, providing an estimated accuracy (RMS error) of less than 1.00° and 1.50°” For RMS errors, smaller is better, so “less than” seems more appropriate than “up to”.

Line 60

Oxford comma after “calibration techniques”

Line 64

Due to “their validity” since we are discussing multiple systems.

Lines 72-77

In modern optical systems having 8+ cameras, occlusion is not generally a problem. It used to be a problem in the past. Where it can still arise is if the subject is wearing loose clothing. Please clarify this or give a specific example.

Lines 89-90

You mention increased accuracy of IMUs. But what is this compared to? Do you mean earlier IMU systems or optical systems? Some Mocap systems can do 1000 fps (see for example: https://www.xcitex.com/procapture-high-speed-cameras.php

https://arxiv.org/pdf/1908.11505.pdf

https://www.norpix.com/blog/high-speed-system-captures-at-1000-fps-x-1080p-from-multiple-synchronized-cameras/

and my own very old Motion Analysis Corp system can do 500 fps.

Line 124

You use the word “biasness.” It is a word, but I suspect that the word “bias” is in more common use. I leave this to your judgment.

Line 133

Oxford comma after “jogging”

Lines 137-138

Be consistent and use either PN system(s) or PNS throughout the paper.

Participants

Line 149

When you start a sentence with numbers, please spell them out unless you have style guidelines from PLOS that say differently.

Line 152

“musculoskeletal injury, illness, or disease” – note the change in plurality and Oxford comma.

Instrumentation

Line 159

Vicon is a very widely used system, but it’s not clear to me that it is used more universally than Qualisys, MAC, or others. Perhaps you could just indicate that it is widely used rather than “the most widely used” unless you are prepared to cite hard data.

Line 161

When you start a sentence with numbers, please spell them out.

Line 163 – This is just a curiosity and does not need to be addressed in the paper – is there any work showing how IMU suits affect soft tissue movement?

Line 164 – Also a curiosity – is there any work looking at how trailing cables interfere with movement? This should not affect your results one way or another, but it could be important using such a system clinically.

Data Analysis

Line 190

Mathworks is in Natick, MA, not Natrick.

Line 202

Please provide a reference. Are these Cohen’s values?

Line 206

Please indicate that X and Y are time series in the above equation and in the text by using notation such as X(t) and Y(t). Indicate summation over time.

Results

Line 227

Distance wrist shot shoulder angles also did not have a strong correlation according to your table.

Line 238

Your Figure 3 only includes data for hip flexion/extension for a wrist shot.

Discussion

Here there are no line numbers. You do not mention hip ab/adduction during walking.

Please explain in more detail why the hip ab/adduction during walking is not so accurate. Does this have to do with the positioning of the IMUs, lower spinal flexion that is not captured by this setup, or some other reason?

Conclusion and recommendations

(use and not &)

Note (end of first paragraph) that hip ab/adduction during walking also does not match Vicon.

Plotting

The plotting comparing joint angles is only for hip flexion in a single activity for a single trial for a single subject. Although there are supporting Bland-Altman plots, I suspect that the biomechanics community would like to see some sort of ensemble comparison of joint angles for each activity. PLOS ONE has no limitation on the number of figures. However, I would consider how to standardize the data so that information for the same activity from multiple subjects could be placed on the same figure. Likewise, it is possible to do multipanel figures – for example – showing multiple ranges of motion across multiple subjects in a single figure. The point of this is to present a convincing case with supporting materials that your results are as advertised.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: PONE-D-20-33330_reviewerComments.pdf

Click here for additional data file.

8 Mar 2021

Thank you for the positive review and constructive comments. Below, we highlight the revisions made to enhance the manuscript. The information stated below has also been attached as a file named: "Response to reviewers".

Pg 3, row 52.

Replace method with methods.

This has been revised. See pg 3, row 52.

Pg 3, row 53.

Replace up to < with less than.

This has been revised. See pg 3, row 53 to 54.

Pg 3, row 60.

Insert comma.

This has been revised. See pg 3, row 60.

Pg 3, row 64.

Replace its with their.

This has been revised. See pg 3, row 64.

Pg 4, row 72-77

Comment: Although true, most systems have enough cameras that this is not a problem… For most lab collections obstruction is not an issue. Is there a specific application you have in mind where it is?

The purpose of the study was to investigate and potentially provide evidence for researchers that the PNS could be a viable tool to collect motion capture data in a more representative environment outside of a lab setting. This is especially so when it is not feasible to set up an optical based system. For example in situations where there is a need to analyse movement over a larger area, if an optical based system is used, a greater number of cameras would be required to cover this increase in volume. In this case, the PNS could be used to overcome the limitations of the lab space and the number of cameras needed.

Pg 4, row 89-90.

Comment: Is this compared to earlier IMUs? What’s the accuracy? You cited the accuracy of the optical mocap systems earlier.

Yes, this is in comparison with earlier IMUs. As the accuracy of the sensors are highly dependent on the sensor specifications, software algorithms, movement type and joint angle analysed there is a large range for this value. Therefore, no exact value was indicated in this section. This was highlighted on pg 5, row 106 to 109.

Pg 6, row 124.

Comment: Uncommon word, bias is probably preferable. This has been revised. See pg 6, row 124.

Pg 6, row 133.

Insert comma.

This has been revised. See pg 6, row 133.

Pg 6, row 138.

Comment: Be consistent with the use of PNS.

This has been revised. Has been changed to PNS for these instances:

Pg 6, row 137; pg 8, row 175: pg 9, row 180; pg 11, row 206.

Pg 7, row 149.

Comment: Spell out the number.

This has been revised. See pg 7, row 149.

Pg 7, row 152.

Change illnesses to illness.

This has been revised. See pg 7, row 152.

Pg 7, row 152.

Change diseases to disease.

This has been revised. See pg 7, row 153.

Pg 7, row 159.

Change from the most widely used to a widely used system?

This has been revised. See pg 7 row 159.

Pg 7, row 160.

Comment: Was the statement “The VICON optoelectronic motion analysis system was chosen as it is a widely used system and is considered as the current laboratory gold-standard with a high accuracy during dynamic trial.” the result of Topley & Richards (2020)?

Topley & Richards (2020) did not specifically mention that the VICON was a widely used system or that it was considered as the current lab gold standard. It did indicate the high accuracy of the VICON and other camera systems, however testing was conducted using the surface of a rigid aluminum arm that rotates. Therefore, this reference was not added at the end of the sentence since it did not support the statement.

Pg 7, row 161.

Spell out the number in the sentence.

This has been revised. See pg 7, row 161.

Pg 7, row 163.

Comment: Does the suit increases or decrease the soft tissue artifact from the markers? Is this a source of uncertainty?

The amount of error resulting from this was not accounted for in the current study. Therefore, It has been included as a limitation, see pg 24, row 374 to 377.

Pg 8, row 164.

Comment: Does the USB connection affect peoples’ movement as old EMG system wires used to? The wires connecting between each sensor uses a coiled design, so it allows participants to stretch and move through the range of movement freely when necessary but at the same time is still neatly kept which prevents any excessive wiring from dangling around to potentially impede movement. The system connects through a single wire to the laptop and this wire can be placed to face different directions and be shifted out of the way to accommodate various movements.

Pg 10, row 190.

Change Natrick to Natick.

This has been revised. See pg 10, row 190.

Pg 10, 202.

Comment: Reference? Is this Cohen?

The citation has been added. See pg 10, row 203.

Pg 11 row, 206.

Comment: Change X to X(t) and Y to Y(t). Indicate that these are time dependent and that you are summing over time.

This has been revised. See pg 11, row 206 to 207.

Pg 12, row 227.

Comment: Distance wrist shot shoulder also not a strong correlation.

This has been added. See pg 12, row 229 to 230.

Pg 12, row 238.

Comment: This is only for flexion/extension for the wrist shot.

This has been revised to state that the figure is only indicative for hip (flexion/extension) during distance wrist shot. See pg 12, row 241 to 243.

Pg 13, row 244.

Comment: How were data normalized?

Data was normalized using z-score transformation. Details have been added to pg 11, row 218 to 221.

Pg 22.

Comment: What about hip ab/ad?

The mean systematic bias for hip flexion/extension during distance walk was found to be -11.18°. This has been included. Hip abduction/adduction was -1.35 for stationary walk and -1.79° for distance walk. See pg 22, row 323 to 324, pg 323 row 328 to 329, and pg 24 row 383 to 384.

Pg 24.

Change & to and.

This has been revised. See pg 24, row 377 and pg 24, row 378.

Submitted filename: Response to reviewers.docx

Click here for additional data file.

26 Apr 2021

PONE-D-20-33330R1

Validation of the Perception Neuron system for full-body motion capture

PLOS ONE

Dear Dr. Choo,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

ACADEMIC EDITOR: As far as we see, you only responded to the reviewer 2's comments in the previous round, so the reviewer 1's comments remain un-responded. In this round, please respond to both reviewers' comments separately. Also, I would like to suggest to copy-paste the reviewers' comments as they are in the response letter, instead of summarizing their comments, such that we can see which comments you are responding.

==============================

Please submit your revised manuscript by Jun 10 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see:  http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at  https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Kei Masani

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: No

Reviewer #2: Partly

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Although this is a re submission, the authors have not addressed any of my comments from the original review. I am re-inserting them for completion.

----------------------------------

This is a paper where the authors have tried to validate the PNS IMU-based system kinematic outpout against that of the VICON optical capture system. The need for their work is that validation of the PNS system is currently limited. The significance of their work is that, if the PNS system kinematic output is validated, then one could conduct motion analysis studies by using a less expensive system which is easier to transport, it does not require a lot of time to set up and can be used outside of a laboratory environment in a real-world setting. However, besides the cost-related and portability aspects of the authors’ arguments in favor of the PNS system, there are multiple examples of optical capture systems used in a real-world setting, either for studying sports performance or occupational factors. Furthermore, the claim of the authors that set up and calibration is very lengthy for optical capture systems, is not in agreement with our experience, assuming knowledge of optical capture principles. Our volume calibration procedure takes approximately 1 minute consistently, and within the calibration volume one can perform any activity. The marker-model calibration, which is specific to the compuatational model used to analyze motion, is done once, also done in less than 1 minute (6 seconds to be exact) rather than re-calibrating every time a new activity is to be performed, which seems to be the caase with the PNS. However, the authors do claim to be calibrating both systems before each activity (page 8, line 167), which does not make sense with respect to the VICON system.

There are some major concerns before this manuscript is considered for publication. Some of them are:

1) It seems that the output from eah system was filtered in the exact same way (page 10, lines 195-197). How did the authors determine that the frequency content to be filtered from both systems for each activity was identical and, therefore, presumably they used the same filtering process and parameters? Data need to be provided towards this end. If this comment is not correct, then how each signal output from each device for each activity was filtered and why? This information is directly related to the quality of the data output being compared.

2) The authors have used the Plug-In-Gait marker set to compute the kinematics for the activities they studied (page 7, lines 161-162). The Plug-In-Gait marker set, however, is tied to several assumptions tied to the Plug-In-Gait computational model. It appears, however, that the authors did not use the Plug-In-Gait model to compute the kinematics of interest (page 10, lines 187-191). Why did the authors use this marker set model? The marker model itself does not ensure 3D segment definition by 3 marker non-colinearity per segment. Therefore, what assumptions did the authors use for the purposes of 3D segment definition (especially for the femur and tibia/fibula) and corresponding 3D motion computation? This information is also directly related to the quality of the data output being compared.

3) Observing Figure 2, it appears that pelvis is defined by the 2 ASIS markers and the 2 PSIS markers. These markers appear to have been placed on the shorts/trunks of the subject, presumably directly over the specific anatomical landmarks (it is not clear, but it appears that a mid-thigh marker is also placed on the subject’s trunks). How did the authors made certain that all these markers on the trunks remained over the anatomical landmarks of interest throughout the duration of all activities after each calibration? This information is also directly related to the quality of the data output being compared.

4) Observing Figure 2, it appears that the mid-thoracic marker (typically placed on the 10th thoracic spinous process) is placed neither on the 10th thoracic spinous process, nor in line with the thoracic spinous processes. This would create a trunk orientation offset. How has such an offset been accounted for during the kinematic computation of the shoulder motion, which, presumably is measured relative to the trunk?

5) Transverse plane data are very important, especially when considering injury prevention (in sports and occupational biomechanics) and in the clinical decision-making process. Given the ambitions of the authors for the PNS outlined in the introduction, the authors need to provide transverse data output comparisons between the 2 systems.

6) Observing Figures 1 and 2, it appears that PNS can provide ankle-related kinematics. Given the importance of ankle kinematics, especially for all lower extremity activities (in walking gait approximately 80% of the ambulatory power generation is related to the ankle), and given the desire of the authors to involve functional activities in this study, ankle kinematic output comparisons need to be made, even if only for the sagital plane and consistently with the study of Nuesch and Ross.

7) From Tables 3 and 4 it appears that for the same activity the r and RMSE increased and decreased, respectively, for most activities as a function of increasing speed. The authors suggest that this may be related to subjects performing these activities at self-selected speeds. Although performing an activity at a self-selected speed, indeed, has been found to decrease the within trial or within condition/speed variability, the mere increase in velocity results in higher kinematic variability in VICON even from a motion artifact point of view. Therefore, the authors’ argument of self-selected speed is ambiguous. On the contrary, this suggests that either PNS’s performance for activities that are static or quasi static is substandard or the earlier arguments regarding the filtering process need to be given serious consideration.

8) What is the normalization process implemented in the Bland altman plots?

Consequently, for all these major reasons I think this manuscript is not appropriate for publication in the current time and in the current form.

Reviewer #2: Thank you for taking the time to make the recommended changes. I have only a couple more points of clarification listed below. I want to reiterate that as motion capture systems are becoming more mobile, there is a need to evaluate them to extend their applicability and accessibility, and so I feel that your work is timely.

I originally had a number of comments on the rewrite for lines 68-77. In particular these were regarding the ease of setup and the occlusion of markers. I suggest that your comments below address my concerns and that this wording, or something similar should be in the actual paper not just in the response should the editors permit the increase in word count.

"The purpose of the study was to investigate and potentially provide evidence for researchers that the PNS could be a viable tool to collect motion capture data in a more representative environment outside of a lab setting."

"This is especially so when it is not feasible to set up an optical based system. For example in situations where there is a need to analyse movement over a larger area, if an optical based system is used, a greater number of cameras would be required to cover this increase in volume. In this case, the PNS could be used to overcome the limitations of the lab space and the number of cameras needed."

Regarding the instrumentation section – I did not see a frame rate for the IMU system, did I miss it?

Line 177 – the correct number of markers is 33, right?

Data analysis - Since the specific algorithm you used is not completely clear, nor is the method used

to synch the data, I would suggest publishing your code or else describing what was done here in more detail. It’s probably easier to just post the MATLAB code, I would think.

Data analysis - The Sovitzy-Golay filter may not be as well known in the biomechanics community as you think. My recollection is that it is a moving average method involving convolution of low order polynomials. I would describe this more clearly, especially since there are questions about your filter frequency which I don't think specifically applies with this type of smoothing. Since different polynomials may be fit to different portions of the data, it seems that there is non-uniform smoothing which can match the local roughness of the data better and that the “frequency” is polynomical dependent. This is discussed in the white-paper found here:

https://www.hpl.hp.com/techreports/2010/HPL-2010-109.pdf

Alternatively, I would cite a source describing these filters and their response, presuming that the other reviewer is content with this sort of explanation.

Table 3: Generally, correlations are done between series of data. In this case, I would expect a correlation calculated between VICON and IMU data per joint per activity and for each subject. The cited SD is then the SD of correlation coefficients across all subjects, right? Each correlation test has a p-value. What is the p-value shown here if multiple correlation tests were done to determine the mean correlation andits standard deviation? Please clarify.

Finally – I did not find your data in the NIE Data Repository – please post a URL!

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

20 Oct 2021

The complete response to reviewers has been attached in the document titled: Response to Reviewers.

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

14 Dec 2021

29 Dec 2021

Thank you for the points raised. Below, we highlight the revisions that have been made.

Reviewer 2 comments: "Thank you for the updates. However, your DOI (https://doi.org/10.25340/R4/T6W5RX) is returning a "DOI not found" from doi.org. Please check this and ensure that it is posted correctly in your final draft so that readers of the paper may access this information. This is my sole reason for selecting "Minor Revision" rather than "Accept" at this point."

Response: Apologies for that. The data can be accessed through this updated link: https://doi.org/10.25340/R4/IZKYZR

However, the link would only be active when the paper is published.

This has been updated in the submission steps as well, under the section ‘Describe where the data may be found in full sentences. If you are copying our sample text, replace any instances of XXX with the appropriate details.’.

Academic Reviewer comments: "Please add the description about the compression garments in the manuscript, which you made in the response to the reviewer 1 with the citation of Mills's paper."

Response: The description about the compression garments have been added into the Manuscript on page 9, line 194-198.

Submitted filename: Responses to Reviewers.docx

Click here for additional data file.

5 Jan 2022

Validation of the Perception Neuron system for full-body motion capture

PONE-D-20-33330R3

Dear Dr. Choo,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Kei Masani

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

11 Jan 2022

PONE-D-20-33330R3

Validation of the Perception Neuron system for full-body motion capture

Dear Dr. Choo:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Kei Masani

Academic Editor

PLOS ONE