Automatic Measurement of Postural Abnormalities With a Pose Estimation Algorithm in Parkinson's Disease.

OBJECTIVE: This study aims to develop an automated and objective tool to evaluate postural abnormalities in Parkinson's disease (PD) patients.
METHODS: We applied a deep learning-based pose-estimation algorithm to lateral photos of prospectively enrolled PD patients (n = 28). We automatically measured the anterior flexion angle (AFA) and dropped head angle (DHA), which were validated with conventional manual labeling methods.
RESULTS: The automatically measured DHA and AFA were in excellent agreement with manual labeling methods (intraclass correlation coefficient > 0.95) with mean bias equal to or less than 3 degrees.
CONCLUSION: The deep learning-based pose-estimation algorithm objectively measured postural abnormalities in PD patients.

Postural abnormalities are a common motor sign in patients with Parkinson’s disease (PD) [1]. In detail, patients with parkinsonism show involuntary forward flexion of the trunk (camptocormia) [2] or lateral flexion of the trunk (Pisa syndrome) [3] and, less frequently, forward neck flexion (dropped head or antecollis) [4]. However, the prevalence of postural abnormalities varies among studies based on differences in measuring methods and criteria [5]. Thus, consensus for reliable measurements of postural abnormalities is required [6].

Previously, abnormal angles were measured by hand or using a wall goniometer [2]. Recently, methods using photos of patients have been widely used [5-7]. For camptocormia in PD patients, methods using the manually labeled positions of L5, C7 spinous process and lateral malleolus of the ankle [6,7] or methods using the greater trochanter of the femur and the acromion have been proposed [5]. Methods for measuring forward neck flexion have been proposed using the acoustic canal, acromion and greater trochanter of the femur [5]. Additionally, the concept of upper camptocormia has been proposed using L5, C7 spinous processes and the fulcrum [6]. There are computer applications that can calculate the postural angle with patient photographs [6]. However, these methods require manual labeling of the positions of interest and thus requires training and effort. Furthermore, it is sometimes ambiguous to mark the joint position (L5, C7, greater trochanter or acromion) in patients with clothes on. Therefore, manual labeling methods might be in part subjective and result in interrater variability [6].

Recent advances in computer vision technology enable automatic labeling of the joint positions of humans in images [8]. From this perspective, we aimed to develop an application that can automatically measure the angles related to postural abnormalities without any attached markers or manual labeling.

MATERIALS & METHODS

Patients and imaging protocol

We prospectively enrolled PD patients with camptocormia who visited the movement disorder clinic at Seoul National University Hospital from June to December 2020. The diagnosis of camptocormia was determined by neurological examination by a movement disorder expert (BJ) without a detailed definition of flexed angle. The patients were clinically diagnosed with PD based on the UK Brain Bank criteria [9]. We excluded patients who were unable to stand unassisted or wore clothes that covered the body shape. We included participants who could simply remove the clothes covering the body shape (e.g., an outer jacket) but excluded those in which the clothes could not be easily changed in the outpatient clinic (e.g., long skirt or loosened sweater).

The patients were asked to stand relaxed in front of a wall at the clinic. We took lateral pictures of the patients with a smartphone camera (iPhone 6s, Apple, Cupertino, CA, USA) The lens was located in a position parallel to the patient’s hip (Figure 1). The vertical reference (vertical line by the door) was used to take pictures as perpendicular to the floor as possible. For validation, we marked the position of the left great trochanter and the left acromion with a detachable sticker (3M, Saint Paul, MN, USA) after manual palpation of the joint (Figure 1D-F). In cases where the patient’s forearm or hand occluded the position of the hip joint, we took an additional picture of the patients as much as possible, with hands cleared from the hip joint while standing in the same posture (Supplementary Figure 1 in the online-only Data Supplement). All participants gave written informed consent, and the institutional review board of Seoul National University Hospital approved this study (1908-175-1059, 2106-099-1228).

Figure 1.

Automatic and manual measurements of the anterior flexion angle (AFA) and dropped head angle (DHA). A: Lateral photos of the patients standing in front of a wall with a vertical reference within the frame. B: The 2D positions of the left ear, acromion, great trochanter and ankle were automatically labeled (labeled with red dots and connected with red line). C: The DHA and AFA were calculated based on the position of the ear, acromion, greater trochanter (labeled with red dots and connected with red line) and vertical line (green line). The calculated angle was noted as orange double-headed arrows. D: The same photograph from Figure 1A. The locations of the acromion and the hip joint were marked with detachable stickers (annotated with red circle). E: The acromion and greater trochanter were manually labeled based on the position of the detachable sticker (labeled with light blue dots and lines). The location of the left acoustic canal was also manually labeled (light blue dots and lines). F: The DHA and AFA were calculated based on the position of the ear, acromion, greater trochanter (labeled with light blue dots and lines) and vertical line (green line). The calculated angle was noted as orange double-headed arrows. G: The same photograph from Figure 1A shows a vertical reference (light blue arrow). H: Binary transformed image with edge detection algorithm (presented at Materials & Methods). Automatically detected vertical reference (blue line) and degree of the slope are shown. I: The AFA (annotated with orange double-headed arrow) calculated with the adjusted vertical reference (shown in blue line). The vertical line of the photo is shown as green line. The AFA was calculated based on the position of the acromion, greater trochanter (labeled with red dots and lines) and vertical reference line (the blue line moved from the original location annotated with a black double-headed arrow).

Automatic joint estimation and vertical reference detection

For the automatic measurements, we used an open-source deep-learning-based position-tracking algorithm (Openpose [8]). The algorithm automatically detects 25 joints (nose and bilateral eye, ear, shoulder, elbow, wrist, hip, knee, ankle, big toe, little toe and heel) without any attached markers. Then, we subtracted the 2D position of the left ear, shoulder, hip and ankle (Figure 1). To detect the angle of the vertical reference (contour of the door) within the frame, the image was transformed to the binary image, and the vertical edge within the image was detected with the Canny method followed by the Hough transform (Figure 1G-I). The vertical line with the longest length was chosen and was validated (JHS) by visual inspection. We calculated the degree of slope and calculated the Anterior flexion angle (AFA) with an adjusted vertical reference (Figure 1I).

Measurement of postural angles with the pose-estimation method and manual labeling method

The AFA was defined as the angle between the vertical reference and the lines connecting the hip joint and shoulder joint. The dropped head angle (DHA) was defined as the angle between the lines connecting the hip-acromion and acromion-ear (Figure 1) [5,10]. Additionally, we used the position of the great trochanter, acromion and lateral malleolus of the ankle to calculate the AFA (Supplementary Figure 2 in the online-only Data Supplement) [6].

The manual measurement of AFA and DHA was performed with CamptoApp [6] (Department of Neurology, UKSH, Kiel University, Kiel, Germany) by a doctor (KAW) who was blinded to any clinical information or automatic measurement of camptocormia. For the manual measurements, the positions of the acromion and great trochanter were marked at the position of the detachable stickers in the photograph (Figure 1).

Statistical analysis

The intraclass correlation coefficient (ICC) was used to validate the image-based automatic detection of the camptocormia angle. We used the two-way mixed-effects model with the relationship defined as absolute agreement [11]. All analyses were performed using MATLAB 2020a (Mathworks, Natick, MA, USA).

RESULTS

Twenty-eight PD patients were prospectively enrolled. Male patients comprised 39.3% (11/28) of the participants. The mean age (standard deviation) of the participants was 59.0 (6.2) years, with a mean disease duration (standard deviation) of 13.9 (7.2) years. The mean levodopa equivalent dose (standard deviation) was 785.96 (373.46).

Examples of automatic and manual measurements of the AFA and DHA are shown in Figure 1A and B, respectively. The mean (standard deviation) anterior AFA and DHA with the manual measurements were 10.57 (9.61) and 21.45 (19.99) degrees, respectively. The ICC [95% confidence interval] was 0.971 [0.889–0.989] and 0.953 [0.895–0.979] with mean (standard deviation) biases of 1.85 (2.87) and 3.00 (6.69) degrees with the AFA and DHA, respectively. Overall, the performance of the algorithm was in excellent agreement with the manual labeling method (ICC > 0.95), and the mean bias was equal to or less than 3 degrees. The bias and standard deviation are also shown with the Bland-Altman plot (Figure 2).

Figure 2.

Scatter plot and Bland-Altman plot of the anterior flexion angle (AFA) and dropped head angle (DHA). A: Scatter plot showing the mean AFA (degree) of manual (x-axis) and automatic (y-axis) measurements. B: The X-axis in the figure denotes the mean value of the two measurements (manual vs. automatic), and the y-axis in the figure (Δ) is the difference between the two measurements of the mean AFA. The mean (plain line) and 1.96 standard deviations of the bias (dotted line) are denoted as horizontal lines. C, D: Scatter plot (C) and Bland-Altman plot (D) for DHA. SD, standard deviation.

We automatically detected the vertical reference within the image, calculated the slope and adjusted the AFA (Figure 1G-I). The mean (standard deviation) bias in the vertical reference was 0.91 (1.26) degrees. We also calculated the AFA with the malleolar-hip joint line (Supplementary Figure 2 in the online-only Data Supplement), and the calculated mean AFA (standard deviation) was 8.69 (12.59) degrees.

In 7 patients whose hip joint was occluded by the forearm, additional photos of the forearm cleared from the hip joint were analyzed (Supplementary Figure 1 in the online-only Data Supplement). We compared the calculated AHA and DHA from two situations (occluded hip joint vs. visible hip joint). The mean (standard deviation) differences in the AHA and DHA were 0.80 (1.94) and 3.59 (5.11) degrees, respectively.

DISCUSSION

In this study, we showed that the deep-learning-based pose-estimation algorithm can reliably measure abnormal postural angles in PD patients (ICC > 0.95). The algorithm is fully automated and does not require attached markers. Additionally, the algorithm can adjust the bias of the vertical reference. Thus, the algorithm has the potential to be widely used in research and clinical practice for a reliable measure of abnormal postures in PD. We have developed this algorithm as an application that can be downloaded online (https://github.com/LuckyFace/Posture-analyzer).

We measured the AFA and DHA using the greater trochanter, acromion and acoustic canal [5,10]. The AFA measured with this method uses a vertical line that may be slightly tilted while taking photos [6]. To solve this issue, we showed that we can measure the AFA with an adjusted vertical axis by the automatic detection of the slope of the vertical reference within the frame (Figure 1G-I). Of note is that the mean vertical bias was approximately 1 degree, which suggests that with an adequate protocol, the vertical bias can be minimal. Additionally, we used the ankle joint to measure the AFA (Supplementary Figure 2 in the online-only Data Supplement) [6]. However, the previous methods using the malleolar method used C7, L5 of the spine and the lateral malleolus and thus did not directly correspond to our methods using the greater trochanter of the femur and the acromion. Measuring the camptocormia angle using the greater trochanter, acromion and lateral malleolus of the ankle has not been validated in PD patients and thus requires further investigation.

The automatic algorithm could measure the AFA in the case of hip joint occlusion by the forearm. The bias between the two situations (occluded hip joint vs. visible hip joint) was within 5 degrees.

There are several limitations of the study. First, our study did not enroll age-matched healthy controls or patients without postural abnormalities. Second, we did not consider whether the patients were in medication on and off states when taking photographs. Third, the methods using the lateral malleolus were not validated with the manual labeling method. As the initial design of the study was to validate the automated algorithm using the joint positions of the acoustic canal, acromion and greater trochanter, the lateral malleolus was not visible in most of the participants (24/28). Fourth, our method cannot measure upper camptocormia as previously reported, as the pose-estimation algorithm cannot detect the fulcrum [6]. Two participants among the 28 patients showed abnormal upper truncal flexion based on previously proposed criteria [7]. Furthermore, our algorithm could not calculate camptocormia angles using C7 or L5 [6,7], as the algorithm cannot detect the C7 or L5 position. A future revision of the pose-estimation technique would enable the expansion of the current algorithm. Last, we did not evaluate lateral flexion of the patients [3,12], but as the pose-estimation algorithm labels bilateral shoulders and hip joints, we expect that the algorithm would successfully measure the lateral flexion angle, which should be evaluated in the future.