Predictive Value of Verbatim Parkinson's Disease Patient-Reported Symptoms of Postural Instability and Falling.

BACKGROUND: Postural instability is an intractable sign of Parkinson's disease, associated with poor disease prognosis, fall risk, and decreased quality of life.
OBJECTIVE: 1) Characterize verbatim reports of postural instability and associated symptoms (gait disorder, balance, falling, freezing, and posture), 2) compare reports with responses to three pre-specified questions from Part II of the Movement Disorder Society Unified Parkinson Disease Rating Scale (MDS-UPDRS), and 3) examine postural instability symptoms and MDS-UPDRS responses as predictors of future falls.
METHODS: Fox Insight research participants reported their problems attributed to PD in their own words using the Parkinson Disease Patient Reports of Problems (PD-PROP). Natural language processing, clinical curation, and data mining techniques were applied to classify text into problem domains and clinically-curated symptoms. Baseline postural instability symptoms were mapped to MDS-UPDRS questions 2.11-2.13. T-tests and chi-square tests were used to compare postural instability reporters and non-reporters, and Cochran-Armitage trend tests were used to evaluate associations between PD-PROP and MDS-UPDRS responses; survival methods were utilized to evaluate the predictive utility of PD-PROP and MDS-UPDRS responses in time-to-fall analyses.
RESULTS: Of participants within 10 years of PD diagnosis, 9,692 (56.0%) reported postural instability symptoms referable to gait unsteadiness, balance, falling, freezing, or posture at baseline. Postural instability symptoms were significantly associated with patient-reported measures from the MDS-UPDRS questions. Balance problems reported on PD-PROP and MDS-UPDRS 2.11-2.13 measures were predictive of future falls.
CONCLUSION: Verbatim-reported problems captured by the PD-PROP and categorized by natural language processing and clinical curation and MDS-UPDRS responses predicted falls. The PD-PROP output was more granular than, and as informative as, the categorical responses.

INTRODUCTION

Of the cardinal motor signs of Parkinson’s disease (PD), postural instability is the least responsive to dopaminergic therapies [1, 2]. Impairment of postural stability is further associated with poor disease prognosis, leading to increased risk of falls, decreased quality of life, and heightened morbidity and mortality [2-6]. The U.S. Preventive Services Task Force has placed special emphasis on fall prevention and clinician-involvement in preventive efforts for older adults at increased fall risk [7, 8]. Direct patient reporting may be beneficial for earlier identification of people at risk of falls, particularly in the absence of in-person clinical examination.

Fox Insight (https://foxinsight.michaeljfox.org) is an online, observational, longitudinal clinical study sponsored by the Michael J. Fox Foundation for Parkinson’s Research (MJFF) [9]. Consenting research participants were recruited online and invited to complete initial and follow-up study visits, comprising a series of questionnaires and modules. Within the Fox Insight study is the Parkinson Disease Patient Reports of Problems (PD-PROP) research module developed by Grey Matter Technologies. PD participants reply to two open-ended questions: 1) What is the most bothersome problem for you due to your Parkinson’s disease? 2) In what way does this problem bother you by affecting your everyday functioning or ability to accomplish what needs to be done? Respondents report up to their 5th most bothersome problem and assign a severity score to each problem, ranging on a scale from 0 (not particularly severe) to 3 (very severe). To accrue longitudinal data, participants were invited at about six-month intervals to repeat the PD-PROP module and selected categorical questions [9]. This study aimed to characterize postural instability responses using the PD-PROP, compare the PD-PROP responses to postural instability-related ordinal responses provided by the same subjects at the same study visit through the MDS-UPDRS Part II questions 2.11, 2.12, and 2.13, and determine if the PD-PROP could predict future falls.

MATERIALS AND METHODS

Participant selection

Fox Insight study participants who had completed a baseline PD-PROP were identified (n = 23,659, data cut February 2020), and 20,246 participants were selected within the first 10 years since PD diagnosis, age < 100 years, and age at diagnosis > 15 years. For the 18,800 participants who completed both PD-PROP and the three MDS-UPDRS Part II questions (2.11, 2.12, 2.13), 17,297 (92%) participants were selected who completed the questions unassisted. For the time-to-falls analysis, the data were further subset to participants who had a minimum of two PD-PROP ‘visits’, spanning at least 24 months, and who completed MDS-UPDRS Part II and fall reporting data (n = 2,237). Participants who reported falling at baseline in the Non-Motor Symptoms Questionnaire (NMSQ) were further filtered out, leaving n = 1,557 for analysis. Participant selection is summarized in Supplementary Figure 1.

Clinical curation

Keyboard-entered verbatim reports from participants were preprocessed using standard natural language processing and data analytics techniques previously described [10-15] to generate a subset of the PROP dataset for curation and classification of the reports into PD clinical domains and symptoms. The verbatim text was reviewed and curated by three clinicians, two non-clinical researchers, and a PD patient ‘experience-expert’ who independently reviewed randomly selected samples of text and in turn categorized and labelled the concatenated problem and functional consequence pair under umbrella domains of ‘Autonomic Dysfunction’, ‘Bradykinesia’, ‘Cognition’, ‘Fatigue’, ‘Mood’, ‘Pain’, ‘Postural Instability’, ‘Rigidity’, ‘Sleep’, and ‘Tremor’. Within the domains, specific clinically relevant symptoms were similarly curated and categorized. The curated dataset was then used to classify the entire PROP cohort at baseline and across multiple longitudinal visits enabled by a proprietary algorithm using an Apache Lucene [16] phrase query approach

For the postural instability domain, five symptom categories of gait disorder, balance, falling, freezing and posture were identified. In order to evaluate the efficiency of the data mining and data extraction process as well as to calculate the specificity and recall of the data extraction algorithm, curators were provided 50 random samples that the machine classified as true positives, and 50 random samples the machine classified as true negatives. The curators were then asked to classify the verbatim based on their knowledge and experience, and the results were then compared back to the machine extracted results.

Categorical questions

The MDS-UPDRS Part II questions in the Fox Insight ‘Your Movement Experiences’ questionnaire rate 13 movement-related activities of daily living on a 5-point scale (normal, slight, mild, moderate, severe). Three MDS-UPDRS Part II questions were pre-selected as the most relevant to the postural instability domain and its symptoms: 2.11 (Over the past week, have you usually had trouble getting out of a bed, a car seat, or a deep chair?), 2.12 (Over the past week, have you usually had problems with balance and walking?), and 2.13 (Over the past week, on your usual day when walking, do you suddenly stop or freeze as if your feet are stuck to the floor?). Time-to-falling was analyzed using the no-yes responses to the NMSQ question 21 (Fox Insight Non Movement Experiences Question “Have you experienced falling in the last month?”.)

Statistical analysis

Demographic data obtained at Fox Insight enrollment were compared between participants who reported postural instability symptoms and those who did not, using a two-tailed, t-test with Welch’s correction or chi-square test, as appropriate. The association between MDS-UPDRS Part II items and reports of the postural instability domain and its symptoms was determined by two-sided Cochran-Armitage trend test. These analyses were performed using GraphPad Prism 8.4. Figures were generated with either GraphPad Prism 8.4 or UpSetR.

Time-to-falling analyses

Participants with PROP data spanning at least 24 months, including at least two corresponding NMSQ visits and three MDS-UPDRS II questions at the baseline PROP were selected for the time-to-falling analysis. Separate Kaplan-Meier survivorship analyses of time-to-falling were performed for each baseline PROP symptom stratified by priority and for each MDS-UPDRS question stratified by impairment severity; statistical significance of the priority and impairment ratings was determined using unadjusted Cox proportional hazards models with the not-reported (PD-PROP) and unimpaired (MDS-UPDRS) groups omitted. A multi-variate Cox proportional hazards model to predict time-to-falling assessed the predictive value of the baseline PROP and MDS-UPDRS variables; baseline age (dichotomized at 65), sex and participant-reported years from PD diagnosis (< 3, 3–5, > 5) were included in the models. Five PD-PROP baseline variables were included in the models as reported or not reported. The baseline MDS-UPDRS 2.11–2.13 variables were dichotomized as 0 (no impairment) vs 1–4 (any impairment). The analysis included a selection process to identify the most predictive variables, but age, sex and years from diagnosis were specified to remain in the model. These analyses were performed in SAS (9.4).

Data availability statement

Data reported in this manuscript are available to qualified researchers through the Fox Insight Data Exploration Network (Fox DEN) at https://foxden.michaeljfox.org.

RESULTS

Curation validation

Independent curator classification of the postural instability domain and its symptoms showed high validation characteristics. Postural instability symptoms averaged 95.1%specificity and 99.3%recall/sensitivity (Supplementary Table 1).

Cohort characteristics

A total of 17,297 Fox Insight participants are included in these analyses. Demographic features are detailed in Table 1. The 9,692 (56.0%) participants who reported problems associated with postural instability at baseline were significantly older, had a longer time from diagnosis and were more likely to be female than those who were not. Symptom reports in the postural instability domain were; gait disorder (n = 6,800, 70.2%of those who reported postural instability), balance (n = 4,213, 43.5%), falling (n = 2,711, 28.0%), freezing (n = 762, 7.9%), and posture (n = 489, 5.0%). Figure 1 shows whether symptoms were reported singly or in combination with other symptoms. Priority and severity of each postural instability symptom are detailed in Table 2. Symptoms were reported most often with a severity of 1 or 2 on a least-most scale of 0–3.

Table 1

Fox Insight participant demographic features at baseline PD-PROP visit

	Overall (n = 17,297)	Postural instability reporters (n = 9,692)	Postural instability non-reporters (n = 7,605)	p
Age	65.2±9.7 (22–97)	66.1±9.6 (23–97)	64.1±9.8 (22–92)	< 0.0001
Years with PD	3.3±2.8 (0–10)	3.5±2.9 (0–10)	3.0±2.7 (0–10)	< 0.0001
Female (%)	8,213 (47.5%)	4,742 (48.9%)	3,471 (45.6%)	< 0.0001
Male (%)	9,084 (52.5%)	4,950 (51.1%)	4,134 (54.4%)

Values shown are the mean, standard deviation, and range. The p-value was determined between postural instability reporters and non-reporters by two-tailed t test with Welch’s correction for age and years since diagnosis and chi-square test for sex.

Fig. 1

Baseline reporting of postural instability symptoms. Dots denote whether symptoms were reported alone (single dot) or in combination. Horizontal bars display total symptom reporting. Gait disorder, reported alone, was the most commonly reported symptom (n = 3,545) followed by gait disorder and balance co-reporting (n = 1,351).

Table 2

Priority and severity of postural instability symptoms at the baseline PD-PROP

PD -PROP	Priority					Severity
	1st	2nd	3rd	4th	5th	0	1	2	3
Postural instability domain (n = 9,692)	5143, 53.1%	2318, 23.9%	1213, 12.5%	676, 7.0%	342, 3.5%	337, 3.5%	3100, 32.0%	4609, 47.6%	1646, 17.0%
Gait disorder (n = 6,800)	3505, 36.2%	1634, 16.9%	874, 9.0%	487, 5.0%	300, 3.1%	229, 2.4%	2044, 21.1%	3288, 33.9%	1239, 12.8%
Balance (n = 4,213)	1899, 19.6%	1010, 10.4%	658, 6.8%	414, 4.3%	232, 2.4%	118, 1.2%	1465, 15.1%	1963, 20.3%	667, 6.9%
Falling (n = 2,711)	1049, 10.8%	741, 7.6%	441, 4.6%	277, 2.9%	203, 2.1%	66, 0.7%	764, 7.9%	1361, 14.0%	520, 5.4%
Freezing (n = 762)	455, 4.7%	120, 1.2%	94, 1.0%	50, 0.5%	43, 0.4%	9, 0.1%	198, 2.0%	369, 3.8%	186, 1.9%
Posture (n = 489)	129, 1.3%	118, 1.2%	105, 1.1%	82, 0.8%	55, 0.6%	43, 0.4%	167, 1.7%	204, 2.1%	75, 0.8%

Priority and severity of postural instability (overall) and associated symptoms among 9,692 postural instability-reporting participants at their baseline PD-PROP completion. Postural instability and its symptoms were most frequently reported as a first most bothersome problem. Severity was predominantly rated as 1 or 2 on a least-most scale of 0–3.

PD-PROP comparison to MDS-UPDRS Part II

The domain of postural instability and its symptom reports were consistently and significantly associated with all three MDS questions 2.11–2.13 (p < 0.0001), except for the posture symptom (2.11 p = 0.0065, 2.12 p = 0.3915, 2.13 p = 0.1283). To understand how the baseline PD-PROP postural instability reports in these subjects corresponded to their MDS-UPDRS 2.11–2.13 responses, heat maps of PD-PROP and MDS-UPDRS 2.11–2.13 responses (Fig. 2 [postural instability domain], Supplementary Figure 2 [postural instability symptoms]) were generated, with a darker color indicating a higher number of participants’ intersecting responses. The highest proportions of postural instability responses fell at the intersection of “not reported” on the PD-PROP and “normal” or “slight” on the MDS-UPDRS questions. Some first priority PROP problems were designated as “slight” or even “normal” on the MDS-UPDRS.

Fig. 2

Heat maps of baseline postural instability reporting and responses to MDS-UPDRS 2.11 (rising up), 2.12 (balance & walking), and 2.13 (freezing) questions. Heat maps show participant baseline postural instability priority and corresponding MDS-UPDRS 2.11–2.13 responses, as a percentage of total participates n = 17,297. A darker color indicates a higher percentage of participants. The strongest intersections were at normal (MDS-UPDRS) and not reported (PD-PROP).

Time-to-falling analysis

Data from 1,557 participants were followed longitudinally for time-to-falling analysis. The Kaplan-Meier plots for balance stratified by priority (p = 0.0392 for ranking of priority) and for MDS-UPDRS question 2.12 stratified by impairment severity (p < 0.0001) are shown in Fig. 3. The order of the priority rankings was not significant for the other four symptoms; the order of the severity rankings was also significant for question 2.11 (p < 0.0001) but not for question 2.13. Using a Cox proportional hazards model to predict time-to-falling (responded ‘Yes’ to NMSQ Question 21), the most predictive variables comprising the final adjusted model were reporting problems with balance on the PD-PROP, and impairment on MDS-UPDRS questions 2.11–2.13 (Table 3). Age, female sex, and > 5 years since diagnosis were also significant predictors.

Fig. 3

Kaplan-Meier analyses of time to new onset falls for PD-PROP balance (A) and MDS-UPDRS 2.12 (balance and walking) (B). N = 1,557 participants. The fall outcome variable was NMSQ Question 21 (no-yes falling).

Table 3

Time-to-falling prediction

Variable	Hazard ratio, 95%confidence interval	p
Age (> 65)	1.419 (1.143, 1.761)	0.0015
Sex (female)	1.427 (1.158, 1.758)	0.0008
Years since diagnosis (3–5) (compared to < 3)	0.930 (0.714, 1.211)	0.5896
Years since diagnosis (> 5) (compared to < 3)	1.262 (0.984, 1.617)	0.0664
PD-PROP: balance (reported)	1.669 (1.323, 2.106)	< 0.0001
MDS-UPDRS 2.11 (1–4)	1.359 (1.041, 1.774)	0.0240
MDS-UPDRS 2.12 (1–4)	1.902 (1.435, 2.521)	< 0.0001
MDS-UPDRS 2.13 (1–4)	1.455 (1.133, 1.870)	0.0033

Cox proportional hazards model for time-to-falling prediction, n = 1,557. The overall 2 degree of freedom p-value for years since diagnosis was p = 0.0517. Balance impairment (PD-PROP) and MDS-UPDRS 2.11 (rising up), 2.12 (balance & walking), 2.13 (freezing) impairment were significant predictors.

DISCUSSION

Patient-reported verbatim replies to open-ended questions, using validated clinical curation and natural language processing, demonstrated that symptoms in the postural instability domain correlated well with corresponding questions on Part II of the MDS-UPDRS. Both are predictors of future falls. The PD-PROP balance symptom contributed independently to the risk of falling. The finding that female participants with PD report falling more frequently than male participants has been reported among the predictors of progression to falling in PD [17, 18]. The explanation for this observation may be due to greater reporting tendencies among female participants; however, the confirmation of this sex difference in Fox Insight by both PD-PROP and UPDRS part II deserves further exploration.

PD genotypes have also been shown to contribute to the onset of postural instability [19], prompting future research to examine the effects of genotype on patient-report PD-PROP symptoms of postural instability. As the Fox Insight study progresses and more longitudinal data are available, clinically-meaningful events beyond 24 months will be examined. Additional ‘deep phenotyping’ efforts of PD are also underway, to relate what patients say in the PD-PROP to what they do using digital monitoring measures [20].

Both the MDS-UPDRS Part II and PD-PROP participant responses represent categorical measures and quantitative output. The PD-PROP requires more time and effort at the front end of patient reporting, but once the curation and classification have been completed and the data dictionary compiled, this approach of analyzing verbatim text can be applied more widely and produce more granular and richer data. The MDS-UPDRS question 2.12 is indeed a strong predictor of falls, but otherwise provides limited information. The key methodological difference is that the MDS-UPDRS Part II categorization is locked within a 0–4 ordinal scale while the PD-PROP classification is based on verbatim replies to open-ended questions about reported problems and their functional consequences, and elaborated further by self-reported priority of the problem (from most bothersome) and the severity of the problem. Further, PD-PROP has the additional advantages of 1) being electronically transcribed, eliminating the need to transcribe from paper to electronic formats, and improving data accuracy [21], 2) capturing patient priorities and functional consequences, and 3) being suited for re-curation (e.g., separating falling from fear of falling).

In the longitudinal context of PD randomized controlled trials, the MDS-UPDRS Part II has shown little change over time in early PD and so may not be ideal to quantify disease progression [22, 23]. The PD-PROP may be more informative but has not yet been applied to longitudinal interventional trials. Natural language processing and clinical curation could also be applied more widely to clinical care in the form of analyzable patient or clinician text to open-ended questions.

The MDS-UPDRS Part II and PD-PROP are both predictive of falls and therefore have value in foretelling the occurrence of a serious outcome, such as falls that portend morbidity and mortality [2-6]. Their predictive value for falling could inform the design of clinical trials aimed at forestalling the occurrence of this clinically relevant outcome, a finding that would be tantamount to disease modification since existing treatments have not been demonstrated to exert therapeutic benefit on the clinical domain of postural instability, the symptom of balance, or the occurrence of falling,

The Fox Insight longitudinal data, accessible to researchers via Fox DEN (https://foxden.michaeljfox.org), are entirely reported by the participant (or care partner), without without investigator assessment or input. In the current COVID-19 era of burgeoning telemedicine and virtual clinical trials, there is lingering concern about examination-related limitations of sole patient reporting, particularly with regard to PD signs of rigidity and postural reflexes [24]. Regardless, patient-reported problems, functional consequences and derived symptoms remain relevant for clinical research and care. The case can be made that what the patient says is as or more important than what the clinician hears or observes [25, 26].

Click here for additional data file.