The Practical Application of Clinical Prediction Rules: A Commentary Using Case Examples in Surgical Patients with Degenerative Cervical Myelopathy.

Study Design Commentary. Objective This commentary aims to discuss the practical applications of a clinical prediction rule (CPR) developed to predict functional status in patients undergoing surgery for the treatment of degenerative cervical myelopathy. Methods Clinical cases from the AOSpine CSM-North America study were used to illustrate the application of a prediction rule in a surgical setting and to highlight how this CPR can be used to ultimately enhance patient care. Results A CPR combines signs and symptoms, patient characteristics, and other predictive factors to estimate disease probability, treatment prognosis, or risk of complications. These tools can influence allocation of health care resources, inform clinical decision making, and guide the design of future research studies. In a surgical setting, CPRs can be used to (1) manage patients' expectations of outcome and, in turn, improve overall satisfaction; (2) facilitate shared decision making between patient and physician; (3) identify strategies to optimize surgical results; and (4) reduce heterogeneity of care and align surgeons' perceptions of outcome with objective evidence. Conclusions Valid and clinically-relevant CPRs have tremendous value in a surgical setting.

What Is a Clinical Prediction Rule?

A clinical prediction rule (CPR) combines signs and symptoms, patient demographics, and other predictive factors to estimate disease probability, treatment prognosis, or risk of complications.1 CPRs are valuable in the medical field because they may inform policy changes, influence clinical practice, and guide the design of future research studies. In terms of public policy, prediction models can help develop preventive strategies and outline protocols for the management of high-risk asymptomatic patients. In a clinical setting, CPRs are useful because they can (1) inform patients of their risk of disease development; (2) provide an evidence-based tool to help clinicians make decisions about screening, diagnostic testing, and therapeutic interventions (i.e., timing, type, and intensity); (3) facilitate shared decision making between patients and physicians; (4) help manage patients' expectations; (5) evaluate the cost-effectiveness of treatment by providing information on prognosis; and (6) reduce heterogeneity of practice across centers. From a research perspective, knowledge of important predictors can help design appropriate inclusion and stratification criteria for future studies and can identify relevant confounders to optimize robust statistical analyses.

The Acute Physiology and Chronic Health Evaluation II (APACHE II) and the model for end-stage liver disease are two examples of common CPRs used in a clinical setting.2 3 The APACHE II is an intensive care unit scoring system that incorporates 12 physiologic measurements, age, and medical history to assess disease severity and risk of mortality.2 This score is used to identify life-threatening physiologic problems and to guide the development of suitable treatment strategies. Furthermore, this knowledge can inform concerned family members about the risk of subsequent hospital death. On the hand, the model for end-stage liver disease score uses a patient's bilirubin and creatinine levels and the international normalized ratio from prothrombin time to evaluate disease severity, estimate the chance of survival, and prioritize recipients of liver transplants.3 These two examples demonstrate that CPRs can be used to identify high-risk patients, assess severity of disease, and provide prognostic information to patients and their families.

A Clinical Prediction Rule in a Surgical Setting

There is an increasing need for clinicians to accurately and objectively quantify a patient's likely surgical outcome. A CPR was recently developed to predict functional status in patients undergoing surgery for the treatment of degenerative cervical myelopathy (DCM).4 The primary objective of this commentary is to describe the clinical application of this CPR and, through the use of examples, explain how it can ultimately improve patient care.

DCM is a progressive, degenerative spine disease and the most common cause of spinal cord dysfunction in adults worldwide.5 6 The term degenerative cervical myelopathy was originally described by Nouri et al to encompass a wide range of pathologies, including cervical spondylotic myelopathy and ossification of the posterior longitudinal ligament.7 Current reports on the natural history have suggested that 20 to 62% of patients with symptomatic DCM will deteriorate over time if they are not treated surgically.8 Surgery is primarily recommended to halt disease progression and prevent the exacerbation of neurologic signs and symptoms. Furthermore, patients undergoing surgical decompression exhibit significant improvements in neurologic outcomes, functional status, and health-related quality of life.9 10 Although the majority of patients do improve postoperatively, some achieve better outcomes than others. Given these varying prognoses, it is critical to accurately predict how well each patient will fare after surgery and who is likely to benefit the most from intervention.

This CPR was developed to discriminate between patients with mild myelopathy at 1 year following surgery (modified Japanese Orthopaedic Association [mJOA] ≥16) and those with substantial residual neurologic impairment (mJOA < 16).4 A score of 16 on the mJOA translates to greater social independence, minimal neurologic signs and symptoms, and an ability to perform activities of daily living. The original CPR included six statistically significant clinical variables and one clinically relevant imaging parameter. Based on the model, patients were more likely to achieve a score of ≥16 if they had milder disease preoperatively, had a shorter duration of symptoms, did not smoke, were younger, did not have psychiatric disorders or impaired gait, and had a larger transverse area. Results from a recent systematic review, however, revealed that transverse area is not predictive of outcome; this imaging parameter was consequently removed from the model to increase its relevance in a clinical setting.11 12 The following equation (Eq. 1) summarizes this CPR:

P=e 1.59+(−0.81)/1+e 1.59+(−0.81)

where P is the probability of achieving an mJOA score ≥16, Ps is depression or bipolar disorder (1: absent, 2: present), mJOA0 is the preoperative myelopathy severity as evaluated by the mJOA (0 to 18), A is age in years, IG is impaired gait (1: present: 2: absent), S is smoking status (1: nonsmoker, 2: smoker), and DS is duration of symptoms (1: ≤3 months; 2: >3 but ≤6 months; 3: >6 but ≤12 months; 4: >12 but ≤24 months; 5: >24 months).

Validation of This Clinical Prediction Rule

The original CPR was developed using data on patients enrolled at spine care centers in Canada and the United States. As a result, this model truly reflects the patient demographics, disease causation, management strategies, and health care systems of North America. Unfortunately, several differences may exist across countries, cities, and centers that could decrease the generalizability of our model and render it invalid at sites outside of North America. Some of these include differences in (1) disease definitions and clinical presentation, (2) average demographics, (3) interpretation of predictors or assessment of outcome, (4) access to care, and (5) management strategies.

To assess external validity, we evaluated the discrimination and calibration of our CPR on a data set of surgical patients enrolled at centers across the world. Based on our results, the model displayed good predictive performance and adequate calibration in the international data set.12

Using Eq. 1 to Predict Outcome

The patient, a 57-year-old nonsmoking woman, presented with severe myelopathy (mJOA = 11) secondary to spondylosis, disk herniation, and hypertrophy of the ligamentum flavum. This patient complained of gait impairment, clumsy hands, and weakness and upon clinical examination displayed evidence of lower limb spasticity, hyperreflexia, broad-based unstable gait, and atrophied intrinsic hand muscles. Her preoperative duration of symptoms was 14 months. The patient also had a heart arrhythmia and moderate hypertension.

The variables relevant to the prediction equation are baseline mJOA, duration of symptoms, impaired gait, smoking status, psychiatric health, and age. Table 1 displays the numbers that must be entered into Eq. 1 to compute the probability of achieving a score ≥16. This probability was calculated as 48.95%.

Table 1

An example of how to use a clinical prediction equation to evaluate outcome

Predictor	Patient information	Entered into Eq. 1
Baseline mJOA	11	mJOA0 = 11
Duration of symptoms (mo)	14	DS = 4
Impaired gait	Present	IG = 1
Smoking status	Nonsmoker	S = 1
Psychiatric comorbidities	No depression or bipolar	Ps = 1
Age (y)	57	A = 57

Abbreviations: A, age in years; DS, duration of symptoms (1: ≤3 months; 2: >3, ≤6 months; 3: >6, ≤12 months; 4: >12, ≤24 months; 5: >24 months); IG, impaired gait (1: present, 2: absent); mJOA0, baseline modified Japanese Orthopaedic Association score (0–18); Ps, depression or bipolar disorder (1: absent, 2: present); S, smoking status (1: nonsmoker, 2: smoker).

P=e 1.59+(−0.81)(1)+(0.19)(1)+(−0.036)(57)+(0.91)(1)+(−0.69)(1)+(0.27)(4)/1+e 1.59+(−0.81)(1)+(0.19)(1)+(−0.036)(57)+(0.91)(1)+(−0.69)(1)+(0.27)(4)=49.0%

Clinical Application of This Clinical Prediction Rule

In a surgical population, a CPR can be used to manage patients' expectations, facilitate shared decision making, identify ways to optimize results, and standardize practice across centers. The following explanations and case examples highlight how this CPR can be used in these four ways to improve patient care.

Manage Patients' Expectations and Improve Overall Satisfaction

In the United States, recent legislative developments require clinicians to report patient satisfaction in their assessment of overall treatment outcomes.13 The Centers for Medicare and Medicaid Services (CMS), hospitals, and insurance providers have identified patient satisfaction as a major component and predictor of overall quality of care.13 As part of a CMS hospital inpatient value-based purchasing program, Medicare reimbursements are increasingly linked to patient satisfaction. Furthermore, as of 2013, CMS has provided “value-based” incentive payments to acute care hospitals in part based on results from patient satisfaction surveys.14

Patient satisfaction is not exclusively based on clinical outcomes and can be strongly influenced by several extraneous factors, including the politeness of the staff, cleanliness of the facilities, surgical wait times, and physician's communication skills. Furthermore, according to a study by Hamilton et al, the extent to which a patient's preoperative expectations are met is also a significant predictor of overall satisfaction of surgery.15

Expectations are “the yardstick by which our patients measure the course of recovery, occurrence of complications and the outcome.”16

Patients' expectations, however, are often influenced by anecdotal evidence provided by friends and family, information obtained from the Internet, and various news sources and past surgical experiences. The attending surgeon must use the consent discussion as an opportunity to outline risks and benefits of the procedure, provide accurate outcome information, and correctly manage patients' expectations. However, these expectations are also affected by how the patient understands the information provided by the surgeon and how he or she translates it into “actionable beliefs,”16 which further depends on several factors such as mental state, attitude, and individual or cultural values. As a result, patients may form different expectations than those the surgeon is attempting to communicate.

A quantitative tool used to predict surgical outcome can help a surgeon more effectively convey prognostic information and give the patient a better understanding of how he or she should expect to fare following intervention. Patients' expectations will therefore be more appropriately managed and, accordingly, overall satisfaction will likely improve.

Facilitate Shared Decision Making and Counsel Concerned Patients as to Potential Treatment Options

Predicting surgical outcome can also facilitate shared decision making between the patient and physician. An estimate of the risks and benefits must be presented to the patient in order for him or her to make informed decisions about the various treatment options. During the consent process, the discussion between the patient and physician should include a prediction of surgical outcome, the consequences of not undergoing surgery, and the probability of achieving social independence. This CPR can provide this information to both the surgeon and the patient and enable shared decision making.

Identify Strategies to Optimize Results

A CPR can also help clinicians identify strategies to optimize surgical results. We must first distinguish between the predictors that clinicians have control over from those that cannot be changed.

Age: A patient's age is unchangeable.

The other five factors are modifiable:

Duration of symptoms: A longer duration of symptoms is associated with a worst postoperative outcome. To improve the prognosis, clinicians can choose to operate earlier on patients with DCM rather than waiting for the disease to progress. This option, however, requires that primary care physicians accurately detect DCM at early disease stages, differentiate between it and mimicking diagnoses, and refer patients immediately for surgical consultation.17

Baseline severity score: Severer preoperative myelopathy is also predictive of a worse outcome. This point can be changed by earlier detection of DCM and by prioritizing consultation and surgery for these patients.

Smoking status: It is unclear why smokers have a worse postoperative outcome than nonsmokers. Previous studies have suggested that smokers have higher rates of nonfusion and wound infections than nonsmokers.18 We, however, speculated that smoking is a surrogate for lower socioeconomic status, poorer dietary choices, the presence of comorbidities, and less access to postsurgical care. Further research is required to confirm these hypotheses; however, until this research is done, surgeons should promote smoking cessation prior to surgery to optimize outcomes.

Psychiatric disorders: Patients with depression or bipolar disorder do not perform as well following surgery. Some surgeons will preselect their patients according to their mental state and will not operate on those with psychiatric comorbidities. It is recommended that these patients seek counseling prior to intervention to improve outcomes, recovery, and satisfaction.

Impaired gait: Patients with gait dysfunction typically have more severe spinal cord dysfunction and score lower on a preoperative mJOA assessment. The presence of impaired gait is also predictive of a worse outcome. This factor is related to a patient's duration of symptoms and the baseline severity score: if a patient is operated on earlier and at a milder disease stage, he or she may not have progressed to displaying evidence of gait impairment.

It is important to note that this model is not intended to evaluate surgical candidacy or identify patients who are more likely to benefit from surgical intervention than conservative management. Furthermore, there are only a few situations where our CPR would recommend against surgery: (1) the patient is very old and frail; (2) the patient is at high risk of experiencing a major complication; or (3) the patient has several comorbidities and poor general health status.

Reduce Heterogeneity of Care and Align Surgeons' Perceptions of Outcome with Objective Evidence

According to a study by Davidson et al, there is substantial variability between clinicians in terms of the information they convey to their patients with spinal cord injuries.19 In this study, a questionnaire was distributed to spine surgeons across North America who specialized in the management of traumatic spinal cord injury. This survey was designed to evaluate the type of prognostic information surgeons provide to their injured patients and to assess the variability of responses across practices. The questionnaire presented various case vignettes and asked questions such as, “How long will it take this patient to return to work?”; “What are the chances this patient will be free of back pain and stiffness 1 year after surgery?”; and “What do you tell your patient concerning the chances for functional recovery in his or her lower extremities?” Ideally, all surgeons would be well informed of the existing prognostic literature and provide similar responses to these questions. This ideal, however, was not the case; the results indicated substantial variability in the information surgeons provide to their patients about how they are expected to fare following intervention.

Similarly, in DCM, surgeons often have different perceptions of surgical prognosis, which is likely due to the controversy in the literature surrounding the most important clinical and imaging predictors of surgical outcome. Our CPR can be used across centers to objectively quantify a patient's likely outcome, align surgeons' perceptions with more objective evidence, and ensure that consistent and accurate information is being conveyed to patients.

Example Cases

In this section, we present example cases to illustrate how this CPR can be used to manage patients' expectations, counsel concerned patients as to the potential treatment options, and align surgeons' perceptions with objective evidence. Table 2 summarizes how the probability of achieving a score of mJOA ≥16 was calculated.

Table 2

Case examples and clinical applications of the clinical prediction rule

Case no.	Patient information to be entered into Eq. 1	Probability of achieving an mJOA score ≥16 (%)
1	mJOA0 = 14, DS = 1, S = 1, Ps = 1, IG = 2, A = 49	92.7
2	mJOA0 = 13, DS = 5, S = 1, Ps = 1, IG = 1, A = 69	41.0
3	mJOA0 = 17, DS = 2, S = 1, Ps = 1, IG = 2, A = 53versus mJOA0 = 15, DS  = 4, S = 1, Ps = 1, IG = 2, A  = 54 versusmJOA0 = 15, DS = 4, S = 1, Ps = 1, IG  = 1, A = 54	93.6 85.0 69.6
4	mJOA0 = 15, DS = 5, S = 1, Ps = 1, IG = 2, A = 62	76.4

Abbreviations: A, age in years; DS, duration of symptoms (1: ≤3 months; 2: >3, ≤6 months; 3: >6, ≤12 months; 4: >12, ≤24 months; 5: >24 months); IG, impaired gait (1: present, 2: absent); mJOA0, baseline modified Japanese Orthopaedic Association score (0–18); Ps, depression or bipolar disorder (1: absent, 2: present); S, smoking status (1: nonsmoker, 2: smoker).

Note: Bolded values indicate a change from the patient's original information.

The following two cases demonstrate how predicting outcome before surgery can aid in managing expectations.

Case 1: A 49-year-old nonsmoking man presented with moderate myelopathy (mJOA = 14) secondary to spondylosis, disk herniation, and congenital stenosis. This patient had numb and clumsy hands, muscular weakness, corticospinal motor deficits, hyperreflexia, and upgoing plantar responses. The duration of symptoms was 2 months. The patient also had coexisting moderate hypertension, mild respiratory disease, and mild diabetes.

Case 2: A 69-year-old nonsmoking man presented with moderate myelopathy (mJOA = 13) secondary to spondylosis, disk herniation, and hypertrophied ligamentum flavum. This patient had numb and clumsy hands, an impaired gait, muscular weakness, corticospinal distribution motor deficits, hyperreflexia, a positive Hoffman sign, upgoing plantar responses, and a broad-based unstable gait. The duration of symptoms was 120 months. The patient had a mild stroke (Fig. 1).

Fig. 1

Case 2.

Based on estimates computed by Eq. 1, case 1 has a 92.7% chance of improving to an mJOA score ≥16 whereas case 2 only has a 41.0% chance of achieving this outcome. These patients should be managed differently during the surgical consent process. The attending surgeon should inform both patients that they are likely to improve following surgery but should notify case 2 that he will still have substantial residual neurologic deficit and may require assistance with activities of daily living. This information will help manage case 2's expectations of outcome and, in turn, help to improve his overall satisfaction. With respect to observed outcome, case 1 was neurologically normal postoperatively (mJOA = 18), whereas case 2 improved from 13 to 15 but did not reach a score ≥16.

Case 3 provides an example how a CPR can be used to facilitate shared decision making and to counsel concerned patients as to potential options.

Case 3: A 53-year-old nonsmoking man presented with mild myelopathy (mJOA = 17) secondary to spondylosis and disk herniation. This patient had numb and clumsy hands, bilateral arm paresthesia, muscular weakness, and atrophy of intrinsic hand muscles. The duration of symptoms was 4 months. The patient had unspecified endocrine comorbidities (Fig. 2).

Fig. 2

Case 3.

This case is an example of a patient with mild myelopathy and a short preoperative duration of symptoms. This patient has an excellent surgical prognosis; however, he may be reluctant to consent to neurosurgery for such mild upper limb symptoms. This CPR can help surgeons counsel this patient and inform him that if he is operated on early and at his current disease state, he will achieve a better outcome than if he were to wait. An example is provided in Table 2. We assumed that if the patient waits 1 year before surgery, he will exhibit a 2-point decline on his preoperative mJOA score, will be 1 year older, and will have a significantly longer duration of symptoms. As a result, his probability of achieving a score ≥16 on the mJOA decreases from 93.6 to 85.0%. Furthermore, if he starts to exhibit signs and symptoms of gait dysfunction, this estimate further decreases to 69.6%. These figures would be valuable to help clinicians counsel their patients and to enable shared decision making.

Case 4: A 62-year-old nonsmoking man presented with mild myelopathy (mJOA = 15) secondary to spondylosis, disk herniation, and congenital stenosis. This patient had numb hands, Lhermitte phenomena, weakness, atrophy of intrinsic hand muscles, and a positive Hoffmann sign. The duration of symptoms was 36 months. The patient had coexisting mild gastrointestinal (stomach/intestine) disorders (Fig. 3).

Fig. 3

Case 4.

For each participant of the study, the surgeon was asked to predict how the subject would fare following surgical intervention: improve from baseline status, remain the same, or worsen. For case 4, the surgeon believed the subject would be the same as baseline. However, the CPR predicted a 76.4% chance the patient would achieve a score ≥16 and therefore improve by at least 1 point on the mJOA. The patient did indeed improve following surgery and was neurologically normal at 1-year follow-up (mJOA = 18). This example demonstrates how a CPR can help align surgeons' perceptions with more objective evidence.

Limitations and Future Directions

This model was developed to predict functional status in patients undergoing surgery for the treatment of DCM. Unfortunately, the mJOA/JOA scales are not highly correlated with patient-reported outcomes that assess overall quality of life. This conclusion was determined by several studies: (1) Kato et al reported moderate correlation between the JOA/mJOA and the Short-Form (SF)-12 Physical Component Score and Neck Disability Index and low correlation with the SF-12 Mental Component Score20; (2) Kopjar et al also demonstrated poor correlation with the Neck Disability Index (r =  − 0.343), SF-36 Physical Component Score (r = 0.300), and Mental Component Score (r = 0.245)21; and (3) King et al showed no association between the JOA and patient reports of outcome.22 Due to this moderate to poor correlation, this model cannot be used to predict quality-of-life outcomes in these patients, which is a limitation of the model, especially because health-related quality-of-life measures have become increasingly important.

A randomized controlled trial is required to assess the health care utility and the true application of this CPR in a surgical spine setting. In this trial, we propose that 50% of the patients will have their outcome objectively predicted by our CPR, whereas the other 50% will have a regular consent conversation with their attending surgeon. Patient satisfaction and other health-utility measures will be evaluated and compared between the two groups to ascertain the real value of this CPR.

Conclusions

By using actual cases examples in a DCM population, this commentary highlights the clinical applications of CPRs and their value in a surgical setting. CPRs can be used to predict disease development, define the treatment prognosis, and estimate the risk of complications in various patient populations. In a surgical setting, these tools can facilitate shared decision making between patients and physicians, manage expectations, and help standardize care across centers. Furthermore, a CPR can help identify ways to optimize the results and assist in the evaluation of cost-effectiveness.

Table 1

Modified Japanese Orthopaedic Association (mJOA) score

I. Motor dysfunction score of the upper extremities
Inability to move hands	0
Inability to eat with a spoon but able to move hands	1
Inability to button shirt but able to eat with a spoon	2
Able to button shirt with great difficulty	3
Able to button shirt with slight difficulty	4
No dysfunction	5
II. Motor dysfunction score of the lower extremities
Complete loss of motor and sensory function	0
Sensory preservation without ability to move legs	1
Able to move legs but unable to walk	2
Able to walk on flat floor with a walking aid (i.e., cane or crutch)	3
Able to walk up and/or down stairs with hand rail	4
Moderate to significant lack of stability but able to walk up and/or down stairs without hand rail	5
Mild lack of stability but walk unaided with smooth reciprocation	6
No dysfunction	7
III. Sensation
Complete loss of hand sensation	0
Severe sensory loss or pain	1
Mild sensory loss	2
No sensory loss	3
IV. Sphincter dysfunction
Inability to urinate voluntarily	0
Marked difficulty with micturition	1
Mild to moderate difficulty with micturition	2
Normal micturition	3