Patient-Reported Outcomes in Foot and Ankle Orthopedics.

Patient-reported outcomes (PROs) are a measure of health care quality that reflect the patient's perceptions of their own health status. Recently, there has been a renewed interest in implementation of PROs into everyday clinical practice. There are many dozens of PROs available to foot and ankle surgeons with little consensus on which measures are most appropriate for a given condition. These measures vary widely in length, validity, and content. When integrating PROs into clinical practice, we recommend that clinicians should collect, at a minimum, validated PRO scores that assess pain, function, and general health. Furthermore, concise instruments should be used wherever possible to minimize patient burden, maximize patient engagement, and ensure meaningful data are collected. In the near future, outcomes registries employing computer adaptive testing will facilitate the routine collection of PRO data from all patients. LEVEL OF EVIDENCE: Level V, expert opinion.

Introduction

Patient-reported outcomes (PROs) are an important tool for assessing health care quality and outcomes. PRO measures (PROMs) reflect the patient’s perceptions of their health status and help quantify the impact of conditions and injuries as well as improvement following treatment. There are many dozens of PROMs available to foot and ankle surgeons (Table 1). These measures can range from simple qualitative assessments of pain or satisfaction to complex computer-adaptive tests (CATs) that utilize questions from large question banks based on previous responses.

Table 1.

Characteristics of the 10 Most Commonly Used PRO Measures in Foot and Ankle Orthopedics.a

Frequency of Usage	PRO Measure	Description	Number of Questions	Score Range	Validated?	MCID Known?
1	American Orthopaedic Foot & Ankle Society (AOFAS)	General measure of foot and ankle symptoms and function; incorporates clinician-entered objective criteria	Variable, ∼6-10 questions	0-100, lower indicating impairment	No. The AOFAS now recommends against their use.	Yes for Hallux Valgus subscale
2	Visual analog scale (VAS)	Global measure of pain	1	0-100 mm, higher indicating impairment	Yes	Yes
3	Short Form–36 (SF-36)	Global measure of health	36	0-100, lower indicating impairment	Yes	Yes
4	Foot and Ankle Outcome Score (FAOS)	General measure of foot and ankle symptoms and function	42	0-68, higher indicating impairment	Yes	No
5	Short Form–12 (SF-12)	Global measure of health; shortened version of SF-36	12	0-100, lower indicating impairment	Yes	Yes in general; no for foot and ankle conditions
6	Foot Function Index (FFI)	General measure of foot and ankle symptoms and function	17	0-100, higher indicating impairment	Yes	No
7	Foot and Ankle Ability Measure (FAAM)	General measure of foot and ankle symptoms and function	31	0-100, lower indicating impairment	Yes	Yes
8	Short Musculoskeletal Function Assessment (SMFA)	General measure of musculoskeletal symptoms and function	46	0-100, higher indicating impairment	Yes	Yes in general; no for foot and ankle conditions
9	Ankle Osteoarthritis Scale (AOS)	Specific measure of osteoarthritis symptoms and function	18	0-100, higher indicating impairment	Yes	Yes
10	Olerud-Molander	Specific measure of ankle fracture symptoms and function	9	0-100	Yes	No

Abbreviation: PRO, patient-reported outcome.

aThe 10 most commonly used PRO measures in the recent foot and ankle orthopedic literature include global measures of health status, measures specific to foot and ankle function, and measures designed to assess foot and ankle patients with a particular condition. The columns on validation status and the minimal clinically important difference (MCID) refer to the measure as it pertains to foot and ankle practice.

Although following medical outcomes has been important to clinicians for decades, most of commonly used PROMs in foot and ankle orthopedics have emerged in the past 25 years. Despite growth in the popularity of implementing PROM assessments into clinical research and the wide variety of options available to foot and ankle surgeons, there remains a lack of consistency in which tools are used for a given diagnosis or treatment regimen. Achieving consensus PRO measures would help ensure providers are tracking patient outcomes using the most appropriate and validated scales for the condition they are treating. Furthermore, consensus measures allow clinicians and researchers to more directly compare outcomes data and establish best practices.

The objective of this review is to outline the current state of patient-reported outcome usage in foot and ankle surgery, examine the quality of some of the most commonly used measures, and to explore the implementation of PROMs in clinical practice, outcomes registries, and consortia.

History of Outcomes in Orthopedics

The concept of assessing and tracking outcomes in medicine can be traced back more than a century and a half to Florence Nightingale. While she was an English field nurse during the Crimean War in the mid-1850s, Nightingale helped pioneer the concept of evidence-based medicine. Through simple observation of causes of mortality, she noted that far more soldiers died from disease than from battlefield injuries. Nightingale was then able to significantly reduce all-cause mortality by simply improving nutrition and hygiene standards. Decades later in 1914, Ernest Codman, MD, an early American orthopedic surgeon, built on Nightingale’s work through the development of the “end result” idea. Dr Codman emphasized the duty of documenting the clinical presentation, treatment regimen, and objective outcomes, or end result, of each patient encounter in an effort to better assess medical outcomes. Following Dr Codman’s lead, many of the early outcome measures in orthopedics consisted of objective measures like loss of life or limb, range of motion, length of stay, or complication rates.

Toward the end of the 20th century, orthopedic surgeons began developing and incorporating PROMs into research and ultimately into clinical practices. One of the prototypical PROMs that remains frequently used today is the Short Form–36 (SF-36), originally developed by the RAND corporation in 1992 as a 36-question assessment of general health status. In the approximately 25 years since the SF-36 was developed, hundreds of patient-reported outcome measures have become available to orthopedic surgeons, including 89 for foot and ankle surgery (E.L. and K.J.H., unpublished data, 2018), 206 for spine surgery, and 73 for pediatric orthopedics. For foot and ankle surgeons, many of the most popular patient-reported outcome measures in use today were developed in the 1990s and 2000s.

Prior to 2004, most PROMs consisted of a fixed set of questions, called short forms (SFs), that are the same for every patient. These early PROMs primarily used classical test theory wherein each individual question is associated with a correct answer and the total percentage correct is recorded at the end of the test. This is in contrast to item-response theory where each individual question is associated with both a correct answer and a calculated difficulty. Using item-response theory, the NIH Patient-Reported Outcomes Measurement Information System (PROMIS) initiative was introduced in 2004 as a series of short computer-adaptive tests (CATs) that assess clinical parameters such as pain, social function, or mental health. CATs use a patient’s previous responses to select future questions. CATs have demonstrated substantial reductions in test burden with minimal loss of precision in comparison to SFs. Another benefit of PROMIS CATs is that they are designed to assess health domains rather than specific disease processes. Domains are measurable clinical parameters with relevance across a variety of diagnoses and clinical specialties. PROMIS measures are classified into one of 4 profiles: Global Health, Physical Health, Mental Health, and Social Health. Each of these profiles is then divided into a series of domains. Altogether, there are more than 100 domains available on the PROMIS platform, allowing providers to assess broadly applicable parameters such as Physical Function, Pain Intensity, Fatigue, Social Function, and Substance Use. The PROMIS initiative has demonstrated promising early results and may help guide the future of patient-reported outcomes collection.

Available PROs in Foot and Ankle Surgery

There are dozens of patient-reported outcome measures that are available to foot and ankle clinicians. An analysis of PRO usage in foot and ankle clinical research from 2012 to 2016 noted 89 unique measures in the field. The most frequently utilized measures in foot and ankle journals included assessments of overall foot and ankle function, assessments of overall health, and questionnaires designed for specific diagnoses and procedures (Table 1). The top 5 scales were the AOFAS scales, the visual analog scale (VAS), the Short Form–36 (SF-36), the Foot and Ankle Outcome Scale (FAOS), and the Short Form–12 (SF-12). Altogether, these top 5 scales constituted 54% of all PRO usage. Custom scales assessing satisfaction and function were also frequently cited in the clinical outcomes literature.

Although the AOFAS scales have been the most popular metrics in foot and ankle surgery over the last 15 years (E.L. and K.J.H., unpublished data, 2018), usage began to decline after the AOFAS published a position statement discouraging further use in 2011. As of 2016, AOFAS scales appeared in 47% of all outcomes articles. Although it is a nonstandardized measure, use of the VAS for pain assessment has been increasing in popularity at an average rate of 4.5% per year over the past 5 years and appeared in 44% of all clinical outcomes articles in 2016, making it the second most commonly used measure in foot and ankle surgery since 2002 (E.L. and K.J.H., unpublished data, 2018). Usage of the SF-36, FAOS, and SF-12 has also increased modestly over the past 5 years.

Validation of PROs

The ideal patient-reported outcome metric is freely available and possesses a variety of characteristics, including precision, validity, responsiveness, and coverage. Precision is the standard error in measurement—for example, how close 2 theoretical patients with the same physical function would score on a test of physical function. Validity, in broad terms, is the extent to which the items in the test accurately assess the variable of interest. Content validity refers to whether the measure reflects the full scope of the given topic. Construct validity is whether the measure aligns with other accepted measures or differences in that variable—for example, whether a measure of physical function is able to detect a difference between injured and uninjured populations. Responsiveness is the sensitivity of a test in detecting a clinically relevant change in health status (eg, minimal clinically important difference [MCID]). Coverage is the range of scores over which a given measure is sensitive. In routine clinical practice, restoration of an average level of health and function is often sufficient. This can be accomplished with most PROMs in use today. However, athletes are an example of a population for whom restoration of an average level of health and function may not be sufficient. For these populations, an optimal metric should seek to minimize floor and ceiling effects to help providers differentiate patients with unusually high or low function. It is important for clinicians to understand the validation and quality of PROM tools incorporated into clinical practice or research assessments (Figure 1). This understanding will aid in the interpretation of clinical data, both in practice and in the literature.

Figure 1.

Patient-reported outcome measure (PROM) categories of interest to the orthopedic provider.

The American Orthopaedic Foot & Ankle Society (AOFAS) scales were introduced in 1994 as a series of 4 subscales covering ankle-hindfoot, midfoot, hallux, and lesser toes. Each of these subscales incorporates patient-reported and physician-reported data to provide a composite functional score. Although AOFAS scales have been among the most commonly used PROs by foot and ankle surgeons since their inception (E.L. and K.J.H., unpublished data, 2018), concerns have been raised about their validity and precision. The AOFAS scales correlate poorly with many commonly used legacy measures such as the SF-36, Musculoskeletal Function Assessment, and the QALY score. Further, the hallux subscale has been found to be inconsistent with repeated administrations. The inclusion of absolute terms such as “always” or “never” on the patient questionnaire and the presence of physician-assessed scores both introduce additional error. Finally, a recent survey noted that up to 50% of the attributes measured in the AOFAS scales were not of primary importance to patients. In response to these concerns, the AOFAS published a position statement in 2011 discouraging further use of the AOFAS subscales in favor of more highly validated foot and ankle metrics.

One of the first patient-reported outcome measures to achieve widespread use in orthopedics was the VAS. The VAS, developed in 1976, is commonly used to quantify a patient’s pain by having them place a mark along a horizontally oriented axis numbered from 0 to 10, where a change of ±30% represents a minimal clinically important difference. The VAS has been among the most popular patient-reported outcomes in foot and ankle orthopedics for decades and is today the second most commonly used measure in foot and ankle orthopedics (E.L. and K.J.H., unpublished data, 2018). Its brevity is an advantage as it is easy to administer, quantify, and interpret. Although the VAS has low sensitivity and may display nonlinear responses to changes in pain level, it has been validated for a wide variety of orthopedic conditions, including chronic stable joint pain and revision hip arthroplasties.

The Short Form-36 (SF-36) was developed in 1992 as a 36-question assessment of general health. Four years later, in 1996, the Short Form-12 (SF-12) was developed as a 12-question assessment modeled off the SF-36 in an effort to reduce burden on the patient while minimizing loss of precision. Today, the SF-36 is the most commonly used PRO to assess general health and the third most commonly used PRO in foot and ankle orthopedics overall (E.L. and K.J.H., unpublished data, 2018).

Although the SF-36 is validated for a variety of orthopedic conditions including total hip and knee replacements, ACL reconstructions, and arthroscopic partial meniscectomies, it can be subject to unrelated medical conditions affecting the score independent of orthopedic outcomes. The SF-36 is an excellent research tool, but the presence of a subscription fee to access and the length of the form may limit utility in routine clinical practice for most providers.

The Short Musculoskeletal Function Assessment (SMFA) was developed in 1999 as a concise measure of overall musculoskeletal function and how musculoskeletal function interferes with daily life. It is unique among the functional PROMs in that the SMFA assesses overall musculoskeletal function rather than focusing specifically on the foot and ankle. Although it is relatively long at 46 questions, the SMFA is a highly validated measure with moderate responsiveness.

The Foot Function Index (FFI) is a 23-question PROM assessing foot and ankle function with a focus on pain and function. Although it was developed in 1991 and is fairly simple, the FFI is a reliable and valid measure for assessing foot and ankle function. However, it may not be as responsive as FAAM and PROMIS CAT tools for many foot and ankle conditions and interventions. In one study of patients undergoing elective surgery for one of 6 foot and ankle disorders, the FAAM and PROMIS Physical Function CAT were responsive for 5 of 6 disorders, the PROMIS Pain Interference CAT was responsive for 4 of 6 disorders, and no disorders showed improvement on the FFI.

The Foot and Ankle Ability Measure (FAAM) is a 28-question PROM that was developed in 2003 with a focus on both daily function and sports function. It is a valid measure for assessing numerous foot and ankle conditions including chronic ankle instability. The FAAM, FFI, and SMFA are all highly correlated measurements.

The Foot and Ankle Outcome Score (FAOS) is a 42-question PROM designed to assess foot and ankle function. It has been found to have good validity for a variety of foot and ankle conditions such as hallux valgus, osteoarthritis, hallux rigidus, adult acquired flatfoot deformity, and ankle ligament reconstruction ; however, the length of the FAOS may lead to decreased response rates and lower patient engagement.

Although the Patient-Reported Outcomes Measurement Information System (PROMIS) was developed in 2004, it is a dynamic system with new domains being released regularly. Between adult and pediatric measures, more than 100 PROMIS domains are currently available. The Orthopaedic Foot & Ankle Outcomes Research Network (OFAR) currently collects the following 5 PROMIS scales: Physical Function CAT, Pain Interference CAT, Pain Intensity SF, Global Health SF, and Depression CAT. When using CAT and item-response theory (IRT), PROMIS domains take around 1 minute or less for the patient to complete. This allows clinicians to administer 3 to 5 specific PROMIS domains while minimizing the burden on their patients. In addition to their brevity, PROMIS measures have several distinct advantages over more traditional legacy measures. PROMIS measures have been validated for a variety of foot and ankle conditions with negligible floor and ceiling effects. PROMIS measures for Pain and Physical Function may also have prognostic value in determining which patients are most likely to benefit from foot and ankle surgery. PROMIS tools have also been well established in other orthopedic subspecialties and other medical specialties, which facilitates a common language for patients with more than just a foot and ankle condition.

Selecting a PRO Platform and Instruments

When implementing PROs into clinical practice, the gold standard should always be the efficient collection of clinically significant data. It is also important for clinicians to recognize the importance of PRO brevity in maximizing patient response rates and minimizing clinical disruptions. For implementation into routine clinical practice, we also suggest using cloud-based or EHR-based surveys administered to patients. The advantage of EHR integration is the availability of PRO scores during a patient evaluation and for inclusion in a patient note. Additionally, EMR integration provides a platform for longitudinal tracking of PRO data as part of a patient’s electronic health record. Cloud-based systems allow storage of searchable data for quality improvement initiatives within the practice or institution. There are an increasing number of vendors available to the orthopedic provider to collect PRO scores from patients by increasingly efficient and secure means. The most commonly used domains in orthopedics include PROMs to assess global health, physical function, pain, and depression for each patient, preoperatively and at 1 or more postoperative time points. However, PROMs assessing patient satisfaction or disease-specific PROs can also be helpful in assessing a patient’s progress and outcome, such as fatigue measures for cancer patients or social function measures for patients with diabetes.

PROs evaluate the entire patient at a point in time and it can be different to isolate one limb when both limbs affect a given PRO score. For example, the PRO scores from a patient with a unilateral ankle fracture will likely be very different from those of a patient with the same ankle fracture on one side and a complex open pilon fracture on the other side. As a result, it is paramount for providers to keep the entire patient context in mind when trending and interpreting PRO scores.

Moving forward, the true promise of PROs in evaluating clinical outcomes can only be achieved by aggregating data on a national scale. Outcomes registries are organizations that provide a platform for PRO collection from multiple sites as well as a central location for data aggregation, analysis, and dissemination. Registries can provide clinicians collecting PRO data with an established infrastructure for EMR integration and HIPAA compliance. The Orthopaedic Foot & Ankle Outcomes Research (OFAR) Network was created by the AOFAS in 2012 to serve as an outcomes registry for foot and ankle orthopedics. Following a successful 10-site pilot study in 2014, OFAR is scaling the operation to include a patient portal with automated alerts to prompt patients to complete assessments, EMR integration, and a cloud-based platform. Registries like OFAR are poised to play a pivotal role in the selection of consensus PROMs, collection and analysis of PRO data, and widespread integration of PRO data collection into routine clinical practice.

Conclusions

Patient-reported outcomes are important indicators of health care quality, and their use in foot and ankle clinical research dates back decades. In addition to their research utility, PROMs are important vehicles in the pursuit of patient engagement, outcomes tracking, prognoses provision, assessing value in health care, and more. The authors recommend that clinicians and researchers should not rely on AOFAS measures as their sole outcome measure, because of their poor validity. PROMIS measures are a series of next-generation PROs that are capable of employing computer-adaptive technology to reduce testing times and allow efficient PRO administration from any electronic device with an Internet connection. Registries similarly increase the efficiency of PRO administration by facilitating automated data collection. As we move toward a value-based health care model, the use of registries, PROMIS tools, and consensus legacy measures may serve as a key mechanism for the efficient, accurate, and consistent collection of patient-reported outcomes data.

Click here for additional data file.

Supplemental Material, FAO852930-ICMJE for Patient-Reported Outcomes in Foot and Ankle Orthopedics by Eric Lakey and Kenneth J. Hunt in Foot & Ankle Orthopaedics