Development and reliability of a standard rating system for outcome measurement of foot and ankle disorders I: development of standard rating system.

BACKGROUND: The aim of this study was to report the five scales comprising the rating system that the Japanese Society for Surgery of the Foot (JSSF) devised (JSSF standard rating system) and the newly offered interpretations and criteria for determinations of each assessment item.
METHODS: We produced the new scales for the JSSF standard system by modifying the clinical rating systems established by the American Orthopaedic Foot and Ankle Society (AOFAS scales) and the Japanese Orthopaedic Association's foot rating scale (JOA scale). We also provided interpretations of each assessment item and the criteria of determinations in the new standard system.
RESULTS: We improved the ambiguous expressions and content in the conventional standard rating systems so they would be easily understood by Japanese people. The result was five scales in total. Four were designed for use specifically for ankle-hindfoot, midfoot, hallux metatarsophalangeal-interphalangeal, and lesser metatarsophalangeal-ineterphalangeal sites; and the fifth was for the foot and ankle with rheumatoid arthritis. Furthermore, we described interpretations and criteria for determinations with regard to evaluation items in each scale.
CONCLUSIONS: Conventionally, the AOFAS scales or the JOA scale have been separately applied depending on the sites or disorders concerned, but it was often difficult to decide on scores during practical evaluations because of differing expressions in different languages and also because of ambiguity in the interpretation of each evaluation item and in scoring standards as well. JSSF improved these scales and added definite interpretations of evaluation items as well as criteria for the rating (to be reported here in part I). Because these steps were expected to improve the reliability of outcomes assessed by each scale, we examined the reliability in scores of the newly developed scales, which are reported in part II (in this issue).

Introduction

To provide a basis for evidence-based medicine (EBM) from estimating the therapeutic usefulness or efficacy in epidemiologic surveys and clinical studies, such estimates must be based on reliable evaluation criteria. In Japan, the Japanese Orthopaedic Association’s foot rating scale (JOA scale)1 developed in 1991 for this purpose has been widely utilized in discussions of the therapy of foot and ankle disorders. The JOA scale assesses almost all foot disorders, including congenital abnormalities, trauma, infection, paralysis, tumors, arthropathies, circulation disorders, and rheumatoid arthritis (RA). Sites assessed range from the toes to the entire foot, making it a convenient tool to assess a wide area of the foot and ankle joints. Problems as to its reliability have been pointed out, however, as some of the items evaluated are unnecessary or unsatisfactory for particular disorders or sites.2

In 1994, the American Orthopaedic Foot and Ankle Society (AOFAS) developed rating scales for the ankle-hindfoot, midfoot, hallux metatarsophalangeal-interphalangeal, and lesser metatarsophalangeal-interphalangeal sites, respectively.3 Unlike the JOA scale, these scales categorized assessment sites into four areas, and the evaluated items were classified into three major categories: pain, function, and alignment. Another distinctive point was that these scales included items for both subjective and objective evaluation. These scales are now widely used and have been established worldwide as benchmark standard rating scales for the foot and ankle. However, during the evaluation process examiners often waver when interpreting each item or when deciding on scores because of ambiguity within the instrument. Also, regrettably, they have not been inspected for validity and reliability; neither have the JOA scales. Therefore, it has been pointed that they could not be used as a basis for practicing EBM.4–7

The JOA, therefore, undertook strategies to establish rating standards that use both doctors’ objective and patients’ subjective rating scores with the hope that the new standards would be appreciated internationally. The Japanese Society for Surgery of the Foot (JSSF) was also involved in this effort.

The purpose of Part I of this endeavor is to describe novel rating standards for foot and ankle disorders, providing explanations of each evaluation item and detailed criteria for ratings.

Materials and methods

The JOA launched the Preparatory Committee of Criteria for Diagnosis and Assessment in September 1999 and entrusted each of its member societies with such tasks as improvement and revision of evaluation standards and inspection of their validity and reliability. JFFS started the The Committee on Rating Standards for Foot Disease of the JSSF in June 2000, and the Committee subsequently had many discussions and meetings.

As a result, we translated into Japanese the four sites-pecific criteria for evaluation3 of the AOFAS and revised expressions and content to be more appropriate for Japanese people. Modifying this system for use in Japan is justified. First, we considered it difficult to apply one criterion in an evaluation to cover the entire foot and ankle. Second, site-specific evaluation criteria were thought to be easy to use; and, lastly, we considered it difficult for the international community to appreciate evaluation criteria that differed greatly from those of the AOFAS clinical rating system,3 which are accepted as the worldwide standard. However, because it seemed more appropriate to assess the foot as a whole rather than in a site-specific way in patients with rheumatoid arthritis (RA) because of the features of that disease, we partially modified and supplemented the JOA1 scale. Furthermore, we wrote interpretations and scoring criteria for each item in each scale to avoid differing interpretations.

Our current four site-specific scales are a completely novel and original Japanese version and are far from a duplicate of the AOFAS clinical rating system, as we modified the expressions and content to suit Japanese people. We also added interpretation criteria for each item and rating criteria, such as a pain scale, which were lacking in the AOFAS clinical rating system. This is why the Committee on Rating Standards for Foot Disease of the JSSF grouped together the five scales — comprised of four site-specific scales and the RA foot and ankle scale — and termed it the JSSF standard rating system.

Results

The total score of each scale was set at 100 points. Each observation item was scored only from clinical findings and included both subjective and objective evaluation items. Each of the four scales (i.e., those for the ankle-hindfoot, midfoot, hallux, and lesser toes) were divided into three major items, which were pain, function, and alignment. The scale for RA had five major items: pain, deformity, motion, walking ability, and activities of daily living (ADL). In addition, interpretation and criteria for scoring accompanied each item.

Ankle-hindfoot scale

Because inversion and eversion of the foot involve the talocrural, talocalacaneal, talonavicular, and/or calcaneocuboid joints, we evaluate the ankle and hindfoot as a complex; this scale deals with disorders or trauma incurred in those areas. A total of 40 points are assigned to pain, 50 to function, and 10 to alignment (Table 1).

Table 1

Ankle-hindfoot scale

Parameter	Points
Pain (40 points)
None	40
Mild	30
Moderate	20
Severe	0
Function (50 points)
Activity limitations
None	10
Limitations on recreational activities	7
Some limitations on daily and recreational activities	4
Severe limitations on daily and recreational activities	0
Maximum continuous walking distance
600m or more	5
400 m to less than 600m	4
100 m to less than 400m	2
Less than 100m	0
Walking surfaces
No difficulty on any surface	5
Some difficulty on uneven terrain, stairs, inclines	3
Severe difficulty or inability to walk on uneven terrain, stairs, inclines	0
Gait abnormality
None or slight	8
Obvious (walking possible but gait abnormality obvious)	4
Marked (walking difficult and gait abnormality obvious)	0
Sagittal motion (flexion plus extension)
Normal or mild restriction (30° or more)	8
Moderate restriction (15°–29°)	4
Severe restriction (less than 15°)	0
Hindfoot motion (inversion plus eversion)
Normal or mild restriction (75%–100% normal)	6
Moderate restriction (25%–74% normal)	3
Severe restriction (less than 25% normal)	0
Ankle-hindfoot stability (anterior drawer, varus-valgus stress)
Stable	8
Unstable	0
Alignment (10 points)
Good, plantigrade foot, well aligned	10
Fair, plantigrade foot, mild to moderate degree of malalignment	5
Poor, nonplantigrade foot, severe malalignment	0

Pain (criteria for scoring pain are shown in Table 2).

Table 2

Pain scale

Degree of pain	Spontaneous or during exercise	During daily life	Playing sports or with heavy work	Measures against pain (remarks)
None	None	None	None	None
Mild	Sometimes while in motion	None	Mild pain	None
Moderate	Always while in motion	With every movement	Moderate	Sometimes required
Severe	Always	Can barely walk	Severe (infeasible)	Always required

Function.

“Maximum continuous walking distance” is the maximum distance a person can walk without a break.

Gait abnormality is graded as follows: “none or slight,” “obvious” (walking possible but gait abnormality obvious), and “marked” (walking difficult and gait abnormality obvious).

Sagittal motion score includes the total points for passive flexion and passive extension in the sagittal plane. Motion is measured at the flexion position of the knee joint with the basic axis on the fibula and with the locomotion axis on the foot sole. Grades are “normal or mild restriction” (range of ≥30°), “moderate restriction” (15°–29°), and “severe restriction” (<15°).

Hindfoot motion score is the total points for passive inversion and passive eversion. Motion is measured at the flexion position of the knee so the basic axis is a vertical line toward the crus and the locomotion axis is on the foot sole. “Normal or mild restriction” denotes 75%–100%, “moderate restriction” 25%–74%, and “severe restriction” <25% of that on the normal side.

Ankle-hindfoot stability is evaluated by the presence or absence of instability at the anterior drawer and under varus-valgus stress. “Stable” indicates a firm endpoint at the anterior drawer and under varus-valgus stress. “Unstable” indicates a soft endpoint.

Alignment. Plantigrade foot indicates that the foot can touch the ground with the sole when walking. “Mild to moderate malalignment” is easily correctable, and “severe malalignment” is uncorrectable.

Midfoot scale

The midfoot scale is for disorders or trauma incurred in areas of the first to third cuneiform, second to third cuneiform, and third cuneiform-cuboid, naviculocuneiform, naviculo-cuboid, or Lisfranc joints. A total of 40 points are assigned to pain, 45 to function, and 15 to alignment (Table 3).

Table 3

Midfoot scale

Parameter	Points
Pain (40 points)
None	40
Mild	30
Moderate	20
Severe	0
Function (45 points)
Activity limitations
None	10
Limitations on recreational activities	7
Some limitations on daily and recreational activities	4
Severe limitations on daily and recreational activities	0
Footwear requirements
Can wear conventional shoes	5
Comfort footwear and shoe insert required	3
Modified shoes or brace required	0
Maximum continuous walking distance
600m or more	10
400 m to less than 600m	7
100 m to less than 400m	4
Less than 100m	0
Walking surfaces
No difficulty on any surface	10
Some difficulty on uneven terrain, stairs, inclines	5
Severe difficulty or inability to walk on uneven terrain, stairs, inclines	0
Gait abnormality
None or slight	10
Obvious (walking possible but gait abnormality obvious)	5
Marked (walking difficult and gait abnormality obvious)	0
Alignment (15 points)
Good, plantigrade foot, well aligned	15
Fair, plantigrade foot, mild to moderate degree of malalignment	8
Poor, nonplantigrade foot, severe malalignment	0

Pain (criteria for evaluation of pain are given in Table 2).

Function.

Footwear. Designations are “can wear conventional (commercially available) shoes,” “comfort footwear (limited selection of commercially available shoes) or shoe insert required,” and “modified shoes (specially ordered) or brace required.”

“Maximum continuous walking distance” is defined as the maximum distance a person can walk without a break.

Gait abnormality is graded as “none or slight” when there is no or minimal gait abnormality, “obvious” when the abnormality is evident but walking is possible, and “marked” when the abnormality is obvious and walking is difficult.

Alignment. Plantigrade foot indicates that the foot can touch the ground by the sole when walking. “Mild to moderate malalignment” indicates easily correctable malalignment, and “severe malalignment” indicates uncorrectable malalignment.

Hallux metatarsophalangeal-interphalangeal scale

The hallux metatarsophalangeal-interphalangeal (MTP-IP) scale was devised for disorders or trauma incurred in areas of the first metatarsal, proximal phalanx of the hallux, distal phalanx, and MTP and IP joints. A total of 40 points are assigned to pain, 45 to function, and 15 to alignment (Table 4).

Table 4

Hallux metatarsophalangeal-interphalangeal scale

Parameter	Points
Pain (40 points)
None	40
Mild	30
Moderate	20
Severe	0
Function (45 points)
Activity limitations
None	10
Limitations on recreational activities	7
Some limitations on daily and recreational activities	4
Severe limitations on daily and recreational activities	0
Footwear requirements
Can wear conventional shoes	10
Comfort footwear and shoe insert required	5
Modified shoes or brace required	0
MTP joint motion (dorsiflexion plus plantarflexion)
Normal or mild restriction (75° or more)	10
Moderate restriction (30°–74°)	5
Severe restriction (less than 30°)	0
IP joint motion (plantarflexion)
No restriction	5
Severe restriction (less than 10°)	0
MTP-IP stability (all directions)
Stable	5
Unstable	0
Callus or clavus
No or asymptomatic callus or clavus	5
Painful callus or clavus	0
Alignment (15 points)
Good, well aligned	15
Fair, mild to moderate degree of malalignment	8
Poor, severe malalignment	0

MTP, metatarsophalangeal; IP, interphalangeal

Pain (criteria for evaluation of pain are listed in Table 2).

Function.

Footwear. Designations are “can wear conventional (commercially available) shoes,” “comfort footwear (limited selection of commercially available shoes) or shoe insert required,” and “modified shoes (specially ordered) or brace required.”

MTP joint motion score is the total of points for passive flexion and passive extension in the sagittal plane. Measurement is made so that the basic axis is on the first metatarsus, and the locomotion axis is on the first proximal phalanx. Designations for restriction are “normal or mild” (range of ≥75°), “moderate” (30°–74°), and “severe” (<30°).

IP joint motion is evaluated only by passive flexion and is measured so the basic axis is on the first proximal phalanx, and the locomotion axis is on the first distal phalanx. With “no restriction” flexion is possible at ≥11°, and with “severe restriction” flexion is possible only at ≤10°.

MTP-IP stability is observed from every direction. “Unstable” indicates ease of dislocation.

Callus or clavus is assessed by the presence or absence of pain. The whole foot sole and hallux are evaluated.

Alignment. “Mild to moderate malalignment” indicates easily correctable malalignment, and “severe malalignment” indicates uncorrectable malalignment.

Lesser metatarsophalangeal-interphalangeal scale

The lesser MTP-IP scale is devised for disorders or trauma incurred in areas of the second to fifth metatarsus, proximal phalanx, middle phalanx, distal phalanx, MTP, and PIP joints. A total of 40 points are assigned to pain, 45 to function, and 15 to alignment (Table 5).

Table 5

Lesser metatarsophalangeal-interphalangeal scale

Parameter	Points
Pain (40 points)
None	40
Mild	30
Moderate	30
Severe	0
Function (45 points)
Activity limitations
None	10
Limitations on recreational activities	7
Some limitations on daily and recreational activities	4
Severe limitations on daily and recreational activities	0
Footwear requirements
Can wear conventional shoes	10
Comfort footwear and shoe insert required	5
Modified shoes or brace required	0
MTP joint motion (dorsiflexion plus plantarflexion)
Normal or mild restriction (75° or more)	10
Moderate restriction (30°–74°)	5
Severe restriction (less than 30°)	0
IP joint motion (plantarflexion)
No restriction	5
Severe restriction (less than 10°)	0
MTP-IP stability (all directions)
Stable	5
Unstable	0
Callus or clavus
No or asymptomatic callus or clavus	5
Painful callus or clavus	0
Alignment (15 points)
Good, well aligned	15
Fair, mild to moderate degree of malalignment	8
Poor, severe malalignment	0

Pain (criteria for evaluation of pain are shown in Table 2).

Function.

Footwear. Designations are “can wear conventional (commercially available) shoes,” “comfort footwear (limited selection of commercially avail- able shoes) or shoe insert required,” “modified shoes (specially ordered) or brace required.”

MTP joint motion score is the total points for passive flexion and passive extension in the sagittal plane, with the measurement so the basic axis is on the second to fifth metatarsi, and the locomotion axis is on the second to fifth proximal phalanges. Ratings for restriction are “normal or mild restriction” (≥75°), “moderate” (30°–74°), and “severe” (<30°).

IP joint motion is evaluated only by passive flexion, with measurements made so the basic axis is on the second to fifth proximal phalanx, and the locomotion axis is on the second to fifth middle phalanx. “No restriction” indicates that flexion is possible at ≥11°, and “severe restriction” indicates that flexion is possible only at ≤10°.

MTP-IP stability is observed from every direction. “Unstable” indicates ease of dislocation.

Callus or clavus is assessed by the presence or absence of pain on the whole foot and the hallux.

Alignment. “Mild to moderate malalignment” is easily correctable, and “severe malalignment” is uncorrectable.

Rheumatoid arthritis foot and ankle scale

All of the disorders and deformities caused by RA in the forefoot, midfoot, and ankle-hindfoot regions are objects of evaluation. Because the entire foot should be evaluated in RA patients, the JOA scale1 formed the basis for the evaluation in this system. We removed “muscle strength” and “sensory disturbance” and added items related to ADL, resulting in five major items: pain, deformity, motion, walking ability, and ADL. A total of 30 points are assigned to pain, 25 to deformity, 15 to motion, 20 to walking ability, and 10 to ADL (Table 6).

Table 6

RA foot and ankle scale

General Pain (30 points)
None or mild	30
Mild, persistent pain when walking	20
Unable to walk because of pain	10
Severe at all times	0

RA, rheumatoid arthritis

a Forefoot (MTP/IP joints) is evaluated at a site that is limited in motion most severely among the hallux and lesser toes

b Hindfoot is evaluated for a motion that is most severely limited among such motions as dorsiflexion/plantarflexion and inversion/eversion

Pain. “Continuous pain when walking” indicates persistent pain beginning at the initiation of ambulation but the ability to walk continuously. “Unable to walk due to pain” means the inability to walk because of severe pain upon physical exercise but no pain at rest. “Severe pain at all times” means continuous severe pain even upon rest.

Deformity.

Deformity is evaluated separately in the forefoot (hallux, lesser toes), midfoot, and hindfoot.

Deformity of the MTP and IP joints of the hallux. “Slight” means slight protrusion of the MTP joint or a flexible (manually correctable) deformity. “Obvious” denotes marked protrusion of the MTP joint or a rigid (manually uncorrectable) deformity of the IP joint. When the deformity is termed “marked,” the hallux overrides the second toe or is a rigid deformity with a callus or ulcer on the IP joint.

Deformity of the MTP, proximal IP (PIP), and distal IP (DIP) joints of the lesser toes. “Slight” designates a flexible (manually correctable) deformity; “obvious” is a rigid (manually uncorrectable) deformity; and “marked” is a rigid deformity with ulceration.

Deformity of the hindfoot. “None” is the presence of only physiological calcaneus valgus. “Obvious” indicates an obvious calcaneus varus or calcaneus valgus, although the entire surface of the foot sole touches the ground. “Marked” deformity means that the medial or lateral side of the foot sole does not touch the ground.

Motion

The forefoot (MTP/IP joints) is evaluated at the site that is limited in motion most severely among the hallux and lesser toes.

Hallux MTP joint score is a total of points for passive extension and passive flexion in the sagittal plane and is measured with the basic axis on the first metatarsal and the locomotion axis is on the first proximal phalanx. “Normal” is ≥75°; “half or more of normal” is 30°–74°; and “less than half” indicates <30°.

Hallux IP joint motion is evaluated only for passive flexion. Upon measurement the basic axis is on the first proximal phalanx, and the locomotion axis is on the first distal phalanx. “Normal”: plantar flexion is possible at ≥11°.

MTP motion among the hallux and lesser toes is evaluated for passive flexion and passive extension in the sagittal plane, the scores of which are totaled. Upon measurement, the basic axis is on the second to fifth metatarsal, and the locomotion axis is on the second to fifth proximal phalanx. “Normal” is ≥75°; “half or more of normal” is 30°–74°; and “less than half” indicates <30°.

Second to fifth IP joint motion is evaluated only for passive flexion. Measurement of motion is made so the basic axis is on the second to fifth proximal phalanx, and the locomotion axis is on the second to fifth middle phalanx. “Normal”: plantar flexion is possible at ≥11°.

Hindfoot is evaluated for a motion that is most severely limited among such motions as dorsiflexion/plantarflexion and inversion/eversion.

Joint motion in the sagittal plane includes total points for passive dorsiflexion and passive plantarflexion. Motion is measured at the flexion position of the knee joint with the basic axis on the fibula and with the locomotion axis on the foot sole. “Normal” is ≥60°; “half or more of normal” is 30°–59°; and “less than half” is <30°.

Hindfoot motion score is the total points for passive inversion and passive eversion. Measurement is made with the knee flexed so the basic axis is a vertical line toward the crus, and the locomotion axis is on the foot sole. “Normal” is ≥60°; “half or more of normal” is 30°–59°; and “half or less than normal” is <30°.

Activities of daily living (the scale for ADL is shown in Table 7).

Table 7

Scale for activities of daily living

Parameter	Easy	Difficult	Unable
Climbing or descending stairs	Placing one foot on stair above/below the other	Placing feet side by side at every step	Unable without help
Sitting on heels	Can touch the entire dorsal foot and ground	Room between dorsal foot on ground	Unable to sit on heels
Standing on toes	Can stand on toes easily for 5 s or more with one leg	Cannot stand on toes with one leg; can stand with both legs for 5s or more	Unable to stand on toes even with both legs
Footwear	Can wear stylish shoes	Shoes become misshapen because of foot deformity	Can wear only modified shoes (orthopedic shoes)
Japanese-style toilet	Can bend knees fully without lifting heels not	Can bend knees fully, but heels are on the floor	Cannot squat down

Discussion

The foot and ankle refer to a wide area from the distal phalanx of the foot to the distal end of the leg. Each portion consists of many bones, joints, ligaments, ligamentous symphyses, and intrinsic and extrinsic muscles. In addition, disorders or trauma of the foot and ankle are so varied that evaluation of a limited area is not always adequate when the evaluated area represents only part of the expression of a systemic condition such as paralytic disorders and RA. Thus, it is difficult to evaluate such conditions appropriately using only one scale.

The AOFAS clinical rating system3 consists of four site-specific scales that enable a focus on target sites or diseases. Observation items are relatively simple and have been translated and interpreted by attending medical staff (examiners) for wide use in Japan. However, some items are not sufficiently straightforward for interpretation or rating in the AOFAS clinical rating system. For instance, for the item pain, to which as many as 40 of the full 100 points are assigned, criteria for evaluation are so ambiguous as to produce a rating bias among examiners that may greatly influence results. For the item function, the maximum walking distance is expressed in blocks, a term not familiar to Japanese. Also, the definition of gait abnormality and stability is not clear. Description of shoes is also difficult to understand in the Japanese translation. Additionally, for the item of alignment, the expression is so confusing as to cause difficulty with evaluation. To solve these problems, in part I of the present study (described herein) we describe in detail the interpretation and evaluation criteria of each item to provide Japanese examiners more understandable standards and to minimize differences in results among examiners.

It is difficult to cover the entire disease spectrum by site-specific rating systems of disorders for which the entire foot must be assessed, such as RA. In such a case, the JOA scale1 is useful because it covers a disorder or sites over a wide range. This scale, however, includes so many items that the number of points assigned to each item is small, making a comparison difficult between treatments or between cases. Interpretation and criteria for rating each item are indefinite, as with the AOFAS clinical rating systems, leaving the final decision to the discretion of the examiner. Therefore, in the RA foot and ankle scale that we devised in this study, we removed the items “muscle strength” and “sensory disturbance,” which are usually not applicable to RA, and adopted five major items: pain, deformity, motion, walking ability, and ADL as evaluation items. We increased the number of points for each item and provided details of interpretation and evaluation criteria. This change is expected to improve the reliability of outcomes evaluated using each scale.

A rating standard must be reliable not only as a rating scale but to perceive the target disease or therapy properly. Defining interpretations and rating criteria of evaluation items cannot always guarantee an improvement in inter- and intraexaminer reliability. It is necessary to test reliability for a standard to be recognized as evidence. Many tests have been conducted on the reliability of rating criteria in the field of orthopedic surgery. 8–16 The Committee on Rating Standards for Foot Disease of the JSSF tested five scales (JSSF standard rating system) for intra- and interexaminer reliability in a multiinstitutional study. The results of this study are reported in part II (in this issue).