A computerized system for the application of Basso, Beattie and Bresnahan scale in Wistar rats.

OBJECTIVES: To develop and test a computer program to assist researchers in assigning scores in the application of the Basso, Beattie and Bresnahan (BBB) scale and to compare these scores when doing so in free, targeted and automated computer-assisted modes.
METHOD: To test the program, the participants used the Impactor methodology recommended by the New York University (USA), in which 12 Wistar rats submitted to spinal cord injury were filmed on the 28(th) day after the injury. Eight researchers from the Laboratory of Medical Investigation, Faculdade de Medicina da Universidade de São Paulo, SP, Brazil took part in the study. The two heads of the laboratory, with 15 years of experience in the application of the scale, were considered the gold standard.
RESULTS: The results of the scale application were not significantly different in relation to the gold standard, considering the mean of the evaluators in each method: free, targeted and automated form (with the help of the computer).
CONCLUSIONS: The application of the BBB scale in the automated mode, using the computer program, did not present any difference in relation to the gold standard for all the evaluators. Level of Evidence II, Diagnostic Studies.

INTRODUCTION

Spinal cord injury is a serious public health issue and one of the most devastating and incapacitating neurological syndromes that affect human beings. It is characterized by severe motor alterations, alterations of superficial and deep sensitivity, and neurovegetative and psychosocial disorders.1

The understanding of the physiopathological mechanisms of spinal cord injury becomes essential, yet no consensus has yet been reached on the best method of analysis of functional recovery. Although some tests that evaluate functional recovery are easy to use, they present limited sensitivity to subjective observations.2 , 3 Most spinal cord injury studies evaluate functional recovery through the analysis of sensory and locomotor reflexes, and use Wistar rats due to their practicability, cost and availability. These tests are also performed in other animals.4 - 6 One of the difficulties referred to in the studies is the establishment of a standardized evaluation system to assess motor function in spinal cord-injured animals.7 - 9

The functional recovery evaluation scale of Basso, Beattie and Bresnahan (BBB)10 , 11 is the main scale used to quantify motor recovery in spinal cord-injured rats, which follows studies carried out by MASCIS (Multicenter Animal Spinal Cord Injury Study). Basso et al. 10 present a scale to evaluate locomotor recovery in rats with spinal cord injury at the levels of T VII, T VIII and T IX, based on the functional, sensory and motor responses, and that ranges from 0 to 21, and demonstrate that this scale is efficient and sensitive. We studied the frequency and the nature of errors in the interpretation of the BBB scale,12 and a combined method of evaluation was also suggested to reduce interpretation errors.

One of the difficulties referred to in experimental studies and in the use of the BBB scale is the lack of establishment of a standardized evaluation system to assess the degree of spinal cord injury, the determination of the most appropriate animal species,13 and the comparison of inter-evaluator results that present discrepancies.1 , 14 In this paper we propose a training facilitator approach, through an automated routine for interpretation of the Basso, Beattie and Bresnahan scale, through a computer program. The objective was to develop a computerized interpretative system that allows the BBB scale to be applied to rats with experimental spinal cord injury, aiming to reduce the discrepancies of scores given among researchers and to allow less experienced researchers, through the use of the scale, to achieve a performance similar to that of researchers with more experience in the application of this scale.

METHOD

This is an experimental prospective trial with Wistar rats. The trial was approved by the Ethics Research Committee of the University as it is in compliance with international ethical principles in animal research.

To test a computer program especially developed for this survey as an auxiliary tool in the issuance of scores from the BBB scale, 12 Wistar rats were submitted to a previous spinal cord injury, using Impactor methodology of the New York University - Impactor as standard in the production of spinal cord injuries.13 Three types of spinal cord injury were produced in the rats, using different heights for weight falling: of mild intensity (rats marked green, weight falling from a 12.5 mm), moderate (rats marked blue, height of 25 mm) and intense (rats marked black, 50 mm). The mild injury was achieved with the impact of a rod on the spinal cord, falling from a 12.5 mm height. In the moderate injury, the height was 25 mm and, in the severe injury, it was 50 mm.

On the 28th post-injury day, all the rats were filmed in free movement. The images of the rats in movement were edited to form four-minute blocks. The video recording was performed using three digital cameras simultaneously, positioned at three different points, at a distance of 50 cm from the animals to avoid losing any details of their movements.

Eight researchers from the laboratory of the School of Medicine Universidade de São Paulo (FMUSP) took part in this survey, and analyzed the images of the 12 Wistar rats. The two heads of the laboratory (AFC and GBS), with 15 years of experience in the application of the scale, initially evaluated the images of the 12 rats and, in mutual agreement, assigned a single score to each animal, based on the side of greater motor deficit or the side with the lowest score. The results of the two more experienced evaluators, with papers published on the subject of spinal cord injury,15 , 16 were considered the gold standard in the evaluation and used as a reference.

The gold standard evaluators assigned values from 0 to 21 according to the BBB scale (Table 1) to each rat, where zero corresponded to total absence of movements and 21, normal movements. The result of this evaluation is shown in Table 2. Afterwards, the six participating researchers received the same filmed images of the rats with the task of applying the BBB scale at three different times, with an interval of 15 days between them. Thus, each researcher evaluated the same rats at three different times.

Table 1.

21-point functional evaluation scale of Basso et al.10

Score	Operational definitions of categories and attributes
0	No observable movement of the hindlimbs.
1	Slight (limited) movement of one or two joints, usually hip and/or knee.
2	Extensive movement of one joint or extensive movement of one joint and slight movement of the other.
3	Extensive movement of two joints.
4	Slight movement of all three joints of the hindlimbs.
5	Slight movement of two joints and extensive movement of the third joint.
6	Extensive movement of two joints and slight movement of the third joint.
7	Extensive movement of the three joints in the hindlimbs.
8	Sweeping without weight bearing or plantar support of the paw without weight bearing.
9	Plantar support of the paw with weight bearing only in the support stage (i.e., when static) or occasional, frequent or inconsistent dorsal stepping with weight bearing and no plantar stepping.
10	Plantar stepping with occasional weight bearing and no forelimb-hindlimb coordination.
11	Plantar stepping with frequent to consistent weight bearing and occasional forelimb-hindlimb coordination.
12	Plantar stepping with frequent to consistent weight bearing and occasional forelimb-hindlimb coordination.
13	Plantar stepping with frequent to consistent weight bearing and frequent forelimb-hindlimb coordination.
14	Plantar stepping with consistent weight support, consistent forelimb-hindlimb coordination and predominantly rotated paw position (internally or externally) during locomotion both at the instant of initial contact with the surface as well as before moving the toes at the end of the support stage or frequent plantar stepping, consistent forelimb-hindlimb coordination and occasional dorsal stepping.
15	Consistent plantar stepping, consistent forelimb-hindlimb coordination and no movement of the toes or occasional movement during forward movement of limb; predominant paw position is parallel to the body at the time of initial contact.
16	Consistent plantar stepping and forelimb-hindlimb coordination during gait and movement of the toes occurs frequently during forward movement of the limb; the predominant paw position is parallel to the body at the time of initial contact and curved at the instant of movement.
17	Consistent plantar stepping and forelimb-hindlimb coordination during gait and movement of the toes occurs frequently during forward movement of limb; the predominant paw position is parallel to the body at the time of initial contact and at the instant of movement of the toes.
18	Consistent plantar stepping and forelimb-hindlimb coordination during gait and movement of the toes occurs consistently during forward movement of limb; the predominant paw position is parallel to the body at the time of initial contact and curved during movement of the toes.
19	Consistent plantar stepping and forelimb-hindlimb coordination during gait and movement of the toes occurs consistently during forward movement of limb; the predominant paw position is parallel to the body at the instant of contact and at the time of movement of the toes, and the animal presents a downward tail some or all of the time.
20	Consistent plantar stepping and forelimb-hindlimb coordination during gait and movement of the toes occurs consistently during forward movement of limb; the predominant paw position is parallel to the body at the instant of contact and at the time of movement of toes, and the animal presents consistent elevation of the tail and trunk instability.
21	Consistent plantar stepping and coordinated gait, consistent movement of the toes; paw position is predominantly parallel to the body during the whole support stage; consistent trunk stability; consistent tail elevation
	Definitions
Slight	Partial movement of the joint, below half the range of motion of the joint
Extensive	Partial movement of the joint, above half the range of motion of the joint
Pedaling movement	Rhythmic movement of the hind limb in which its three joints are extended, then fully flexed and once again extended. The animal generally leans sideways, the plantar surface of the paw may or may not touch the ground, no body weight bearing is evident over the entire surface of the rear paw
Without weight bearing	In the contraction of the extensor muscles of the hind limb during plantar stepping of the paw or no thigh elevation
With weight bearing	Contraction of the extensor muscles of the hind limb during plantar stepping of the paw or thigh elevation
Plantar stepping	The paw is in plantar contact with weight bearing, followed by the forward movement of the limb until plantar contact is re-established with weight bearing
Dorsal stepping	The weight is borne by the dorsal surface of the paw at any point of the step cycle
Coordination of the fore and hind limbs	For every step of the fore limb a step is taken with the hind limb and the hind limbs alternate
Occasional	Less than or equal to half of the times, < 50%
Frequent	More than half, but not always, 51- 94%
Consistent	Almost always or always, 95 – 100%
Trunk instability	Lateralization of weight that causes oscillation from one side to another or partial collapse of the trunk

Table 2.

Evaluations carried out by the researchers considered gold standard of reference for the others, according to the side.

Black rat	Left	Right	Lowest value
11	0	1	0
12	4	4	4
14	1	1	1
15	0	1	0
Blue rat	L	R	Lowest value
3	0	1	0
4	0	1	0
5	4	1	1
6	1	1	1
Green rat	L	R	Lowest value
12	4	1	1
15	11	10	10
16	13	13	13

These six evaluators carried out three analyses: a free application of the scale; a targeted application of the scale; and an automated application of the scale. The "free" evaluation" (FA) was based on the free classification of the motricity detected in the rat graded from 0 to 21, according to the intensity of the injury presented. The "targeted" evaluation" (TA) was based on 14 questions (especially formulated for this survey) about the normality of the selected segments, i.e., for every analysis segment, the evaluator is first asked to analyze the image, then to reply about the normality or non-normality of each segment, in sequential form. In the "automated" evaluation (AE), they used a computer program with the same questions as the TA. When these questions were answered, the computer program automatically issued a score from 0 to 21. (Table 3)

Table 3.

Computer program containing 14 questions

Program issues score from 0 to 21Automated Ev. AE	Left side	Answer below
Movement of the hind limb	1. Hip	1. None (  )Slight (  ) Extensive (  )
	2. Knee	2. None (  )Slight (  ) Extensive (  )
	3. Ankle	3. None (  )Slight (  ) Extensive (  )
4. Rhythmic circular movement such as pedaling, either touching the ground or not, but without weight bearing (conditional macro activated if 3 joints with extensive movement)		4. Conditional macro activated if Items 1,2,3 extensiveYes (  ) No (  )
5. Plantar support		5. Yes (  ) No (  )
6. Weight bearing		6. None (  )Occasional < 50% (  )Frequent 51 to 94% (  ) Consistent 95 to 100% (  )
7. Dorsal stepping (= step)		7. None (  )Occasional < 50% (  )Frequent 51 to 94% (  )Consistent 95 to 100% (  )
8. Plantar stepping		8. None (  )Occasional < 50% (  )Frequent 51 to 94% (  )Consistent 95 to 100% (  )
9. Coordination of fore leg stepping with hind leg stepping		9. None (  )Occasional < 50% (  )Frequent 51 to 94% (  )Consistent 95 to 100% (  )
10. Detach toes		10. None (  )Occasional < 50% (  )Frequent 51 to 94% (  )Consistent 95 to 100% (  )
11. Position paw initial contact		11. Internal rotation (  )External rotation (  )Parallel (  )
12. Elevation of paw		12. Internal rotation (  )External rotation (  )Parallel (  )
13. TAIL *down: tail touches the ground during steps		13. Lowered all the time (  ) Lowered most of the time (  ) Raised (  )
14. Trunk instability		Yes ( )                   No (  )
Name of researcher                                   Rat                       Color

The order in the mode of application of the scale and the order of the rats in the filming analyses were altered to avoid memorization by the researchers. Thus, while the first researcher could evaluated the rats in the order of 1, 5 and 8 with AE, after 15 days he or she could evaluate rats 8, 1 and 5 in the TA and after 15 days could evaluate rats 8, 5 and 1 in the FA, while the other researchers could simultaneously evaluate other rats in another order of evaluation, for example.

The participants adopted as an analysis standard the lowest value between the sides or the highest motor deficit value according to the international guidelines of the Ohio State University17 and in compliance with the rules of MASCIS;17 accordingly, rat number 13 presented a difference of more than three points between the sides, hence this rat was disregarded in the analysis.

Statistical analysis

The repeated measures analysis of variance with transformation by posts was used to compare evaluators and methods, and the Student's t-test for paired tests to compare methods, while the paired Wilcoxon test was used when the test assumptions were not satisfied. A significance level of 5% (p ≤ 0.05) was used and the checking of normality of the distributions was executed using the Kolmogorov-Smirnov and Shapiro-Wilk tests, while the statistical program adopted was the Statistical Package for Social Sciences (SPSS) version 15.0.

Box-plot graphs containing descriptive information were used to present non-parametric data.

RESULTS

According to Table 4, there is no significant difference between the automated method and the gold standard for each evaluator, from 1 to 6, since in all the comparisons, p > 0.07.

Table 4.

Comparison of the scores obtained with the automated, targeted and free method with the gold standard values.

Comparison between the automated method and the gold standard
	Automated	Gold standard	p-value
Evaluator 1
Mean (SD)	2.09 (3.3)	2.82 (4.49)	0.7335*
Median	1	1
Minimum – Maximum	0 – 9	0 - 13
Total	11	11
Evaluator 2
Mean (SD)	2 (3.35)	2.82 (4.49)	0.5580*
Median	1	1
Minimum – Maximum	0 - 12	0 - 13
Total	11	11
Evaluator 3
Mean (SD)	4 (4.47)	2.82 (4.49)	0.0760*
Median	1	1
Minimum – Maximum	1 - 13	0 - 13
Total	11	11
Evaluator 4
Mean (SD)	2.18 (3.87)	2.82 (4.49)	0.4911*
Median	1	1
Minimum – Maximum	0 - 13	0 - 13
Total	11	11
Evaluator 5
Mean (SD)	2.18 (2.79)	2.82 (4.49)	0.6845*
Median	1	1
Minimum – Maximum	0 - 10	0 - 13
Total	11	11
Evaluator 6
Mean (SD)	4.36 (4.39)	2.82 (4.49)	0.1583*
Median	3	1
Minimum - Maximum	0 - 13	0 - 13
Total	11	11
Comparison between the targeted method and the gold standard.
	Targeted	Gold standard	p-value
Evaluator 1
Mean (SD)	2.91 (4.32)	2.82 (4.49)	0.4784*
Median	1	1
Minimum - Maximum	0 - 12	0 - 13
Total	11	11
Evaluator 2
Mean (SD)	2.18 (3.95)	2.82 (4.49)	0.9416*
Median	1	1
Minimum – Maximum	0 - 14	0 - 13
Total	11	11
Evaluator 3
Mean (SD)	5 (4.65)	2.82 (4.49)	0.0836*
Median	5	1
Minimum – Maximum	1 - 15	0 - 13
Total	11	11
Evaluator 4
Mean (SD)	3.09 (4.99)	2.82 (4.49)	0.3441*
Median	1	1
Minimum – Maximum	0 - 17	0 - 13
Total	11	11
Evaluator 5
Mean (SD)	2.91 (3.21)	2.82 (4.49)	0.9163*
Median	2	1
Minimum – Maximum	0 - 10	0 - 13
Total	11	11
Evaluator 6
Mean (SD)	5.73 (5.27)	2.82 (4.49)	0.0145†
Median	3	1
Minimum – Maximum	0 - 14	0 - 13
Total	11	11
Comparison between the free method and the gold standard
	Free	Gold standard	p-value
Evaluator 1
Mean (SD)	2.45 (3.75)	2.82 (4.49)	0.3653*
Median	1	1
Minimum - Maximum	0 - 12	0 - 13
Total	11	11
Evaluator 2
Mean (SD)	2 (2.79)	2.82 (4.49)	0.3428*
Median	1	1
Minimum - Maximum	0 - 10	0 - 13
Total	11	11
Evaluator 3
Mean (SD)	4.27 (4.03)	2.82 (4.49)	0.1361*
Median	1	1
Minimum - Maximum	1 - 11	0 - 13
Total	11	11
Evaluator 4
Mean (SD)	4.55 (5.09)	2.82 (4.49)	0.0368*
Median	2	1
Minimum - Maximum	0 - 17	0 - 13
Total	11	11
Evaluator 5
Mean (SD)	2 (3.38)	2.82 (4.49)	0.5961*
Median	1	1
Minimum – Maximum	0 - 12	0 - 13
Total	11	11
Evaluator 6
Mean (SD)	5.64 (5.16)	2.82 (4.49)	0.0115†
Median	3	1
Minimum - Maximum	0 - 14	0 - 13
Total	11	11

Wilcoxon test;

Paired t test.

Considering the lower value between right and left sides, a significant difference was observed between the targeted method and the gold standard only for evaluator 6 (evaluator 6; p = 0.0145) as shown below in Table 4.

Significant difference was observed between the free method and the gold standard for evaluators 4 and 6 (evaluator 4; p = 0.0368; evaluator 6, p = 0.0115; Table 4).

The boxplot from Figure 1 shows that there was discrepancy of scores applied by evaluator 4 in relation to the gold standard when the latter applied the score freely; and discrepancy of scores applied by evaluator 6 in relation to the gold standard when the latter does so in free and targeted mode (FE, evaluator 4, p = 0.0368; evaluator 6, p = 0.0115; TE, evaluator 6, p = 0.0145).

Figure 1.

Boxplot showing the comparison os scores obtained by the evaluators 4 and 6 and the gold standard values.

According to the results of Table 5, there is no significant difference between the automated, targeted and free methods when compared with the gold standard for the mean of evaluators 1 to 6 (for AE, p = 0.5147, for TE, p = 0.0856 and for FE, p = 0.2132).

Table 5.

Comparison between the methods: mean of the evaluators.

Mean of the Evaluators	Automated	Gold standard	p-value
Mean (SD)	2.8 (2.88)	2.82 (4.49)	0.5147*
Median	1.5	1
Minimum – Maximum	0.5 - 9.83	0 - 13
Total	11	11
Mean of the Evaluators	Targeted	Gold standard	p-value
Mean (SD)	3.64 (3.42)	2.82 (4.49)	0.0856*
Median	2	1
Minimum - Maximum	0.33 - 10	0 – 13
Total	11	11
Mean of the Evaluators	Free	Gold standard	p-valor
Mean (SD)	3.48 (3.34)	2.82 (4.49)
Median	1.67	1	0.2132*
Minimum - Maximum	0.5 - 10.83	0 - 13
Total	11	11

The results of Table 4 show that, in the comparisons between free method and gold standard and targeted method and gold standard, the scores of evaluators 4 and 6 did not appear similar to the gold standard. The only method that obtained results similar to the gold standard for all the evaluators, from 1 to 6, was the automated analysis.

DISCUSSION

The development of surveys with reproducibility, accuracy and low cost leads to the acceptance and diffusion of various experimental models,1 , 14 , 18 yet some models that are used present problems in the production of spinal cord injuries3 for being low cost, controlled and standardized at all the injury levels.19 In 1995, Basso et al. 10 presented the scientific community with a scale for the evaluation of functional recovery of locomotor capacity in rats after spinal cord contusion, and they affirmed that the scale is a predictive measure based on specific observation criteria of the animal's movement, which assigned sequential and cumulative scores, corresponding to points from 0 to 21, called the BBB scale. This scale is currently the method of evaluation of functional recovery most commonly used in experimental research due to its simplicity, ease of application and practicality, having been adopted by MASCIS.17 Although widely used, it presents important discontinuities in its scaling: the levels of recovery from 0 to 6 are not of the same intensity as the levels of recovery from 7 to 14 and present different characteristics in their scores. Moreover, there is controversy regarding the best statistical methodology to be used.1 The scores obtained in the upper or lower range of the scale present different characteristics, i.e., the improvement of two points in the low part of the scale is different from the improvement of two points in the high part, which hinders accurate comparisons of surveys between laboratories.1 , 14 , 18 As it presents such discontinuities, in the distribution of scores, its interpretation is difficult. For it to be used in a standardized manner, we observe the need for specific training, with specialized professionals and a detailed statistical study.

As there is no "gold standard" of direct or indirect evaluation to determine the efficacy of the scale, different complementary or combined methods are used with the BBB to improve its sensitivity and reproducibility. In this study, the two heads of the laboratory of FMUSP, with 15 years of experience in the application of the scale and who published sereval papers in this line of research, were considered the gold standard in the evaluation.15 , 16 The BBB scale involves difficulty in the assignment of scores.14 A computer program was developed to help reduce the discrepancy of scores assigned to the same rat by different observers, in an attempt to bring the evaluation of a researcher with limited experience in the scale closer to that of researchers experienced in its application. The results of this study show that, in the comparisons between the "free" method and the reference of the gold standard measures and the "targeted" method and the gold standard, evaluators 4 and 6 did not appear similar to the gold standard. The only method that obtained results similar to the gold standard for all the evaluators was the automated computer-assisted analysis.

The elimination of discrepancies in relation to the gold standard, when the computer program was used, paves the way for its use as an auxiliary tool in the issuance of scores, especially for researchers who are either beginners or being trained, but does not eliminate the need for prior knowledge of the items analyzed in the BBB scale to enable the researcher to carry out a detailed analysis of the animal's movement.

CONCLUSIONS

The application of the BBB scale in the automated mode, using the computer program, did not present any difference in relation to the gold standard for all the evaluators.