Interobserver Reliability of Pirani Clubfoot Severity Scoring between an Orthopedic Surgeon, a Resident Doctor, and a Nonmedical Counsellor at a Clubfoot Clinic.

BACKGROUND: Ponseti method is the gold standard treatment for idiopathic congenital clubfoot. Pirani clubfoot severity score is a vital tool in assessing treatment of clubfoot. This study determines whether, after a short training in the Pirani scoring, a nonmedical personnel can be as accurate as a doctor in assessing the degree of deformity in clubfoot.
MATERIALS AND METHODS: This was a prospective observational study from January to September 2016. Pirani scoring of all children ≤6 months was done by the counsellor, an orthopedic resident, and a consultant separately. All the three members of the team were blinded of the other's score. The Pirani scoring of the consultant was taken as the most correct. The data were analyzed for interobserver reliability using the kappa statistic and point-by-point interobserver agreements.
RESULTS: One hundred and fifteen clubfeet in 75 children (48 males and 27 females) with an average age of 96 days were included in the study. Differences between the means of scores for each severity component of the deformity including the sum of midfoot scores, hindfoot scores, and total foot scores were <0.1. There was fair-to-substantial interobserver reliability of all the subcomponents when scores from the three independent observers were analyzed.
CONCLUSION: Our study successfully demonstrates that after a short training, Pirani score can be successfully used in assessing clubfoot severity by a nonmedical counsellor.

Introduction

Ponseti method of manipulation and serial plaster casting is the gold standard treatment for idiopathic congenital clubfoot.12 It is a simple and cost-effective method which can be executed by any trained medical or paramedical personnel. It is particularly suitable for health-care settings with scarce resources and limited professionals.3

Our institute is a tertiary referral center for patients from all across the region. We run a clubfoot specialty clinic at our institute which operates twice a week. Our clubfoot clinic team which comprises an orthopedic consultant, two postgraduate students on a 6-monthly rotation, and a nonmedical counsellor caters to almost 50 patients in a month. All our team members form an integral and essential part of the clinic. In an effort to streamline the working of our clinic and increase the number of patients to whom we can efficiently cater, we trained our counsellors in assessing the severity of clubfoot by Pirani scoring.

Pirani scoring is a vital tool for the initial assessment and followup of clubfoot patients.45 The counsellor is a key figure in managing the clubfoot program; from counseling the parents to carrying out field visits in assigned communities, they extend services in remote health centers and district hospitals where trained human power may be lacking. This study was carried out with the purpose of assessing the efficiency of a trained counsellor in Pirani scoring so that their services can be taken more efficiently in situations where trained medical personnel are not readily available. There are studies comparing the reliability of Pirani scoring by paramedical personnel,67 but no such study has been undertaken to review the reliability of a nonmedical professional.

This study determines whether, after a short training in the Pirani scoring, a nonmedical personnel can be as accurate as a doctor in assessing the degree of deformity in clubfoot.

Materials and Methods

We conducted a prospective observational study at our clubfoot clinic from January to September 2016. All idiopathic clubfoot children with age ≤6 months were included in the study. Syndromic, neurogenic, neglected, and clubfoot children of >6 months’ age were excluded from the study.

A written informed consent was taken from the parents of all children enrolled in the study, in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki Declaration and its later amendments.

Pirani clubfoot severity score

Pirani et al.8 devised a scoring system for clubfoot based on clinical signs of contracture. The Pirani scoring system includes six clinical signs; three in the midfoot and three in hindfoot [Table 1].

Table 1

Pirani clubfoot severity score

Each clinical sign is scored either 0, 0.5, or 1 depending on its severity. Thus, each foot can receive a HFS between 0 and 3, a MFS between 0 and 3, and a total score (TS) between 0 and 6.

Severity scoring of all children included in the study was first done by the counsellor using the Pirani method, along with their regular record keeping and counseling sessions. Thereafter, the child was sent to the consultant in outpatient department where a regular checkup of the feet was done. The consultant did his independent Pirani scoring on the patient. Finally, at the time of casting, the resident doctor in the clubfoot team did the Pirani scoring before applying the corrective casts. All the three members of the team were blinded of the other's score. The Pirani score of the consultant was taken as the gold standard.

Statictical analysis

The data were analyzed for interobserver reliability using the kappa statistic and point-by-point interobserver agreement. Kappa statistic is a chance-corrected measure of agreement for particular data.9 It compares the observed agreement with the level of agreement expected by chance alone. The maximum value of 1.0 means that every assessor agrees on the point. A value of 0 indicates no more agreement than expected by chance alone. The kappa statistic for interobserver reliability (strength of agreement) was judged, values ≤0 as indicating no agreement and 0.01–0.20 as none to slight, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost perfect agreement.9

Results

Pirani severity score of 115 clubfeet in 75 children (48 males and 27 females) were measured independently by three different observers at the clubfoot clinic. The average age of children was 96 days with a range from 12 days to 6 months. Difference between the mean of scores for each severity component of the deformity including the sum of MFS, HFS, and total foot scores was <0.1 [Table 2].

Table 2

Means of scores for each parameter for all the three observers

The scores were analyzed by the kappa statistic for reliability between any two independent observers [Table 3]; interobserver reliability among all the three observers combined was also analyzed [Table 4]. PC had the highest kappa value (0.763) and EH had the lowest value (0.493) when the scores of the consultant and the counsellor were compared. CLB showed substantial agreement with a kappa value of 0.691 and EH had moderate agreement with a score of 0.468, between consultant and residents. When the resident and the counsellor scores were analyzed, EH had a perfect agreement, whereas LHT had the lowest score and had only fair agreement. There was fair-to-substantial interobserver reliability of all the subcomponents when scores from the three independent observers were analyzed together.

Table 3

Agreement between consultant, counsellor, and resident in assessment of various parameters

Table 4

Overall agreement between three raters (i.e., consultant, counsellor, and resident) in assessing different parameters

Percentages of point-by-point agreement between consultant and the resident doctor had a mean value of 84.9%. The mean percentage was 83.8% between consultant and counsellor whereas it was 85.6% between resident and the counsellor [Table 5].

Table 5

Percentage of point-to-point agreement for each Pirani score parameter between both observers

Discussion

Congenital talipes equinovarus or clubfoot is one of the most common congenital orthopedic conditions with an incidence of approximately 1:1000 live births.1210 Grading the severity of deformity and assessing its response to treatment, form an important component in clubfoot management. Many classification systems have been proposed811121314151617 for this purpose. The reliability of these systems has been evaluated in several studies. Wainwright et al.18 assessed the reliability of four classification systems, namely, Ponseti and Smoley,10 Harrold and Walker,11 Catterall,12 and Diméglio et al.13 Nine children (13 clubfeet) were assessed by four examiners at different stages in the first 6 months of life (=180 examinations). The results showed kappa values varying between 0.14 and 0.77. Even though Dimeglio classification system had the greatest reliability, none of the systems were found to be entirely satisfactory. Ghanem et al.12 evaluated the results of treatment of 35 patients with unilateral clubfoot and analyzed them according to 13 different rating scores available in literature. They concluded that none of the rating systems were ideal and suggested a new and more objective classification system should be developed. Cummings et al.,19 Catterall,12 and Steck and Robertson20 also had concerns about the lack of consistent results in clubfoot severity scoring.

Munsh et al.21 evaluated fifty idiopathic clubfeet by the scoring systems of Laaveg and Ponseti,15 McKay,16 Magone et al.,17 and Ghanem and Seringe.14 Although there was a good correlation between the scores, there was very poor agreement between the grading of feet by these different systems.

Pirani et al.8 developed a method of clinically evaluating the degree of deformity in clubfoot. This scoring system has become widely accepted and popular among the health personnel owing to its simplicity, reliability, and ease of application.22 A foot can be assessed in less than a minute and no technical equipment is required.482223 In addition, there is a significant correlation between the initial score and the number of casts required for correction.22 A higher score also correlates with a higher chance of the patient requiring tenotomy for complete correction.4

Several studies67202122232425262728 have been undertaken to study the validity and interobserver reliability of Pirani scoring system among trained health personnel [Table 6]. Cosma andVasilescu24 in a study to determine the reproducibility of Pirani and Dimeglio classification systems found a high Pearson's coefficient correlation (>0.85) for both the systems. They even suggested a simultaneous use of both systems as they are different and complemented each other. Flynn et al.25 performed an independent assessment of Pirani and Dimeglio classification systems. The correlation coefficients were found to be 0.90 (P = 0.0001) for the Pirani classification and 0.83 (P = 0.0001) for the Dimeglio classification. Both classification systems had a good interobserver reliability after an initial learning phase.

Table 6

Studies on interobserver reliability of Pirani scoring system for clubfoot severity

Pirani et al.22 found the interobserver strength of agreement in Pirani scoring to be substantial or almost perfect among three independent observers, with kappa score of TS, MFS, and HFS to be 0.92, 0.91, and 0.86, respectively. In a similar study, Jain et al.26 measured the interobserver reliability of Pirani score between five orthopedic surgeons at a foot deformity correction camp. Interobserver variability was calculated using kappa statistic for each of the signs. There was substantial agreement for TS (TS kappa-0.71) and also for MFS (0.68) and HFS (0.66) separately.

Majority of the clubfoot cases (approximately 80%) are reported from developing countries.27 There is a significant shortage of trained health-care personnel in developing countries to manage the clubfoot program. This shortage is mostly compensated by taking help from various nongovernmental organizations (NGOs), paramedical staff, and allied health-care personnel. Pirani scoring, which has become an integral part of the clubfoot management, has a proven substantial interobserver reliability among trained health-care personnel, but its efficiency in the hands of nonmedical and paramedical staff is still a matter of debate. Shaheen et al.6 measured the interobserver reliability of Pirani score between a physiotherapy assistant and an orthopedic surgeon in 91 virgin clubfeet in 54 infants at the Sudan clubfoot clinic. Overall, there was a moderate-to-substantial interobserver reliability for the Pirani clubfoot severity score and all its subcomponents.

Jillani et al.7 compared the reliability of Pirani score between an orthopedic surgeon and a plaster technician who had 2-year operation theater technician diploma. They found interobserver reliability to be fair to substantial with point-to-point interobserver agreement for all components of deformity to be 82%.

Counsellors form an essential part of the clubfoot treatment program.28 These are nonmedical personnel usually associated with an NGO who lend their support to the working of a clubfoot clinic. Their efficiency in effective and accurate Pirani scoring can be of great advantage to the clubfoot team. The present study was undertaken to determine whether, after a short training in the Pirani scoring, a nonmedical personnel can be as accurate as a doctor in assessing the degree of deformity in clubfoot.

The mean scores for each parameter were comparable (difference <0.1 of each other) [Table 2]. Overall, there were substantial agreement for PC and CLB and fair agreement for EH and MC when scores from the three independent observers were analyzed [Tables 3 and 4].

The interobserver reliability of a nonmedical counsellor was found to be comparable to a resident doctor keeping the scores of the consultant as gold standard. The nonmedical counsellor included in our study had no previous training in medical or paramedical branches. They had been a part of our team from the past 4 years and had been actively assisting the doctors in managing clubfoot patients as far as their record keeping and counseling is concerned. The counsellor as a part of the job was aware of the various deformities and scoring system of clubfoot. It was an integral part of their duty to fill the Pirani scores given by the doctors in their records and to follow them up in next visits. We provided a short training in Pirani scoring and they developed the scoring ability equal to a trained medical personnel.

A counsellor's efficiency in effective scoring of clubfoot children can help the program in a big way. The counsellors are required to make frequent field visits in nearby areas for creating awareness regarding clubfoot and followup the patients under treatment. During these visits, they can easily counsel the parents about the approximate number of casts that will be required the probability of tenotomy in the patient and can easily detect any relapse at the earliest, all this with a simple knowledge of Pirani scoring. In the clinic, the treating doctor develops a confidence in their ability to assess the degree of deformity in a clubfoot. Therefore, the trained counsellor can assist in performing this aspect of care in the clubfoot clinic, relieving the doctor of the time burden associated with the assessment.

There are a few key questions which may arise regarding the appropriateness of the research. There may be doubts whether the counsellors will apply the instruments in practice, whether study patients represent the population that will be rated and whether data have been analyzed using appropriate statistics.29 In this study, all these were taken into consideration. There were a few limitations of the study, the sample size was small, and it is difficult to generalize the efficiency of a counsellor which may vary from center to center, person to person, and upon the duration of association with the clubfoot team. Intraobserver reliability was not analyzed as it would have required multiple examinations of the same child, which logistically would have been difficult, and also, it would not have contributed to the research question as the researcher would not have known whether the scoring has been done correctly or not; further multicentric trials on this research questions may be needed to validate the findings.

Conclusion

Our study suggests that after a short training, Pirani score can be effectively used in assessing clubfoot severity by a nonmedical counsellor. This is particularly useful in developing countries, where orthopedic surgeons are few. Effective use of Pirani scoring in the hands of counsellors will go a long way in increasing the number of children which a clinic can cater and also improve the quality of service provided.

Declaration of patient consent

The authors certify that they have obtained all appropriate patient consent forms. In the form the patient(s) has/have given his/her/their consent for his/her/their images and other clinical information to be reported in the journal. The patients understand that their names and initials will not be published and due efforts will be made to conceal their identity, but anonymity cannot be guaranteed.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.