Reliability of American Society of Anesthesiologists physical status classification.

INTRODUCTION

The American Society of Anesthesiologists Physical Status (ASA-PS) classification is a widely used grading system for the pre-operative health of a surgical patient. It was originally developed in 1941 by Saklad et al. and then modified in 1961 by Dripps et al.[1] into a five-class version.

According to other researchers, the ASA-PS classification should be modified and adapted to the paediatric population because there are many differences between the physiology and pathology of adults and children. Many studies have tested the relation between the ASA classification and several outcomes[23] such as mortality, cardiac arrest, morbidity, length of stay and predictors of blood loss. The reliability of the ASA-PS classification has been widely evaluated,[45] but there are different conclusions on the ASA-PS classification reliability. There is no agreement on the level of reliability of the scale.

We conducted a systematic review on the state of studies on the reliability of the ASA-PS classification. To our knowledge, there is only one review on the ASA-PS classification,[6] and there are no systematic reviews on its reliability.

The primary aim was to check the state of studies on the reliability of the ASA-PS classification for the broad population of adults and children waiting for surgery.

METHODS

The questions for the review were as follows: (1) What is the level of reliability of the ASA-PS system among the selected studies? (2) How is the quality of reporting among published studies on reliability of the ASA-PS system? (3) How is the quality of statistical methodology among published studies on reliability of the ASA-PS system?

We used the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Guideline for the first part of the review protocol, the selection of studies. Then, we used a modified version of the Standards for the Reporting of Diagnostic Accuracy (STARD)[7] and Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Guidelines (http://www.strobe-statement.org) to analyse the quality of reporting among studies selected [Appendix 1 and 2]. Finally, we used the Statistical Analyses and Methods in the Published Literature (SAMPL) Guidelines (http://www.equator-network.org/reporting-guidelines/sampl) to test the quality of statistical methodology among the collected studies.

The outcome measures were reliability tested using the k statistic, Cohen's kappa or the intra-class correlation coefficient; the percentage of STARD, STROBE, SAMPL items respected.

The inclusion and exclusion criteria and the methodology to assess the risk bias (of individual studies and of the cumulative evidence of included studies) are shown in Appendix 3.

The systematic search of the international literature published from 1941 through 30 November 2014 was performed using keywords and strategy as shown in Appendix 4. After literature searches, we found 693 records. The selection of articles included in the review was performed in a three-phase process [Figure 1] according to PRISMA Guidelines and is shown in the Appendix 3.

Figure 1

Review process

RESULTS

We collected 13 studies for final analysis Table 1. Three of the studies collected were conducted on a paediatric population and only two were done on real patients. The prevalent study design was observational with scenarios. We found only one multi-centric study.

Table 1

Characteristics of studies selected

The ASA-PS classification reliability was tested among anaesthesiologists and nurses from public and private hospitals.

Eight of the 13 studies tested the inter-rater reliability using the kappa statistic, but the researchers used a very heterogeneous statistical methodology namely, un-weighted, weighted, quadratic kappa. According to the Landis and Koch terminology and classification of kappa value,[8] two of the eight studies found fair inter-rater reliability (k range = 0.21–0.40), three had moderate reliability (k range = 0.47–0.53) and three had good reliability (k range = 0.61–0.82).

All studies conducted on children found a moderate (k range = 0.47–0.50) inter-rater reliability.

Seven studies respected less than 60% of STARD items, and eight respected less than 60% of STROBE and SAMPLE items [Table 1].

None of the studies selected met all 25 items of the STARD, STROBE and SAMPL guidelines.

The studies that met more items of the STARD and STROBE checklists were those of Ringdal et al., Sankar et al. and Cuvillon et al.[5910] All of these studies respected more than 80% of items of both checklists Table 1; however, only Sankar et al. and Cuvillon et al. respected more than 80% of SAMPL items.

DISCUSSION

In this review, the ASA-PS classification shows a wide inter-rater agreement range among all studies included, from fair to very good agreement; however, there was a prevalence of moderate agreement. Seven of the nine studies reported a k inter value higher than 0.4.

Because there are limited data on intra-rater reliability for ASA-PS classification, we think future studies should be planned on these topics. Finally, there are few data on the reliability for patients included in ASA Classes V and VI and limited data on the ASA-PS scale performance with younger children, but it shows moderate agreement with the research available.

We chose to plan a review on the ASA-PS classification reliability because the reliability is a fundamental characteristic of clinical scale.

There is an inter-rater reliability and intra-rater reliability for clinical scores. They are usually analysed using the k statistic, Cohen's kappa.

The wide inter-rater agreement range among all studies included could be explained by the fact that there is a wide discrepancy on the statistical methodology used.

Among the studies collected, we found a prevalence of moderate agreement for the ASA-PS classification; this does not mean that the classification has bad performance, but it could be caused by the bad educational training of the raters.

Furthermore, in our opinion, many previous studies on ASA-PS reliability have several limitations in the methodology: very few studies used a statistical methodology to estimate the right sample size of scenarios or patients; many studies used very few scenarios; finally, almost all of the previous studies used paper scenarios instead of real patients.

The quality of future research on ASA-PS classification should improve: there need to be prospective multi-centre studies based on real patients, planned with a better statistical methodology.

CONCLUSION

The ASA-PS classification seems to have a wide range of inter-rater agreement with a prevalence of moderate value. The administrative staff should be careful to use the ASA-PS classification for administrative billing procedures because of its heterogeneous reliability. Moreover, the physicians should consider the moderate and wide range of agreement of the classification when they use it for general communications. Ideally before using the ASA-PS classification, a test on its reliability among the users should be performed.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.