Assignment of ASA-physical status relates to anesthesiologists' experience: a survey-based national-study.

BACKGROUND: The American Society of Anesthesiologists physical status (ASA-PS) is a grading system adopted worldwide by anesthesiologists to classify the overall health status of patients. Its importance is demonstrated not only by its routinely use in clinical practice, but also by its deployment in other healthcare-related environments. However, a weak/moderate inter-observer agreement for ASA-PS has been previously shown and, although definitions and clinical examples of each class were provided by ASA, doubts remain on the individual factors influencing the assignment to an ASA-PS class. The aim of this study was to investigate whether and how an anesthesiologist's experience conditions the classification into a specific ASA-PS class.
METHODS: An online survey presenting 8 fictitious patients was administered to a group of Italian anesthesiologists and residents of different experience. Respondents were asked to assign each one of the 8 patients to a specific ASA-PS class. For the comparisons, anesthesiologists were subdivided into 5 classes according to the experience as anesthesiologists.
RESULTS: Six hundred one surveys were correctly completed. The highest mean number of correct answers was obtained by residents, and this number decreased progressively with increasing work experience. The lowest value was recorded in the most experienced group (≥ 20 years of experience).
CONCLUSIONS: Low inter-reliability and experience-dependence of ASA-PS must be taken into account when evaluating a patient, particularly in settings where wide differences in experience are present.

Introduction

The American Society of Anesthesiologists physical status (ASA-PS) is a grading system first developed in 1941 to “devise a system for the collection and tabulation of statistical data in anesthesia,” allowing anesthesiologists to record the overall health status of patients before surgery and to properly stratify outcomes [1]. The ASA-PS was profoundly modified in 1962 [2] when changes proposed by Dripps et al. [3] were adopted. The document has since been modified, with the latest amended version was released in 2014 [4].

Although definitions and clinical examples of each class are provided by ASA, doubts remain on the individual factors such as personal experience and workplace habits influencing assignment to an ASA-PS class. Several studies have investigated the inter-rater reliability of the ASA-PS in adult [5-8] and pediatric [9] populations, but only weak/moderate inter-rater agreements were found (k-values between 0.21 and 0.53). Nevertheless, possible associations between the ASA-PS scores and specific intraoperative and postoperative complications have been widely reported in the literature [10].

In fact, the ASA-PS has an important role for anesthesiologists: it is routinely used in clinical practice and in policy-making, performance evaluation, resource allocation, and reimbursement of anesthesia services. It is also frequently adopted as a reference parameter in clinical research. The ASA-PS score consists of six classes [4] (I to VI), with the addition of an “E” to any ASA-PS class that denotes patients needing emergency surgery.

To date, previous studies [11-13] have focused on inter-rater reliability, both within and between anesthesiologists and non-anesthesiologists, but no data are available on the effect of the level of anesthesiology experience and knowledge on assigning ASA-PS classes.

The primary aims of our study were to understand whether ASA-PS scores are properly assigned by anesthesiologists, in simulated, real-life clinical cases, and to investigate a possible link between the assigned ASA-PS score and the clinical experience/knowledge of the assigning anesthesiologists.

The secondary aim of our study was to investigate the inter-rater reliability between participants in relation to different clinical experiences.

Materials and Methods

An online questionnaire was created using SurveyMonkey internet-polling software (http://www.surveymonkey.com). The survey consisted of nine multiple-choice questions. The first question asked for the years of experience as an anesthesiologist: “anesthesiology resident,” “less than five years,” “five to nine years,” “ten to nineteen years,” “twenty or more years,” or other (emergency department physicians). The following eight questions pertained to clinical cases.

The questionnaire was created to allow respondents to remain anonymous. Electronic informed consent was obtained from each participant at the beginning of the survey and both the informed consent and questionnaire were written in Italian. The survey was administered to an Italian closed-group of anesthesiologists present on Facebook (http://www.facebook.com) and consisted of 4,901 members at the time of the survey. Admission to this group is allowed by a small group of administrators and only after each member’s identity and workplace have been identified and confirmed. Membership is permitted only to anesthesiologists, anesthesiology residents, and rarely to emergency department physicians. Questionnaires received from non-anesthesiologists were excluded because Italian emergency department physicians do not receive extensive anesthesiology training. Only one submission per computer, based on IP address, was allowed. The study was approved by the Institutional Ethical Committee of Padua (ref. AOP/2018).

The questionnaire presented eight clinical questions consisting of fictitious clinical cases. The task of each respondent was to assign the correct ASA-PS score, from I to VI, based on personal experience and knowledge. All cases were built following the ASA-PS Classification System and ASA-approved examples (Table 1) [10] to elicit an univocal correct ASA-PS score in each case.

Table 1.

The ASA-Physical Status Classification System Assignment [10]

ASA-PS classification	Definition	Examples
I	A normal healthy patient	Healthy, non-smoking, no or minimal alcohol use
II	A patient with mild systemic disease	Mild diseases only without substantive functional limitations. Examples include (but not limited to): current smoker, social alcohol drinker, pregnancy, obesity (30 < BMI < 40), well-controlled DM/HTN, mild lung disease
III	A patient with severe systemic disease	Substantive functional limitations; One or more moderate to severe disease. Examples include (but not limited to): poorly controlled DM or HTN, COPD, morbid obesity (BMI ≥ 40), active hepatitis, alcohol dependence or abuse, implanted pacemaker, moderate reduction of ejection fraction, ESRD undergoing regularly scheduled dialysis, premature infant, history (> 3 months) of MI, CVA, TIA, or CAD/stents
IV	A patient with severe systemic disease that is a constant threat to life	Examples include (but not limited to): recent (< 3 months) MI, CVA, TIA, or CAD/stents, ongoing cardiac ischemia or severe valve dysfunction, severe reduction of ejection fraction, sepsis, DIC, ARD or ESRD not undergoing regularly scheduled dialysis
V	A moribund patient who is not expected to survive without the operation	Examples include (but not limited to): ruptured abdominal/thoracic aneurysm, massive trauma, intracranial bleed with mass effect, ischemic bowel in the face of significant cardiac pathology or multiple organ/system dysfunction
VI	A declared brain-dead patient whose organs are being removed for donor purposes

ASA: American Society of Anesthesiologists, BMI: body mass index (kg/m2), DM: diabetes mellitus, HTN: hypertension, COPD: chronic obstructive pulmonary disease, ESRD: end-stage renal disease, MI: myocardial infarction, CVA: cerebrovascular accident, TIA: transient ischemic attack, CAD: coronary artery disease, DIC: disseminated intravascular coagulation, ARD: acute renal disease.

The cases presented in the questionnaire are presented below. To further support reliance on the anesthesiologists’ own experience in completing the questionnaire, the ASA-PS classification table (Table 1) was not presented and responders were explicitly asked not access any ASA-PS guidelines before or while taking the survey.

Case 1

An 18-year-old patient presents with acute appendicitis. He is healthy, non-smoking with minimal alcohol use (correct answer: ASA-PS IE).

Case 2

A 55-year-old female patient presents for total gastrectomy. She has a past medical history of hypertension and insulin-dependent type 2 diabetes mellitus; furthermore, she has a recent history of myocardial infarction 80 days ago. The coronary angiography reported non-significant coronary lesions and echocardiography (executed ten days ago) reported only a significant reduction in ejection fraction (EF 40%). The day before surgery you find the patient in good condition at physical examination (correct answer: ASA-PS IV).

Case 3

An 89-year-old man presents for cholecystectomy. He denies any present or past medical problem and denies any medication intake. He lives alone on the second floor without a lift. Physical exam and vital signs are within normal limits on the day of surgery (correct answer: ASA-PS I).

Case 4

A 40-year-old patient presents for sleeve gastrectomy. He has a past medical history of hypertension in good control and obesity (body mass index [BMI] 38 kg/m2) (correct answer: ASA-PS II).

Case 5

A 70-year-old patient presents for umbilical hernia. He has past medical history of hypertension in good control, he has a stage 1 chronic kidney disease and he takes statins for hypercholesterolemia. He is in good health status. He is able to walk up at least two flights of stairs without any fatigue or dyspnea. Preoperative blood findings are all normal (correct answer: ASA-PS II).

Case 6

A 61-year-old man weighing 70 kg presents at the emergency room for a ruptured abdominal aneurysm. Poorly controlled hypertension, insulin/dependent type 2 diabetes mellitus and hypercholesterolemia are known from the history. He had previously undergone surgery for appendectomy, cholecystectomy and bilateral carotid endarterectomy. Preoperative blood pressure is 80/60 mmHg, heart rate 110 beats/min, and pulse-oximetric saturation is 90% (correct answer: ASA-PS VE).

Case 7

A 40-year-old male patient presents for total right knee replacement. Past medical history is relevant only for obesity (BMI 41 kg/m2). He has no other health problems; he denies any medication intake. Laboratory findings are normal (correct answer: ASA-PS III).

Case 8

A 25-year-old female patient presents with acute appendicitis. She is 30 weeks pregnant and no problems related to pregnancy were revealed during prenatal care. She is healthy, non-smoking, without anything notable in her past medical history (correct answer: ASA-PS IIE).

We considered Case 1, Case 6, and Case 8 as emergency clinical case and for this reason the respondents were asked to assign a score from IE to VIE.

Statistical analysis

The independent variable for statistical analysis was the number of years of experience as an anesthesiologist and was divided into five categories: 1) residents in anesthesiology; 2) anesthesiologists with less than 5 years of experience; 3) 5 to 9 years of experience; 4) 10 to 19 years of experience; and 5) equal or more than 20 years of experience. Data were tested for normality with the Shapiro-Wilk test and the mean number of correct answers (possible range 0–8) was determined for each respondent group. Group differences were assessed using one-way ANOVA followed by post-hoc Tukey-HSD tests. Inter-rater agreement was analyzed utilizing Fleiss’s Kappa, fitting for a comparison between multiple raters. All analyses were performed using Microsoft Excel 2016 software (Microsoft Inc., USA). P values < 0.05 were considered statistically significant.

Results

Six hundred twenty-five questionnaires were submitted but 24 were excluded (3.9%). Specifically, 23 submissions were incomplete (3.7%) and one was submitted by an emergency medical doctor (0.2%). Thus, 601 respondents were included in the analysis: 514 were anesthesiologists, of which 150 (25%) had less than 5 years of work experience, 132 (22%) had 5 to 9 years of experience, 130 (21.6%) had 10 to 19 years of experience, and 102 (17%) had 20 or more years of experience. Eighty-seven respondents (14.7%) were anesthesiology residents.

Table 2 presents the distribution of the ASA-PS Classification System assignment for each group and each case. The correct answer is highlighted in asterisk.

Table 2.

Distribution of Answers among Cases and Participants

	ASA I	ASA II	ASA III	ASA IV	ASA V
Case 1
Resident	98.8 (86)*	1.2 (1)	0.0 (0)	0.0 (0)	0.0 (0)
< 5 yr	96.0 (144)*	4.0 (6)	0.0 (0)	0.0 (0)	0.0 (0)
From 5 to 9 yr	98.5 (130)*	1.5 (2)	0.0 (0)	0.0 (0)	0.0 (0)
From 10 to 19 yr	93.0 (121)*	7.0 (9)	0.0 (0)	0.0 (0)	0.0 (0)
≥ 20 yr	98.0 (100)*	2.0 (2)	0.0 (0)	0.0 (0)	0.0 (0)
Case 2
Resident	0.0 (0)	12.6 (11)	80.4 (70)	7.0 (6)*	0.0 (0)
< 5 yr	0.0 (0)	10.0 (15)	79.3 (119)	11 (16)*	0.0 (0)
From 5 to 9 yr	0.0 (0)	12.9 (17)	76.5 (101)	9.1 (12)*	1.5 (2)
From 10 to 19 yr	0.0 (0)	13.0 (17)	78.5 (102)	8.5 (11)*	0.0 (0)
≥ 20 yr	0.0 (0)	10.8 (11)	86.3 (88)	2.9 (3)*	0.0 (0)
Case 3
Resident	34.5 (30)*	63.2 (55)	2.3 (2)	0.0 (0)	0.0 (0)
< 5 yr	35.3 (53)*	59.3 (89)	4.7 (7)	0.7 (1)	0.0 (0)
From 5 to 9 yr	40.9 (54)*	56.8 (75)	2.3 (3)	0.0 (0)	0.0 (0)
From 10 to 19 yr	32.3 (42)*	57.7 (75)	10.0 (13)	0.0 (0)	0.0 (0)
≥ 20 yr	15.7 (16)*	69.6 (71)	14.7 (15)	0.0 (0)	0.0 (0)
Case 4
Resident	0.0 (0)	49.4 (43)*	50.6 (44)	0.0 (0)	0.0 (0)
< 5 yr	0.0 (0)	47.3 (71)*	51.3 (77)	1.4 (2)	0.0 (0)
From 5 to 9 yr	0.0 (0)	51.5 (68)*	46.2 (61)	2.3 (3)	0.0 (0)
From 10 to 19 yr	0.7 (1)	53.0 (69)*	44.6 (58)	1.5 (2)	0.0 (0)
≥ 20 yr	0.0 (0)	55.0 (56)*	44.1 (45)	0.9 (1)	0.0 (0)
Case 5
Resident	0.0 (0)	38.0 (33)*	57.4 (50)	4.6 (4)	0.0 (0)
< 5 yr	0.0 (0)	48.0 (72)*	48.0 (72)	4.0 (4)	0.0 (0)
From 5 to 9 yr	0.0 (0)	50.0 (66)*	47.0 (62)	3.0 (4)	0.0 (0)
From 10 to 19 yr	0.0 (0)	53.1 (69)*	45.4 (59)	1.5 (2)	0.0 (0)
≥ 20 yr	5.0 (5)	45.0 (46)*	44.0 (45)	6.0 (6)	0.0 (0)
Case 6
Resident	0.0 (0)	0.0 (0)	6.9 (6)	46.0 (40)	47.1 (41)*
< 5 yr	0.0 (0)	1.3 (2)	8.7 (13)	54.0 (81)	36.0 (54)*
From 5 to 9 yr	0.0 (0)	0.0 (0)	9.8 (13)	58.0 (76)	32.6 (43)*
From 10 to 19 yr	0.0 (0)	0.8 (1)	12.3 (16)	64.0 (83)	23.1 (30)*
≥ 20 yr	0.0 (0)	1.0 (1)	17.6 (18)	61.0 (62)	20.6 (21)*
Case 7
Resident	1.1 (1)	50.6 (44)	48.3 (42)*	0.0 (0)	0.0 (0)
< 5 years	0.7 (1)	58.0 (87)	40.6 (61)*	0.7 (1)	0.0 (0)
From 5 to 9 yr	2.3 (3)	58.3 (77)	38.6 (51)*	0.8 (1)	0.0 (0)
From 10 to 19 yr	4.6 (6)	66.9 (87)	27.7 (36)*	0.8 (1)	0.0 (0)
≥ 20 yr	7.8 (8)	63.7 (65)	27.5 (28)*	1.0 (1)	0.0 (0)
Case 8
Resident	1.1 (1)	50.6 (44)*	48.3 (42)	0.0 (0)	0.0 (0)
< 5 yr	26.0 (39)	62.0 (93)*	12.0 (18)	0.0 (0)	0.0 (0)
From 6 to 9 yr	30.3 (40)	62.1 (82)*	6.8 (9)	0.8 (1)	0.0 (0)
From 10 to 19 yr	35.4 (46)	55.4 (72)*	7.7 (10)	0.8 (1)	0.8 (1)
≥ 20 yr	40.2 (41)	49.0 (50)*	10.8 (11)	0.0 (0)	0.0 (0)

Values are expressed as percentage (number). The correct answer is highlighted in asterisk.

The mean number of correct answers was significantly different among groups (P < 0.001). Post hoc analysis indicated that the highest mean number of correct answers was obtained by residents (3.95 ± 1.13) and was significantly higher compared to the “10 to 19 years” group (3.46 ± 1.24, P = 0.034) and “≥ 20 years” group (3.14 ± 0.96, P < 0.001). In fact, the “≥ 20 years” group recorded the lowest mean number of correct answers, performing significantly worse than the “< 5 years” group (3.76 ± 1.37, P < 0.001) and “6 to 9 years” group (3.83 ± 1.31, P < 0.001) (Fig. 1).

Fig. 1.

Mean number of correct answers per group. Respondents were divided into five groups according to years (yr) of experience as an anesthesiologist. Group differences were assessed using one-way ANOVA followed by post-hoc Tukey-HSD tests. *P < 0.05, †P < 0.01.

No other statistically significant differences were observed. Table 3 reports the Fleiss’ k for all raters and for each group.

Table 3.

Inter-rater Reliability Tested with Fleiss’ k

Groups	k
All raters	0.38
Resident	0.41
< 5 yr	0.37
From 5 to 9 yr	0.38
From 10 to 19 yr	0.36
≥ 20 yr	0.38

Discussion

The ASA-PS classification is used routinely by anesthesiologists worldwide. Recently, it was found to be an independent predictor of post-operative complications and mortality [11]. Thus, the ASA-PS could potentially be employed as an effective risk stratification metric for medical complications and mortality after surgery, but additional data are needed to confirmed these findings.

Nevertheless, the ASA-PS has often been criticized for its weakness in both adult and pediatric settings, and a moderate inter-observer reliability has been reported in several studies [5-9]. A recent systematic review concluded that physicians should always consider the moderate value of the inter-rater agreement of this classification (confirmed by wide variability throughout different studies), while administrative staff should rely less on the ASA-PS classification for billing procedures because of its heterogeneous reliability [12]. In fact, the current healthcare system emphasizes a pay-for-performance model instead of a fee-for-service model, which was used in the past, and hospitals have to develop performance metrics in order to receive reimbursement payments.

Nonetheless, as observed by Ranta et al. [7], the ASA-PS classification has some favorable features (such as ease of use and the simplicity at the bedside without the need for any technical device), which has also allowed its diffusion outside the field of anesthesiology.

The main finding of our study was that residents and less experienced anesthesiolgists were more accurate in assigning ASA-PS scores than those with more experience, i.e. according to our data, the more experienced anesthesiologists recorded the lowest number of correct answers.

In fact, according to our data, the more experienced anesthesiologists recorded the lowest number of correct answers. Also, the inter-rater reliability (estimated by Fleiss’ k) was weak/moderate in all experience level groups. In these less-experienced groups, better performance was observed in both the total number of correct answers and the mean difference between the assigned ASA-PS score and the correct one. The weak/moderate inter-rater reliability is consistent with previous studies [5-9].

Moreover, the inter-rater agreement varied widely among the cases. For example, almost all responders assigned the correct ASA-PS in Case 1, while their answers ranged from ASAPS I to IV in Case 7. Specifically, while the assignment of the correct ASA-PS to a healthy subject is easy (Case 1), the correct assignment to a multi-morbid patient could be potentially more difficult (Case 7). Indeed, while the ASA rigorously defines ranges and specific information for some clinical variables used to assign the ASA-PS (such as BMI or interval since myocardial infarction to operation), many other clinical variables (often related to as ‘systemic diseases’) are much more loosely defined. This leads some anesthesiologists to miss the more rigorously defined items (especially if they do not frequently refresh their knowledge), while many others are left alone with their clinical judgment when deciding whether a systemic disease should be classified as mild, severe, or a constant threat to life.

Moreover, the ASA provides a clear definition for each ASAPS class but only a few examples for each class, stating that “there is no additional information that will help you further define these categories” [4]. Hurwitz et al. [13] speculated that the ASA felt that anesthesiologists would use their training and own experience to supplement the definitions when assigning the ASAPS.

Based on this assumption, our findings showed that the more experienced anesthesiologists relied more on their clinical experience than on the strict ASA criteria and ‘official’ given examples when assigning a ASA-PS score. The ASA criteria and ‘official’ examples are free and easily available on the internet [4]. We did not insert explanations of the ASA-PS in the questionnaire, nor did we invite participants to refresh their knowledge before answering the questionnaire; on the contrary, responders were even explicitly asked not to refresh their knowledge before taking the survey. With this strategy, we tried to highlight the “experience” factor. As a final result, more experienced anesthesiologists were less accurate in classifying patients compared to less experienced colleagues.

As noted in a previous work [13], adding more clinical examples to the ASA-PS classification could possibly lead to an improvement of correct patient assignment. We speculate that a more exhaustive variety of examples of ASA-PS could improve accuracy in more experienced anesthesiologists.

Previous literature has shown that the inter-rater reliability of the ASA-PS designations is suboptimal [5-9]. In addition, our study showed that “younger” anesthesiologists performed better than “older” colleagues when use of ASA grading system was required. We hypothesize that less experienced physicians are probably more confident with clinical examples provided by the ASA compared to more experienced anesthesiologists who probably rely more on their experience.

This study has limitations. First, the survey was administered to a closed-group of Italian anesthesiologists on Facebook and not to all Italian anesthesiologists. Secondly, we only analyzed completed questionnaires and the exclusion of incomplete forms is a potential source of bias. Thirdly, we used fictitious clinical cases that may not be representative of the respondents’ clinical practice. Finally, we divided participants into five groups based on experience but the experience time period was not the same in all groups. Our investigation was conducted using a social network commonly used by the younger generation. Considering this potential source of bias, we aimed to obtain a similar sample size between the five groups. Thus we arbitrary divided the youngest population, excluding anesthesiology residents, into two groups with a shorter global experience, while the oldest population was divided into two groups with a longer global experience. We acknowledge that using the same experience period for all groups may have yielded different results.

Further studies, possibly with a larger sample size, are needed to clarify the clinical implication of our results. In particular, it would be interesting to re-evaluate the inter-rater agreement of the ASA-PS classification, even using the same sample, when participants are allowed to review the most recent guidelines before taking the survey.

In conclusion, the ASA-PS remains an important but often uncertain grading system that is routinely used by anesthesiologists despite its low inter-rater reliability and, as shown by our data, a strong reliance on work experience. Thus, a rational and careful application of the ASA-PS should be recommended, especially in clinical settings where wide differences in experience exist.