Survey of Glasgow Coma Scale and PaO2/FIO2 ratio assessment methods for the Sequential Organ Failure Assessment score in Japanese intensive care units.

Aim: Accurately calculating the Sequential Organ Failure Assessment (SOFA) score is essential for medical resource allocation and decision-making. This study surveyed Japanese intensive care units regarding their assessment of the Glasgow Coma Scale (GCS) and PaO2/FIO2 ratio, components of the SOFA score.
Methods: A cross-sectional, web-based survey was conducted among healthcare workers. The survey consisted of questions about the intensive care units where they work and questions for respondents. It was distributed to healthcare workers by e-mail through the Japanese Society of Intensive Care Medicine mailing list and social networking service.
Results: Among 414 responses, we obtained 211 valid responses and 175 survey results from unique intensive care units. When assessing GCS in patients under the influence of sedatives, 45.1% (95% confidence interval, 37.6-52.8) of intensive care units assessed GCS assuming that the sedatives had no influence. For the PaO2/FIO2 ratio in the SOFA score, calculation based on the Japanese Intensive Care Patient Database definition document and mechanical ventilator settings were the most common methods in patients with oxygen masks and on extracorporeal membrane oxygenation, respectively. Approximately 60% of respondents indicated that it was difficult to assess GCS assuming that sedatives had no influence.
Conclusion: In patients under the influence of sedatives, approximately half of the intensive care units assessed assumed GCS. There was variation in the methods used to assess the PaO2/FIO2 ratio. Standardized assessment methods for GCS and the PaO2/FIO2 ratio are needed to obtain valid SOFA score.

INTRODUCTION

Accurate calculation of the Sequential Organ Failure Assessment (SOFA) score is essential in medical resource allocation and decision‐making. In Japan, the SOFA score calculation is a requirement to be reimbursed for medical service fees in intensive care. Intensive care units (ICUs) participating in the Japanese Intensive care Patient Database (JIPAD) need to calculate SOFA score.

Some components of the SOFA score, such as the Glasgow Coma Scale (GCS) and PaO2/FIO2 (P/F) ratio, are difficult to assess in some clinical situations. For example, Vincent et al. recommend that the GCS be assessed assuming that sedatives have no effect in patients under the influence of sedatives (assumed GCS). , JIPAD also requires assessment of assumed GCS ; however, GCS is often unknown in emergency admissions. It has also been reported that many hospitals evaluate GCS based on the actual condition of the patient, in which GCS is assessed as the lowest value (actual GCS) when the patient is deeply sedated. In this situation, there can be up to a 4‐point difference between assumed GCS and actual GCS in the SOFA score, which has a total score of 24 points. A previous study reported that the most common errors in evaluating GCS in the context of the SOFA score resulted from patient sedation. Regarding the P/F ratio, there is no fixed rule for assessing FIO2 when an oxygen mask or extracorporeal membrane oxygenation (ECMO) is used. There is concern that each ICU has a different method for calculating the P/F ratio. In this study, we further explored this issue by quantitatively clarifying the actual situation of GCS and P/F ratio and difficulties in calculating the SOFA score.

This study aimed to survey GCS and P/F ratio assessment methods for the SOFA score in Japanese ICUs and revealed difficulties in assessing assumed GCS. We examine differences in GCS and P/F ratio assessment methods among JIPAD‐participating ICUs of different types and discuss ways in which to improve SOFA score calculation.

METHODS

Study design

A cross‐sectional, web‐based survey was conducted among healthcare workers (HCWs) in Japanese ICUs. Participation was voluntary and anonymous. The inclusion criterion was being a HCW who worked in one of the 603 ICUs that met the criteria for medical fees of the Ministry of Health, Labor, and Welfare (MHLW) based on data from the Regional Bureaus of Health and MHLW in February 2021. , , , , , , ICUs or children's hospitals where only pediatric patients are admitted were not included. The survey was distributed to prospective participants by e‐mail using the Japanese Society of Intensive Care Medicine (JSICM) mailing list and the authors' official accounts of social networking services (Facebook and Twitter). The response period was August 23, 2021 to September 23, 2021. A working group that consisted of eight critical care nurses with clinical ICU experience and one nursing researcher developed the questionnaire. The questionnaire was housed in the research electronic data capture (https://projectredcap.org/about/) system at Osaka City University. The questionnaire consisted of questions about the respondent's ICU and questions about the respondent. The content of the questionnaire is shown in Appendix S1. Responses that did not answer all questions were regarded as incomplete and were excluded from the main analysis. This survey was approved by the Ethics Review Committee of Osaka City University and the Clinical Trial Group Committee of the JSICM.

Questions about ICUs

Respondents selected their ICU from a list of target ICUs or wrote it in a free‐text field if their ICU was not on the list or the ICU name was unknown. The research team checked the free‐text responses to determine if the ICU met the inclusion criteria by visiting that institution's website. Questions about the respondent's ICU included the name of the institution; name of the ICU; type of ICU; GCS assessment method in the situation when a patient was sedated or had impaired consciousness before or during sedation; and P/F ratio calculation method when an oxygen mask or ECMO was used. If more than one response about each ICU was obtained, only the first response was used in the description of the questions about the respondent's ICU. For sensitivity analyses, we also used responses by physicians or nurses.

Questions about respondents

All responses from HCWs working in the target ICUs were used in the analysis of responses to questions about respondents. Questions about respondents included the type of work; level of difficulty in assessing assumed GCS; and whether the respondent knew that assumed GCS should be assessed in patients under the influence of sedatives. The difficulty level in assessing assumed GCS was answered if the responder chose to assess GCS inferred for the situation. The level of difficulty was rated on a 0–100 visual analogue scale (VAS), ranging from “easy” to “difficult.” The difficulty level was categorized as easy when VAS ≤50 and difficult when VAS >50.

Statistical analysis

Descriptive statistical analysis was performed. Continuous and ordinal variables were presented as medians and interquartile ranges. Categorical data were expressed as counts and percentages. For results with percentages, 95% confidence intervals (CI) were calculated using the Clopper‐Pearson method. For questions about respondents' ICUs, percentages were calculated with the number of respondents' ICUs as the denominator. For questions about respondents, percentages were calculated with the number of responses as denominators. Data analysis was conducted with R version 4.1.0 (https://cran.r‐project.org/).

RESULTS

We obtained 414 responses, of which 203 were excluded because they were incomplete (n = 194) or because the ICU did not meet the study criteria (n = 9). As a result, the final dataset included information from 211 HCWs and 175 unique ICUs (Fig. 1). The geographic distribution of the respondents' ICUs is shown in Figure 2.

Fig. 1

Survey flow diagram. Participants were required to be a HCW who worked in one of the 603 ICUs that met the criteria for medical fees of the MHLW based on data from the Regional Bureaus of Health and MHLW in February 2021. HCWs, Healthcare workers; ICU, intensive care unit; MHLW, Ministry of Health, Labor, and Welfare.

Fig. 2

Distribution of respondents' ICUs by prefecture. Darker blue indicates a higher proportion or count of respondents' ICUs at the prefecture level. ICU, intensive care unit.

GCS assessment methods and P/F ratio calculation methods for the SOFA score: Questions about ICUs

Responses were obtained from 29.0% (175/603) of target ICUs. It was reported that 94 ICUs participated in JIPAD in 2021, and 54.3% (51/94) of the JIPAD‐participating ICUs responded to this survey (Table 1). SOFA score measurements had been conducted most by intensivists (63.4%) and nurses (24.0%), and final checks had been conducted most by intensivists (46.3%; Table S1). When assessing GCS in patients under the influence of sedatives, assumed GCS was assessed in 45.1% (95% CI, 37.6–52.8) of ICUs (situation A in Fig. 3). In cases where the pre‐sedation GCS was unknown, 89.9% of respondents who evaluated assumed GCS, and stated that they made their determination after collecting as much information as possible (Table S2). In ICUs that participated in JIPAD, assumed GCS was assessed in 62.7% of ICUs for patients under the influence of sedatives (Table 2). If consciousness impairment occurred before or during sedation, the proportion of ICUs assessing assumed GCS decreased (situation B and C in Fig. 3). In the responses stratified by ICU type, the “assumed GCS” proportion in situations A to C was highest in closed ICUs and lowest in open ICUs (Table 2). For GCS‐Verbal, approximately half of the ICUs reported that they would use “VT” if the patient was intubated. For the P/F ratio in the SOFA score, in the case of oxygen masks, the most common response was that the method was calculated according to JIPAD definition document. In the case of ECMO, the most common response was a method based on FIO2 from the mechanical ventilator and clinical judgment. In a sensitivity analysis, the results were similar regardless of whether the priority response was the first response or when it was changed to either the physician or the nurse (Table S3). The questionnaire results, including incomplete data, are shown as Table S4.

Table 1

Characteristics of respondent's ICU and respondent

Respondent's ICU		N (%)	Missing
ICU type	Open ICU	24 (13.7)	0
	Closed ICU	29 (16.6)
	Mixed ICU	122 (69.7)
Presence of intensivist	Yes	138 (78.9)	0
	No	33 (18.9)
	Unknown	4 (2.3)
Participating JIPAD	Yes	51 (29.1)	0
	No	85 (48.6)
	Unknown	39 (22.3)

If more than one response about each ICU was obtained, only the first response was used in the description of ICU characteristics.

“N” indicates the number of respondent's ICU; “n” indicates the number of respondent.

ICU, Intensive Care Unit; JIPAD, Japanese Intensive care Patient database; intensivist, a board‐certified intensivist.

Fig. 3

GCS assessment and P/F ratio calculation methods for the SOFA score. Situations in which GCS was assessed in patients under the influence of sedatives were as follows. Situation A, no consciousness impairment before sedation. Situation B, consciousness impairment before sedation. Situation C, consciousness impairment during sedation. JIPAD rule, calculation of the P/F ratio according to the JIPAD definition document. ICU rule, calculation of the P/F ratio as defined by each ICU. ECMO, extracorporeal membrane oxygenation; P/F, PaO2/FIO2; GCS, Glasgow Coma Scale; JIPAD, Japanese Intensive Care Patient Database; ICU, intensive care unit.

Table 2

GCS assessment methods by JIPAD participation and ICU type

		Overall	JIPAD participation			ICU type
			Yes	No	Unknown	Open ICU	Closed ICU	Mixed ICU
Situation A (%)	Actual GCS	87 (49.7)	17 (33.3)	44 (51.8)	26 (66.7)	19 (79.2)	7 (24.1)	61 (50.0)
	Assumed GCS	79 (45.1)	32 (62.7)	36 (42.4)	11 (28.2)	2 (8.3)	21 (72.4)	56 (45.9)
	GCS assessment with sedation interrupted or reduced	6 (3.4)	2 (3.9)	2 (2.4)	2 (5.1)	1 (4.2)	1 (3.4)	4 (3.3)
	Unknown	3 (1.7)	0 (0.0)	3 (3.5)	0 (0.0)	2 (8.3)	0 (0.0)	1 (0.8)
Situation B (%)	Actual GCS	93 (53.1)	17 (33.3)	50 (58.8)	26 (66.7)	19 (79.2)	8 (27.6)	66 (54.1)
	Assumed GCS	76 (43.4)	31 (60.8)	32 (37.6)	13 (33.3)	3 (12.5)	20 (69.0)	53 (43.4)
	Evaluated as E4, V5, M6	3 (1.7)	3 (5.9)	0 (0.0)	0 (0.0)	0 (0.0)	1 (3.4)	2 (1.6)
	Unknown	3 (1.7)	0 (0.0)	3 (3.5)	0 (0.0)	2 (8.3)	0 (0.0)	1 (0.8)
Situation C (%)	Actual GCS	113 (64.6)	26 (51.0)	58 (68.2)	29 (74.4)	20 (83.3)	12 (41.4)	81 (66.4)
	Assumed GCS	56 (32.0)	24 (47.1)	23 (27.1)	9 (23.1)	1 (4.2)	17 (58.6)	38 (31.1)
	Only affected GCS items received the lowest score	3 (1.7)	0 (0.0)	2 (2.4)	1 (2.6)	3 (12.5)	0 (0.0)	0 (0.0)
	Evaluated as E1, V1, M1	3 (1.7)	1 (2.0)	2 (2.4)	0 (0.0)	0 (0.0)	0 (0.0)	3 (2.5)

In patients under the influence of sedatives, JIPAD requires assessment of assumed GCS.

Situations in which GCS was assessed in patients under the influence of sedatives were as follows.

Situation A: no consciousness impairment before sedation.

Situation B: consciousness impairment before sedation.

Situation C: consciousness impairment during sedation.

GCS, Glasgow Coma Scale; ICU, intensive care unit; JIPAD, Japanese Intensive care Patient Database.

Difficulties in assessing assumed GCS: Questions about respondents

Of 211 HCWs, 134 (63.5%) were physicians, 70 (33.2%) were nurses, and seven (3.3%) were other (e.g., physical therapists and clinical engineers; Table 1). When assessing GCS in patients under the influence of sedatives, approximately 60% of HCWs who assessed assumed GCS indicated that doing so was difficult. The reported level of difficulty was higher for nurses than for physicians (situation A in Table 3). For more complex situations, the percentage of responses that corresponded to “difficult” increased (situations B and C in Table 3). The results of the VAS plots showed similar results (Fig. S1).

Table 3

Difficulty in assumed GCS assessment for patients under the influence of sedatives

		Overall	Physician	Nurse	Other
Situation A, n (%)	Easy	36 (36.4)	33 (40.2)	3 (18.8)	0 (0.0)
	Difficult	63 (63.6)	49 (59.8)	13 (81.2)	1 (100.0)
Situation B, n (%)	Easy	23 (25.8)	21 (29.2)	2 (13.3)	0 (0.0)
	Difficult	66 (74.2)	51 (70.8)	13 (86.7)	2 (100.0)
Situation C, n (%)	Easy	16 (24.6)	15 (28.3)	1 (8.3)	0 (0.0)
	Difficult	49 (75.4)	38 (71.7)	11 (91.7)	0 (0.0)

Situations in which GCS was assessed in patients under the influence of sedatives were as follows.

Situation A: no consciousness impairment before sedation.

Situation B: consciousness impairment before sedation.

Situation C: consciousness impairment during sedation.

GCS, Glasgow Coma Scale.

Knowledge of recommended GCS assessment methods for patients under the influence of sedatives: Questions about respondents

There were 129 (61.1%; 95% CI, 54.2–67.8) HCWs who knew that assumed GCS should be assessed in patients under the influence of sedatives (Table 4). If stratified by work type, 104 (77.6%; 95% CI, 69.6–84.4) physicians and 23 (32.9%; 95% CI, 22.1–45.1) nurses knew that assessment of assumed GCS is recommended in patients under the influence of sedatives.

Table 4

Knowledge of assumed GCS assessment for patients under the influence of sedatives

		Overall	Physician	Nurse	Other
Assumed GCS (%, 95% CI)	Known	129 (61.1, 54.2–67.8)	104 (77.6, 69.6–84.4)	23 (32.9, 22.1–45.1)	2 (28.6, 22.1–45.1)
	Unknown	82 (38.9, 32.2–45.8)	30 (22.4, 15.6–30.4)	47 (67.1, 54.9–77.9)	5 (71.4, 54.9–77.9)

It is recommended that the SOFA score is assessed by assumed GCS, which is to assume that patients under the influence of sedatives have no sedative effect.

CI, confidence interval; GCS, Glasgow Coma Scale; SOFA. Sequential Organ Failure Assessment.

DISCUSSION

Our survey revealed the actual GCS and P/F ratio assessment methods for the SOFA score and quantified the percentages of each method used in the Japanese ICU setting. Approximately half of the ICUs were using assumed GCS for the SOFA score, as recommended by Vincent et al.5 Furthermore, approximately 60% of HCWs reported that it was difficult to assess GCS. Approximately 40% of HCWs were not aware of the recommended methods. The reported difficulty in assessing assumed GCS and lack of knowledge about the recommended method might have been related to the finding that only half of the ICUs were assessing assumed GCS. In the SOFA score, GCS was reported to be the least accurate item; the most common error was because of sedation. Our study further supports existing data and highlights that GCS is incorrectly assessed for the SOFA score in patients under the influence of sedatives. ICUs that participated in JIPAD and closed ICUs tended to assess GCS using the recommended method. JIPAD states that assumed GCS should be assessed in patients under the influence of sedatives. Therefore, standardized methods of obtaining assumed GCS are needed.

Recently, to overcome the difficulty in assessing GCS, a modified SOFA score was suggested. , , A new item called “Glasgow inability to assess” was added to Acute Physiology and Chronic Health Evaluation IV. Such an alternative GCS item might resolve the difficulty in assessing assumed GCS.

Overall, approximately 60% of the respondents knew about assumed GCS. More than 70% of physicians knew about it, but only approximately 30% of nurses or other HCWs knew about it. Because GCS recorded by nurses might be used to calculate the SOFA score, it is important to educate nurses and other HCWs about the recommended method to assess GCS.

Interestingly, approximately half of the ICUs in our study used “VT” when the GCS‐Verbal item could not be calculated for intubated patients. A survey of major head injury centers in the United States reported that 47% of neurosurgeons used the "VT” designation in intubated patients, which was similar to the findings in the present study. However, it is unclear how "VT" is calculated in the GCS. Previous research suggests two ways to estimate GCS‐Verbal. One method is to use the E and M items of the GCS. The other method is to use the following scale: 5, if a patient seems able to talk; 3, if the ability to talk is questionable; and 1, if a patient is generally unresponsive. If the verbal item of the GCS is evaluated as VT, it is difficult to determine the GCS and the SOFA score cannot be calculated accurately. The verbal function assessed by the verbal item in the GCS can sometimes be evaluated in intubated patients by written communication and other means. Therefore, V5, V3, and V1 might be better than VT.

In patients undergoing ECMO, methods for calculating the P/F ratio varied. This might have been because of the lack of an appropriate method to calculate FIO2 in such patients, resulting in variations in the responses. In a previous study of ICU patients that used the SOFA score to predict mortality after cardiac surgery, ECMO patients were assigned the worst score for the respiratory component of the SOFA score. However, it is unclear if this assignment can be used to predict prognosis. In ECMO patients, the P/F ratio is occasionally higher than in non‐ECMO patients, and the calculated SOFA score is, therefore, lower. However, ECMO patients are more severely ill. The SOFA score calculated from the P/F ratio might be inconsistent with the clinical status of ECMO patients. Further research is needed to determine how to calculate the P/F ratio that best predicts prognosis in patients undergoing ECMO.

There are several limitations of this study that must be noted. First, there was no denominator for the responder response rate in our analysis because we administered the survey widely, but did not conduct systematic sampling. Therefore, our results might reflect a small number of biased opinions, particularly if a high percentage of responses were obtained from HCWs who used the recommended methods. In addition, the use of the society's mailing list and social networking service (SNS) might have skewed sampling according to the geographic location of their members. Although respondents had to state the names of the ICUs they worked at, there is a concern that the responses received via SNS might have been answered by non‐HCWs. In addition, bias may have arisen because of our survey collection method, because SNSs might obtain responses from a broader population. Second, in the analysis by ICU, the response rate was 29.0%, which led to the possibility of selection bias. However, the respondents' ICUs had a wide range of characteristics, including different regions and hospital types (university or non‐university affiliated). Third, this survey was voluntary and the responses reflected opinions and perceptions only. They have not been verified by an audit and might not necessarily reflect actual operations. Finally, this survey was conducted during the coronavirus disease‐19 (COVID‐19) pandemic, which may have resulted in a smaller number of responses from ICUs (emergency ICUs, etc.) treating COVID‐19 patients.

CONCLUSION

In patients under the influence of sedatives, approximately half of the ICUs assessed assumed GCS. There was variation in the methods used to assess the P/F ratio in patients being treated with ECMO or oxygen masks. Standardized assessment methods for GCS and the P/F ratio are needed to obtain valid SOFA score.

DISCLOSURE

Approval of the research protocol with approval No. and committee Name: This study was approved by the Ethics Review Committee of Osaka City University (approval no. 2021–117).

Informed Consent: Not applicable.

Registry and Registration Number of the Study: None.

Animal Studies: Not applicable.

Conflict of Interest: The authors declare no conflicts of interest for this article.

Appendix S1. SOFA score survey.

Click here for additional data file.

Table S1. Timing and method of SOFA score measurement.

Table S2. GCS evaluation method when GCS before sedation is unknown.

Table S3. GCS assessment and P/F ratio calculation methods for the SOFA score by differences in priority responses.

Table S4. GCS assessment and P/F ratio calculation methods for the SOFA score by differences in priority responses.

Fig. S1. Difficulty of assumed GCS among physicians and nurses in different situations.

Click here for additional data file.