Comparative evaluation of the bispectral index (BIS) and BISpro during propofol anaesthesia.

OBJECTIVE: To compare the accuracy, correlation and agreement between the bispectral index (BIS) and BISpro during propofol anaesthesia.
METHODS: The BIS, BISpro, heart rate, target-concentration of propofol and Observer's Assessment of Alertness and Sedation (OAA/S) score were recorded every 30 s in female patients scheduled for hysteroscopic surgery. Propofol anaesthesia was induced by an initial target-controlled concentration (1.0 μg/ml) followed by a stepwise increase (0.5 μg/ml) until the patient was unresponsive. Spearman's correlation coefficient and prediction probability were calculated for the association between sedation levels and the above parameters. The ability of investigated parameters to distinguish between OAA/S scores was analysed. Bland-Altman analysis was used to compare the agreement between BIS and BISpro. The BIS and BISpro cut-off values for lost response were also determined.
RESULTS: Out of 30 patients in total, a high correlation was found between BIS and BISpro, and both correlated well with OAA/S score. Only BIS was able to distinguish all investigated OAA/S states accurately, but the ability to predict OAA/S score 5 to loss of response was comparable between BIS and BISpro. The calculated cut-off values were 68 for BIS and 70 for BISpro.
CONCLUSION: BISpro and BIS are reliable monitors of general anaesthesia during sedation.Trial registration number: Chinese Clinical Trial Registry (URL: www.chictr.org.cn): ChiCTR1900024037 (retrospectively registered).

Introduction

The measurement of anaesthesia depth remains an ongoing challenge in clinical practice, because it includes three components: sedation, analgesia and muscle relaxation. Several indices for measuring depth of anaesthesia have been commercialized and applied to clinical settings.[1-3] The bispectral index (BIS) was the first value approved by the United States Food and Drug Administration for monitoring the depth of anaesthesia and is the most widely used clinically.

The BISpro monitor, developed by Maygreen Co. Ltd (Shenzhen, China), measures two indices of anaesthesia depth: BISpro and nociception index (NOX). The BISpro value is similar to the BIS value, which is based on the correlation of the phase between different frequency components of the electroencephalogram (EEG) with bispectral analysis.[4] The BIS ranges from 0 (silent) to 100 (fully awake) and was developed using a database of EEG values and behavioural scales. Instead of using this previous database, the BISpro is based on raw physical principles (www.maygreen.com.cn/). Similar to BIS, the manufacturer of the BISpro monitor suggests that BISpro values ranging from 40–60 may prevent the adverse incidence of awareness during general anaesthesia.

The first aim of the present study was to compare BIS and BISpro values using the Observer's Assessment of Alertness and Sedation (OAA/S) score,[5] by analysing the accuracy of each variable in distinguishing between different clinically defined states during propofol anaesthesia. Specifically, the investigation focussed on whether the BISpro value is as reliable and accurate as the BIS value in reflecting changes during increasing propofol target concentration and sedation levels. A further aim was to determine an appropriate BISpro cut-off value that provided the optimal combination of sensitivity and specificity for predicting an OAA/S score of ≤2 (loss of response), using receiver operating characteristic (ROC) curve analysis.

Patients and methods

Study population

This prospective study was conducted at Peking University Shenzhen Hospital, Shenzhen, China between August 2019 and June 2020, and included sequentially enrolled female patients, aged between 18–55 years and with an American Society of Anaesthesiologists physical status classification of I or II, who were scheduled for hysteroscopic surgery. Exclusion criteria were: a history of neurological deficits; congestive heart failure; arrhythmias; bundle-branch block; severe liver and/or kidney dysfunction; pregnancy; antidepressant treatment within 1 year of enrolment; or dementia. The study was approved by Peking University Shenzhen Hospital Medical Ethics Committee (08/06/2019) and retrospectively registered at ChiCTR (ID: ChiCTR1900024037). Written informed consent was obtained from all participants. All procedures performed in this human study were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Study procedures and data collection

No sedative or analgesic was administered preoperatively. On arrival in the operating room, an intravenous cannula was inserted in the dorsum of a hand, and Ringer's lactate infusion was started. Electrocardiograms, SpO2, non-invasive blood pressure, BIS values (BIS VISTA™ Monitoring System, version 3.20; Aspect Medical Systems, Newton, MA, USA), and BISpro values (BISpro smart monitor, version V1.0.0.336; Maygreen) were monitored. The EEG was registered for BIS using a BIS® sensor XP electrode (Aspect Medical Systems) and for BISpro using the relevant Maygreen electrode over the left or right area of the forehead, according to the manufacturers’ instructions. The side for temporal placement of BIS and BISpro electrodes was randomly selected by flipping a coin. The BIS was monitored using a two-channel referential montage, and calculated with a smoothing rate of 30 s. Both electrode impedances were kept below 10 kΩ. Supplemental oxygen was administered via a venturi mask at a flow rate of 3 l/min throughout the surgery.

After preoperative preparation, propofol anaesthesia (serial No. 16MM2728, Fresenius Kabi Austria GmbH, Austria) was induced by an initial target-controlled effect-site concentration (Ces) of 1.0 μg/ml, using the population pharmacokinetic sets by Schnider et al.[6] and the Orchestra Base Primea (Fresenius Vial, Brézins, France), followed by a stepwise increase of 0.5 μg/ml until the patient was unresponsive. Based on the time to peak effect of propofol (1.6 min), every propofol concentration level was maintained for >2 min.[7] The OAA/S scores were assessed on a scale from 1 (no response to mild prodding or shaking) to 5 (readily responds to name spoken in normal tone) (Table 1). To ensure that each patient received the same intensity of tone and stimulus, OAA/S scores were assessed by a specified investigator (ZL) for all patients every 30 s from the start of infusion. BIS, BISpro, heart rate (HR) and Ces of propofol was also recorded every 30 s. The loss of response was defined as an OAA/S value ≤ 2.

Table 1.

Observer’s Assessment of Alertness/Sedation (OAA/S) scale.

Score	Responsiveness	Speech	Facial expression	Eyes
1	Does not respond to mild prodding or shaking
2	Responds only after mild prodding or shaking	Few recognized words
3	Responds only after the name is spoken loudly and/or repeatedly	Slurring or prominent slowing	Marked relaxation (slack jaw)	Glazed and marked ptosis
4	Lethargic response to name spoken in a normal tone	Mild slowing or thickening	Mild relaxation	Glazed or mild ptosis
5	Responds readily to name spoken in normal tone	Normal	Normal	Clear, no ptosis

The final score is the sum of the Responsiveness, Speech, Facial expression, and Eyes component scores. Thus, a ‘wide awake’ score = 5 and a deeply sedated score = 1.

Sample size calculation

Based on previously published research, a difference of < 0.05 for prediction probability (PK) would not be of clinical significance, and with a t quotient calculated between the chosen difference in clinical importance and the standard error, the present study was calculated to require at least 30 patients.[8]

Statistical analyses

All data are presented as mean ± SD, median (range) or n patients. Changes in BIS, BISpro, HR, and Ces at different OAA/S scores were compared using the non-parametric Kruskal–Wallis H-test.

Two types of significant error for sedative assessment were defined and the number of times they occurred was calculated. A type I error was defined as BIS < 60 and BISpro > 80 or BIS < 40 and BISpro > 60, and a type II error was defined as BIS > 80 and BISpro < 60 or BIS > 60 and BISpro < 40.

Associations between OAA/S scores and BIS, BISpro, HR, and Ces were analysed using Spearman’s correlation coefficient. Moreover, the correlation between BIS and BISpro for all data pairs was evaluated by the nonparametric Spearman’s rank correlation coefficient. Bland–Altman analysis was used to compare the consistency between BIS and BISpro.[9]

Associations between OAA/S scores and BIS, BISpro, and HR were also assessed by determining PK.[10] The accuracy of BIS, BISpro, and HR in distinguishing between different depths of sedation, such as an OAA/S score of 5 versus 4, 3, and loss of response, was analysed. PK was calculated and compared using the PKMACRO and PKDMACRO spreadsheets, as described by Smith et al.[10] The jackknife method was used to compute the standard error of the estimate. A PK value < 0.5 indicates that discordance is more likely than concordance. To enable a comparison of PK, 1 –PK was used when the PK value was < 0.5. To evaluate the accuracy in distinguishing all investigated conditions, the following formula was used: PKall = PKOAA/S 5 versus 4 × PKOAA/S 5 versus 3× PKOAA/S 5 versus loss of response. All steps were weighted equally.

To compare the abilities of BIS and BISpro to identify different conscious states, and to obtain the BIS and BISpro cut-off values that estimate a hypnotic status, ROC curve analysis was used to determine the optimal combination of sensitivity and specificity for predicting an OAA/S score of ≤2. The closer the ROC curve is to the upper left corner, the higher the accuracy of the test.

Statistical analyses were performed using SPSS software, version 22.0 (IBM, Armonk, NY, USA), MedCalc statistical software, version 19.0.5 (MedCalc Software Ltd, Ostend, Belgium), GraphPad Prism, version 5.01 (GraphPad Software, San Diego, CA, USA), PKMACRO, and PKDMACRO.[10] A P-value < 0.05 was considered statistically significant.

Results

Thirty-one patients were enrolled, and one patient was excluded for a poor signal quality index bar at 10 min after induction. Thus, a total of 30 patients were included in the final analysis (mean age, 36.2 ± 8.4 years; height, 158.7 ± 6.8 cm; and weight, 53.8 ± 8.8 kg). The Ces for loss of response was 3.1 ± 0.3 μg/ml.

The two EEG parameters significantly decreased as OAA/S scores decreased, in that BIS and BISpro values were statistically lower in patients with OAA/S scores of 1 and 2 versus scores of 3–5 (P < 0.05; Table 2). One pair of values exhibited a type I error for sedative assessment and there were no type II errors (Table 3). BIS and BISpro also correlated well with OAA/S scores (r = 0.898, PK = 0.943 for BIS; and r = 0.806, PK = 0.879 for BISpro), but the PK of BIS was significantly better than the PK of BISpro (P < 0.01; Table 4). The correlation coefficient (r) for Ces (r = 0.761) was similar to BISpro, and no statistically significant differences in PK values were found between them (Table 4). Correlation coefficients and PK values for sedation levels were higher for BIS and BISpro than for HR (Table 4). A high correlation (r = 0.839, P < 0.001) was found between BIS and BISpro (Figure 1).

Table 2.

Changes in bispectral index (BIS) and BISpro in 30 patients with different Observer’s Assessment of Alertness/Sedation (OAA/S) scale scores.

Parameter	OAA/S Score
	1	2	3	4	5
BIS	57.3 ± 8.1	57.6 ± 8.9	69.8 ± 8.4a,b	81.3 ± 4.7a,b	92.7 ± 5.0a,b
BISpro	60.4 ± 11.5	59.0 ± 9.2	67.8 ± 8.7a,b	78.8.2 ± 8.4a,b	89.9 ± 5.2a,b
Statistical significance	NS	NS	NS	P < 0.001	NS

Data presented as mean ± SD.

aVersus OAA/S score of 1 (P < 0.05); bversus OAA/S score of 2 (P < 0.05).

NS, no statistically significant difference between BIS and BISpro.

Table 3.

Bispectral index (BIS) values and corresponding BISpro values.

BIS	Pairs	BISpro (Pairs)
		100–80	79–60	59–40	< 40
100–80	198	144	54	0b	0b
79–60	172	18	131	23	0b
59–40	52	1a	14	36	1
<40	1	0a	0a	1	0

aType I error (BIS < 60 and BISpro > 80, or BIS < 40 and BISpro > 60); btype II error (BIS > 80 and BISpro < 60, or BIS > 60 and BISpro < 40).

Table 4.

Spearman’s correlation coefficient and prediction probability (PK) for the Observer’s Assessment of Alertness/Sedation (OAA/S) scale in 30 patients who underwent propofol anaesthesia.

Parameter	OAA/S
	Spearman’s coefficient		PK
	r	Statistical significance	Value	SE
BIS	0.898	P < 0.001	0.943	0.007
BISpro	0.806	P < 0.001	0.879	0.010a
HR	0.271	P < 0.001	0.617	0.019a,b
Ces	0.761	P < 0.001	0.852	0.011a

BIS, bispectral index; HR, heart rate; Ces, effect-site concentration of propofol; SE, standard error.

aVersus BIS (P < 0.01); bversus BISpro (P < 0.01).

Figure 1.

Scatter plot of bispectral index (BIS) and BISpro values recorded every 30 s in 30 patients who underwent propofol anaesthesia during hysteroscopic surgery. A good correlation between BIS and BISpro is shown (r = 0.839; P < 0.01).

The results of Bland–Altman analysis are shown in Figure 2. The mean difference between BIS and BISpro was 1.7 with a standard error of the estimate of 7.91. The upper limit of agreement was 17.2 and the lower limit of agreement was –13.8. Only 0.04% of points were outside of the 95% limit of agreement, but BIS and BISpro differed by more than 10 units, suggesting a poor agreement between them.

Figure 2.

Bland–Altman analysis plot of the difference between bispectral index (BIS) and BISpro values in 30 patients who underwent propofol anaesthesia during hysteroscopic surgery, showing mean difference (continuous line) and upper and lower limits of agreement (two dashed lines). OAAS, Observer’s Assessment of Alertness/Sedation scale; SD, standard deviation.

The probability of BIS, BISpro, HR, and Ces to predict change in OAA/S scores was assessed by PK (Table 5). Only BIS was able to distinguish all investigated states accurately (PK > 0.90 for each state change), and analysis revealed a PKall of 0.933 for BIS and a PKall of 0.840 for BISpro (Table 5). A lower PK for the state between an OAA/S score of 5 and 4 was shown for BISpro (0.868) compared with BIS (0.937), however, the ability to predict the state change from OAA/S of 5 to loss of response was equivalent between BIS and BISpro.

Table 5.

Prediction probability (PK) for the different stages of propofol anaesthesia in 30 patients.

Parameter	OAA/S 5 versus 4		OAA/S 5 versus 3		OAA/S 5 versus loss of response
	PK	SE	PK	SE	PK	SE	PKall
BIS	0.937	0.017	0.996	0.002	1.000	0.000	0.933
BISpro	0.858	0.026a	0.980	0.007	0.999	0.002	0.840
HR	0.614	0.038a,b	0.701	0.033a,b	0.719	0.045a,b	0.617
Ces	0.835	0.026a	0.936	0.013a	0.993	0.003c	0.776

BIS, bispectral index; HR, heart rate; Ces, effect-site concentration of propofol; OAA/S, Observer’s Assessment of Alertness/Sedation; SE, standard error.

aVersus BIS (P < 0.01); bversus BISpro (P < 0.01); cversus BIS (P < 0.05).

The ROC curve analysis identified BIS and BISpro values with the best level of discrimination to obtain an OAA/S score of ≤2 during propofol sedation (Figure 3). The calculated BIS cut-off value was 68 (sensitivity of 92.4%, specificity of 86.3%, and area under the curve [AUC] of 0.940, P < 0.01); and the BISpro cut-off value was 70 (sensitivity of 86.4%, specificity of 73.1%, and AUC of 0.879, P < 0.001). The BIS value was closer to the upper left corner.

Figure 3.

Receiver operating characteristic curves showing the relationship between Observer’s Assessment of Alertness/Sedation scale value ≤2 and bispectral index (BIS) or BISpro value.

Discussion

The BIS is derived from recordings of large-volume EEGs and associated levels of consciousness and sedation of patients receiving different sedative infusions, combined into a database. The mixed information of the power and frequency of the EEG is then fitted with the bispectral analysis to obtain an optimal value, ranging from 0 to 100. When the number is reduced, cortical inhibition is deepened. In other words, BIS compares the EEG signal with a previous database to produce a dimensionless number that reflects the level of hypnosis.[4] BISpro is similar to BIS in that it also originates from EEG data, and is based on four basic waves: δ, θ, α, and β. However, BISpro uses a different algorithm from BIS to quantify the depth of anaesthesia. Symbolic dynamics are used to divide the acquired EEG signals into linear and nonlinear modules. Each part is marked by a symbol and converted into time series. EEG energy is calculated by a special frequency band, and the ratio of α and β is determined by calculating the difference in energy between high and low frequency during anaesthesia. At the same time, the total number of suppression episodes during the 30 s of EEG is calculated in real-time, and the total amount of anaesthesia depth in the stationary phase and the quiescent period is quantified. The logical classification of the four-parameter values of brain waves is performed to obtain an anaesthesia depth score (www.maygreen.com.cn/).

The OAA/S score is widely used for assessing the level of sedation and provides a good correlation with different degrees of sedation. Decreases in the sedation level found in the present study were associated with decreases in BIS and BISpro, and both BIS and BISpro correlated well with the OAA/S. Despite the good correlation between BIS and BISpro, the agreement between these two methods was poor. Good agreement is indicated by values within a range of 10 units above and below the mean difference.[11,12] An unsatisfactory disagreement, shown by more than 10 units between BIS and BISpro, was found in the present study, with upper and lower agreement limits of 17.2 and –13.8, respectively.

A highly effective index of sedation should accurately discriminate between the investigated states of anaesthesia. A PK value of 0.5 indicates that the variable predicts the states no better than a chance of 50%, whereas a PK value of 1.0 means that the variable correctly predicts the states 100% of the time, with no overlap. Furthermore, a parameter that perfectly distinguishes all conditions with a PK of 1.0 results in a PKall of 1.0 × 1.0 × 1.0 = 1.0; while a variable that predicts all conditions with a PK of 0.5 results in a PKall of 0.5 × 0.5 × 0.5 = 0.125. Only BIS was able to accurately distinguish all investigated states with a PK > 0.90. Based on the present data, BISpro was found to be limited by a worse PK for OAA/S score of 5 versus 4. This may be caused by an overlap of clinical signs between OAA/S scores of 5 and 4 that may have affected the researcher’s ability to differentiate between a score of 4 or 5. Another explanation for these discrepancies may be the differences between algorithms used in BIS and BISpro monitors. The algorithms differ between BIS and BISpro values, and different artifact rejection methods might play a role, in addition to intra-patient variability.[13] Findings similar to the present results were reported in a previous study, in which entropy index and BIS displayed no linear correlation outside a BIS range of 40–60.[14] BIS is mainly calculated by the Beta Ratio during awake or light sedation, SyncFastSlow (bispectral component) predominates during the surgical level of hypnosis (BIS 40–60), and the Burst Suppression Ratio/Quazi algorithm detects a deep level of anaesthesia (BIS < 40).[4,11] However, due to the lack of a specific algorithm for BISpro, we cannot comment on this phenomenon with any degree of certainty. Although the BIS and BISpro were not identical, the PK during transition from awake to lost response was equivalent (1.000 for BIS, and 0.999 for BISpro), suggesting that both were able to detect awareness during the operation.

The closer the ROC curve is to the upper left corner, the higher the accuracy of the test. The point closest to the ROC curve in the upper left corner is the best threshold with the least error and the fewest false positives and false negatives. BIS was more effective than BISpro in estimating the state of loss of response in the present findings, as its curve was closer to the upper left corner. BIS values at an OAA/S score of ≤ 2 were similar to those reported in previous research, and approximately 2 points less than BISpro.[5]

The present study results may be limited by the fact that the investigated states were divided into five scales only by OAA/S assessment, which may have resulted in some overlap between adjacent scales, particularly between OAA/S score 5 and 4. This might have weakened the power of the monitor to distinguish between the different hypnotic levels. Previous studies have defined six anaesthetic states, comprising awake versus loss of response, awake versus loss of eyelash reflex, awake versus steady-state anaesthesia, steady-state anaesthesia versus the first reaction, and steady-state anaesthesia versus extubation, which might contribute to a more definite conclusion and more closely represent clinical settings. Another limitation was that the effect of electromyography was not considered, which might have played an important role in impairing the accuracy of BIS and BISpro.[15-17]

In summary, BISpro showed a good correlation with BIS and OAA/S during propofol anaesthesia, but a poor agreement with BIS. BISpro provided the lowest accuracy in distinguishing the states between entirely awake and light sedation, but was close in power to BIS in distinguishing between awake and loss of response. Therefore, BISpro was shown to be a reliable monitor for general anaesthesia, but not an accurate index for light sedation.