The utility of three screening questionnaires for obstructive sleep apnea in a sleep clinic setting.

PURPOSE: The aim of this study was to determine the diagnostic value of three screening questionnaires in identifying Korean patients at high risk for obstructive sleep apnea (OSA) in a sleep clinic setting in Korea.
MATERIALS AND METHODS: Data were collected from 592 adult patients with suspected OSA who visited a sleep center. All patients completed the Sleep Apnea of Sleep Disorder Questionnaire (SA-SDQ), the Berlin questionnaire, and the STOP-Bang questionnaire. Estimated OSA risk was compared to a diagnosis of OSA. The sensitivity, specificity, positive predictive value, and negative predictive value were calculated for each questionnaire.
RESULTS: The prevalence of OSA was 83.6% using an apnea-hypopnea index (AHI) ≥5/h and 58.4% for an AHI ≥15/h. The STOP-Bang questionnaire had a high sensitivity (97% for AHI ≥5/h, 98% for AHI ≥15/h), but the specificity was low (19% and 11%, respectively). In contrast, the sensitivity of the SA-SDQ was not high enough (68% for AHI ≥5/h, 74% for AHI ≥15/h) to be useful in a clinical setting, whereas the specificity was relatively good (66% and 61%, respectively). The sensitivity and specificity values of the Berlin questionnaire fell between those of the STOP-Bang questionnaire and the SA-SDQ.
CONCLUSION: The STOP-Bang questionnaire may be useful for screening OSA in a sleep clinic setting, but its specificity is lower than the acceptable level for this purpose. A new screening questionnaire with a high sensitivity and acceptable specificity is therefore needed in a sleep clinic setting.

INTRODUCTION

Obstructive sleep apnea (OSA) is a sleep-related breathing disorder characterized by full or partial occlusion of the upper airway during sleep. The prevalence of OSA was estimated from large population studies, such as the Sleep Heart Health study, to be up to 18%,1 although overt prevalence is reported to be 4% in men and 2% in women.2 Untreated OSA is often associated with cardiovascular disease.3,4,5,6 OSA can be effectively treated in many patients with continuous positive airway pressure (CPAP).7,8 Therefore, it is important that OSA should formally be diagnosed to identify patients who could benefit from CPAP. However, it is not easy to diagnose OSA, largely because of problems in accessibility. The gold standard for the diagnosis of OSA remains in-laboratory polysomnography,9 but this is a cumbersome and expensive test.

Accurate screening for OSA is essential and commonly involves multiple screening questionnaires 10,11,12,13,14,15,16,17,18 that are based on clinical characteristics because such tools are easy to use. Characteristics of an ideal screening test differ according to the population to which it is applied. An ideal screening test in a general population should have a relatively high specificity as well as sufficient sensitivity.19 However, in patients suspected of having OSA in a clinical setting, an ideal screening test should have a high sensitivity with an acceptable specificity in order to avoid missing cases, as this disease can lead to adverse health outcomes.4,5,6

Most screening questionnaires have been developed and validated in Western countries to detect patients with high risk for OSA. The impact of factors known to affect the severity of OSA differs between Asian and Western patients.20 Asians have a lower incidence of obesity and are at a higher risk for more severe OSA than Western populations.20 Also, craniofacial structures contribute more to OSA in Asian than in Western patients.20,21 Hence, the utility of various screening questionnaires to determine OSA risk may need to differ between Asian and Western populations.

Recently, screening questionnaires for OSA have been systemically reviewed.20 In studies on patients in sleep clinics or in those with habitual snoring, the sensitivity and specificity of different questionnaires in predicting OSA ranged from 59-81% and 46-80%, respectively.20 The Berlin questionnaire and the Sleep Apnea of Sleep Disorder Questionnaire (SA-SDQ), which are commonly used in Western countries, were included in those studies. The STOP-Bang questionnaire is a simple 8 point tool for screening patients who may have OSA. It has previously been validated for screening preoperative surgical patients, showing a sensitivity of 92.9% in a validation cohort.11 However, it has not been extensively evaluated and compared to other screening questionnaires in a sleep clinic setting. The purpose of this study was to determine the diagnostic value of three screening questionnaires including the Berlin questionnaire, the SA-SDQ, and the STOP-Bang questionnaire in identifying Koreans at high risk for OSA in a sleep clinic setting.

MATERIALS AND METHODS

Subjects

Data were collected from adult patients who visited the sleep center of the Asan Medical Center for the evaluation of suspected OSA between January 2009 and June 2010. Criteria for inclusion were as follows: age over 18 years, results of an in-laboratory polysomnography performed for suspected OSA, and completion of three OSA screening questionnaires. Patients were excluded if they had received treatment for OSA or if they had an active psychiatric or medical disorder. The diagnosis of OSA was based on an apnea-hypopnea index (AHI) of 5 or more events per hour. The severity of OSA was classified based on the AHI values: 5-15/h as mild OSA, 15-30/h as moderate OSA, and ≥30/h as severe OSA.

Screening questionnaires

All patients completed three screening questionnaires including the SA-SDQ, the Berlin questionnaire, and the STOP-Bang questionnaire. The risk of OSA in each patient was estimated from all three questionnaires. The cut-off values for the SA-SDQ and the STOP-Bang questionnaire were recalculated for our Korean patient cohort, and the modified values were used to evaluate the utility of these questionnaires.

The Berlin questionnaire has been most commonly evaluated in previous studies.12,13,15,20,22 It includes 11 questions organized into three categories. The first category includes five items on snoring, the second category includes three items on daytime sleepiness, and the third category includes one item on history of hypertension. The first and second categories are considered positive if responses indicate persistent symptoms (>3-4 times per week) on two or more questionnaire items. A positive score on the third category requires a history of hypertension or a body mass index (BMI) >30 kg/m2. Patients who scored positively on more than two categories are classified as being at high risk of OSA.12,13,22

The STOP-Bang questionnaire was originally developed as a quick tool to screen OSA as part of a preoperative assessment of surgical patients.14 STOP questionnaire consists of four yes-no questions on snoring, tiredness, observed apnea, and blood pressure.11 The STOP-Bang questionnaire incorporates the STOP questions and assigns points for each yes answer on the following: BMI >35 or 30 kg/m2 (depending on the study), age >50 years, neck circumference >40 cm, and male gender.11,14 In the present study, we used the 30 kg/m2 as the cut-off value for BMI. A score of three or more positive responses was considered high risk for OSA. The cut-off point was recalculated for Korean patients and compared to the original score of 3.

The SA-SDQ consists of eight questions and four items related to weight, smoking status, age, and BMI, which are calculated to generate a raw score.16 The eight questions address the specific symptoms of sleep apnea syndrome including snoring loudly, breathing arrest during night sleep, awaking suddenly gasping for breath, sweating during night sleep, high blood pressure, blocked nose when wanting to sleep, snoring/breathing worsening when sleeping on the back, and snoring/breathing worsening when sleeping after alcohol use. Total scores range from 0 to 60. The suggested cut-off points for identifying patients with OSA are 36 for men and 32 for women.16 The cut-off points were recalculated for Korean patients and compared to the original ones.

In-laboratory polysomnography

An OSA diagnosis was made using standard polysomnography (Grass-Telefactor twin version 2.6, West Warwick, RI and RemLogic version 2.0, Embla system Inc., Broomfield, CO, USA), that consisted of electroencephalography (F3-A2, F4-A1, C3-A2, C4-A1, O1-A2, O2-A1), electrooculography (ROC-A1, LOC-A2), electromyography, electrocardiography (modified V2 lead), pulse oximetry for O2 saturation, and a microphone to detect snoring. Electrodes for electromyography are placed on the chin and on the bilateral tibialis anterior muscles. Airflow is assessed using an oronasal thermistor and a nasal pressure transducer. Thoracoabdominal movement is assessed using respiratory inductance plethysmography.

Sleep and associated events were scored according to the American Academy of Sleep Medicine manual.17 Sleep stages were identified for each 30-s epoch by well-trained registered polysomnographic technologists. Apnea was defined as a drop in peak thermal sensor excursion by at least 90% of the baseline value for at least 10 s. Hypopnea was defined as a drop in the nasal pressure signal by at least 30% of the baseline value for at least 10 s, with at least a 4% reduction in O2 saturation from the pre-event baseline. The AHI was defined as the number of apnea and hypopnea events per hour. The AHI was taken as indicator of OSA severity.

Statistical analysis

Patient characteristics were compared between patients with and without OSA using a chi-square test and Student's t-test. Estimated OSA risk, as assessed by the Berlin, SA-SDQ, and STOP-Bang questionnaires, was compared to the diagnosis of OSA (at an AHI of ≥5 and ≥15) as determined by in-laboratory polysomnography. The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated for high risk for each questionnaire. The area under the receiver operating characteristic (ROC) curve was calculated for high risk for the SA-SDQ and the STOP-Bang questionnaire, and then cut-off values were calculated for Korean subjects in this study. The ROC curve is a graphical plot displaying true positives versus false-positives across a range of cut-off values for clinical use. The area under the ROC curve (AUC) is a measure of the overall accuracy of a diagnostic test. It can be any value between 0 and 1, since both the x and y axes have values ranging from 0 to 1. The closer AUC is to 1, the better the overall diagnostic accuracy of the test. Thus, ROC curve is helpful to select possibly optimal cut-off values and to discard suboptimal ones for diagnostic decision making. The significance level was set at p<0.05. All statistical analyses were two-sided and performed using SPSS Version 19.0 (SPSS Inc., Chicago, IL, USA).

RESULTS

Patient characteristics

Of the 683 patients who were eligible for our study during the inclusion period, 592 patients were finally included in our cohort. The remaining 91 patients were excluded due to a recent history of uvulopalatopharyngoplasty (n=9) and active medical disorders [psychiatric (n=12), neurological (n=15), gastrointestinal (n=18), cardiac (n=15), malignancy (n=6), and others (n=16)]. This final cohort of 592 patients (Table 1) included 493 men (83.5%) and 99 women (16.7%), with a mean age of 47.6 years (range 18-79 years) and mean BMI of 24.7 kg/m2 (range 13.4-40.5 kg/m2). The prevalence of OSA (AHI ≥5/h) was 83.6% (495 patients). According to AHI, 149 (25.2%) had mild OSA (AHI 5-15/h), 133 patients (22.5%) had moderate OSA (AHI 15-30/h), and 213 patients (37.0%) had severe OSA (AHI ≥30/h). Patients with OSA were significantly older and had a higher BMI than those without OSA (p<0.001). Patients with OSA also had a higher prevalence of hypertension (p<0.05) and OSA symptoms such as snoring and chocking (p<0.001) (Table 1).

Table 1

Patient Characteristics

	All	OSA absent (AHI <5/hr)	OSA present (AHI ≥5/hr)	p value*
Number	592	97	495
Age, yrs (mean±SD)	47.6±12.7	40.9±13.2	48.9±12.2	<0.001
Male:female, n	493:99	65:32	428:67	<0.001
Body mass index, kg/m2 (mean±SD)	24.7±3.5	23.8±3.02	26.1±3.5	<0.001
Neck circumstance, cm (mean±SD)	39.2±3.5	36.9±3.3	39.7±3.4	<0.001
AHI, /hr (mean±SD)	25.9±21.8	2.8±1.6	30.4±21.1	<0.001
Hypertension, n (%)	224 (37.8)	28 (28.9)	196 (39.6)	0.046
Snoring, n (%)	529 (89.4)	76 (78.4)	453 (91.5)	<0.001
Choking, n (%)	328 (55.4)	26 (26.8)	302 (61.0)	<0.001

AHI, apnea-hypopnea index; OSA, obstructive sleep apnea; SD, standard deviation.

*Difference between patients with and without obstructive sleep apnea.

Cut-off value of the SA-SDQ for Korean patients

The ROC curves were constructed separately for men and women using the SA-SDQ scores and the OSA status determined by polysomnography (Fig. 1). The ROC curves graphically illustrate the ability to distinguish between subjects with and without a disease. Overall accuracy of the ROC curves is expressed as the AUC, which ranged between 0 and 1. The closer AUC to 1, the better the overall diagnostic accuracy of the test. The ROC curve is useful in assigning the best cut-off value for clinical use. In this study, the cut-off value for the SA-SDQ test was recalculated based on the ROC curve. With an AHI of 5/hour, the modified cut-off value formen was 36, which was the same as the original value. However, the modified cut-off value for women was 30, which was lower than the original value of 32. The cut-off values based on an AHI of 15/hour were the same as the values based on an AHI of 5/hour. The sensitivity, specificity, PPV, and NPV using the new cut-off values of the SA-SDQ were compared to the original scores (Table 2). The predictive values using the new cut-off values were somewhat better than the original values, especially the specificity and PPV for identifying patients at high risk for moderate-to-severe OSA. Hence, the modified cut-off values were used in the present study. The cut-off values of the STOP-Bang questionnaire were recalculated using a ROC curve and were found to be the same as the original values. Therefore, the original cut-off values were used in our current analyses.

Fig. 1

Receiver-operator characteristic (ROC) curves for the SA-SDQ for an AHI cutoff of ≥5/hour and ≥15/hour. The ROC curve is a graphical plot displaying the ability to distinguish between subjects with and without a disease. Overall accuracy of the ROC curve is expressed as the area under the ROC curve (AUC), which ranges between 0 and 1. The closer AUC to 1, the better the overall diagnostic accuracy of the test. The AUC is given in the legend of each graph. SA-SDQ, Sleep Apnea of Sleep Disorder Questionnaire; AHI, apnea-hypopnea index.

Table 2

Prediction Parameters for the SA-SDQ, Berlin, and STOP-Bang Questionnaires

	Berlin	STOP-Bang	SA-SDQ	Modified SA-SDQ
AHI ≥5/hr (%)
Sensitivity	71.5	97.0	66.9	67.5
Specificity	32.0	18.6	65.0	66.0
PPV	84.3	85.9	90.7	76.4
NPV	18.0	54.6	27.8	55.2
AHI ≥15/hr (%)
Sensitivity	75.5	98.0	73.1	74.0
Specificity	35.4	10.6	54.5	60.8
PPV	62.1	60.6	69.3	75.3
NPV	50.6	78.8	59.0	58.1

SA-SDQ, Sleep Apnea of Sleep Disorder Questionnaire; AHI, apnea-hypopnea index; PPV, positive predictive value; NPV, negative predictive value.

Performance of screening questionnaires

The prevalence of OSA (AHI ≥5/h) was 83.6%. Using an AHI ≥5/h, the sensitivity of the three questionnaires ranged from 67.5-97.0%, with the STOP-Bang questionnaire showing the highest sensitivity, followed by the Berlin questionnaire and the SA-SDQ (Table 2). The specificity of the three questionnaires ranged from 18.6-66.0%, with the SA-SDQ showing the highest specificity. The specificity of the Berlin (32.0%) and STOP-Bang (18.6%) questionnaires were much lower than that of the SA-SDQ. The PPV ranged from 76.4-85.9%, and the NPV ranged from 18.0-55.2%.

The prevalence of moderate-to-severe OSA (AHI ≥15/h) was 58.4%. Using an AHI ≥15/h, the sensitivity of the three questionnaires ranged from 74.0-98.0%, with the STOP-Bang questionnaire showing the highest sensitivity (Table 2). The specificity of the three questionnaires ranged from 10.6-60.8%, with the SA-SDQ showing the highest specificity. The specificity of the Berlin (35.4%) and STOP-Bang (10.6%) questionnaires were significantly lower than that of the SA-SDQ. The PPV ranged from 60.6-75.3%, and the NPV ranged from 50.6-78.8% (Table 2).

DISCUSSION

We evaluated the diagnostic value of three screening questionnaires in Korean patients with suspected OSA who visited a sleep laboratory. The prevalence of OSA was 83.6% for an AHI ≥5/h and 58.4% for an AHI ≥15/h. The mean BMI in our study population was 24.7 kg/m2. We used 30 kg/m2 as the cut-off value for BMI in the STOP-Bang questionnaire. The modified cut-off value of the SA-SDQ specific to Korean women was lower than the original cut-off value, and was therefore used in our present evaluation. Our data showed that the STOP-Bang questionnaire and the SA-SDQ had the highest sensitivity and specificity, respectively, for identifying Korean patients with a high risk for OSA (AHI ≥5/hour and AHI ≥15/hour). However, none of the three screening questionnaires studied was found to have a high sensitivity with an acceptable level of specificity for the use in a sleep clinic setting.

Patients who visit a sleep clinic usually have a high probability of a sleep disorder. Hence, an ideal questionnaire in such a setting should be sensitive enough to detect any patients with moderate and severe OSA. We found in our Korean subjects that the STOP-Bang questionnaire had a very high sensitivity (97% for AHI ≥5/h, 98% for AHI ≥15/h) with an acceptable PPV (86% for AHI ≥5/h, 61% for AHI ≥15/h), but that its specificity was low (19% for AHI ≥5/h, 11% for AHI ≥15/h). These findings are similar to those of recent studies 14,18 that showed high sensitivity of the STOP-Bang questionnaire in identifying sleep clinic patients at high risk for OSA. Ong, et al.14 found previously that the STOP-Bang questionnaire using a BMI cut-off value of 30 kg/m2 had a sensitivity of 94% for identifying OSA for AHI ≥15/h, and Vana, et al.18 also showed that the sensitivity of STOP-Bang questionnaires using a BMI cut-off point of 30 kg/m2 was very high, reporting 97% for OSA [AHI ≥5/h or respiratory distress index (RDI) ≥5/h] and 95% for moderate-to-severe OSA (AHI ≥15/h or RDI ≥15/h). However, the specificities of these tests were low, ranging from 18-40% for OSA or moderate-to-severe OSA. Although the low specificity was not the major concern in patients referred to a sleep clinic, we and others have found that the specificity of the STOP-Bang questionnaire may be significantly below acceptable levels.

In contrast to the STOP-Bang questionnaire, the sensitivity of the SA-SDQ was not high enough (68% for AHI ≥5/h, 74% for AHI ≥15/h) to be used in the clinic, but its specificity was relatively good (66% for AHI ≥5/h, 61% for AHI ≥15/h). These findings are consistent with those of a previous study,23 in which the authors evaluated the validity of the SA-SDQ in epileptic patients and found a sensitivity of 80% and a specificity of 66%. In that study population, 45% of patients had coexisting OSA with an AHI >5/h.

The Berlin questionnaire is most commonly used in studies of OSA screening.12,13,15,20,22 This questionnaire is usually evaluated as a screen for OSA in patients with no history of sleep disorders. Netzer, et al.12 reported that the Berlin questionnaire had a high sensitivity and moderate specificity (0.97 and 0.54, respectively) in a primary care population. In a previous study of patients referred to a sleep clinic,13 however, the Berlin questionnaire was found to show low sensitivity and specificity (0.62 and 0.43, respectively) using a RDI level of >10, which is in agreement with our present results. Hence, the Berlin questionnaire was not recommended as an appropriate instrument to identify patients with sleep apnea in a sleep clinic population.13

Another issue considered in our present study was whether the cutoff values used by original validation studies would be applicable in Asian patients who are known to have more severe OSA at a lower BMI. The mean BMI in our current study population was 24.7 kg/m2, which was much lower than that reported by Netzer, et al.12 in Americans and Germans (29 kg/m2) and by Chung, et al.11 (30 kg/m2) in a Canadian cohort. Based on the findings of another previous study on an Asian population,14 we here used a cutoff BMI value of >30 kg/m2 for a positive score in the STOP-Bang questionnaire, which was consistent with the cutoff BMI value used in the Berlin questionnaire.

We calculated ROC curves for the SA-SDQ from which we could recommend cutoff values of 36 for men and 30 for women to identify Korean patients at a high risk for OSA (AHI ≥5/h and AHI ≥15/h). Our modified cutoff for Korean men was the same as the originally reported value,16 but that for Korean women was lower than the original value. Our predictive values using a cutoff of 30 for women were somewhat better than the original values, especially in the specificity and PPV for identifying patients at high risk of moderate-to-severe OSA. The modified cutoff values for the STOP-Bang questionnaire in our present study to identify Korean patients with a high risk for OSA were the same as the originally reported values.

This study has several limitations that should be noted. First, our subjects were Koreans who visited a sleep center due to suspected OSA and were therefore considered to be at high risk for this disorder. Therefore, a bias may have been introduced that could markedly increase the apparent sensitivity of the questionnaire and reduce its specificity. Our current findings therefore cannot be generalized to patients with OSA in primary care clinics or in the general population.

In conclusions, the STOP-Bang questionnaire showed the highest sensitivity for identifying Korean patients with a high risk for OSA (AHI ≥5/h and AHI ≥15/h). It may be useful for screening OSA in a sleep clinic setting, but its specificity is lower than the acceptable level for this purpose. A new screening questionnaire with a high sensitivity and acceptable specificity is, therefore, needed for the use in a sleep clinic setting.