A Nomogram for Predicting the Likelihood of Obstructive Sleep Apnea to Reduce the Unnecessary Polysomnography Examinations.

BACKGROUND: The currently available polysomnography (PSG) equipments and operating personnel are facing increasing pressure, such situation may result in the problem that a large number of obstructive sleep apnea (OSA) patients cannot receive timely diagnosis and treatment, we sought to develop a nomogram quantifying the risk of OSA for a better decision of using PSG, based on the clinical syndromes and the demographic and anthropometric characteristics.
METHODS: The nomogram was constructed through an ordinal logistic regression procedure. Predictive accuracy and performance characteristics were assessed with the area under the curve (AUC) of the receiver operating characteristics and calibration plots, respectively. Decision curve analyses were applied to assess the net benefit of the nomogram.
RESULTS: Among the 401 patients, 73 (18.2%) were diagnosed and grouped as the none OSA (apnea-hypopnea index [AHI] <5), 67 (16.7%) the mild OSA (5 ≤ AHI < 15), 82 (20.4%) the moderate OSA (15 ≤ AHI < 30), and 179 (44.6%) the severe OSA (AHI ≥ 30). The multivariable analysis suggested the significant factors were duration of disease, smoking status, difficulty of falling asleep, lack of energy, and waist circumference. A nomogram was created for the prediction of OSA using these clinical parameters and was internally validated using bootstrapping method. The discrimination accuracies of the nomogram for any OSA, moderate-severe OSA, and severe OSA were 83.8%, 79.9%, and 80.5%, respectively, which indicated good calibration. Decision curve analysis showed that using nomogram could reduce the unnecessary polysomnography (PSG) by 10% without increasing the false negatives.
CONCLUSIONS: The established clinical nomogram provides high accuracy in predicting the individual risk of OSA. This tool may help physicians better make decisions on PSG arrangement for the patients referred to sleep centers.

INTRODUCTION

With the improvement of the social awareness about obstructive sleep apnea (OSA), the number of patients seeking treatment at sleep centers has drastically increased worldwidely. The polysomnography (PSG), though expensive and time-consuming, has significantly benefited patients in terms of diagnosis and treatment in past few decades. However, it would be infeasible to offer each patient the PSG with regards to effectiveness. The substantially growing demand will also cause pressure to the sleep specialists with the expertise to perform PSG and possibly prevent OSA patients from receiving early diagnosis and timely treatment.[12] One likely solution to improve the value of each PSG testing would be linked to the prediction of individual risk.

Sleep medicine literature has revealed some plausible risk factors for OSA based on well-established statistical models. A prediction model will further be helpful for optimizing the PSG arrangement and to use the health resources rationally, assisting the doctors to educate the patients and improving patients’ compliance.[3] Whereas, the reported OSA prediction models so far have only very limited impact on the clinical decision-making.[456789] Most recently, novel statistical methods have been proposed and soon applied in some field,[1011121314] such as the urology.[15161718192021] Particularly, nomogram has been deemed a proven reliable tool to quantify the hazard of disease based on multivariable modeling procedures.[22] With a visualized and intuitive graph, the satisfactorily precise prediction for an individual patient would be available, greatly facilitating easier diagnosis and providing various strategies to deal with a large number of sleep-disordered patients referred to hospitals. The goal of our study was to set up a simple yet accurate nomogram that allows the individualized OSA risk assessment, taking into account important clinical syndrome as well as the demographic and anthropometric characteristics. This system, supported by the transparent and informative nomogram,[232425] may help clinicians better make decisions whether the patient is needed to be further examined with PSG without increasing the workload.

METHODS

The protocol for this study was approved by the Ethics Committee of Nanfang Hospital. The medical records of 401 patients who had undergone PSG due to sleepiness, snoring, or other symptoms suspicious for OSA from September 2009 to June 2011 at Sleep Center, Nanfang Hospital, Southern Medical University were reviewed. The inclusion criteria included: Age ≥18 years; the medical history and demographic information, the anthropometric indicators, and the PSG data were complete. The exclusion criteria were: Pregnant female and patients with central sleep apnea syndrome. All the data were recorded by physicians after performing the standard disease history inquiry and physical examination. The demographic characteristics included gender and age. The potential explanatory variables included duration of disease, presence of snoring, restless sleep, choking, daytime sleepiness, lack of energy, dizziness, torpid reaction, hearing loss, morning headache, memory loss, concentrating difficulty, early awakening, chest tightness, mouth dryness, mouth pain, acid reflux, drooling, night sweating, nocturia (≥2 times), loss of libido, irritability, difficulty of falling asleep, and smoking status (currently smoking and having smoking history were both considered as smoking). Other plausible indicators included height, weight, neck circumference, chest circumference, waist circumference, presleep blood pressure, and postsleep blood pressure. All the PSG were conducted under the supervision of the sleep center medical staff. The monitoring lasted for at least 5 h. The results were manually reviewed by the experienced sleep specialists.

Statistical analysis

Data were presented as mean ± standard deviation (SD) or median with interquartile range. Risk factors were studied through an ordinary logistic regression modeling technique. The dependent variable was defined by categorizing apnea-hypopnea index (AHI), based on the admittedly professional criteria, into four classes: None OSA (AHI < 5), mild (5 ≤ AHI < 15), moderate (15 ≤ AHI < 30), and severe (AHI ≥ 30) OSA. Explanatory variables for the ordinal logistic regression model were: Duration of disease, lack of energy, difficulty of falling asleep, smoking status, and waist circumference. All covariates were chosen with a priori-based clinical relevance, and they stayed in the model regardless of their statistical significance from the univariate analysis. We employed bootstrap method to establish the regression equation with 1000 replications.

To assess the model calibration, we followed Copas's proposal that regression smoothing method was used to produce calibration plots where the relationship between observed and predicted probabilities of OSA was described graphically.[2627] A nomogram was therefore developed for identifying patients at risk of OSA, and further the score for each patient was calculated. It should be noted that the original data from 401 observations were fully considered as the model validation data and the score derived from nomogram as the outcome; then, the areas under the receiver operating characteristic curve area under the curve (AUC) were calculated, respectively, to evaluate the performance of the nomogram.

To illustrate the clinical effects application of the nomogram, the positive predictive value, the number of PSG, and missed diagnosis that can be reduced using nomogram, were calculated based on various cut-off values. The decision curve analysis was used to examine whether this prediction model is useful for medical decision-making.[10] In current case, this method estimates the net benefit of the prediction model by summing up the benefits (true positives) and subtracting the harms (false positives), where the latter is weighted by a factor related to the relative harm of a missed OSA compared with an unnecessary PSG. A model is of clinical value if it has a higher net benefit across the full range of threshold probabilities; it is able to aid decision-making upon PSG arrangement. That is, an assessment can be made on whether the net benefit of the prediction model is better than the blanket policy of treating or not treating all patients. According to the decision curve analysis, the theoretical number of unnecessary PSG examinations, which can be reduced by utilizing nomogram, under the same missed diagnosis rate was also calculated. The formulas for calculating the net benefit[12] and the number of unnecessary examinations that can be evitable are, respectively, given as follows:[1012]

The formula for calculating the number of unnecessary examinations that can be reduced is:[10]

SPSS 13.0 (SPSS Inc., Chicago, IL, USA) was used for statistical analysis and calculating the AUC. R 2.15 (2 May 2012, R Core Team, http://www.r-project.org/) was applied to build the ordinal logistic regression equation. The plotting and calibration of the nomogram were conducted using R with the “rms” package. The decision curve analysis was performed using STATA 11SE (StataCorp., TX, USA). A P < 0.05 was considered statistically significant.

RESULTS

A total of 401 patients with snoring, sleepiness, night breath difficulty, or other symptoms were included in the analysis. The clinical and demographic characteristics of these patients were shown in Table 1. There were 73 patients with AHI <5, 67 patients with 5≤ AHI <15, and 82 patients with 15≤ AHI <30; additionally, 179 patients were with AHI ≥30. The condition of the male OSA patients was generally severer: A longer the duration of disease indicated a worse condition. The symptoms of tiredness (or, say, the lack of energy), choking, nocturia, mouth dryness, and early awakening were more common to the moderate or severe OSA patients; in contrast, the symptom of difficulty in falling asleep appears more prevailing among the mild patients. The difficulty in falling asleep roots in the self-consciousness of the patients, and the time for awakening after sleep onset of the severe patients was shortest (data not shown). Smokers seem to have a higher risk of OSA since they accounted for a large proportion of moderate or severe patients. The greater the weight, the neck circumference, the chest circumference, and the waist circumference were, the severer the condition was. The postsleep blood pressure of the moderate or severe OSA patients was also relatively higher.

Table 1

Demographic, clinical characteristics of the study population

Characteristics	AHI <5 (n = 73)	5≤ AHI <15 (n = 67)	15≤ AHI <30 (n = 82)	AHI ≥30 (n = 179)	P
Age, years, mean ± SD	41 ± 12	47 ± 11	48 ± 10	44 ± 11	0.001
Duration of disease, months, mean (range)	60 (34–120)	96 (36–120)	108 (48–120)	120 (72–180)	<0.001
Male, n (%)	52 (71.2)	55 (82.1)	72 (87.8)	171 (95.5)	<0.001
Snoring, n (%)	65 (89.0)	63 (94.0)	80 (97.6)	173 (96.6)	0.051
Restless sleep, n (%)	8 (11.0)	11 (16.4)	7 (8.5)	18 (10.1)	0.443
Choking, n (%)	39 (53.4)	48 (71.6)	63 (76.8)	142 (79.3)	<0.001
Daytime sleepiness, n (%)	41 (56.2)	22 (32.8)	33 (40.2)	90 (50.3)	0.018
Lack of energy, n (%)	19 (26.0)	35 (52.2)	48 (58.5)	133 (74.3)	<0.001
Dizziness, n (%)	34 (46.6)	31 (46.3)	36 (43.9)	75 (41.9)	0.884
Torpid reaction, n (%)	1 (1.4)	4 (6.0)	4 (4.9)	4 (2.2)	0.298
Hearing loss, n (%)	1 (1.4)	2 (3.0)	1 (1.2)	1 (0.6)	0.505
Morning headache, n (%)	21 (28.8)	17 (25.4)	22 (26.8)	40 (22.3)	0.711
Memory loss, n (%)	23 (31.5)	22 (32.8)	28 (34.1)	44 (24.6)	0.329
Concentrating difficulty, n (%)	10 (13.7)	12 (17.9)	11 (13.4)	25 (14.0)	0.853
Early awakening, n (%)	8 (11.0)	3 (4.5)	0 (0)	2 (1.1)	<0.001
Chest tightness, n (%)	12 (16.4)	5 (7.5)	11 (13.4)	13 (7.3)	0.100
Mouth dryness, n (%)	43 (58.9)	44 (65.7)	60 (73.2)	138 (77.1)	0.023
Mouth pain, n (%)	7 (9.6)	7 (10.4)	9 (11.0)	24 (13.4)	0.810
Acid reflux, n (%)	0 (0)	0 (0)	1 (1.2)	1 (0.6)	0.666
Drooling, n (%)	0 (0)	0 (0)	0 (0)	2 (1.1)	0.477
Night sweating, n (%)	6 (8.2)	7 (10.4)	4 (4.9)	12 (6.7)	0.595
Nocturia ≥2, n (%)	11 (15.1)	23 (34.3)	28 (34.1)	64 (35.8)	0.011
Loss of libido, n (%)	1 (1.4)	1 (1.5)	1 (1.2)	3 (1.7)	0.993
Irritability, n (%)	1 (1.4)	1 (1.5)	0 (0)	1 (0.6)	0.663
Difficulty of falling asleep, n (%)	10 (13.7)	10 (14.9)	5 (6.1)	2 (1.1)	<0.001
Height, cm, mean (range)	168 (162–171)	168 (164–172)	168 (162–172)	170 (166–174)	0.003
Weight, kg, mean (range)	68 (60–75)	76 (65–82)	75 (70–81)	81 (75–88)	<0.001
Neck circumference, cm, mean (range)	37 (35–40)	39 (37–41)	40 (38–42)	41 (39–43)	<0.001
Chest circumference, cm, mean (range)	94 (89–100)	100 (93–104)	98 (96–104)	103 (99–108)	<0.001
Waist circumference, cm, mean (range)	90 (85–98)	99 (92–103)	98 (94–104)	103 (98–110)	<0.001
Presleep systolic blood pressure, mmHg, mean (range)	126 (116–135)	128 (120–137)	128 (118–139)	130 (120–138)	0.188
Presleep diastolic blood pressure, mmHg, mean (range)	79 (73–86)	78 (72–85)	80 (74–86)	81 (77–88)	0.036
Postsleep systolic blood pressure, mmHg, mean (range)	122 (112–131)	127 (118–137)	127 (120–138)	130 (122–142)	<0.001
Postsleep diastolic blood pressure, mmHg, mean (range)	80 (72–85)	81 (74–88)	82 (78–88)	85 (78–93)	<0.001
Smoking, n (%)	17 (23.3)	24 (35.8)	27 (32.9)	87 (48.6)	0.001

SD: Standard deviation; AHI: Apnea hypopnea index.

To assess the risk factors of OSA, we developed the modeling with ordinal logistic regression procedure, taking into account the effective predictors: Duration of disease, smoking status, difficulty in falling asleep, the lack of energy, and the waist circumference. The bootstrap resample method was applied to derive the mathematical equation and to establish the nomogram for predicting the probability of OSA [Figure 1]. The nomogram was used by firstly locating a patient with regards to each predictor on the horizontal scale. A score for an individual patient was then calculated by summing up the points derived from each item in the assessment system. This score will be corresponding to the risk of OSA, the risk of moderate-severe OSA or severe OSA.

Figure 1

Nomogram for predicting OSA on PSG. Instructions for physicians: To obtain the nomogram-predicted probability of PSG, locate patient values on each axis. Draw a vertical line to the “Point” axis to determine how many points should be attributed for each variable. Sum the points for all variables to obtain the total point. Locate the total point on the “Total Points” line, so that the individual probability of OSA on PSG can be assessed on the “Predicted probability of OSA” line. There are three probability lines corresponding to having any OSA, having moderate-severe OSA and having severe OSA, respectively. For example, for a patient with the following characteristics: Ten years of duration of disease, no smoking, no difficulty of falling asleep, lack of energy, waist circumference equal to 101 cm, his/her corresponding total score is 6 + 0 + 13 + 13 + 27 = 59; his/her corresponding probability of having OSA is 0.92, his/her probability of having moderate-severe OSA is 0.76 and his/her probability of having severe OSA is 0.53. OSA: Obstructive sleep apnea; PSG: Polysomnography.

We fulfilled the graphical assessment of calibration, portraying the relationship between observed and predicted probabilities of OSA. Specifically, the calibration curves were generated for the nomogram of: (1) Any OSA, (2) moderate-severe OSA, and (3) severe OSA. When compared with the ideal curve (45° line), there was a good agreement between the predicted and observed probabilities [Figure 2]. The Hosmer-Lemeshow goodness-of-fit test indicated that the difference between the probabilities predicted by the model and the observed probabilities was insignificant.

Figure 2

Calibration plot of nomogram by bootstrapping with 1000 resamples. Instructions for readers: The 45 line refers to perfect predictions. Points estimated below the 45 line refer to nomogram overprediction, whereas points situated above the 45 line refer to nomogram underprediction. A nonparametric, smoothed curve indicates the relationship between predicted probability and observed frequency of obstructive sleep apnea on polysomnography. Vertical lines indicate the frequency distribution of predicted probabilities.

Considering whether the patient has OSA, whether the patient has moderate-severe OSA, and whether the patient has severe OSA as the state variables, and taking the nomogram scores of the 401 sets of original data as the test variables, the AUC was analyzed [Figure 3]. An AUC of 0.8 indicated that the nomogram had a good discriminative ability for the three statuses.

Figure 3

Receiving operating characteristic curves of the nomogram.

The positive predictive values of different cut-off values were calculated according to the nomogram scores of the 401 patients [Table 2]. The positive predictive value referred to the percentage of truly sick patients whose diagnostic result was positive; it suggested the ability of the model for selecting correctly the true OSA patients, helping clinicians arrange necessary examination subsequently based on the predicted risk of disease. Table 2 showed that for those patients, whose score was ≥72, likely 100% of them had OSA, and 93.6% of them had severe OSA. For those patients, whose score was ≥50, 93.6% of them had OSA and 80.5% of them had moderate-severe OSA.

Table 2

The positive predictive values of the nomogram at different cut-off levels of score

Cut-off values	Positive predictive values
	Any OSA	Moderate-severe OSA	Severe OSA
≥20	0.822	0.656	0.450
≥25	0.831	0.668	0.458
≥30	0.842	0.678	0.467
≥35	0.869	0.707	0.494
≥40	0.886	0.724	0.520
≥45	0.907	0.758	0.553
≥50	0.936	0.805	0.594
≥55	0.971	0.846	0.659
≥60	0.987	0.903	0.716
≥65	0.989	0.935	0.839
≥70	0.984	0.902	0.869
≥72	1	0.936	0.936

OSA: Obstructive sleep apnea.

Figure 4 showed the decision curves. The decision curve analysis revealed that the nomogram, with a probability threshold of >45%, was able to substantially reduce the unnecessary examinations. Table 3 displayed the values of the net benefit plotted in Figure 4 and reflected the advantage of using nomogram to determine further PSG compared to the current strategy (performing PSG for every patient). The net PSG reduction was shown in Table 3. In particular, difference of 10 for a probability threshold of 82% can be interpreted as follows:

Figure 4

Decision curve for the outcome of any obstructive sleep apnea using the nomogram.

Table 3

Net benefit of conducting PSG for all patients or according to nomogram, using a threshold of patient

Pt (%)	Net benefit		Advantage of nomogram
	PSG all	Nomogram	Net benefit	Reduction in avoidable PSG per 100 patients
45	0.660	0.662	0.002	0
50	0.626	0.643	0.017	2
55	0.584	0.623	0.039	3
60	0.532	0.595	0.063	4
65	0.465	0.551	0.086	5
70	0.376	0.507	0.131	6
75	0.251	0.483	0.232	8
80	0.064	0.424	0.360	9
82	−0.040	0.414	0.454	10

The reduction of unnecessary PSG per 100 patients was calculated as: (net benefit of the nomogram-net benefit of conducting PSG for all)/(pt/1−pt])×100. This value was net of false negatives, and was therefore equivalent to the reduction of unnecessary PSG without a decrease of the number of patients with OSA who duly have PSG, pt referred to the threshold probability at which a clinician will assign patient to PSG. PSG: Polysomnography; OSA: Obstructive sleep apnea.

The nomogram would markedly reduce about 10% unnecessary PSG examinations, but did not miss any patient with OSA. This did not imply that the nomogram was free of false-negative rate issue. Instead, considering a new prediction model that produced no false negatives and reduced false positives by 10%, the nomogram would have equivalent net benefit compared to this new prediction model.

DISCUSSIONS

With a sufficient study population, we constructed a nomogram tailored to assessing the individual OSA risk for patients based on some clinical syndromes and demographic as well as anthropometric characteristics. The quantification of risk factors was confirmed through an ordinal logistic regression modeling: We identified the duration of disease, lack of energy, difficulty of falling asleep, smoking status, and waist circumference as predictors. Our investigation would help an easy individual risk assessment of OSA at the bedside before an expensive, time-consuming PSG testing, and would therefore markedly cut down the number of unnecessary tests. More specifically, the established nomogram would theoretically reduce the number of patients with threshold probability 82% who required PSG by 10% without a decrease in the number of patients with OSA, who duly have PSG.

The 2009 Guidelines suggested that the risk assessment for OSA patients should be incorporating concerns upon the medical history as well as the physical examination. High-risk patients are expected to undergo a timely PSG examination, yet others could be considered according to the risk degree of OSA and the complications.[28] A variety of prediction models have so far been developed to identify patients with OSA and guarantee them the examination priority.[293031] These models are useful for predicting the risk of OSA, but some of them require the use of special tools,[32] some of them are time-consuming and labor-consuming,[33] some of them require invasive examinations,[34] some of them are expensive,[30] some of them are not accurate enough,[31] and some had rather a low specificity.[353637]

In this study, the classifier system for patients potentially with/without (high) risk of OSA has shown a satisfactory performance, as was demonstrated by the AUC larger than 0.8. The calibration plots further illustrated that there was a good agreement between the predicted and observed probabilities of OSA. Notably, the nomogram is of great value in clinical applications. The nomogram is able to reduce the unnecessary PSG examinations by about 10% if using the selection threshold 82%. Table 3 provided more results of positive predictions from the nomogram with different thresholds. The sensitivity and specificity would not be given the same importance during the clinical decision-making. In fact, in order to reduce the number of “false patients” during the PSG examination, the specificity must be given a greater weight, as it could be positively associated with correct predictions.

From our clinical experience, patients coming for a sleep consultation have frequently described their feelings as “fatigue” or “lack of energy”.[38] In this study, an estimated 33% of the patients without sleepiness reported a lack of energy. The average AHI of those patients was 30, which indicated that “lack of energy” might suggest a high risk of severe OSA. Our study also offered a look that the OSA patients with different clinical severity might vary in terms of some symptoms. The severe OSA patients, for example, have rarely been recorded with difficulty in falling asleep; instead, they may be noticed with a longer duration of disease and a greater likelihood to feel fatigue (the alleged “lack of energy”). Such symptoms would be difficult to quantify, as feelings are usually subjective and cryptic. In this regard, we set up a dichotomous variable (yes or no) to describe. In the regression model, we included the duration of disease as a predictor simply because the duration of disease would indicate to some extent the severity of the OSA condition. It also revealed that OSA emerges gradually, and it takes long period from mild stage to severe stage. Intervening at an early stage can prevent or delay the occurrence of severe OSA. The difficulty in falling asleep is the reflection of the sleep debt. The severe OSA patients rarely have the difficulty of falling asleep. A possible reason is that their sleep requirement is even stronger. Concerned about the clinical significance and the collinearity issues, the ordinal logistic regression model only included the waist circumference out of options including chest, neck, and waist circumferences. The results of this study were also consistent with the previous studies.[39]

The decision curve analysis estimated the “net benefit” of the prediction model by summing up the benefits (true positives) and subtracting the harms (false positives), where the latter was weighted by a factor reflecting the relative harm of a miss-specified OSA versus an unnecessary PSG. Specifically, the weight could be derived from the threshold probability of OSA and required to be defined with a value that a physician would choose PSG. The decision curve analysis aimed at weighing the benefit and risk by comparing the net benefit of using the model with different threshold probabilities versus performing PSG examination immediately for all patients without selection. It confirmed whether the model was superior to the strategy of unselectively performing PSG for all patients. From our study, the nomogram provided a higher net benefit that it is superior to the strategy of examining all patients.

This study had several advantages and novel features: (1) All the predictors were the routine clinical examination data; it required no additional equipment, no invasion, no extra cost, and no complicated calculation; according to the nomogram paper, it took only seconds to calculate the risk of OSA for a patient; (2) The calculation process of nomogram was transparent, having a potential educational role; the patients can clearly see the impacts of the disease risk factors, so that they will change their lifestyle, such as quitting smoking and controlling weight; (3) It can improve the clinical decision-making. The main limitations of this study were that this model was established on the basis of a single center sample, and it was only internally validated by bootstrapping techniques. The validity of the model should be further validated at external institutions, and though our questionnaire had a higher level of discrimination, there were indeed false positive and false negative results.

In conclusion, the nomogram established in our study to predict the risk of OSA for an individual patient would be of great value to be used at sleep centers. Largely relying on clinical syndromes and the demographic and anthropometric characteristics, this nomogram would easily be applied to help a clinical decision-making, as it requires no extra workload and/or cost to accumulate patient information. The nomogram in this study was developed based on patients of the sleep center, its sensitivity and specificity in the general population need to be further investigated. Although this nomogram is insufficient for diagnosis, it assesses the pretest probability of sleep disordered breathing and prioritizes patients for evaluation.