Predicting the night-to-night variability in the severity of obstructive sleep apnea: the case of the standard error of measurement.

STUDY
OBJECTIVES: Night-to-night variability in the apnea-hypopnea index (AHI) may affect the accuracy of the diagnosis of obstructive sleep apnea (OSA) and treatment selection. This study was conducted to assess the utility of the standard error of measurement (SEM) in predicting the night-to-night variability in the OSA.
METHODS: Ninety nine patients underwent a 3-consecutive nights of sleep monitoring with a validated home portable monitoring devise (BTI-APNiA, BTI Biotechnology Institute, Vitoria, Spain). The night-to-night variability in apnea- and hypopnea-related measures and blood desaturation were assessed. The agreement between the three nights was also assessed. The SEM and the AHI of the first night were used to calculate a range for the severity of the OSA. This range was then challenged to predict the most frequent OSA severity, the OSA severity in nights 2 and 3, and the OSA severity in the three nights.
RESULTS: Ninety nine patients (mean age: 56±14 years) participated in the study. The mean body mass index was 25.4±4.0 Kg/m2 and the mean score of Epworth questionnaire was 8±5. The AHI of the first, second and third nights were 13.96±13.46, 13.76±12.76 and 13.52±12.91 events/h, respectively. The night-to-night variability in the AHI and the sleep time in supine position over the three nights were not statistically significant. However, the differences in the severity of the OSA was statistically significant (range of agreement in the diagnosis: 41.7%-83.3%). The standard error of measurement (SEM) considering the AHI was 4.64 events/h.. The SEM was efficient in predicting the most frequent OSA severity (among the three nights) in more than 96% of the cases.
CONCLUSIONS: The night-to-night variability in the AHI might affect the diagnosis of OSA. The use of standard error of measurement and the AHI of one single night would be of interest to predict the night-to-night variability in the severity of OSA.

INTRODUCTION

The apnea and hypopnea index (AHI) is the primary measure in the diagnosis of obstructive sleep apnea and its severity[1],[2]. It is also considered in determining the eligibility of the patient to receive treatment and as one of the criteria to select the type of treatment[3]. This index is also an indicator of the risk of health problems associated with the obstructive sleep apnea[4]-[10].

However, several studies have reported the presence of considerable individual night-to-night variability in the AHI[11]-[19]. On one hand, this variability may affect the accuracy of diagnosis and treatment decision. Also it would affect the assessment of the efficacy of non-CPAP treatments like the intraoral appliances and the titration of the mandibular advancement (on the bases of AHI value)[3]. On the other hand, may affect the cumulative effect of oxygen desaturation and the likelihood of OSA-related health problems[17].

In practice, the diagnosis of the OSA is based on the AHI measured from a sleep study of a single night[20]. This could be related to the long waiting list in the sleep disorder units, the high cost of sleep study and the shortage in the health professional[21]-[23]. For that, the numeric value of the AHI is presented without any dispersion measurement.

It would be useful for the clinician to be able to estimate the variability in the diagnosis of OSA from the outcomes of a sleep study of a single night[20]. This would reduce cost and facilitate the efforts to give an adequate response for the social need of the diagnosis of OSA. Standard error of measurement is related to both the standard deviation of the test outcomes and the reliability of the test. It is a subtle and complex measure that are useful in describing the likely range of actual scores such a patient might achieve as a result of the unreliability of the assessment[24].

The hypothesis of this study was that the use of SEM would be useful in predicting the night-to-night variability in the OSA. For that, home sleep monitoring with a respiratory polygraphy was performed over three consecutive nights. The study objectives were 1) assess the differences between the three nights in regard to apnea- and hypopnea-related measures, sleep time in supine position, and oxygen saturation measurements, 2) assess the agreement in the AHI between the three nights, 3) estimate the SEM, 4) assess the differences in the OSA diagnosis between the three nights, and 5) assess the ability of AHIfirsth night ± SEM in predicting the night-to-night variability of OSA.

MATERIALS AND METHODS

This article was written following the STROBE (Strengthening the Reporting of Observational studies in Epidemiology) guidelines[25]. To address the research purpose, the investigators designed and implemented a retrospective cohort study. The study population was composed of consecutive patients attending a private Sleep Disorders Unit (Vitoria, Spain) between May 2015 and February 2016 for the assessment of OSA. To be included in the study sample, patients had to be older than 18 years and have a valid sleep study with home portable monitoring (PM) device over three consecutive nights. The exclusion criteria were previous diagnosis of obstructive sleep apnea (OSA), presence of other sleep disorders, use of bimaxillary occlusal splints, use of continuous positive airway pressure (CPAP) machine, use of any medicament that may alter sleep, and ASA III-IV.

All procedures performed involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.

For all patients anthropometric data were collected for sex, age, weight, height, body mass index (BMI). Sleep questionnaire and scoring the Epworth sleepiness scale were carried-out[26].

All sleep studies were carried out at the patient’s home with a validated respiratory polygraphy (BTI-APNiA) over three consecutive nights. This device has five channels: respiratory flow, oxygen saturation, cardiac frequency, corporal position, and snoring. It corresponds to classification III of the AASM[27]. The device measured the nasal air flow with a probe connected to a transducer, and the oxygen saturation with cutaneous pulse oximetry via a finger probe (model 7000A and Modl XPOD 3012LP; Nonin Medical; Plymouth, MN). All sleep studies were analyzed automatically by BTI-APNiA according to the criteria of the American Academy of Sleep Medicine[28],[29]. The sleep analysis was performed by a single sleep technician and was supervised by a specialist in sleep medicine.

A respiratory event was identified as apnea if it had duration ≥ 10 sec and the drop in respiratory signal was > 90% of the amplitude of the reference respiratory air flow. However, if the drop in the respiratory signal was between 30% and 90%, accompanied by a drop in oxygen saturation ≥ 3%, the respiratory event was identified as hypopnea. AHI was then calculated by adding the total number of hypopnea and apnea events and then divided by the sleep duration in hours. To accept the studies for analysis the minimal time of recording with good signals should be at least of 240 min.

Statistical analysis

Absolute and relative frequency distributions were calculated for qualitative variables and mean values and standard deviations for quantitative variables.

Mauchly’s Test of Sphericity was used to asses normality before the performance of general linear model of repeated measures. This test was performed to assess the significance of the differences between the three nights in regard to apnea- and hypopnea-related measures, sleep time in supine position, and oxygen desaturation measurements. The reliability of the AHI between the three nights was assessed by the intraclass correlation coefficient (ICC) of average measures and the ICC of single measures. The 95% confidence intervals of the individual differences in the AHI between nights were calculated by ± 1.96 × the square mean of the differences due to error[30]. The χ2 test was used to assess the significance of differences in the diagnosis of OSA severity between the three nights. The OSA categories were mild (5 ≤ AHI < 15 event/h), moderate (15 ≤ AHI < 30 events/h), and severe (AHI ≤ 30 events/h). An AHI < 5 events/h exclude the diagnosis of OSA. Kappa as measurement of agreement in the diagnosis of OSA was also calculated.

The standard error of measurement (SEM) and was then calculated according to Fleiss and Kingman[30]. An estimated range of OSA severity was calculated considering the AHIfirst night ± SEM. These new values of AHI determined the OSA categories as described earlier and also into two main OSA categories: mild-to-moderate (AHI < 15 events/h) and moderate-to-severe (AHI ≥ 15 events/h). Then, several proportions was calculated to asses 1) the possibility of the OSA diagnosis in nights 2 and 3 to be within the estimated range, 2) the possibility of the most frequent diagnosis of OSA of the three nights to be within this range, and 3) the possibility of the diagnosis of OSA of the three nights to be within this range. The statistical significance was set at p<0.05. SPSS v15.0 for Windows statistical software package (SPSS Inc., Chicago, IL, USA) was used for statistical analysis.

RESULTS

The study sample was formed by 99 patients (44% females) with a mean age of 56±14 years (range: 16 to 86 years). The mean body mass index of 25.4±4.0 Kg/m2. The neck and waist perimeters were 38±5 and 94±14 cm, respectively. The mean score value of the Epworth questionnaire was 8±5.

The statistical analysis showed no statistically significant differences between the 3 nights in regards to the mean values of the variables of AHI, AHI in supine, AHI in no supine, central apneas and hypopneas, sleep time in supine position, and oxygen desaturation measurements (Table 1). The ICC of the average measure was excellent (≥ 0.9) for all variables except for the AHI in non supine position which was optimum (≥ 0.8) (Table 2). However the ICC of single measure was lower being optimum for all variables except for the AHI in non supine position and time in supine position which were good (≥ 0.7).

Table 1

Comparison between the home PM devise over the three nights.

	Nights	Mean	SD	95% Confidence Interval for Mean		p value
				Lower Bound	Upper Bound
AHI (events/h)	1	13.96	13.46	11.28	16.64	0.940
	2	13.76	12.76	11.22	16.31
	3	13.32	12.91	10.75	15.90
AHIsup (events/h)	1	23.37	20.14	19.35	27.39	0.745
	2	25.70	22.03	21.30	30.10
	3	24.63	22.04	20.23	29.03
AHInosup (events/h)	1	7.39	8.61	5.67	9.10	0.772
	2	6.64	7.23	5.20	8.08
	3	6.78	7.46	5.29	8.27
%timesup	1	37.35	24.79	32.40	42.29	0.680
	2	38.35	25.39	33.28	43.41
	3	40.93	24.95	35.95	45.91
SaO2_min	1	85.71	5.81	84.56	86.88	0.584
	2	85.15	6.28	83.90	86.40
	3	84.99	6.26	83.74	86.24
SaO2_average	1	93.62	1.65	93.29	93.94	0.539
	2	93.33	1.98	92.94	93.73
	3	93.53	1.88	93.16	93.91
CT90	1	4.31	10.23	2.27	6.35	0.402
	2	6.83	15.49	3.74	9.92
	3	6.09	14.09	3.28	8.90

Table 2

Agreement measurements between the three nights.

	Cronbach's Alpha	ICC single measures	95% Confidence Interval for Mean		ICC average measures	95% Confidence Interval for Mean
			Lower Bound	Upper Bound		Lower Bound	Upper Bound
AHI	0.954	0.874	0.829	0.909	0.954	0.936	0.968
AHIsup	0.928	0.810	0.748	0.862	0.928	0.899	0.949
AHInosup	0.878	0.705	0.618	0.780	0.878	0.829	0.914
Timesup (%)	0.915	0.779	0.708	0.838	0.914	0.879	0.939
Centrals apneas and hypopneas (%)	0.923	0.800	0.727	0.859	0.923	0.889	0.948

Even more, the lower limit of the ICC of single measure was below 0.7 for the variable the AHI in non supine position (Table 2). The Bland-Altman plot supported the presence of wide individual variability in the values of the AHI between the three nights (Figure 1). The Bland-Altman plot shows the individual difference in the AHI between the three nights with the 95% confidence intervals (± 9.1). The 83%, 87% and the 84% of the individual AHI difference between nights 1 and 2, nights 1 and 3 and nights 2 and 3 were within this confidence interval.

Figure 1

A, B and C) Bland-Altman plot demonstrating individual differences in AHI between the three nights. D) The differences in the individual AHI between nights. The solid lines represent the 95% confidence interval of the change in the AHI between nights.

Tables 3 to 5 show that the individual variability in the AHI between the three nights resulted in a significant differences in the diagnosis of the severity of the OSA. The Kappa as measurement of agreement was low (< 0.6) between the three nights. The coincidence in the diagnosis between the three nights had a wide variation between 41.7% and 83.3%. The standard error of measurement (SEM) considering the AHI was 4.64 events/h.

Table 3

Variability in the severity of OSA between nights 1 and 2.

		OSA in night 2				Total
		free of OSA	mild	moderate	severe
OSA in night 1	free of OSA	25	10	1	0	36
	mild	5	16	7	0	28
	moderate	0	7	11	5	23
	severe	0	0	5	7	12
Total		30	33	24	12	99

p value (χ2 test) = 0.000

Kappa = 0.442

Table 5

Variability in the severity of OSA between nights 2 and 3.

		OSA in night 3				Total
		free of OSA	mild	moderate	severe
OSA in night 2	free of OSA	23	7	0	0	30
	mild	5	24	4	0	33
	moderate	0	10	10	4	24
	severe	0	0	3	9	12
Total		28	41	17	13	99

p value (χ2 test) = 0.000

Kappa = 0.536

In order to assess the possibility of predicting the variability in the OSA between the three nights, the values of AHI of the first night was modified by ± SEM. The range of OSA severity was then calculated considering 4 categories of OSA (No OSA, mild, moderate and severe). Table 6 shows that 84% of the OSA severity at nights 2 and 3 were within the calculated range. If we consider the three nights, the possibility of the most frequent OSA severity to be within the calculated range was of 96%. It is important to mention that the estimated range of OSA was narrow and spanned at the most 2 OSA categories (for example OSA range between mild to moderate). This range was the same as the range of OSA between the three nights in 48%. These data were very similar if the AHI of the second night was considered (85%, 95% and 44%, respectively).

Table 6

Efficacy of standard error of measurement to estimate a range for OSA variability based on data of apnea-hypopnea index of the first night.

	OSA in nights 2 and 3	The most frequent OSA between the three nights	The OSA of the three nights
Out of range (%)	16	4	53
Within range (%)	84	96	48

Recoding of the OSA categories in two main types (mild and moderate-to-severe) showed event better predictability. Table 7 shows that 94% of the OSA severity at nights 2 and 3 were within the calculated range. The possibility of the most frequent OSA severity to be within the calculated range was of 99%. This range was the same as the range of OSA between the three nights in 84% of the cases.

Table 7

Efficacy of standard error of measurement to estimate a range for OSA variability (considering 2 OSA categories: mild and moderate-to-severe) based on data of apnea-hypopnea index of the first night.

	OSA in nights 2 and 3	The most frequent OSA between the three nights	The OSA of the three nights
Out of range (%)	6	1	16
Within range (%)	94	99	84

DISCUSSION

Although the polysomnography (PSG) is the gold-standard method in the diagnosis of OSA, home sleep studies with portable monitoring devices (PM) are increasingly used as a first-choice diagnostic test by many health care systems[17]. Home PM would reduce costs and facilitate the diagnostic process[22],[23]. It is important then to assess the night-to-night variability of the AHI assessed through home portable monitoring devices. As the potential of missed diagnosis and misclassification of the OSA severity by PM remains uncertain[17].

This study has shown that the differences have not been statistically significant between the three consecutive nights in regard to apnea- and hyoponea-related variables as well as oxygen desaturation variables. However, a considerable individual variability in the AHI has been observed.

This is in agreement with the findings of previous reports that have assessed the night-to-night variability in the AHI[11]-[19]. There is no clear explanation of the variability in the AHI but one study has shown that the spontaneous changes in daytime leg fluid accumulation and overnight rostral fluid shift were related to the changes in NREM and supine AHI[19].

Thus, the variability in the AHI should be considered when this parameter is used in the clinical practice for the diagnosis of OSA and the selection of treatment[11]. There is a need to assess the impact of night-to-night variability in treatment titration and the assessment of its efficacy, particularly for the non-CPAP treatments. The present study has the focus on how to manage this night-to-night variability in the clinical practice rather than finding an explanation for it. From statistical point of view the true value of AHI could not be measured and the variation in the AHI could be related to the unreability of the test. Smith et al has estimated the predicted true score (PTS) of AHI in a consecutive series of 1338 patients[20]. The 95% CI for the PTS for the true AHI can span conventional AHI severity categories. In that study, for example, the 95% CI for the PTS based on an observed AHI of 15 events/h was 5.6 - 38.0 events/h[20].

In the present study a range of OSA severity has been estimated for each patient considering the AHIfirst night ± SEM. For example, the intervals of an observed AHI of 15 events/h were 10.36 to 19.69 events/h. The OSA severity of nights 2 and 3, and the most frequent OSA severity of the three nights have been within the estimated OSA range in 84% and 96% of the cases, respectively. This estimated range has spanned at the most two categories (for example, OSA range between mild to moderate). This would implicate that most of the observed AHI values were within 4.64 units from the limits of the OSA categories. This method would thus be helpful in saving the need for home sleep monitoring for several nights. This would increase the potential of home PM in saving resources and facilitate the diagnostic process.

The standard error of measurement describes the likely range of actual values such a patient could have as a result of the unreliability of the assessment[24]. Aarab et al.[11] have estimated that SEM for PSG based on AHI values over 4 nights. The SEM in that study had the value of 4.6 events/h which is very close to the value obtained in the present study (4.64 events/h). Although this similarity is promising, there is a need to assess the variability of SEM in different populations and differed devices. This is will be an important aspect for the use of SEM to predict the night-to-night variability of the severity of OSA.

The limitations of the present study have been the retrospective design, the absence of baseline PSG study, one sleep technician assessed the sleep studies, and the number of monitored nights (3 consecutive nights). However, Prasad et al. have observed that increasing the number of nights from 2 to 8 has not affected the night-to-night variability[17]. The absence of PSG study has no allowed the assessment of the AHI variability according to the severity of the OSA. However, the settings of the present study have simulated the real picture of the clinical practice when home PM devise is used. Moreover, the mean AHI of the study population has been relatively low. This would indicate the need of more studies where the study population has higher mean values.

CONCLUSIONS

The night-to-night variability in the AHI between the home PM over three consecutive nights has not been statistically significant. However the differences in the OSA severity has been statistically significant. A considerable individual variability in the AHI has also been observed. The variability in the AHI should be considered when this parameter is used in the clinical practice for the diagnosis of OSA. The use of the standard error of measurement and AHI of a single night would be helpful in suggesting a narrow range for OSA severity that would predict the most frequent OSA severity among the three nights.

Table 4

Variability in the severity of OSA between nights 1 and 3.

		OSA in night 3				Total
		free of OSA	mild	moderate	severe
OSA in night 1	free of OSA	25	11	0	0	36
	mild	2	21	5	0	28
	moderate	1	8	11	3	23
	severe	0	1	1	10	12
Total	28	41	17	13	99

p value (χ2 test) = 0.000

Kappa = 0.554