Association of hypoglossal nerve stimulator response with machine learning identified negative effort dependence patterns.

BACKGROUND: Hypoglossal nerve stimulator (HGNS) is a therapeutic option for moderate to severe obstructive sleep apnea (OSA). Improved patient selection criteria are needed to target those most likely to benefit. We hypothesized that the pattern of negative effort dependence (NED) on inspiratory flow limited waveforms recorded during sleep, which has been correlated with the site of upper airway collapse, would contribute to the prediction of HGNS outcome. We developed a machine learning (ML) algorithm to identify NED patterns in pre-treatment sleep studies. We hypothesized that the predominant NED pattern would differ between HGNS responders and non-responders.
METHODS: An ML algorithm to identify NED patterns on the inspiratory portion of the nasal pressure waveform was derived from 5 development set polysomnograms. The algorithm was applied to pre-treatment sleep studies of subjects who underwent HGNS implantation to determine the percentage of each NED pattern. HGNS response was defined by STAR trial criteria for success (apnea-hypopnea index (AHI) reduced by > 50% and < 20/h) as well as by a change in AHI and oxygenation metrics. The predominant NED pattern in HGNS responders and non-responders was determined. Other variables including demographics and oxygenation metrics were also assessed between responders and non-responders.
RESULTS: Of 45 subjects, 4 were excluded due to technically inadequate polysomnograms. In the remaining 41 subjects, ML accurately distinguished three NED patterns (minimal, non-discontinuous, and discontinuous). The percentage of NED minimal breaths was significantly greater in responders compared with non-responders (p = 0.01) when the response was defined based on STAR trial criteria, change in AHI, and oxygenation metrics.
CONCLUSION: ML can accurately identify NED patterns in pre-treatment sleep studies. There was a statistically significant difference in the predominant NED pattern between HGNS responders and non-responders with a greater NED minimal pattern in responders. Prospective studies incorporating NED patterns into predictive modeling of factors determining HGNS outcomes are needed.

Introduction

The hypoglossal nerve stimulator (HGNS), Inspire®, is a therapeutic option for patients with moderate to severe obstructive sleep apnea (OSA) who are intolerant of positive airway pressure therapy (PAP). However, HGNS requires surgical implantation and is associated with considerable cost. Thus, patient selection criteria to target therapy to those most likely to benefit are important.

Currently, patient selection is based on inclusion criteria developed for the STAR trial, the investigation that led to FDA approval of Inspire® HGNS [1]. These criteria include age > 18 years, body mass index (BMI) < 32 kg/m2 (increased to < 35 kg/m2 for some insurance carriers), and moderate to severe OSA defined as an apnea–hypopnea index (AHI) between 15 and 65 events/h. Subsequently, drug-induced sleep endoscopy (DISE), which assesses the site and type of airway obstruction, is performed for further phenotyping [2].

The pattern of airway obstruction on DISE has been associated with HGNS outcome. Anterior–posterior rather than concentric collapse at the level of the velopharynx is associated with better response [3, 4]. However, other studies concluded that DISE may not be a reliable indicator of therapeutic efficacy [5]. Current clinical and DISE selection criteria remain suboptimal as successful improvement of the apnea–hypopnea index (AHI) with HGNS is achieved in only 60–70% of patients according to international registry data [1, 6, 7]. Additionally, DISE is invasive and requires conscious sedation.

Consequently, there is a paucity of reliable, non-invasive predictors of response to HGNS. The Mallampati and Friedman scores have limited predictive value [5]. Imaging of the upper airway (UA) using computed tomography and other modalities is also an inadequate indicator of response [8, 9]. Some studies have shown that higher baseline AHI may be associated with a better response [10] while others have reported the opposite [6]. Additionally, neck circumference, age, and sex are also unreliable predictors of response [6]. A more consistent finding is that lower pre-operative and intra-operative PAP required for airway opening [11, 12] is associated with a greater reduction in the AHI with HGNS. The association of lower airway opening pressures with HGNS efficacy suggests that responders have more compliant airways, particularly at the level of the soft palate, which may be more amenable to airway opening with HGNS. Therefore, it is reasonable to hypothesize that HGNS efficacy may be dependent, at least in part, on the predominant site of UA obstruction. A non-invasive means to identify the pattern and primary site of UA obstruction during sleep may be useful to better select patients for HGNS.

Polysomnographic (PSG) signals can identify features of OSA that may lead to a personalized approach to therapy [13]. In this regard, different patterns of inspiratory flow limitation have been observed in the nasal pressure signal, a surrogate for airflow typically recorded during sleep studies. Negative effort dependence (NED) is present when inspiratory airflow decreases despite increasing driving pressure. This results in distinct patterns of inspiratory flow limitation that have been associated with different anatomic levels of UA collapse [14] (Fig. 1). Building on the relationship between NED pattern and level of UA collapse [14, 15], we hypothesized that the predominant pattern of inspiratory flow limitation on the nasal pressure signal in pre-operative sleep studies could contribute to the characterization of OSA phenotypes associated with HGNS response.

Fig. 1

NED patterns. Signals obtained from the nasal pressure signal (surrogate for airflow) of a sleep study (PSG and HST) demonstrating A non-flow limited inspiratory pattern, B NED minimal, which is defined by a less than 34% reduction from the peak to the plateau of the inspiratory flow signal, C NED non-discontinuous pattern defined as more than a 34% decrease from the peak to the plateau of the inspiratory flow signal, and D NED discontinuous which also has more than a 34% decrease from peak to plateau of the inspiratory signal as well as an abrupt disruption in flow

However, manually identifying and quantifying NED patterns on a breath-by-breath basis on overnight sleep studies is impractical. Artificial intelligence and machine learning (ML) have been utilized to assist with PSG/HST scoring and approach “human-level” accuracy [16]. Furthermore, ML has been suggested as an efficient high-dimensional tool to assess physiological waveforms to define OSA subtypes [17-20]. In this research, we trained an ML model to identify NED patterns on the nasal pressure signal of diagnostic sleep studies. We then retrospectively applied the model to a separate test data set to assess the association of NED pattern predominance for HGNS therapy responders and non-responders.

Methods

Dataset

All patients (n = 50) were evaluated and treated for OSA with the Inspire® HGNS at the Northwell Health Sleep Disorders Center as per standard clinical practice. Surgical implantation of Inspire® was performed by one otolaryngologist (MS). The Northwell Health Institutional Review Board approved this study. This analysis is a retrospective assessment of preimplantation diagnostic sleep studies as well as post-implantation sleep studies performed after optimization of HGNS settings. All sleep studies were scored per AASM guidelines. Hypopneas were defined as a 30% reduction in nasal flow accompanied by a ≥ 4% decrease in SpO2.

The development dataset contained 5 patients encompassing 5 pre-therapy PSG sleep studies acquired with Natus Sleepworks. PSG’s that contained a representation of all 3 NED patterns were chosen for the development set. The test dataset contained a total of N = 45 patients encompassing 90 sleep studies, consisting of 45 pre-therapy studies and 45 post-therapy studies. All 45 pre-therapy studies were manually reviewed for technical adequacy resulting in the exclusion of 4 studies. The remaining test dataset N = 41 consisted of 5 PSG recordings acquired with Natus Sleepworks and 36 HSATs acquired with Noxturnal T3. Both Natus Sleepworks and Noxturnal T3 incorporate clinically equivalent nasal pressure transducers. Forty-five post-therapy studies included 6 in-laboratory PSGs and 39 HSATs. Post-therapy HSATs were performed on a single HGNS voltage throughout the night. Post-therapy PSGs were performed as titration studies with AHI and oxygenation data taken from the segment of the study with the therapeutic voltage that resulted in the lowest AHI. Respiratory indices were calculated using total recording time (TRT) defined as the time from “lights out” to “lights on” for both PSGs and HSATs so that indices derived from both study types would be comparable. Post-implantation adjustments were made according to the standard Inspire® post-implantation pathway [21]. Pre-therapy and post-therapy HSATs were performed instead of PSG in some patients because of insurance reasons, patient preference, and COVID-19 restrictions.

Negative effort dependence (NED) analysis

NED definition

As described by Genta et al. [14], three NED patterns were identified and were characterized by the percent reduction from the peak to the plateau of the inspiratory portion of the nasal pressure waveform. The NED minimal pattern was characterized by < 34% reduction, NED non-discontinuous > 34% decrease, and NED discontinuous > 34% decrease with abrupt disruptions in flow (Fig. 1). These three NED patterns were used to characterize breaths in 5 pre-therapy PSG sleep studies, which in turn were used to develop the NED identification model.

NED dataset

A total of 2690 NED events across 5 PSGs used for the development set were manually classified by two experienced sleep physicians into one of the three NED patterns based on the NED pattern criteria described above. Overall, 953/2690 (35.4%) were annotated as NED minimal; 744/2690 (27.7%) as NED discontinuous; and 993/2690 (36.9%) as NED non-discontinuous.

NED modeling

This characterization of NED events was used to train a 15-layer convolutional neural network (CNN) model. The model was trained to classify each manually annotated NED event into one of the three patterns. The input to the CNN was a 30-s window of the nasal pressure flow signal centered at the end of each annotated NED event (supplemental Fig. 1). Leave-one-patient-out (LOPO) cross-validation was used to evaluate the CNN model.

HGNS response analysis

Response definition

A successful HGNS response was initially defined by criteria used in the STAR trial as a 50% decrease of the AHI and a post-therapy AHI < 20/h [1]. In addition to AHI, other oxygenation indices were analyzed as HGNS response endpoints including both ≥ 3% and ≥ 4% desaturation criteria (ODI3, ODI4), percent of study time with SpO2 below 90% (T90), and the hypoxic burden index (HBI) defined by the area under the desaturation events with associated respiratory events [22]. These indices were analyzed as a decrease of greater than or equal to 50% between the pre-therapy and post-therapy sleep study as additional endpoints of HGNS response denoted as ΔAHI, ΔODI3, ΔODI4, ΔT90, and ΔHBI.

NED variables

The NED event model was leveraged to generate three NED pattern variables for each patient in the HGNS response statistical analysis (Fig. 2). The NED model was evaluated on all breaths of the test set during the total recording time. A signal processing-based method was used to detect each breath and the input window was centered at the inspiratory portion of each individual breath flow waveform. The 3 resulting NED variables represent the proportion of each predicted NED event type as a ratio of all analyzed breaths for each patient.

Fig. 2

Dataset description. Description of the datasets used in the study as well as the NED pattern classification and HGNS response modeling. A dataset of 50 patients was split into two sub-datasets. The first (N = 5) is used for the development of a NED pattern classification model which is trained and validated using a leave-one-patient-out (LOPO) cross-validation methodology. The second (N = 45) is used to generate HGNS response variables consisting of a combination of NED-based variables (calculated utilizing the trained NED classification model), demographics, and sleep study indices. A p-value is then calculated for each feature to determine the statistical significance associated with HGNS response

Demographic, clinical, and sleep study variables

Demographic, clinical, and sleep study variables were evaluated in the statistical analysis for HGNS response in addition to NED variables. Demographic variables included gender and age; clinical variables included pre-therapy BMI and Epworth Sleepiness Scale (ESS) scores; sleep study variables included pre-therapy AHI, ODI3, ODI4, T90, and HBI.

Statistical analysis

A two-sided t-test was used to evaluate all NED, demographic, clinical, and sleep study variables for HGNS response. All variables were evaluated with endpoints defined by STAR Trial, ΔAHI, ΔODI3, ΔODI4, ΔT90, and ΔHBI. Statistical significance was determined at p < 0.05.

Results

NED event results

The NED classification model showed a classification accuracy of 84% among 2690 NED events (Fig. 3). The high agreement between the model and manual annotation suggests that the annotators were consistent in manual classification and that different NED subclasses contained distinct flow patterns that could be discerned by the machine learning model. Furthermore, there was a low level of disagreement in the model between NED discontinuous and NED minimal with only 25/1727 (1.4%) events being misclassified as the other class suggesting a strong phenotypic differentiation between these NED subclasses.

Fig. 3

NED classification. A machine learning model was trained to distinguish between different NED patterns: NED minimal, NED non-discontinuous, and NED discontinuous. There was a high agreement between the model and manual annotation with an accuracy of 84%

HGNS effect of therapy

The STAR Trial’s definition of response is based solely on a specific change in AHI. However, a link between the severity of nocturnal oxygen desaturation and cardiovascular consequences has been demonstrated [22, 23]. As such, we hypothesized that expanding the definition of therapy response to include nocturnal oxygenation metrics may yield a more robust and relevant assessment of HGNS outcome. Therefore, metrics describing sleep-related oxygenation including T90, ODI, and HBI were also incorporated into the response definition. The effect of HGNS on AHI, ODI3, ODI4, T90, and HBI was assessed by Wilcoxon signed-rank test. All indices showed statistically significant improvement with HGNS (Table 1).

Table 1

Effect of HGNS therapy

	Baseline Mean	Baseline 0.25 quartile	Baseline 0.75 quartile	HGNS Mean	HGNS 0.25 quartile	HGNS 0.75 quartile	p-value
Endpoint
AHI	36.0	23.7	45.6	17.05	6.8	22.1	p < 0.001
ODI3	40.1	30.7	53.1	26.9	13.7	37.0	p < 0.001
ODI4	31.0	20.3	42.0	17.6	7.8	25.4	p < 0.001
T90	0.17	0.06	0.23	0.11	0.03	0.13	p < 0.001
HBI	1.49	0.69	1.86	0.67	0.15	0.83	p < 0.001

Effect of HGNS therapy on our patient cohort was compared using Wilcoxon signed-rank test and included AHI as well as metrics of nocturnal oxygen saturation (ODI, T90, and HBI). There is a statistically significant (p < 0.001) difference in this patient cohort in AHI, ODI3, ODI4, T90, and HBI

HGNS results

The test dataset consisted of 35 men and 6 women. Other demographic information with standard deviations includes age 61.5 ± 11.8 years, BMI 29.9 ± 3.4 kg/m2, pre-treatment AHI 36.6 ± 14.9/h, and ESS 10.6 ± 5.8. We classified patients as HGNS responders or non-responders based on the STAR Trial definition of success, ΔAHI, ΔODI3, ΔODI4, ΔT90, and ΔHBI. Comparison of responders and non-responders revealed no statistically significant differences in image, gender, BMI, pre-treatment ESS, AHI, ODI3, ODI4, HBI, or percentage of NED discontinuous and NED non-discontinuous patterns for the majority of endpoints (Table 2). However, the percentage of NED minimal pattern across endpoints defined by STAR Trial (p = 0.01), ΔAHI (p = 0.01), ΔODI3 (p = 0.01), ΔODI4 (p = 0.01), and ΔT90 (p = 0.02) differed significantly between responders and non-responders (Table 1). Overall pre-therapy characteristics that appear to define responders are those with a greater amount of NED minimal pattern. Specifically for the STAR trial criteria, responders exhibited an 18% higher proportion of NED minimal breaths (48% vs. 30%).

Table 2

Test set feature analysis (STAR Trial Responder vs. non-responder characteristics)

	STAR Trial	ΔAHI-50%	ΔODI3-50%	ΔODI4-50%	ΔT90-50%	ΔHBI-50%
N (# responder, # non-responder)	26, 15	28, 13	22, 19	18, 23	19, 22	18, 23
Demographic & clinical variables
Age, yr	0.90 (60.6, 61.1)	0.64 (61.4, 59.5)	1.00 (60.8, 60.8)	0.67 (61.5, 59.9)	0.01 (65.3, 55.6)	0.01 (64.7, 55.8)
Gender, M:F	0.86 (22:4, 13:2)	0.40 (23:5, 12:1)	0.85 (19:3, 16:3)	0.15 (13:5, 22:1)	0.50 (15:4, 22:2)	0.58 (14:4, 19:2)
Body mass index, kg/m.2	0.11 (29.4, 31.2)	0.06 (29.3, 31.5)	0.19 (29.2, 30.7)	0.08 (29.2, 31.1)	0.85 (29.9, 30.1)	0.71 (29.8, 30.2)
Epworth Sleepiness Scale	0.99 (10.3, 10.3)	0.45 (9.89, 11.3)	0.16 (11.6, 9.23)	0.12 (11.5, 8.83)	0.55 (9.86, 10.9)	0.28 (9.52, 11.4)
Sleep study variables
AHI, events/h	0.84 (35.7, 36.6)	0.91 (36.2, 35.6)	0.83 (36.5, 35.6)	0.85 (36.4, 35.5)	0.51 (37.5, 34.3)	0.61 (37.1, 34.7)
ODI3, events/h	0.42 (41.7, 37.2)	0.20 (42.4, 35.0)	0.23 (43.5, 37.1)	0.13 (43.6, 35.5)	0.15 (43.6, 36.0)	0.16 (43.4, 35.8)
ODI4, events/h	0.65 (31.9, 29.3)	0.32 (32.8, 27.1)	0.24 (34.4, 28.0)	0.18 (34.1, 27.0)	0.13 (34.7, 26.6)	0.10 (34.8, 26.1)
T90, %time	0.05 (0.13, 0.22)	0.16 (0.14, 0.21)	0.05 (0.12, 0.21)	0.01 (0.12, 0.23)	0.54 (0.15, 0.18)	0.65 (0.18, 0.15)
HBI, (%min)/h	1.00 (89.1, 89.0)	0.20 (100.2, 65.1)	0.69 (94.6, 84.4)	0.77 (92.4, 84.9)	0.22 (103.5, 72.4)	0.02 (114.2, 57.1)
NED variable
No discontinuous, %	0.41 (41, 45)	0.25 (40, 47)	0.17 (38, 46)	0.19 (39, 46)	0.89 (42, 43)	0.91 (43, 42)
Discontinuous, %	0.18 (12, 18)	0.56 (13, 16)	0.34 (12, 16)	0.31 (12, 16)	0.03 (10, 19)	0.13 (11, 18)
Minimal, %	0.01 (48, 30)	0.01 (47, 30)	0.01 (50, 34)	0.01 (49, 32)	0.02 (48, 33)	0.08 (46, 35)

Responders and non-responders from the test set were identified by the STAR trial definition (AHI reduction by 50% and AHI < 20 events/hr). A two-sided t-test was used to analyze differences in demographic, clinical, sleep study, and NED variables distinguishing responders from non-responders. Other endpoints include reduction by > 50% in AHI (but not fulfilling < 20 events/hr), ODI3, ODI4, T90, and HBI are listed. p-values are listed for each variable associated with each metric. Aside from gender, numbers in parentheses are the mean value (responders, non-responders)

Among these various metrics, the percentage of breaths with a NED minimal pattern of inspiratory flow limitation was consistently significantly different between responders and non-responders regardless of whether we defined success based on reduction in AHI, ODI3, ODI4, or T90. Thus, NED minimal pattern emerged as a significant factor distinguishing responders and non-responders with more NED minimal pattern breaths in the responders.

Discussion

Current patient selection criteria for Inspire® HGNS are associated with successful outcomes in 60–70% of cases using the STAR trial definition of success [7]. More precise selection criteria may increase success rates. We hypothesized that the pattern of inspiratory flow limitation seen on the nasal pressure signal of pre-operative sleep studies could contribute to the prediction of response to HGNS therapy. Negative effort dependence (NED) develops during inspiratory flow limitation when inspiratory airflow decreases despite increasing driving pressure. This results in distinct patterns of inspiratory flow limitation which have previously been described as NED minimal, NED non-discontinuous, and NED discontinuous. As these NED patterns have been associated with different anatomic levels of UA collapse, they may have value in predicting response to HGNS therapy [14]. We developed an ML model to detect these different NED patterns on pre-therapy sleep studies in a development set. We then applied this model to a subsequent test data set and evaluated the association of the predominant NED pattern on the pre-therapy sleep study with response to HGNS therapy. Our data revealed a greater percentage of NED minimal pattern of inspiratory flow limitation during pre-operative sleep studies in responders compared with non-responders to HGNS therapy.

A prior study assessed the relationship of various NED patterns with the level of upper airway collapse determined by endoscopic visualization as well as pharyngeal flow and pressure measurements. The NED minimal pattern of inspiratory flow limitation has been shown to indicate that the retroglossal region is a major site of UA collapse. The non-discontinuous pattern may reflect palatal or lateral pharyngeal wall collapse. While abrupt discontinuities in the NED pattern (NED discontinuous) has been associated with an epiglottic site of collapse [14]. While these associations may not rule out collapse in other segments of the UA, they suggest that the NED pattern may reflect the predominant site of airway collapse. Our study is novel in that it demonstrated the accuracy of ML in identifying NED patterns on sleep studies, a task that is impractical to perform manually. Furthermore, all signals analyzed by the ML model are those already in PSG or HST recording montages, making it easy to implement clinically. All indices were calculated with TRT as the denominator, which eliminates differences in the calculation of indices between PSG and HST. Our data demonstrated that the percentage of inspiratory flow limited breaths demonstrating the NED minimal pattern was greater in HGNS responders. While this does not rule out collapse in other regions of the UA, it suggests that the retroglossal region is a predominant site of obstruction in responders. This finding is consistent with the major effect of HGNS which produces anterior movement of the tongue.

We recognize that the definition of “success” is evaluated typically in the binary sense, although HGNS “success” may be better characterized as a continuous spectrum. We also acknowledge that there may be a night to night variation in OSA severity and our post-therapy data is derived from a single night sleep study. There are several other limitations to our study including a small sample size which can limit ML performance and accuracy. Furthermore, our metrics defining therapeutic success were based on objective measures from PSG and HST and did not incorporate subjective outcomes such as improvement in excessive daytime sleepiness, functional status, or cardiovascular outcomes.

The present findings demonstrate the feasibility of our ML-based model to identify predominant NED patterns as well as their relevance to HGNS outcomes. Future studies are needed to prospectively evaluate the role of NED pattern determination, possibly as part of a multivariable endotypic and phenotypic model, to predict HGNS outcomes.

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 523 KB)