Mechano-acoustic sensing of physiological processes and body motions via a soft wireless device placed at the suprasternal notch.

Skin-mounted soft electronics that incorporate high-bandwidth triaxial accelerometers can capture broad classes of physiologically relevant information, including mechano-acoustic signatures of underlying body processes (such as those measured by a stethoscope) and precision kinematics of core-body motions. Here, we describe a wireless device designed to be conformally placed on the suprasternal notch for the continuous measurement of mechano-acoustic signals, from subtle vibrations of the skin at accelerations of around 10-3 m s-2 to large motions of the entire body at about 10 m s-2, and at frequencies up to around 800 Hz. Because the measurements are a complex superposition of signals that arise from locomotion, body orientation, swallowing, respiration, cardiac activity, vocal-fold vibrations and other sources, we exploited frequency-domain analysis and machine learning to obtain-from human subjects during natural daily activities and exercise-real-time recordings of heart rate, respiration rate, energy intensity and other essential vital signs, as well as talking time and cadence, swallow counts and patterns, and other unconventional biomarkers. We also used the device in sleep laboratories and validated the measurements using polysomnography.

Natural processes of the human body yield a multitude of mechano-acoustic (MA) signals, many of which strongly attenuate at the skin-air interface[1-5]. Motions with amplitudes and frequencies ranging from subtle vibrations to full-body kinematics contain diverse and important physiological health information. Examples include vocal fold vibrations (~100 Hz), cardiac activity (~10 Hz), gait and locomotion (~1 Hz), respiration (~0.1 Hz) and body orientation (~0 Hz). Digital stethoscopes and inertial measurement units (IMU) represent clinical-grade tools that quantitatively capture some of these and other MA data[6-9]. The former acquires signals confined to a frequency range of 20 to 1900 Hz, non-continuously and episodically[8].

Traditional IMUs can be used continuously but are most effective at low frequencies (0–100 Hz) due mainly to their high inertial mass and loose coupling to the body[9,10]. Such types of sensors cannot capture both mechanical and acoustic aspects of MA signals simultaneously, with high fidelity. Compact, skin-mounted accelerometers and/or gyroscopes based on microelectromechanical systems (MEMS) offer important capabilities in this context, with demonstrated examples in capturing signatures of cardiac mechanics such as seismocardiograms or ballistocardiograms[11-14], respiratory rate and sounds[15-21], sounds of swallowing[22-25], vocals[26,27], changes in body position and motion[28-30] and others[31]. Certain of these measurements have direct relevance to medical applications, particularly when implemented in devices that monitor multiple physiological signals[1,3,32].

Conventional wearable devices for such purposes couple to the body in forms of straps, adhesive patches or bands, where the chest or the wrist serves as the measurement interface[33-35]. Other body locations can be considered, but with significant practical limitations that follow from the rigid, planar form factors of the devices[26,36,37]. For example, microphones strapped to the neck can capture acoustic signals associated with speech and throat sounds for detection of dietary behavior and respiratory physiology[38]. Accelerometers affixed by wax to the skin of the neck can record a range of body processes for tracking patterns of sleep[36]. These kinds of systems cannot, however, support persistent, comfortable interfaces to the skin during natural daily activities. Soft, lightweight skin-compatible wireless devices with the ability to mount on unusual parts of the body can continuously track a full spectrum of mechanical and acoustic signatures of body processes. They have implication in monitoring health status and social interactions throughout the day, quantifying sleep behaviors, measuring athletic performance, guiding rehabilitation protocols and many others.

Recent advances in soft electronics[1,39-43] serve as the foundations for skin-compliant, lightweight devices that incorporate accelerometers based on MEMS technologies[1]. Here, an elastomer encapsulates and supports a collection of functional components interconnected with serpentine conductive traces that maximizes the measurement sensitivity by mechanically decoupling the sensing element from the supporting electronics[1]. The resulting ‘epidermal’ MA sensors are highly responsive to movements and vibratory processes of the body, with the ability to capture high-quality signals across frequencies from 0 Hz to the audible band, with minimal interference from ambient noise[1].

The advances presented here build on these initial findings through (1) the design of wireless systems optimized for a comfortable skin-interface and high precision, high bandwidth MA measurements, powered with small-scale rechargeable batteries, (2) the use of the suprasternal notch (SN) as a unique anatomical mounting location that offers a rich blend of MA information related to diverse classes of physiological processes and core body motions, (3) the development of data analysis techniques for extracting quantitative physiological insights from the resulting multi-modal data, (4) the combined demonstration of these unusual mechanics of the device, unique mounting locations and advanced analytic approaches in continuous or semi-continuous monitoring during routine daily activities and physical exercise and (5) the clinical validation of results captured in sleep laboratories through quantitative comparisons to recordings obtained by gold-standard polysomnography systems.

Results

Engineering mechanics of the device.

The thin, soft form factors of the systems introduced here allow skin-interfaced measurements of MA signals continuously and wirelessly at nearly any location across the body, including sensitive regions such as the SN. High data fidelity and comfortable, non-irritating interfaces are key features. Figs. 1a and b outline the overall device layout, with images that demonstrate its ability to deform naturally with movements of the neck when mounted on the SN. The design incorporates deformable, non-coplanar serpentine interconnects, a strain isolation layer at the base, a soft encapsulation overlayer and a hollow air-pocket configuration. Together, these features provide low modulus, elastic mechanics despite the incorporation of conventional rigid electronic components and flexible printed circuit board technologies with layouts that are compatible with high volume manufacturing.

Fig. 1 |

Images, schematic illustrations, functional flow charts and mechanical modeling results for a wireless, skin-interfaced mechano-acoustic (MA) measurement technology designed for mounting on the suprasternal notch (SN).

a, Images that demonstrate soft device mechanics during movements of the neck while interfaced to the SN. b, Exploded schematic illustration of the active components, interconnect schemes and enclosure architectures. c, Block diagram of the system operation (Note S1). d, Finite element modeling of the mechanics during uniaxial tensile and twisting deformations. e, Images of a device in undeformed (top), stretched (middle) and twisted (bottom) configurations.

Fig. 1b presents, more specifically, the overall structure of the system. The device consists of a flexible printed circuit board (fPCB) based on a 25 μm thick middle polyimide (PI) support layer with patterned traces of 12 μm thick rolled, annealed copper (Cu) on the top and bottom surfaces (AP7164R, DuPont), each encapsulated with an insulating layer of PI (25 μm, FR1510, DuPont)[44]. The main electronic sub-systems include (1) a three-axis digital accelerometer (BMI160, Bosch) for measuring motions with a sampling frequency and resolution of 1600 Hz and 16 bits, respectively, a broad bandwidth response (0–1600 Hz) and a sufficient dynamic range (+/− 2 g) (g is the gravitational acceleration, 9.8 m/s2), (2) a microcontroller (nRF52832, Nordic Semiconductor) for acquiring data from the accelerometer and communicating the results wirelessly via Bluetooth Low Energy (BLE) protocols and (3) a wireless inductive charging circuit to support a rechargeable 45 mAh lithium-polymer battery (Fig. 1c).

Because these sub-systems rely exclusively on rigid, planar off-the-shelf components, they must be carefully integrated in a manner that simultaneously offers soft, skin-compatible mechanics as well as effective mechanical coupling of the accelerometer to the body. The schemes used here exploit advanced versions of design concepts in stretchable electronics[43], adapted for use with the fPCB generally, and to its interconnects between the sub-systems specifically. As shown in Fig. 1b, serpentine-shaped interconnects mechanically and electrically join two rectangular regions of the fPCB (islands; 1 cm × 1 cm). One island supports the microcontroller and charging circuit. Here, the fPCB folds onto itself to minimize the area consumed by the rigid components (Fig. 1b). The other island includes the charging coil and connections to the battery. The coil includes two Cu traces (120 μm widths) in a rectangular spiral design (8 turns, 100 μm pitch) on both top and bottom sides of fPCB. This minimizes the overall size of the device while allowing larger electromagnetic flux to go through the coil as compared to a single layer coil with the same dimension. The top and bottom coils share the same polarity and dimensions (9.55 mm by 10.7 mm). Electroplating (Contac S4) creates a conductive path across the two sides of fPCB through holes located at the ends of the coils.

Small pieces of rigid printed circuit board material (Garolite G-10/FR4; 381μm thick, 22 GPa modulus, McMaster Carr 1331T37) support each of the two main islands to increase their bending stiffness by three orders of magnitude (49 μm thick, 4.8 GPa modulus). This design effectively eliminates bending in these regions, thereby enhancing the robustness of the solder bonding joints between the components and the fPCB. These islands comprise 30% of the overall area of the system, leaving ~70% for freely deformable regions including the serpentine-interconnect structures.

The accelerometer rests on a cantilever that extends from the component island by a thin, narrow region of the fPCB to allow effective coupling to the skin with minimal mechanical constraints from other parts of the system (dotted outline in Fig. 1b). To prevent entanglement, the interconnects consist of two layers of serpentine wires embedded in silicone gel (Silbione 4717 Gel A/B, Elkem, 0.4mm thick) with a low Young’s modulus (6 kPa) to minimize constraints on deformation of the serpentine wires (Fig. S1a). A pre-buckled, out-of-plane, arc-shaped geometry allows the serpentine interconnects to assume traction-free architectures that absorb tensile deformations in two distinct modes (Figs. S2 and 1dii) in a manner largely decoupled from the device’s bottom substrate. Finite element analysis (FEA) results highlight significant mechanical advantages of these non-coplanar interconnects compared to conventional planar serpentine layouts (Fig. S1,Note S2). At the optimized arc angle θ = 270° and with pre-buckling, the elastic stretchability, ε = (Ly-L*)/L*, where L* is initial length (Fig. 1di) and Ly is the length where the Cu layer reaches the limit of plastic yielding, improves to 43% from 14% stretchability with arc angle θ = 210o and without pre-buckling (Figs. 1dii and S1b). Additional FEA results indicate that the maximum effective strain in the Cu layer is significantly less than the yield strain (0.3%) under various mechanical loads (Figs. 1diii and S3). These results highlight the range of robust, low-modulus, elastic responses (Figs. S4andS5) necessary to accommodate realistic physiological motions with little constraint on the underlying skin (Fig. 1e). Fig. S5 also shows that the fPCB thickness has little effect on the overall device modulus and stiffness. In addition to the mechanical characterization of the complete device, Figs. S6–8 illustrates the effect of the packaging materials and device structure on the signal fidelity. Fig. S6 compares z-axis acceleration recordings across a frequency range of 0 to 800 Hz, measured by a complete wireless system and by a wired, isolated accelerometer. The ratio between the two measurements is close to unity across the measurement range, demonstrating that the effects of the packaging materials are negligible. As shown in Figs. S7,8, results from a simple simulation model confirms these findings.

Data and analysis approaches for measurements from the suprasternal notch.

Natural physiological processes generate diverse MA signals at the surface of the skin, from subtle vibrations on the order of (Fig. 2a) to large-scale motions with amplitudes of ~2 g, across a band of frequencies (0 to 800 Hz) that can be captured with the accelerometers used here. The SN represents a unique anatomical location for recording such signals, as a direct soft tissue MA interface to vital organs related to cardiovascular, respiratory, and digestive systems and their interconnections between the head and torso. Fig. 2a shows the typical sensitivity of the device (z-axis, axes defined in Fig. 1b) characterized when placed on a vertical slab of elastomer (4 mm thick, 60 kPa) and when interfaced to the SN (Fig. 1a) of a subject while sitting quietly. The sensor on the slab shows nearly uniform noise power density across the entire frequency range.

Fig. 2 |

Representative mechano-acoustic (MA) data in the form of accelerations measured along three orthogonal axes from a device mounted on the suprasternal notch (SN) of a healthy normal subject.

a, Power spectral analysis of data (z-axis acceleration) collected from a device vertically resting on an elastomer and interfaced to the SN of a subject sitting quietly. The power spectrum of the measurement from the SN shows high power below 100 Hz associated with various involuntary physiological events. b, 3-axes time series data simultaneously recorded over a 60 second interval as a subject engages in various activities that include sitting at rest, talking, drinking water, changing body orientation, walking and jumping. c, Sample time series data, spectrograms, and spectral information corresponding to cardiac activity, talking, swallowing, and walking. The frequency analysis uses a Hanning window with a width of 0.1 s moving in time steps of 0.02 s.

At the SN, proximity to the carotid artery results in vibratory signatures related to the pulsatile flow of blood. The periodic nature of these signals allows determination of heart rate and variability. The cross-correlation of pulses measured from the chest near the pulmonic area and the SN defines a time lag between these two signals (Fig. S9). The lag of ~13 ms is consistent with recordings of vibratory signatures from the carotid artery itself, as opposed to chest/body vibrations. The amplitudes yield information on the intensity of cardiac activity and, indirectly, stroke volume. Passage of air through the trachea and movements of the chest wall produce MA data related to swallowing, talking, breathing, coughing, sighing, snoring and other responses. Furthermore, the device simultaneously responds to chest wall and full-body movements, including orientation referenced to the gravity vector.

All such signals rise well above the noise floor of the measurement system. To demonstrate the collective capabilities, Fig. 2b presents representative three-axis acceleration data recorded from the SN of a healthy normal subject engaged in a sequential series of activities. The signals associated with each of these and other activities exhibit distinct features in time and frequency, thereby conveying a rich set of information related to body processes. The following analysis focuses, for simplicity, on accelerations measured along the direction normal to the surface of the skin (z-axis).

The first 10 seconds of the data feature signals that arise from respiration. Here, expansion and contraction of the chest wall induce periodic rotations of the device around the y-axis at the neck, along with some translational motions. The main effect is to change the magnitude of the projection of gravity along the z-and x-axes. Mounted on the SN, the device includes grounded points (e.g. trachea) that do not move with the chest wall. Thus, the angular range of rotations (~1.3°) from respiration is larger than that associated with chest wall movement alone (~0.5°). As the subject begins to hold one’s breath at the ~10 s mark, these periodic changes cease. Talking and swallowing generate high-frequency signals associated with acoustic vibrations and rapid motions, as shown in the data from 16–34 s. When the subject leans in different directions through the time period from 34–45 s, quasistatic 3-dimensional accelerations provide instantaneous measurements relative to the gravity vector indicative of body orientation. Walking (45–53 s) and jumping (53–60 s) induce large amplitude accelerations with significant projections along all axes.

Fig. 2c shows quantitative details of the high-frequency features (>10 Hz) of individual physiological events. Cardiac activities – both systole and diastole10– give rise to paired pulses with peak amplitudes of ~0.05 g, and power concentrated in a frequency band of 20–50 Hz. The speech signals feature high-quality harmonic structures with fundamental frequencies in the range of 85 to 255 Hz for typical adults[45,46]. Swallow events initiate with slow motion (~0.1 s) of the vocal folds and with larynx mechanics during the pharyngeal phase and end with a high-frequency ring-down associated with flow of water or food during the esophageal stage[6]. Large impact forces that span a broad frequency ranges up to ~100 Hz characterize walking motions.

The characteristic frequencies and temporal structures of these features do not depend critically on the subject (gender, body type, ethnicity and age) (Fig. S10) or the mounting location in the vicinity of the SN (Fig. S11), although their relative amplitudes can vary slightly. For instance, the magnitudes of talking/swallowing signals increase as the mounting location approaches the larynx and upper esophageal sphincter. The magnitudes of walking signals are invariant with position as might be expected. Biological features across different subjects, including the fundamental frequencies of speech[47], the thickness of the skin, and other aspects lead to differences in the signal, in ways that are expected and do not affect approaches in signal analysis and interpretation.

In practice, MA measurements consist of data streams that superpose physiological information from a multitude of sources. A processing flow that exploits the characteristic time-frequency features demonstrated in Fig. 2 enables separation of key events, each of relevance in medical and/or fitness monitoring, i.e. energy expenditure (EE), heart rate (HR), respiration rate (RR), swallow counts (SC), and talking time (TT) (Fig. 3a). Application of these strategies on the data in Fig. 2 demonstrates the scheme (Figs. 3b–xf). Fig. 3b summarizes the EE over 2 s, 50% overlapping moving averaging window as a sum of all-axis low-frequency (1–10 Hz) band-limited root-mean square (BLRMS) data[48]. The result allows classification of activity levels from low (sitting ~10−2g) to medium (walking~10−1 g) to high (jumping ~100 g) on a logarithmic scale. Analysis of HR begins with the application of a bandpass filter (f = 20 Hz, f = 50 Hz) to the z-axis acceleration data to suppress noise outside the frequency range of interest. Cardiac pulses correspond to local maxima greater than 0.005 g in the time series of these band-passed signals, ignoring intervals shorter than 0.33 s (~180 beats per minute, or BPM) and longer than 1.2 s (~50 BPM). Applying a 5 s, 50% overlapping moving window average to peak-to-peak intervals yields a running estimate of HR (Fig. 3c). The peak-detection algorithm, however, cannot operate reliably with motion artifacts that involve large impacts and associated temporal and spectral features that coincide with those due to comparatively subtle cardiac mechanics.

Fig. 3 |

Flow diagram of signal processing and corresponding results from representative mechano-acoustic (MA) data acquired from healthy normal subjects.

a, Block diagram of post-processing analytics for energy expenditure (EE), heart rate (HR), respiration rate (RR), swallow count (SC) and talking time (TT); blue arrow indicates a use of three-axis accelerometer data and black arrow feeds only z-axis data. EE: The window-averaged 1–10 Hz band-limited root-mean-square (BLRMS) sum of data from all three axes (AEE) indicates the intensity of activities. HR: Detection of heartbeat peaks relies on identification of local maxima (NHR) in 20–50 Hz band-passed waveforms (ZHR). RR: Zero-crossing nodes (N0) of the decoupled, 0.1–1 Hz band-passed chest-wall motion (Z′′) from three-axis measurements serve as the basis for RR estimation (Note S3). TT: The talking signals feature pronounced harmonics of fundamental frequencies in the spectrogram analysis (P(f)). SC: The broadband swallow-like events (NMatch) correspond to occurrences of peaks (Ns) in both low-passed and high-passed signals (S1 0.1–5 Hz; S2, >100 Hz). The algorithm outputs swallow events Nsc that do not overlap with talking or activity periods. b-f, Application of the signal processing flow to the representative MA data (Fig. 2) for EE (b), HR (c), RR (d), TT (e), and SC (f) analysis.

Measurements of RR are particularly sensitive to ambulatory signals due to overlaps of these two types of signals in the frequency domain (0.1 – 1 Hz). Traditional methods rely on three-axis accelerometers mounted on the chest or the abdomen and allow determination of RR only during periods that exclude effects of locomotion[15,16,49]. Simple/weighted sum methods (e.g. Principal Component Analysis) can make use of the multi-axis information[17]. In the approach reported here, a noise subtraction algorithm exploits time-synchronized three-axis acceleration measurements to extract respiration signals at all activity levels. The detection mechanism relies critically on the SN mounting location and orientation (Figs 1.a,b), where the z-axis and x-axis measurements are both sensitive to chest-wall motion, while the y-axis acceleration is mainly associated with core-body motions. A wavelet transform projection yields common mode z′ values between z-and x-axis measurements. The differential mode z′′ between z′ and y-axis measurements decouple motions from respiration. In each pair of projection, we retain wavelet transform coefficients with coherence larger than 0.8 (Note S4).

Fig. 3d compares the band-passed (f = 0.1 Hz, f = 1 Hz) z-axis measurement with z′′. As opposed to traditional time-frequency analysis approaches for RR[50,51], a search for zero-crossing nodes of z′′ determines the average time-span of inspiration-expiration cycles in one minute, as the basis for estimating RR as breaths-per-minute (BPM) (Fig. 3d). The direct counting method accounts for the time non-stationary nature of respiration during physical activities. An adaptive threshold of 10% of the standard deviation of the data helps reduce miscounting associated with small-amplitude and fast ripple features on top of the overall 1-min respiration pattern (Fig. 3d, Note S4).

Signals that arise from speech involve the prominent presence of a second harmonic of the fundamental frequency in the expected frequency range and magnitude (Fig. 3e). The talking time detected in this manner appears as shaded regions in Fig. 3e. Using this device to determine TT has distinct advantages over methods relying on microphones due to its insensitivity to airborne acoustics. Tests in controlled, acoustically isolated rooms show that external sounds at 100 Hz and 90 dB(Z), appear as signals with amplitudes of only (Fig. S12) while speech at a similar frequency and at 65 dB(Z) on microphone (near the audible threshold) appears on the device with maximum amplitude of (Fig. S13). The effects of ambient sounds can, therefore, be neglected entirely in almost all practical scenarios. Swallowing events feature both low-frequency mechanical motions (0.1–5 Hz) and high-frequency acoustic components (100–800 Hz). Based on this observation, the algorithm for SC detects swallow events as high-frequency and low-frequency peaks that occur, coincidentally, within a 2 s time window (Fig. 3f). For the purpose of this algorithm, swallow events are only considered if they are separated by more than 0.2 s from talking events and by more than 0.5 s from active periods (EE > 0.05 g).

Human subject evaluations in practical scenarios.

Field studies in physical exercise and in dining demonstrate these data acquisition and processing schemes. The first involves cycling and resting on a stationary bike, with HR between 50 to 180 BPM. A Polar® hand-grip monitor yields reference HR values every 5 s (Fig. 4a). In a separate set of experiments, subjects manually count the number of breathing cycles per five minutes during normal sitting, walking and running activities as a reference for RR. For dining, each subject talks and swallows for five minutes according to a prescribed protocol. Here, periodicity (with a time scale of ~10 s, significantly longer than the time scale of event detection of ~ 0.1 s) in the activities facilitates tracking of numbers of events. In each minute of the nth test (n = 0, …,5), the subject talks for n × 10s, then swallows at (n + k) × 10 s, k = 1,…, 6 – n. During data acquisition, each subject marks the start and end of the talking period as well as swallowing instances using logging buttons on an app that runs on the smartphone used for data acquisition (Fig. 1d). The data set includes five subjects for each scenario (Table S1). For the cycling test, each subject cycle for 5 min. Applying a 5 s, zero overlapping moving window generates a total of 301 samples for comparison. RR experiments yield 56 samples across different activity levels, wherein each sample is an average RR over one 5-min test. A total of 26 dining tests generate average values for TT and SC over each 5-min test.

Fig. 4 |

Results from mechano-acoustic (MA) data recorded at the suprasternal notch (SN) in field studies with comparisons to reference measurements.

a, HR measurements during a 5-minute interval during exercise to increase the HR using MA signals and a Polar® hand-grip monitor. The cardiac amplitude, measured as the peak acceleration amplitude, exhibits a correlation with the HR measurement. b, RR measurements during a 2-minute interval using MA data and manual counting. The subject counts peak-to-peak intervals but the algorithm counts zero-to-zero intervals, thereby leading to a difference that appears as a time lag. c, A sample 1-min dining-scheme experiment comparing the reference labeling of events (cross) with the MA device detection (dot). For reference labels, label 1 and 2 mark the start and end of talking, while label 3 marks the occurrence of swallowing. d, The Bland Altman analysis for HR, RR, TT and SC. The solid and dashed lines represent mean difference and standard deviation × 1.96, respectively. Different colours represent the five different healthy normal subjects.

Figs. 4a–c shows the example results of MA measurements versus reference data. In the exercising scheme, the MA HR follows the reference HR from 100 to 160 BPM over a time period of 5 min under cycling activity, as in Fig. 4a. In addition to HR, the MA device captures the amplitude of the cardiac activity, which exhibits an approximate linear correlation to HR. In Fig. 4b, the MA RR follows the manually counted RR for this 2-min segment during walking. The mean RR values in both cases, marked by the dashed lines, closely match. Fig. 4c is a 1-min demonstration of the dining scheme. The MA talking and swallowing events agree with the label reference.

Figs. 4d shows Bland-Altman plots for HR, RR, TT, and SC. The solid and dashed lines mark the mean and 1.96 times the standard deviation of the difference between MA measurements and reference values, respectively. HR has a mean difference of 2.8 BPM and a standard deviation ARMS of 6.5 BPM. This limit of agreement is comparable to that observed with some of the best commercial devices for HR monitoring during physical activity (A= 4–29 BPM). RR has a mean difference of 0.3 BPM and a standard deviation of 2.5 BPM. As with HR, this difference is comparable to that of conventional monitors of RR (A = 2–3 BPM). A categorized statistical analysis indicates that the standard deviation error increases slightly with intensity of the activity (Fig. S14). TT has a mean difference of −2.0 s/min and a standard deviation of 2.2 s/min. SC has a mean difference of −0.7 counts/5 min and a standard deviation of 2.8 counts/5 min.

Sleep studies.

Use of MA devices to quantify patterns of sleep represents a potential application in advanced clinical diagnostics. Fig. 5a shows a subject equipped with an MA device on the SN and with a complete suite of conventional polysomnography (PSG) sensors for gold-standard reference measurements. A sleep lab technician performs visual observations alongside throughout the study to record changes in body orientation. In addition to HR and RR detection, MA device also monitors quasi-static body orientation continuously, following calculations for a rotation matrix that transforms gravity measurements in the canonical frame = [0,0,−g0] to the device frame ′ = [a, a, a], i.e. g′ = (Note S5).

Fig. 5 |

Application of mechano-acoustic (MA) sensing from the suprasternal notch (SN) in clinical sleep studies.

a, Image of the MA device on the SN (red box) along with a gold-standard polysomnography (PSG) sensor ensemble, including devices for recording Electrocardiograms (ECG), Electroencephalograms (EEG), and Electrooculograms (EOG) and for Pressure Transducer Airflow (PTAF) measurements, along with an Abdomen Strain Gauge, Thorax Strain Gauge and Thermistor. b, Avatar representation of a subject with the associated device frame and canonical frame. c, Body orientation calibration test. The arrows indicate the position of the nose. d, Comparisons of heart rate determined with the MA sensor and with the ECG recordings during sleep. e, Comparisons of the respiration rate determined with the MA sensor and with the nasal Pressure Transducer Airflow (PTAF) recordings during sleep. f, Comparisons of the body orientation determined with the MA sensor and by visual inspection by a sleep technician. g, Inference of sleep stages based on multi-band z-axis signal power of MA measurements in comparison to clinically determined sleep stages

Fig. 5b shows an avatar representation of the subject in a left recumbent position, along with the corresponding device and global frames of reference. Rotation matrix to the Euler angles conversion around the body-fixed (intrinsic) axis in z-y-x sequence (MATLAB™ function “rotm2eul”) makes intuitive sense of the rotation operation. The rotation angle θ around the longitudinal x-axis characterizes the major body orientation of interest during sleep, where we define zero degree as supine and the positive sense as turning right (left-hand rule). Fig. 5c presents a calibration test on rotation angles, in which a subject roll into a series of body orientations. In addition to the supine, prone, left recumbent, and right recumbent positions, the MA signal at the SN reveals additional information associated with the relative rotation of the head against the torso (Figs.1a and 5c). Figs. 5d–f compare MA devices vs HR derived from electrocardiography (EKG) (1-min window, 50% overlap), RR derived from pressure transducer air flow (PTAF) (2-min window, 75% overlap) and sleep technician inspection measurements of body orientation (0.1-s window, 0% overlap) respectively throughout a ~7-h sleep study on a healthy male (Age 26, Asian/Korean) subject. Bland-Altman analysis for HR and RR gives a standard deviation error of 3.9 BPM and 2.6 BPM, respectively (Fig. S15).

As a routine part of clinical care, segments of sleep are classified into four different stages, excluding the wake stage. The duration and frequency of each stage determines the sleep quality. The four stages are Rapid Eye Movement (REM), and non-REM Stages N1, N2, and N3. REM sleep comprises irregular and high RR and HR, along with the large amplitude electrooculogram (EOG) signals recorded using the PSG suite[52]. Non-REM sleep stages are defined by characteristic features on electroencephalogram (EEG), and although the MA device does not record EEG or EOG data, the results in Fig. 5g suggest some capabilities in quantifying sleep stages by applying machine learning algorithms on MA data obtained during stages determined manually by a clinical expert.

Here, a Hidden Markov Model (HMM) determines sleep stages as generative sequences characterized by a set of observable sequences with an underlying probabilistic dependence (Note S6). The algorithm exploits multi-band z-axis signal power on logarithmic scales as the observable clustering features, with a multi-band choice featuring the 0.1–0.8 Hz for respiration signals, sub-bands in the range 0.8–20 Hz for motions, as well as 20–80 Hz for cardiac signals. As shown in Fig. 5g, the optimized inferred hidden states capture the overall trend of gold-standard sleep scoring. The success rate for a simple binary wake/asleep classification is 82%. Fig. S16 includes a complete confusion matrix for multi-classes. Advanced analytics, a subject of on-going work, may enable further classification.

The wireless operation and the ability to track sleep with a single device facilitates use in home settings and in a way that does not alter natural patterns of sleep, as significant advantages over PSG systems. Insights obtained in this way can guide behaviors to optimize sleep quality. For example, Fig. 6a shows the cumulative distributive function (CDF) of HR and RR statistics, extracted from data collected from eight healthy normal subjects over five nights of sleep in the home, classified into four body orientations (supine: −45° < ϕ < 45°, left: − 135° < ϕ < −45°, right: 45° < ϕ < 135°, prone: ϕ > 135° or ϕ < −135°). The solid lines are mean CDF values for all subjects. The shaded regions mark the standard deviation variance between subjects (Fig. 6a). The inset compares the mean and standard deviation of HR and RR respectively generated from the averaged CDF for different body orientations. The results indicate that the HR is highest, on average, in Left-recumbent/Prone-approximate positions. The RR is higher in recumbent positions but lower in prone positions, with supine as a reference. According to previous studies[53-55], such changes may relate to reductions in venous flow, with resulting blood pressure reductions and/or increases in reflexive sympathetic nervous activity.

Fig. 6 |

Insights into sleep patterns determined by mechano-acoustic (MA) sensing from the suprasternal notch (SN).

a, Cumulative distribution function (CDF) of HR and RR in supine, prone, left, and right body orientations. The inset indicates mean (line) and standard deviations (bars) of the measurements between subjects. b, Sample data that illustrate the transition from snoring to quiet periods, plotted along with body orientation. c, Comparisons of different types of snoring mechanisms and their characteristics in acceleration and frequency.

As expected, the device is sensitive to snoring signals as well. Fig. 6b shows time series and time-frequency analysis of a representative ~50 s period of snoring followed by a transition to a ~50 s period of quiet sleep from a healthy female subject. The time series superimposes the band-stopped (f = 1 Hz, f = 60 Hz) z-axis acceleration measurement with the band-passed (f = 0.1 Hz, f = 1 Hz) respiration signal, to show that snoring occurs during exhalation. The snoring-to-quiet transition in the sample of Fig. 6b correlates with a slight left-ward head versus torso rotation of ~10°. The time-frequency analysis shows a clear presence of harmonics with fundamental frequency of ~50 Hz, which falls in the range of the natural frequencies of the soft palate and tongue structures[56,57]. The TT algorithm can be adapted to search for snoring time (ST) by shifting the fundamental frequency search range to the lower frequencies (). The auto-detected ST appears as shaded regions of the spectrogram plot. A tendency for snoring occurs during inspiration due to the Bernoulli effect[58]. As the throat starts to narrow, the velocity of flow may increase which drops the pressure in the airway behind the tongue and soft palate, thereby drawing the tissues together. The timing relative to the inspiration period may indicate the anatomical origin of snoring: tongue, or soft palate. Fig. 6c compares the snoring signal generated by these two different mechanisms. From 0 to 16 s, the subject artificially obstructs the airway by pushing the tongue to the back of the throat, then the subject artificially generates the snoring sound by vibrating the soft palate. The results clearly demonstrate that tongue snoring has inconsistent timing relative to the respiration cycle as compared to soft palate snoring. Moreover, the spectrogram illustrates that snoring generated by the tongue appears at a higher frequency range than that of the soft palate, as might be expected based on the mechanics of these processes.

Discussion

This paper summarizes a comprehensive set of concepts in hardware design and data analytics as the basis for an unusual class of soft, wireless MA sensor designed to operate on the SN, as a location that enables collection of a unique set of multimodal data related to a range of physiological processes. Specific demonstrations in various practical scenarios show capabilities for simultaneously monitoring HR, TT, RR, SC, EE through data algorithms that exploit time and frequency domain representations, with additional possibilities in tracking body orientation, steps and gait, coughing events, respiratory sounds and many others. By design, the technologies and methods described here align well with current manufacturing practices and commercial components, thereby offering a high technology readiness level. However, increasing the range and sensitivity of the accelerometers or designing systems that incorporate multiple different types of accelerometers has the potential to improve the functionality and to expand the types of addressable applications.

Unresolved challenges for continuous monitoring include those that involve motion artifacts in cardiac signal and respiratory rate detection. Semi-continuous monitoring can, however, be accomplished by opportunistically sensing during times where such artifacts are absent[14]. Incorporating multinodal MA detection approaches represents an alternative route for signal decomposition and precise measurements of cardiac mechanics[14,59]. The use of improved reference measurement methods in an ambulatory environment under intense and natural activities may aid in further development of data analytics approaches. Multimodal sensing enabled by the addition of strain gauges, for example, could enhance the accuracy in determining parameters, such as the respiration rate, that can be affected by low frequency noise and motion artifacts. Evaluating signals under an expanded variety of daily activities with additional participants, improving analysis algorithms for enhanced performance, and developing classification methods for a broader range of MA signals, are interesting directions for future research.

Monitoring of sleep appears to be an interesting potential area of application, with significant advantages compared to both polysomnography systems and conventional wrist mounted wearables. Others include uses in hospitals for monitoring surgical recovery, particularly in pediatric populations; in assisted living environments to track social engagements; in speech and physical therapy to treat aphasia and dysphagia. The straightforward addition of sensor modalities beyond accelerometry, such as pulse oximeters, could further expand the fields of use.

Methods

Fabrication of the Electronics.

Fabrication began with patterning a sheet of fPCB (12 μm thick top and bottom Cu layer, 25 μm thick middle PI layer, AP7164R, DuPont) into the necessary shapes using a ultraviolet (UV) laser cutter (LPKF U4). A CO2 laser cutter (VLS3.50) defined pieces of Flame Retardant 4 (FR-4) board (0.381 mm, McMaster Carr 1331T37) into geometries matched to half of the sizes of the two electronic islands (Fig. 1b). Folding the island regions of the fPCB around the FR-4 and bonding them in place using an adhesive (Loctite Tak Pak 444) yielded a dual-sided structure. Solder paste (Chip Quik TS391LT) joined the various electronic and sensor components onto the fPCB by reflow using a heat gun (AOYUE Int866).

Design of the Encapsulating Enclosure.

A thin (~0.3 mm) elastomeric membrane of silicone (modulus, 60 kPa; toughness, 500 MPa) molded into a preformed shape bonds on top of the planar silicone substrate around its perimeter to form a hollow enclosure that protects the system from the environment. This design provides a waterproof encapsulation structure that also allows free movement of the buckled serpentine interconnects (Fig. S17). The device shows no degradation in performance after complete immersion in phosphate buffered saline solution at 70°C for ten days. By comparison to a conventional, solid core encapsulation strategy, the hollow air-pocket layout reduces the equivalent tensile modulus and bending/twisting stiffnesses by ~ 3.4, ~ 1.6, and ~ 2.5 times, respectively (Figs. S4, S5d, e and f). In fact, the equivalent tensile modulus is only ~18 kPa, nearly seven times lower than that of human skin (120 kPa) [60] The bending and twisting stiffnesses (~17.2 Nmm2 and ~59.4 Nmm2, respectively) are ~2.5 and ~3.9 times lower than those of the skin with comparable thickness.

A layer of a soft silicone gel (Silbione 4717 Gel A/B, Elkem) at the base of the system, but within the encapsulating structure, provides a degree of mechanical isolation from the underlying skin, where stresses would otherwise accumulate at the locations of the islands during motions and/or deformations of the body and skin. Simulations indicate that the shear and normal interfacial stresses remain below the threshold for sensation (~20 kPa) for deformations of the skin to tensile strains of up to 30% when the thickness of the gel is 400 μm (Figs. S18 andS19). Without the strain isolation layer, the stresses reach 20 kPa at strains of only 10% (Fig. S20). This strain isolation layer does not, however, extend to the region of the device that supports the accelerometer, thereby ensuring its intimate mechanical coupling to the skin. The result enables high fidelity in MA measurements, as demonstrated by comparing data recorded using this device against those captured using the same accelerometer but in an isolated form connected by fine wires to external data acquisition electronics. Evaluations involve both systems on a vibrational stage or on a planar speaker, programmed to move in a sinusoidal manner with a frequency between 1 Hz to 100 Hz and between 100 Hz to 800 Hz, respectively (Fig. S6). The measured responses are nearly identical in the frequency range of interest (0–800 Hz) except for a maximum of ~13% drop in the relative response around 91 Hz. Similar measurements show that the responses of the device are not significantly affected by stretching to tensile strains of 12% (Fig. S21). Fig. S7 summarizes an analytic model for the device on the skin (Note S3). For frequencies between 1 and 50 Hz (Fig. S8), the difference in the acceleration of skin with and without the device is less than 3%, due to the small mass (4.56 g) of the device.

Assembly of the Devices.

A 3-axis milling machine (Roland MDX 540) created aluminum molds in geometries defined by three-dimensional computer-aided design (CAD) drawings created using ProE Creo 3.0. Casting a silicone thermoset polymer into the gap formed by matching pairs of molds defined the capping membrane (Ecoflex, 00–30, Smooth-on) with well controlled thickness (300 μm). Curing occurred in an oven at 70° C for 15 minutes. Bonding this membrane to a planar silicone substrate film around the perimeter defined an air cavity that enclosed the electronics.

Silicone gel (Silbione RT Gel 4717 A/B, Bluestar Silicone, E = 5 kPa) served as a strain isolation layer at the base of the device. A manual screen-printing process delivered this material onto the substrate film in a pattern to match geometries of the islands. Heating on a hot plate at 100° C for 5 minutes cured the gel. Delivering the electronics in aligned fashion onto this gel and capping the entire structure with the membrane completed the fabrication.

Characterization of the Device Performance.

Characterization focused on fully integrated devices and isolated accelerometers connected to external data acquisition electronics by fine wires (36 Ga.). An acrylic silicone served as a bonding adhesive (3M 2477p) to the center of a speaker (JBL Go Portable 1) or to a vibration stage (3B Scientific) in both cases. The vibration stage and speaker were programmed to execute frequency sweeps from 1 to 100 Hz and from 100 to 800 Hz, respectively.

Data Collection.

The SN served as the mounting location in all cases. For the swallowing experiment, the duration of each session was 5 min. The subject talked and swallowed water for a prescribed number of times per minute throughout the session. The talking duration was 10-s per session, and the swallowing occurred every 10 s from the end of a previous talking session to the start of a subsequent one.

For the HR monitoring session, the subject rode a cycling machine with a HR sensor interfaced to electrodes on the handle. The session began when the HR reached over 90 BPM. The subject cycled to increase the HR by 10 BPM each min until reaching 170 BPM as shown in the Fig. 4a.

For the sleep study, in addition to the MA device, the subject wore a suite of PSG sensors, including a three channel system for electroencephalography (EEG) on the forehead, two channel leads for electrocardiography (EKG) on the chest, a pair of leads for electromyography (EMG) on the chin, a pair of channels for electrooculography (EOG) on the side of upper left eye and lower right eye, a pressure transducer air flow (PTAF) with thermistor in the nostrils, two strain gauge bands around the chest and abdomen.

For all the studies, the participant gave informed consent.

Data Analytics.

All analysis used MATLAB (R2018b) technical computing languages. All digital manipulation uses a 4th order Butterworth infinite impulse response (IIR) filter followed by a non-causal, zero phase filtering approach (MATLAB™ function “filtfilt”). Identification of sleep stages used Python 3.0, with sci-kit learn and hmm learn packages.

Voice Signal Detection Using the Harmonics.

Given that is a local maximum of power spectral density P(f) in the range of the human voice (< 160 Hz for male subjects, 150 Hz < < 400 Hz for female subjects) (Fig. 3e), the algorithm searches for a local maximum in the frequency range and identifies speech if the search matches the anticipated harmonics behavior within a tolerance distance Instances with are not considered.

Classifying Active and In-active States.

Identification of active versus in-active states uses a fixed threshold of 0.05 where are the characteristic mean and standard deviation of the first 30-s of EE measurement as the subject sits quietly with minor movements.

Mechanics of Swallow Event Detection.

This criterion allows for only one high-frequency peak to be paired with the temporally closest low-frequency peak. Evaluating the stochastic differential of the signal zeroes the quasi-static offset before the data pass through the low-pass filter. The algorithm then locates local maxima in the resulting time series of minimum peak prominences of 5 × 10−4 g, a minimum peak distance of 1s and a maximum peak width at half maximum of 0.5 s. High-passed peaks are detected with a minimum peak height of 0.024 g~5A, where A is the high-frequency (100–800 Hz) BLRMS of quiet-time signals.

Reporting summary.

Further information on research design is available in the Nature Research Reporting Summary linked to this article.