Differential cardiopulmonary monitoring system for artifact-canceled physiological tracking of athletes, workers, and COVID-19 patients.

Soft, skin-integrated electronic sensors can provide continuous measurements of diverse physiological parameters, with broad relevance to the future of human health care. Motion artifacts can, however, corrupt the recorded signals, particularly those associated with mechanical signatures of cardiopulmonary processes. Design strategies introduced here address this limitation through differential operation of a matched, time-synchronized pair of high-bandwidth accelerometers located on parts of the anatomy that exhibit strong spatial gradients in motion characteristics. When mounted at a location that spans the suprasternal notch and the sternal manubrium, these dual-sensing devices allow measurements of heart rate and sounds, respiratory activities, body temperature, body orientation, and activity level, along with swallowing, coughing, talking, and related processes, without sensitivity to ambient conditions during routine daily activities, vigorous exercises, intense manual labor, and even swimming. Deployments on patients with COVID-19 allow clinical-grade ambulatory monitoring of the key symptoms of the disease even during rehabilitation protocols.

INTRODUCTION

The emergence of wearable technologies capable of multimodal, clinical-grade monitoring of physiological health increases the demand for sensors, systems, and data analytics approaches that enable reliable, continuous operation during natural daily activities. By comparison to traditional devices that loosely couple to the wrist, skin-mounted technologies offer vastly superior measurement capabilities due to their persistent, intimate interfaces to the body (–). This mode of operation can support a range of clinically standard diagnostic assessments, such as those based on electrocardiography (, –), photoplethysmography (–), arterial tonometry (–), and others (–). A recent set of important capabilities follows from wide-bandwidth measurements of subtle motions and vibrations of the surface of the skin [i.e., mechano-acoustic (MA) responses] (–) that arise from activities of internal organs and accelerations due to global movements of the body (, , ). Skin-interfaced devices for such purposes use precision, high-bandwidth accelerometers based on microelectromechanical system technologies in layouts that optimize sensitivity to motions of the surface of the skin across a broad range of frequencies, from nearly zero to several thousand hertz. The resulting data reflect not only bulk motions of the body, as with conventional wearable devices, but also features from a broad range of body sounds, as with digital stethoscopes, but impervious to ambient sounds. Additional information appears in a range of frequencies between these limits. For example, when mounted on the neck or the chest, the recordings enable detailed assessments of cardiac activity from motions of the heart and from pulsatile flow of blood through near-surface arteries, of respiratory cycles from chest wall movements, of respiratory sounds from airflow through the lungs and trachea, of swallowing behaviors from laryngeal motions and actions of the esophagus, of vocalization patterns from vocal fold activation, and of movements and changes in orientation of the core body. Distinct features in the temporal and spectral characteristics of these processes yield insights into physical activity and health status via a rich range of conventional [e.g., heart rate (HR)] (, ) and unconventional (e.g., coughing frequency) metrics (, , –), in a seamless manner, without privacy concerns that would follow from use of microphones or other recording devices.

Through these mechanisms, a single device in a sealed, waterproof package that requires only mechanical coupling to the skin can produce a powerful breadth of health-related information. An important caveat is that the diverse range of MA signals contribute to a single stream of time series data in a temporally overlapping fashion. Advanced data filtering and analytics approaches can separate and quantify different characteristic events on the basis of unique temporal and spectral features, but they fail to operate reliably in many scenarios of practical interest. Particular challenges arise when different activities with similar spectral content occur simultaneously. These circumstances render digital signal processing approaches ineffective. For example, respiration rate cannot be determined accurately while running. Related types of motion artifacts are fundamental limitations to both consumer wearables mounted on the wrist and clinical-grade wired monitoring systems.

The work presented here overcomes this limitation through advanced concepts in system designs and optimized choices in anatomical mounting locations, at the hardware level without the need for complex and often ineffective digital signal processing strategies. The approach exploits a pair of time-synchronized, high-bandwidth accelerometers [inertial measurement units (IMUs)] at opposite ends of a skin-interfaced device that locates one of the IMUs at the suprasternal notch (SN) and the other at the sternal manubrium (SM). Differences in movements of the skin associated with cardiac and respiratory activity between these regions lead to differences in signals captured by these IMUs. By contrast, overall movements of the neck and the core of the body produce nearly identical responses. As a consequence, simple differential measurements cleanly eliminate common mode features, thereby separating signals associated with cardiopulmonary and related processes from those due to body movements. An additional benefit of this architecture is that temperature sensors integrated in these IMUs can be used in a similar differential manner to yield estimates of core body temperature, largely independent of the ambient. Here, careful choices in thermal aspects of the device layout, rather than intrinsic anatomical gradients, produce the necessary differential responses.

The following sections present (i) designs of automated devices that incorporate matched pairs of high-bandwidth IMUs with optimized soft mechanics for high measurement sensitivity and accurate time synchronization across the SN and SM; (ii) results of spatiotemporal mapping of movements of the skin at this region of the anatomy during cardiac and respiratory activity; (iii) examples of modeling and design approaches for exploiting these IMUs in dual temperature sensing of core body temperature, with minimal influence of the thermal ambient; (iv) demonstrations of continuous, differential measurements of temperature, HR, and respiratory rate (RR) across a range of vigorous activities and conditions, with benchmarking against the most accurate commercial sensors; and (v) illustrations of the use on patients recovering from COVID-19 infections to track key symptoms of the disease even during intense physical rehabilitation protocols.

RESULTS

Design and characterization

The platform exploits a thin, flexible printed circuit board (fPCB) in an open architecture, with an elastomeric encapsulation structure that completely seals the system to physically isolate the electronics from the environment and to facilitate sterilization for reuse. The layouts yield soft mechanical characteristics for comfortable mounting on the skin, even at sensitive regions of the body. A touch-free docking interrogator supports wireless charging and initiates data downloads in an automated fashion to eliminate user burden.

Figure 1A shows a device mounted on the base of the neck, positioned to span the SN and SM. This unique anatomical location allows for measurements of a rich range of biophysical information, from activity of the cardiopulmonary system and movements of the core body to a diverse collection of processes across the thoracic cavity, esophagus, pharynx, and oral cavity related to respiration, speech, swallowing, wheezing, coughing, and sneezing.

Fig. 1

Images, schematic illustrations, functional flow charts, and mechanical modeling results for a wireless, skin-interfaced device designed for dual MA measurements at the SN and the SM.

(A) Image of the device mounted on the base of the neck, positioned with one end at the SN and the other at the SM. (B) Exploded-view schematic illustration of the active components, interconnect schemes, and enclosure architectures. (C) Image of a device next to a U.S. quarter (diameter, 24.26 mm). (D) Images of the device during various mechanical deformations: a twisting angle of 90°(left), 45% uniaxial stretching (middle), and a bending angle of 180° (right). (E) Finite element modeling of the mechanics for the deformations in (D). The contour plots show the maximum principle strains in the metal layer of the serpentine interconnects for twisting (left), stretching (middle), and bending (right). (F) Block diagram of the system operation. A tablet provides an interface for operating the device, wirelessly downloading the data from the device, and transmitting these data to a cloud server through a cellular network. Processing on the cloud platform yields vital signals (HR, respiration, and body temperature) and other metrics of interest (cough count and physical activity). Photo credit: Hyoyoung Jeong, Northwestern University.

Figure 1B presents an exploded-view schematic illustration of the soft enclosure and the fPCB with passive/active chip-scale components. Top and bottom encapsulating films of a silicone elastomer (thickness, 0.3 mm; Silbione RTV 4420) mechanically isolate the active parts of the systems in a sealed enclosure that allows operation even when submerged in water or exposed to sweat. The design is also compatible with U.S. Centers for Disease Control and Prevention guidelines for cleaning and disinfecting using 70% alcohol solutions (fig. S1) (). The fPCB exploits a copper (12 μm)–polyimide (PI; 25 μm)–copper (12 μm) laminate (DuPont, AP7164R) patterned to define conductive traces with widths of 80 and 150 μm. The layout (fig. S2) includes separate islands for the circuit components (main body), each of the two IMUs (IMU1 and IMU2), and a wireless charging coil. Serpentine-shaped traces interconnect these islands to mechanically decouple the IMUs from one another, as necessary in precision, differential measurements of MA signals at the surface of the skin. Specifically, two pairs of narrow, filamentary serpentine structures electrically connect the IMUs to the main body in a manner that minimizes mechanical constraints. The fPCB also includes multiple zones to allow for static bending during an assembly process that folds the system into a compact configuration. The image in Fig. 1C shows the overall size relative to a U.S. quarter (diameter, 24.26 mm). The dimensions of the encapsulated device are 46 mm by 22 mm; its thickness is less than 9 mm, and its weight is less than 6.35 g.

Experimental studies (Fig. 1D) and finite element analysis (FEA) computations (Fig. 1E) confirm that the strains in the copper of the serpentine interconnects remain below fracture limits (ε = 1%) throughout the assembly process and during operation under different types of external loads: 90° torsional [the left frame in Fig. 1 (D and E)], 45% tensile [the middle frame in Fig. 1 (D and E)], and 180° bending [the right frame in Fig. 1 (D and E)] deformations. This soft, stretchable design can accommodate deformations of the skin at the SN without fatigue or fracture and with minimal irritation and discomfort. Stiffening layers beneath critical regions of the platform reduce the probability for bending/stretching-induced damage of the solder joints. The result is a mechanically robust platform that enables high sensitivity measurements of both subtle motions of the skin and full body kinematics across a broad range of frequencies. Descriptions of additional design features, including those related to thermal measurements, appear subsequently.

The block diagram in Fig. 1F summarizes the overall system operation. The three main components include the device, a tablet with a customized app as a user interface, and a cloud platform for data storage and analytics. The device uses a BLE SoC (Bluetooth Low Energy System on a Chip) (Nordic Semiconductor, nRF52840), a PMIC (power management integrated circuit) (Texas Instruments, BQ25120), a 4-gigabit NAND flash memory (Micron, MT29F4G01), and two identical IMUs, each with an embedded temperature sensing unit (STMicroelectronics, LSM6DSL). Wireless charging involves voltage and current protection as support for a 75-mA·hour lithium polymer battery. The user interface allows time-synchronized operation of up to 12 devices, simultaneously. Although not explored in the following experiments in this study, this feature supports monitoring of social interactions and/or capture of MA signals at multiple body locations (fig. S3). Figure S4 describes the state diagram of the system, to illustrate behaviors before and after configuration, followed by deployment on a subject. The diagram also shows operation during data collection, charging, and data transfer.

Full automation of the key operational steps minimizes user burden, of particular importance for use with patients with COVID-19, as described subsequently. The user simply mounts the device during use and places it on the wireless charging platform when removed. The sensor continuously stores data from both accelerometers onto the internal memory module when not on the charging platform; when on this platform, the device charges and simultaneously streams data to a user interface device via Bluetooth protocols. The user interface then passes data to a cloud hub for signal processing to extract various physiological information, including cough count, RR, HR, activity level, body orientation, and calibrated body temperature. The cloud hub is HIPAA (Health Insurance Portability and Accountability Act) compliant, and the interface application uses HTTPS transport layer security (TLS 1.2) with an algorithm for encryption/decryption for the application programming interface and a standard for in-storage encryption (AES-256).

Core body temperature estimation with dual temperature sensing

The simplest consequence of the dual-sensing architecture is in temperature measurements that approximate the temperature of the skin (Tskin), largely unperturbed by the ambient (Tamb), following schemes described previously in other contexts (, ). Here, sensors embedded in IMU1 and IMU2, in a configuration illustrated in Fig. 2A, yield temperatures with repeatability of 0.004°C every 5 s (adjustable up to a 52-Hz sampling rate). IMU1 rests directly adjacent to the skin, separated only by the thin bottom encapsulation layer (0.3-mm-thick silicone elastomer). A 6-mm-thick thermally insulating foam (polyurethane mixture) with a metallic film (12-μm-thick aluminized polyethylene) minimizes coupling to the environment via convection, conduction, and radiation. The temperature at IMU1 depends strongly on the core body temperature, modulated by the effective thermal properties of the tissues and the ambient conditions (). IMU2 resides on the outward-facing side of the device, with only the top encapsulation layer above, to maximize and coupling to the environment (Fig. 2, B and C). The multiple underlying layers, including the adhesive film, bottom encapsulation, fPCB, battery, and thermal insulating foam limit heat transfer from the skin to IMU2.

Fig. 2

Dual-sensing platform for differential temperature and MA sensing.

(A) Exploded-view and (B) cross-sectional schematic illustrations of the device. (C) Side view of a completed device next to a U.S. quarter. (D) Finite element results for the temperature distribution in the skin and outside the device for skin and ambient temperatures of 37° and 22°C, respectively, with a convection coefficient of 10 W m−2 K−1. Cross-sectional profile of temperature along the A-B axis (inset). (E) Temperature profile along the A-B cross section for different ambient temperatures and convection coefficients. (F) Differential temperature measured using the temperature sensors in IMU1 and IMU2. (D) to (F) correspond to the case of a core body temperature of 37°C. (G) Representative results determined as the subject moves through rooms at various ambient temperatures. Dual temperatures (first row), differential temperature (second row), and the calibrated and measured core body temperatures (third row). Photo credit: Hyoyoung Jeong, Northwestern University.

Transient heat transfer analysis associated with three-dimensional (3D) thermal conduction and natural convection quantifies these effects. The boundary conditions include a constant temperature at the bottom surface of the tissue layer (Tcore) and convective coupling to the ambient air at the free surfaces (Tamb). The parameters include the room temperature, Tamb = 18° to 24°C, and the convection coefficient, h = 5 to 30 W m−2 K−1. Figure 2D highlights temperature distributions across the skin and regions surrounding the device for Tcore = 37°C, Tamb = 22°C, and h = 10 W m−2 K−1. The temperature profile along the AB cross section shows results inside the device with Tcore = 37°C (Fig. 2E) and with Tcore = 38° to 40°C (fig. S5). Figure 2F summarizes the differences in temperature between the two IMUs (Tdiff) for Tcore = 37°C. As might be expected, reducing Tamb and/or increasing h increases Tdiff. As specific examples, for h = 5 W m−2 K−1, Tdiff = 1.72°C when Tamb = 24°C and Tdiff = 2.52°C when Tamb = 18°C. In addition, for h = 30 W m−2 K−1, Tdiff = 3.80°C when Tamb = 24°C, but Tdiff = 5.55°C when Tamb = 18°C. In the same way, fig. S6 shows temperature differences under the condition of Tcore = 38° to 40°C, with associated temperature distributions in fig. S7. The analysis also quantifies the effects of changes in the ambient temperature, the core body temperature, the convection coefficient, and other key parameters. Measurements of differential temperature together with subject-specific thermal models yield robust estimates of core body temperature.

A simple demonstration involves a subject wearing a device in an environment with an ambient temperature of 18.2°C, then moving between areas with temperatures of 21.3° and 19.5°C every 3 to 8 min, and lastly remaining in place as the ambient temperature rises from 19.5° to 24.2°C for 7 min. The results for temperatures recorded from IMU1 and IMU2 appear in the top graph in Fig. 2G. The middle graph shows the differential temperature. A subject-specific model converts these temperature measurements into estimates of core body temperature (third row in Fig. 2G), determined by eq. S17 in the “1-D analytical model for the thermal characteristics” section in Supplementary Materials and Methods, where Tamb is the ambient temperature inferred from IMU2, TIMU1 is the temperature from IMU1, TIMU2 is the temperature from IMU2, and is a quantity that depends on the thickness and heat transfer coefficients of the skin and the various material layers of the device. Details of the structures, values, equations, and the modeling approaches appear in fig. S8 (see the 1-D analytical model for the thermal characteristics section in Supplementary Materials and Methods). A thermocouple placed under the tongue (green curve in the bottom graph of Fig. 2G) yields reference values that approximate the core body temperature. Figure S9 compares the results to the core temperature estimated from measurements at IMU1 and IMU2. The differences remain less than ~0.5°C across ambient temperatures from 19.5° to 24.2°C. Figure S10 shows Bland-Altman plots of the data. Sensing with only IMU1 (red) yields a mean difference of 3.81°C and an SD of 0.22°C compared to the oral measurement; the dual-sensing approach (blue) yields a mean difference of 0.01°C and an SD of 0.18°C. A 1D heat transfer model for analytics and 3D FEA model of the temperature dynamics (fig. S11) can capture essential aspects of these demonstrations. The analytical and 3D FEA results agree well over the range of different ambient temperature scenarios with relevant heat convection coefficients and the core temperature (36.3°C), similar to the experimental results in Fig. 2G (fig. S11B).

Dual sensing from the SN and the SM

Dual temperature sensing relies critically on design choices that yield different levels of sensitivity to temperatures of the body and the ambient for IMU1 and IMU2. For dual MA sensing, differential responses arise mainly from spatial gradients in motions across the mounting location, specifically those from the SN, the location of IMU1, and from the SM, the location of IMU2 (2.5 cm below the IMU1). Spatiotemporal maps of motions of this region of the anatomy determined by 3D particle tracking velocimetry (3D-PTV) provide quantitative insights into differential motions associated with respiratory and cardiac activities at these two locations and adjacent regions. 3D-PTV relies on optical techniques to track the Lagrangian paths of fiducial marks on the skin, in 3D using stereoscopic imaging (, ), in a way that recapitulates the point-measurement modality of the IMUs. Here, 3D-PTV can capture the essence of dual sensing from the SN and SM by recording from four time-synchronized, high-speed cameras, each at a frame rate of 200 frames per second (fps) (fig. S12), and track motions across the neck, including regions of the SN and SM (Fig. 3A). Displacement and vector contour fields follow from interpolation of fiducials at each frame based on Delaunay triangulation (Fig. 3B). The results featured here correspond to representative velocities measured near the peak of a cardiac cycle, relative to those between cycles, for a subject at rest. The results reveal notable differences between motions at the SN and SM, as the basis for differential detection.

Fig. 3

Distributions of displacements across the neck and surrounding regions determined by 3D-PTV during natural respiratory and cardiac activities, with a focus on the SN and the SM.

(A) 3D vector and contour fields of displacements, superimposed on the neck image. (B) 3D view of (A). The color denotes the velocity along the z axis, w, during a cardiac cycle. (C) Displacement along the z axis, ΔZ, as a function of time at the SN and SM during a breath hold, highlighting cardiac activity. (D) Differential displacement between the SN and SM determined from the data in (C). (E) Color contour of ΔZ at the peak of a cardiac cycle highlighted by the blue arrow in (D). (F) ΔZ as a function of time at the SN and SM during breathing and slight body motions. (G) Differential displacement between the SM and SN determined from the data in (F). (H) Color contour of ΔZ at the peak of inhalation, highlighted by the blue arrow in (G). Photo credit: Jin-Tae Kim, Northwestern University.

Figure 3 (C to E) shows additional detail, corresponding to z-axis displacement profiles through several cardiac cycles during a breath hold after a brief period of exercise (20 push-ups). The peak displacements at the SN are ~50% larger than those at the SM (Fig. 3C). By contrast, the displacements associated with body motions are nearly identical (as expected, but not explicitly shown here), thereby allowing for efficient subtraction. The differential result appears in Fig. 3D. Figure 3E shows a color contour plot of z-axis displacements at the peak of the cardiac cycle highlighted by the arrow in Fig. 3D (fig. S12, C and D, and movie S1).

Similar considerations apply to differential dynamics associated with respiration. Figure 3 (F to H) and fig. S12 (E and F) summarize the displacement distributions for three cycles of breathing while slightly swinging back and forth along the z axis. Figure 3F shows motions at the SN and SM, where responses include contributions from body motions and respiration for each case. The differential result (Fig. 3G) largely isolates the respiratory signals (fig. S12, E and F). Note that the small periodic features in these data arise from cardiac activity. A color contour plot of z-axis displacements at peak inhalation further highlights the spatial gradients that enable differential detection (Fig. 3H and movie S2). Data captured with the devices show similar trends (fig. S13), and simple 1D analytical models (fig. S14; see the “Analytical modeling of differential accelerometry” section in Supplementary Materials and Methods) can capture essential aspects of these behaviors (fig. S15) that reveal a clear basis for differential detection at the SN and SM.

Differential MA sensing minimizes motion artifacts in respiratory and cardiac monitoring

As verified by 3D-PTV, cardiac and respiratory activities create motions that have different amplitudes at the SN and the SM. Similar amplitudes result from movements of the core body. By consequence, simple subtraction of MA signals measured at these two locations greatly improves the accuracy and reliability of measurements of respiratory and cardiac activity by eliminating large, common-mode features that result from body motions. Figure 4 summarizes results captured using a device platform that incorporates IMUs with capabilities in high-fidelity three-axial accelerometry.

Fig. 4

Representative data collected during various ambulatory motions and measurements of controlled RR and normal HR.

(A) The subject sat quietly for 7 min, walked for 14 min with resting intervals, ran for 8 min with resting intervals, and jumped for 7 min with resting intervals under controlled RRs (6 to 35 RPM). (B) Magnified views of walking and running signals from (A), highlighting baseline fluctuations associated with respiration. The far-right box in green outline is a further magnified view from the data in the middle frame, highlighting cardiac activities S1 and S2. (C) Single-accelerometer data (black dot) yield reliable values of RR while the subject sits still. During ambulatory motions, the single-accelerometer data yield unreliable values of RR. The differential signals (blue dots) yield accurate respiration rates, consistent with ground truth (green triangles). The red arrow indicates the time frame of (B). (D) Single-accelerometer data provide the HR reliably while the subject sits still. During ambulatory motions, the single-accelerometer data (black dot) yield unreliable values compared to those from differential signals (blue dot) and from ground truth (green triangle). Signals associated with tapping between transitions cause aberrant values. The red arrow indicates the time frame of (B).

The flow chart in fig. S16A illustrates the approach for calculating the RR [respirations per minute (RPM)] from the differential data. The algorithm selects and performs a weighted average of the five highest energy components (minimum threshold of 50% of the maximum energy) in the frequency spectrum across the range of interest for the RR (6 to 60 RPM) within 1-min time windows (fig. S16F). As in fig. S16 (B and C), the differential signal largely eliminates common-mode “noise” associated with walking. Other signal components such as those due to cardiac activity lie outside the frequency range associated with respiration and/or have power below the threshold (fig. S16, D and E).

The flow chart in fig. S17A highlights the corresponding algorithm for HR [beats per minute (BPM)]. As with the RR, the differential data remove features from walking, running, jumping, and related activities. Band-pass filtering of the frequency spectra for 1-min time windows with cutoff frequencies of 45 and 170 BPM eliminates low-frequency signals from slow body processes and high-frequency content from vocalization and related events (fig. S17, B and C). This frequency envelope captures essential features associated with the S1 peaks associated with cardiac sounds, equivalent to those observed in seismocardiograms (SCGs) (fig. S17D). The frequency with the maximum energy and those with at least 80% of the maximum energy serve (fig. S17E) as the basis for a weighted average to determine the HR.

Figure 4 (B to D) highlights results obtained during sitting, walking, running, and jumping. The first instance involves resting in a chair with a controlled RR of 6, 10, 12, 15, 20, 30, and 35 RPM (0 to 7 min). The subject intentionally controls the exhale/inhale (1:1 ratio) time with a timer while moving. Next, the subject walks (8 to 21 min, 90 steps/min with 50-cm average stride lengths), runs (22 to 29 min, 180 steps/min with 85-cm average stride lengths), and jumps (31 to 36 min, vertical jumps every 2 to 3 s at approximately 40-cm height), all under similar controlled RR. Walking and running generate repetitive sequences of high-amplitude, impulse signals that dominate the data from IMU1 (red) and IMU2 (black). The differential signals, by contrast, feature a clear, periodic response associated with respiration (15 BPM; blue), as shown in the left frame in Fig. 4B (purple dashed region in Fig. 4A) and the middle frame in Fig. 4B (yellow shaded region in Fig. 4A). This differential signal also contains information on cardiac activity, as prominent S1 and S2 peaks of an SCG (1.5 s; green dashed region in the middle frame in Fig. 4B).

Figure 4 (C and D) compares RR and HR results extracted on the basis of normal and differential approaches. In the absence of body motions (e.g., sitting), the values are similar [blue shaded region in Fig. 4 (C and D)]. During ambulatory motions (walking, running, and jumping), results from single-accelerometer data (black dot) are highly variable compared to those from differential data (blue dot) for both HR and RR (yellow, red, and green shaded region in Fig. 4 (C and D)]. Differential sensing yields accurate results not only for walking and running but also for jumping. Figure S18 shows Bland-Altman plots for RR [single sensor, fig. S18 (A and B); dual sensor, fig. S18C]. The results with IMU1 (red) and IMU2 (black) exhibit a mean difference of −0.84 RPM (IMU1) and −0.90 RPM (IMU2) and an SD of 8.72 RPM (IMU1) and 10.49 RPM (IMU2). The differential results (blue) show a mean difference of 0.27 RPM and SD of 1.93 RPM. Likewise, for HR, results with IMU1 (red) and IMU2 (black) show a mean difference of −2.23 BPM (IMU1) and −4.12 BPM (IMU2) and an SD of 13.92 BPM (IMU1) and 13.18 (IMU2), and those with differential data (blue) show a mean difference of 0.01 BPM and an SD of 2.71 BPM (fig. S19). When comparing single- and dual-sensing performance based on the SD of the extracted RR and HR, the differential signal from dual sensing shows an improvement of 77 and 79% over RR and HR from single-sensing data, respectively.

Examples during vigorous activities in sports

Athletic competition, fitness training, manual labor, and related activities create daunting challenges for accurate measurements of RR and HR because of fast, dynamic, and highly variable large-amplitude accelerations of the body. The dual-sensor platform offers powerful capabilities in these and other contexts. Figure 5 highlights examples in cycling, playing basketball, and swimming. For exercise on a stationary bike (24 min; Fig. 5, A and B, and figs. S20 and S21), both the single-sensing (IMU1) and dual-sensing (differential) data yield HR results that match those obtained with a reference device (General Electronics, Dash3000). The RR values are also similar because of the limited effects of motion artifacts in this scenario. By contrast, for subjects playing basketball (11 min), differential sensing uniquely provides reliable measurements of HR and RR, as might be expected on the basis of controlled studies described previously (Fig. 5, C and D). Further benefits appear during extreme motions (3.5 to 5 min in Fig. 5D). The water-tight encapsulation and internal nonvolatile memory allows use in aquatic sports, as illustrated during swimming [5 min in Fig. 5 (E and F)]. When motion artifacts and the target signal are in a similar frequency range, the differential measurement approach is particularly valuable. During swimming, RR calculations that use the signal from a single IMU are dominated by responses associated with swimming strokes. Our algorithm processes the results as outliers because of the large amplitudes of these accelerations, which are inconsistent with respiration (Fig. 5F). The differential signal from the dual sensor greatly minimizes signals associated with swimming strokes, thereby yielding clear features associated with cycles of exhalation and inhalation and enabling calculations of the RR. The differential signal yields accurate respiratory activity, although the patterns of breathing and swimming occur in the same frequency range, as shown in Fig. 5 (G and H) and movie S3. A similar demonstration, highlighted in fig. S22, involves push-ups performed with controlled breathing out of phase with the push-up cycle, such that the RR signal cannot be distinguished from the periodic body motions using data from either IMU1 or IMU2. The differential signal, by contrast, shows clear features associated with exhalation/inspiration, well matched to the periodicity and the amplitude of controlled breathing.

Fig. 5

Tracking of cardiopulmonary activity during intense physical activities.

(A) Image of the dual-sensing device at the SN/SM along with reference devices for SpO2 and electrocardiogram recording and thermocouples for oral and ambient temperature measurements while cycling. (B) Comparisons of RR and HR determined by the dual-sensing (blue square) and single-sensing (red circle) and reference devices (green triangle, for HR only) while cycling for 24 min. (C) Image of the dual-sensing device on the SN/SM while playing basketball. (D) Comparisons of RR and HR determined from the dual- and single-sensing data while playing basketball for 11 min. (E) Image of the dual-sensing device on the SN/SM while swimming. (F) Comparisons of RR and HR determined with the dual- and single-sensing data while swimming for 5 min. (G) Representative z-axis acceleration data acquired from the dual-sensing device during swimming. Accelerations measured from IMU1 (red), IMU2 (black), calculated differential signal (blue), and baseline of the differential signal (light blue). (H) Magnified data associated with the differential signal (blue) and its baseline (light blue) from the area highlighted by the green box (G). Photo credit: Hyoyoung Jeong and Yoonseok Park, Northwestern University.

Examples during vigorous activities in manual labor

Worker health represents another area of opportunity given the need to continuously monitor key cardiopulmonary parameters in hostile environments (). Demanding occupations that involve work in construction, mining, firefighting, and related areas could benefit from noninvasive, high-fidelity monitoring systems to detect fatigue, heat exhaustion, and performance in ways that are seamless and compatible with high motion artifacts and extreme ambient conditions in temperature, sounds, and other. Figure S22 highlights examples of manual labor including hammering nails, carrying boxes, and shoveling dirt. Data in fig. S23A show that the differential signal exhibits clear features of cardiac activity, otherwise hidden by the strong, impulsive features associated with hammering. Similarly, respiratory features can be easily extracted even during large and irregular signatures of body movements in these cases (fig. S23, B and C).

Clinical deployment for monitoring COVID-19 patients recover

An area of urgent interest is in digital monitoring of the key symptoms of patients with COVID-19 to track the progress of recovery and the response to therapies in the hospital and the home. In addition to RR, HR, body movements, and body temperature, measurements also capture the intensity and frequency of coughing, talking, and laughing events. Collectively, these factors are important for symptomatic evaluation of the disease () and for indirect assessments of aerosol production. Studies reported here involve a COVID-19–positive patient (49 years old; female; height, 170 cm; weight, 107 kg; type 2 diabetes mellitus, obesity, hypertension, and cerebrovascular accident in 2018) provided with the dual-sensing platform and instructions for use in recording over the course of 8 days, as shown in Fig. 6. The patient captured 171 hours of data from 5 days within this time frame, including periods of dry cough and shortness of breath with oxygen therapy during the recovery phase. Figure 6 (A and B) shows a decreasing trend in cumulative cough count, along with the RR during this same interval, where the blue dots represent 5-min averages and the orange line shows the data after processing with Savitzky-Golay smoothing. Figure 6C summarizes the HR, where the black dots and red line show similar averages and smoothed results, respectively. Figure 6D presents the activity level (green bar) calculated by integrating the spectral power across a frequency range from 1 to 10 Hz (, ) and the estimated core body temperature (red line). Daytime corresponds to the time interval between 6 a.m. and 6 p.m., while other times are considered night. Average body temperature recorded from the first day (37.5°C) compared to the eighth recovery day (37.0°C) shows a decrease of 0.5°C. This recovery period includes a regimented and intense set of physical rehabilitation protocols. The ability track vital signs and key symptoms throughout could provide actionable clinical information on recovery and patient readiness to return home. Widescale deployment of this technology could improve patient care, aid in managing the pandemic, and also enhance our understanding of the disease.

Fig. 6

Data collected from a COVID-19 patient in the form of cough count, RR, HR, activity level, and estimated core body temperature.

(A) Variation of cough frequency from the patient while recovering over a period of 8 days. The first set was measured from 1 to 7 p.m. on the first day. The second set was measured from 8 a.m. to 8 p.m. on the second day. The third set was measured from 1 to 9 p.m. on the fourth day. The fourth set was measured from 9 a.m. to 8 p.m. on the seventh day, and the fifth set was measured from 8 a.m. to 8 p.m. on the eighth day. The purple line shows the cumulative number of coughs. (B) Variation of respiration rate and results from Savitzky-Golay smoothing (orange line). (C) Variation of HR and results from Savitzky-Golay smoothing (red line). (D) Activity level (green bar) and estimated core body temperature (red) during day (yellow shaded region) and night (blue shaded region). a.u., arbitrary units.

DISCUSSION

This paper presents a low-profile, lightweight, flexible, and wireless sensor that intimately couples to the skin as a dual measurement interface to the SN and SM with modalities for differential sensing of temperature and MA signatures of body processes. The results allow for measurements of a broad range of physiological parameters and activity behaviors that overcome a fundamental challenge in nearly every existing monitoring system: motion artifacts. Specific examples reported here include tracking of cardiac activity, respiratory activity, respiratory sounds, body temperature, and overall activity across a range of controlled settings and natural activities in sports, manual labor, and clinical medicine. These technologies and underlying ideas have many other implications. Examples include rehabilitation for patients with aphasia and/or dysphagia, where measurements of vocal activity and swallowing are possible during daily life, outside hospitals or rehabilitation clinics, of particular relevance to stroke survivors and patients with chronic obstructive pulmonary disease. Capabilities in tracking these processes without privacy concerns associated with microphone recordings and in a manner that is independent of ambient sounds represent key features of the approach. The availability of multiaxial information, including three-axis acceleration measurements, three-axis gyroscope data, and three-axis magnetometer information, suggests additional opportunities for these same platforms. Examples include quantitative measurements of neck movements (fig. S24) for patients recovering from cervical spine surgery by using accurate vector data between IMU1 and IMU2, as well as full-body motion detection followed by full-body motion reconstruction for rehabilitation or early-stage atypical motion diagnosis for cerebral palsy.

MATERIALS AND METHODS

Encapsulating the electronics in a soft enclosure

Top and bottom molded layers of a low-modulus elastomer (Silbione 4420; each 300 μm thick) form a soft encapsulating structure for the electronics. The fabrication process involved placing the electronics onto the bottom layer and then casting a uniform overcoat of a liquid prepolymer to a silicone elastomer (Ecoflex 0030). Mounting the top molded layer with a spacer on each of the short sides of the mold and clamping the assembly together enclosed the system for thermal curing at 70°C in an oven for 20 min. Cooling to room temperature, removing the device, and eliminating excess elastomer from the perimeter using a die cutter completed the process.

Forming the thermal insulating foam

A three-axis milling machine (Roland MDX 540) created an aluminum mold with a concave shape. Casting a liquid precursor to a polyurethane foam material (mixing ratio of A to B is 2:3; FlexFoam-iT! III, Smooth-On, USA) on the mold after coating its surface with a releasing agent (Ease Release 200, Smooth-On, USA) and then pressing a flat aluminum plate on top side produced insulation foams upon curing on a hot plate at 100°C for 30 min. A reflective film (thermal blanket; Swiss Safe Products) attached to the flat bottom surface of the foam layer using a 5-μm-thick double-sided tape (No. 5600, Nitto Denko Co., Japan) further improved the insulating properties. The final step of the process involved a CO2 laser (Universal Laser System Inc.) to cut the perimeter of the material into the final geometry.

Modeling the mechanical characteristics

The commercial software ABAQUS (ABAQUS Analysis User’s Manual 2010, version 6.10) defined the strain ε in the metal layers of the system. The simulations allowed selection of design parameters to ensure that the strain in the copper (Cu) remains below the fracture limit (ε = 1%), to avoid mechanical failure during assembly of the device and during different types of deformations (stretching, bending, and twisting). The thin Cu and PI films were modeled by composite shell elements (S4R). The number of elements in the model was ~2 × 105, and the minimal element size was 1/8 of the width of the narrowest interconnects (100 μm). The mesh convergence of the simulation was guaranteed for all cases. The elastic modulus (E) and Poisson’s ratio (ν) of are ECu = 119 GPa and νCu = 0.34 for copper and EPI = 2.5 GPa and νPI = 0.34 for PI.

3D FEA modeling for the thermal characteristics

Transient heat transfer analysis determined the effects of thermal conduction and natural convection on the responses of the temperature sensors. The tissue and internal sensor components were modeled by hexahedron elements (DC3D8). The encapsulation layer was modeled using tetrahedron elements (DC3D4). The number of elements in the model was ~6 × 105, and mesh convergence of the simulation was ensured for all cases. The boundary conditions included a constant temperature (Tcore) at the bottom surface of the tissue layer and convection conditions with the ambient air (Tamb) at the free surfaces. The following parameters were used in the computations: room temperature Tamb = 18° to 24°C; convection coefficient h = (5 to 30) W m−2 K−1; thermal conductivity, heat capacity, and mass density of 0.3 W m−1 K−1, 1460 J kg−1 K−1, and 960 kg m−3 for the tissue; 0.21 W m−1 K−1, 1090 J kg−1 K−1, and 1420 kg m−3 for the thermoplastic chips; 0.343 W m−1 K−1, 1150 J kg−1 K−1, and 1850 kg m−3 for FR4; 0.03 W m−1 K−1, 1200 J kg−1 K−1, and 85 kg m−3 for the urethane foam; 0.21 W m−1 K−1, 2100 J kg−1 K−1, and 909 kg m−3 for PI; 0.2 W m−1 K−1, 1460 J kg−1 K−1, and 1070 kg m−3 for Ecoflex 00-30; and 0.15 W m−1 K−1, 1460 J kg−1 K−1, and 970 kg m−3 for Silbione 4420.

Measuring the displacement distributions by 3D-PTV

The experiments involved recordings from four synchronized high-speed area scan cameras (2048 × 1088 resolution; HT-2000M, Emergent) with 35-mm imaging lenses (F1.4 manual focus; Kowa) at the frame rate of 200 fps. The process focused on tracking of 300 fiducial points marked in a grid pattern across the neck covering the SN, the SM, and adjacent areas. The investigation volume was 10 cm by 8 cm by 10 cm illuminated by six arrays for 600 lumen light-emitting diode light bars. Preprocessing, calibration, 3D reconstruction, tracking, and postprocessing used customized 3D-PTV code. Image sequences were preprocessed by subtracting the background noise and enhancing the contrast. 3D calibration exploited the structure-from-motion technique from multiple views. After removing effects of lens distortion, intrinsic parameters of a single camera were estimated using the checkboard calibration method. Extrinsic parameters of all four cameras, including 3D translation and rotation matrices, were obtained by using a sparse set of points matched across the views. Once all camera parameters were estimated, a dense set of fiducial points across multiviews were detected in a subpixel level and reconstructed in 3D coordinate. 3D reconstructed fiducial points were tracked using the Hungarian algorithm and linked by performing a five-frame gap closing to produce long trajectories. Displacement, velocity, and Lagrangian acceleration were filtered and computed using fourth-order B splines. 3D displacement and vector contour fields were obtained by interpolating scattered fiducial points at each frame based on the Delaunay triangulation. Image sequences during cardiac activities (movie S1) were magnified using the Eulerian video magnification method (). More details on 3D-PTV can be found in (, ).

Procedures for dual-sensing temperature and motion measurements

A double-sided medical silicone adhesive (3M, 2477P) secured the sensors to the neck area (aligned IMU1 on the SN and IMU2 on the SM). The authors affirm that all subjects in the study provided written informed consent for study images to be published with faces blurred. All data in this study were captured using IMUs with sensitivity of 0.061 mg (gravitational acceleration), 1666-Hz sampling rates (adjustable up to 6664 Hz), and ± 2g acceleration measurement range (adjustable up to ±16g).

Protocols for human subject studies

The studies were approved by the Northwestern University Institutional Review Board, Chicago, IL, USA (STU00202449 and STU00212522) and were registered on ClinicalTrials.gov (NCT02865070 and NCT04393558). All study-related procedures were carried in accordance with the standards listed in the Declaration of Helsinki, 1964. For COVID-19–positive patients, double-sided medical silicone adhesive (3M, 2477P) secured the sensor to the neck area (aligned IMU1 on the SN and IMU2 on the SM) for more than 12 hours. For multiple days of use, medical-grade transparent film (Tegaderm, 3M) was applied between the skin and the double-sided adhesive to eliminate irritation from the adhesive. Clinical staff assisted the patient in placing the sensor. After each data measurement session, the device was sterilized with 70% isopropyl alcohol and left to dry at room temperature, and the sterilization process was repeated twice.

Classifying signal features by machine learning to extract coughing events

Figure S25 shows a flow chart of the algorithm. Training data included time series z-axis acceleration data with features associated with tapping, coughing, laughing, and throat clearing. Training of this classifying algorithm used 10 datasets from each class (subjects SP1 to SP4) (fig. S26). Feature extraction used peak detection, spectral information, and spectrograms. The first step identified events associated with tapping, coughing, laughing, and throat clearing using adapted thresholds according to the input signal levels evaluated across sliding windows with widths of 0.5 s. Each extracted event was then aligned to the center of corresponding time frames to maximize the energy of the signal for postprocessing (first row of fig. S25C) based on continuous wavelet transformations. Resulting images within a 0.12-s window formed the basis for further analysis and classification (third row of fig. S25C). Specifically, a binary tree architecture using a support vector machine (SVM) classified these extracted features into four activities (fig. S25D). First, in the SVM1 classifier, throat clearing activities were removed by negative values of the SVM1 hyperplane. Next, tapping activities were classified from SVM2 with a specific decision boundary (SVM2 result value: 2.5). Last, SVM3 separated the classes (coughing and laughing) with another decision boundary as described in fig. S25E.