| Literature DB >> 35884312 |
Maritza Bustos-López1, Nicandro Cruz-Ramírez1, Alejandro Guerra-Hernández1, Laura Nely Sánchez-Morales2, Nancy Aracely Cruz-Ramos3, Giner Alor-Hernández3.
Abstract
Appropriate teaching-learning strategies lead to student engagement during learning activities. Scientific progress and modern technology have made it possible to measure engagement in educational settings by reading and analyzing student physiological signals through sensors attached to wearables. This work is a review of current student engagement detection initiatives in the educational domain. The review highlights existing commercial and non-commercial wearables for student engagement monitoring and identifies key physiological signals involved in engagement detection. Our findings reveal that common physiological signals used to measure student engagement include heart rate, skin temperature, respiratory rate, oxygen saturation, blood pressure, and electrocardiogram (ECG) data. Similarly, stress and surprise are key features of student engagement.Entities:
Keywords: engagement detection; learning environments; physiological signals; sensors; wearables
Mesh:
Year: 2022 PMID: 35884312 PMCID: PMC9312492 DOI: 10.3390/bios12070509
Source DB: PubMed Journal: Biosensors (Basel) ISSN: 2079-6374
Figure 1Physiological Variables for Engagement Detection and Associated Body Parts.
Figure 2Distribution of Information Sources.
Figure 3PRISMA Flow Diagram of the Search Strategy.
Classification of Biosensors for Cognitive Engagement Detection.
| Type of Wearable Biosensor | Description | Wearable Biosensor Technologies | Wearable Device | Applications |
|---|---|---|---|---|
| Mechanical | Accelerometers and motion sensors require the integration of another wearable physiological monitoring device as well as some type of computer software interface equipped with specific algorithms for signal manipulation and analysis. | Accelerometer with ECG necklace | Leap Motion | Tracking gait |
| Physiological [ | Physiological sensors can be used for predicting obstructive sleep apnea and monitoring heart rate, oxygen saturation, heart rate variability, breathing rate, and oxygen saturation. Further, these sensors can measure stress levels and mental fatigue. | PPG ring sensor | Ring | Concentration monitoring |
| Biochemical [ | Biochemical sensors can be used for non-invasive sweat monitoring through epidermal tattoo potentiometric sodium sensors with wireless signal transduction. Further, they are used for one-point wireless ECG acquisition with flexible polydimethylsiloxane (PDMS) electrodes. | Epidermal tattoo potentiometric sodium sensor | Finger and limb motion detection |
Commercial Wearable Devices for Engagement Detection.
| Manufacturer | Model | Form Factor | Sensors Used | Parameters | Physiological Signs | API | FDA |
|---|---|---|---|---|---|---|---|
| Biovotion™ [ | Everion | Armband | HR sensor, PPG sensor | ST, SpO2, RR, HR, HRV, Sleep | Stress, relaxation | Everion device | Approved |
| Abbott [ | FreeStyle Libre | Semi-invasive | Continuous Glucose Monitoring (CGM) sensor | ST, BP | Distress | Ambrosia | Approved (2020) |
| Halo Sport [ | Halo Sport 2.0 | Headphones | Electro neurostimulator | Neuropriming | Halo Sport | Approved | |
| Scosche™ [ | Scosche Rhythm24 | Armband | HR optical sensor | PPG, HR, HRV, cadence, step tracking, burned calories, distance, speed | Stress, postures, surprise | ScoscheSDK24 Framework | Not Approved |
| Equivital™ [ | LifeMonitor | Chest belt | ECG biosensor, HR sensor, medical-grade thermometer, and tri-axis accelerometer | ST, SpO2, RR, HR, HRV, GSR | Stress, engagement | Equivital | Approved |
| Med/Wise [ | Gluco Wise ® | Clip (thumb, | CGM radio wave | Continuous glucose monitor (CGM) | No specified | Gluco Wise ® | - |
| Biobeat™ [ | Biobeat™ Chest patch | Chest patch, wrist monitor | PPG | ST, SpO2, RR, HR, HRV, BP, ECG | Stress, surprise | Biobeat | Approved (2019) |
| G-Tech Medical™ [ | G-Tech Medical™ | Chest patch | EMG | EMG | Surprise | G-Tech Medical | - |
| Health Care Originals™ [ | ADAMM-RSM | Chest patch | Acoustic sensor, HR sensor, | HR, ST, RR, cough | Stress, surprise | Health Care Originals | - |
| iRhythm™ [ | Zio patch | Chest patch | HR sensor | ECG | Surprise | iRhythm™ | Clear (2021) |
| Preventice™ [ | Bodyguardian Heart | Chest patch | Accelerometer, ECG sensor | ECG | Surprise | Preventice™ | Clear (2012) |
| VitalConnect™ [ | Vital Patch | Chest patch | Accelerometer, ECG sensor, | ECG, HR, HRV, RR, ST, body posture, activity, BP, SpO2 | Stress, surprise | VitalConnect™ website | Cleared |
| BioTelemetry™ [ | BioTelemetry™ | Chest patch | ECG sensor | ECG | Surprise | BioTelemetry™ | Cleared |
| Kenzen™ [ | Kenzen™ | Chest patch | HR biosensor | Sweat, HR, ST | Stress, surprise | Kenzen™ | No Approved |
| Theranica Nerivio Migra™ [ | Theranica Nerivio Migra™ | Chest patch | EMG sensor | EMG | Surprise | Theranica | Approved (2019) |
| Medtronic™ [ | Zephyr BioHarness | Chest Strap | CGM sensor | HR, HRV, RR, body posture, activity intensity, acceleration, accelerometry, ST, burned calories, speed, distance, elevation, BP, SpO2 | Stress, surprise | Zephyr: Developer and User Tools | Approved (2012) |
| Beddit™ [ | Beddit Sleep Monitor | Contactless in-bed sensor | PPG sensor | RR, HR, sleep measures | Stress, relaxation | Beddit™ | - |
| Beurer™ [ | Beurer SE80 | Contactless In-bed sensor | Respiratory rate sensor, HR sensor | RR, HR, sleep measures | Stress, relaxation | Beurer™ | - |
| Cosinuss™ [ | Cosinuss Two | Earbud | HR sensor, body temperature sensor, 3D accelerometer | HR, HRV, SpO2, activity | Stress | Cosinuss™ | - |
| Yono™ [ | Earbud | Earbud | Thermometer | ST | Stress | Yono™ | - |
| BioIntellisense™ [ | BioIntellisense Epidermal patch | Epidermal patch | HR sensor | ST, RR, HR, coughing, sneezing | Stress | BioIntellisense™ | Approved (2019) |
| Bose® [ | SoundSport ® Pulse | Wireless headphones | HR sensor | HR, PPG | Stress | Bose® Connect | - |
| VivaLNK™ [ | Fever Scout | Epidermal patch | ECG and HR sensors | ST | Stress, relaxation, surprise | VivaLINK | Approved (2017) |
| Vital Scout | Epidermal patch | ECG and HR sensors | HR, HRV, RR, activity, sleep, stress levels | Stress, relaxation | VivaLINK | Approved (2019) | |
| Spire Health™ [ | Spire Health Tag | Fitness Tracker | HR, ECG, and RR sensors | HR, RR, breathing pattern, activity | Stress, relaxation, surprise | Spire Health™ | Not Approved |
| Muse™ [ | Muse S | Headband | EEG sensor | EEG, PPG, SpO2, breathing pattern, sleep tracking | Relaxation, concentration, postures, surprise, frustration, interest, laugh | Muse Developers | - |
| Motiv™ [ | Motiv Ring | Ring | Accelerometer and PPG and HR sensor | PPG, HR | Stress, surprise | Motiv™ | - |
| Oura™ [ | Oura Ring | Ring | Body temperature sensor, optical, infrared sensors, 3D accelerometer and gyroscope sensor | PPG, HR, HRV, ST, RR, activity, sleep | Stress, surprise, relaxation | Oura Cloud API | - |
| Komodo Technologies™ [ | AIO smart sleeve | Sleeve | ECG sensor | ECG, HR, HRV, activity intensity, SpO2, step tracking, distance | Stress, postures, surprise, interest, laugh | AIO Sleeve App | No Approved |
| Kinsa™ [ | Kinsa | Smart thermometer | ST sensor | ST | Stress | Kinsa™ | Approved (2013) |
| Orpyx™ [ | Surro Gait Rx | Smartwatch, shoe insert, shoe pod | Pressure sensor | BP | Stress, surprise | Orpyx™ | - |
| Surro Sense Rx | Watch, shoe insert, | Pressure sensor | BP | Stress, surprise | Orpyx | Cleared | |
| Apple™ [ | Watch Series 3,4,5 | Smartwatch | Oximeter, electrical HR sensor, optical HR sensor, accelerometer, gyroscope sensor | Fitness and activity-tracking, ECG, PPG, HR, HRV, sleep quality, stress levels, RR | Stress, relaxation, postures, surprise, laugh, interest | Apple™ Developer | Approved |
| Empatica™ [ | Embrace 2 | Smartwatch | EDA sensor, peripheral temperature sensor, 3-axis accelerometer, gyroscope sensor | HR, HRV, EDA, ST, activity | Stress, engagement, laugh | Empatica™ for Developers | Approved (2018) |
| E4 | Bracelet | PPG sensor, 3-axis accelerometer, EDA sensor (GSR Sensor), infrared thermopile sensor | BVP, GSR, SC, HR, HRV | Stress, relaxation, arousal, excitement | Empatica™ for Developers | Not Approved | |
| Fitbit™ [ | Charge 4 | Smartwatch | 3-axis accelerometer, optical HR monitor, altimeter | PPG, HR, SpO2, activity, sleep | Stress, surprise, relaxation | Fitbit™ | Not Approved |
| Ionic | Smartwatch | 3-axis accelerometer | HR, SpO2, activity, sleep | Relaxation | Fitbit™ | Approved | |
| Versa 2 | Smartwatch | 3-axis accelerometer, optical HR monitor, altimeter, ambient light sensor, relative SpO2 sensor, built-in microphone | HR, guided breathing, SpO2, step tracking, distance, stress level, sleep | Stress, relaxation, postures, surprise | Fitbit™ | Approved | |
| Gyenno [ | Gyenno Spoon | Spoon | Accelerometer | No specified | Stress | Gyenno | - |
| Gl Logic [ | AbStats | Abdominal device | Vibration sensor, acoustic sensor | A telemetry monitor | GI Logic | - | |
| Garmin™ [ | Fenix 5 | Smartwatch | HR sensor | HR, SpO2, activity, sleep | Stress, relaxation | Garmin™ Connect Developer | - |
| Forerunner 945 | Smartwatch | HR sensor | HR, SpO2, RR, activity, sleep | Stress, relaxation | Garmin™ Connect Developer | - | |
| Venu | Smartwatch | Pulse oximeter, HR sensor | HR, SpO2, RR, activity, sleep | Stress, relaxation | Garmin™ Connect Developer | - | |
| Vivoactive 4 | Smartwatch | Pulse oximeter, HR sensor | HR, SpO2, RR, activity, sleep | Stress, relaxation | Garmin™ Connect Developer | - | |
| Vivomove 3 | Smartwatch | Pulse oximeter, HR sensor | Step tracking, sleep quality, HR, stress levels, body composition, SpO2, intensity minutes, details of physical activity, breathing frequency | Stress, relaxation, postures, surprise | Garmin™ Connect Developer | - | |
| Holter [ | Stat-On™ | Portable sensor | ECG sensor | HR, HRV | Stress | No specified | - |
| Honor™ [ | Honor Watch Magic 2 | Smartwatch | Accelerometer, gyroscope, magnetometer, optical HR sensor, ambient light measurement, and barometer | HR, stress levels, sleep quality, distance, speed, SpO2 | Stress, relaxation, surprise | Huawei™ Developers | - |
| Huawei™ [ | Huawei Watch fit | Smartwatch | 6-axis IMU sensor (accelerometer sensor, gyroscope sensor), Optical HR sensor, capacitive sensor | HR, SpO2, sleep quality, stress levels, step tracking, distance | Stress, relaxation, postures, surprise | Huawei™ Developers | - |
| Band 6 | Smart Watch | Accelerometer, three electrodes, ECG sensor, barometric altimeter | ECG, SpO2 | Stress, relaxation, postures, surprise | Huawei™ Developers | - | |
| Mobvoi™ [ | TicWatch Pro 2020 | Smartwatch | HR sensor | HR, step tracking | Stress, postures, surprise | Mobvoi™ Developers | - |
| LifeBeam [ | LifeBeam diy kit | Helmet | Optical sensor | HR, blood flow, and oxygen saturation | LifeBeam | - | |
| Kuaiwear Kuai [ | KUAI-Sport Headphones | Headphones | HR sensor and accelerometer | HR | Stress | No specified | - |
| Omron™ [ | Heart Guide | Smartwatch | Accelerometer, PPG HR and | PPG, HR, BP, ECG, step tracking, distance | Stress, postures, surprise, laugh, interest | OMRON API for Developers | Approved (2019) |
| OnePulse™ [ | OnePulse™ | Smartwatch | ECG sensor | HR, activity, sleep patterns | Stress, relaxation, surprise | Not specified | Approved |
| Samsung™ [ | Samsung™ Gear Sport | Smartwatch | Accelerometer, Gyro Sensor, Barometer, HR monitoring sensor | HR, step tracking, sleep quality | Stress, relaxation, postures, surprise | Samsung™ Developers | - |
| Samsung™ Galaxy Watch Active 2 | Smartwatch | Accelerometer, barometer, gyroscope sensor, HR sensor | HR, sleep quality, stress levels, BP, distance, step tracking | Stress, relaxation, postures, surprise | Samsung™ Developers | - | |
| SmartMonitor™ [ | SmartMonitor™ | Smartwatch | Accelerometer | Detects repetitive shaking motions, HR, activity | Stress, surprise | SmartMonitor™ | - |
| Verily Life Sciences™ [ | Verily Study Watch | Smartwatch | CGM sensor | Wireless monitor for pulse, HR, ECG, ST | Stress, surprise, laugh, interest | Verily | Approved (2019) |
| Viatom Technology™ [ | Viatom Checkme O2 | Smartwatch | Oximeter, HR sensor | HR, ECG, SpO2, activity tracker, ST, sleep monitoring | Stress, relaxation, surprise, laugh, interest | Viatom | Approved |
| Withings™ [ | Withings™ ScanWatch | Smartwatch | ECG, oximeter | ECG, HR, SpO2, step tracking, distance, sleep quality | Stress, relaxation, postures, surprise, laugh, interest | Withings™ Developer | Approved |
| Move ECG | Analog watch | Heart rate sensor, 3-axis accelerometer, 3-axis gyroscope sensor | HR, ECG | Stress | Withings Developer | - | |
| Xiaomi™ [ | Huami Amazfit Health Band | Smartwatch | ECG sensor, pedometer | HR, movement tracking | Stress, postures, surprise | Mi Developer | Approved (2019) |
| Mi Smart Band 5 | Smartwatch | ECG sensor | HR, sleep quality, step tracking, stress level, BP | Stress, relaxation, postures, surprise | Mi Developer | Not Approved | |
| Sensoria™ [ | T-Shirt Short Sleeve + HRM | T-Shirt | HR monitor | HR, speed, distance, step tracking | Stress, postures, surprise | Sensoria™ Platform | - |
| Ambiotex™ [ | Ambiotex Smart Shirt | T-Shirt | ECG and HR sensors | Stress level, ECG, HR, HRV, step tracking | Stress, postures, surprise, laugh, interest | Ambiotex™ | - |
| Tempdrop™ [ | Tempdrop™ | Underarm armband | Thermometer | ST | Stress | Tempdrop™ | - |
| Carré Technologies™ [ | Hexoskin Smart Garments | Vest | ECG sensor | ECG, HR, HRV, RR, stress level, effort, fatigue, activity intensity, acceleration, step tracking, sleep quality, SpO2 | Stress, relaxation, postures, surprise, laugh, interest | Hexoskin Developers | - |
| Nuubo™ [ | Nuubo Wearable ECG | Vest | ECG sensor | ECG | Surprise, laugh, interest | Nuubo™ Wearable ECG | Approved |
| Zoll™ [ | Lifevest | Vest | Temperature sensor | ECG | Surprise, laugh, interest | Lifevest | Approved (2018) |
| AliveCor™ [ | Kardia Band | Wristband | Electrodes | ECG | Surprise, laugh, interest | AliveCor™ | Approved (2019) |
| Ava Science™ [ | Ava Wristband | Wristband | 2-wavelength optical PPG sensor | EDA, PPG, HR, ST | Stress, surprise, engagement, laugh | Ava | Approved |
| Sentio Solutions™ [ | Feel | Wristband | EDA, PPG HR, and skin | EDA, PPG, HR, ST | Stress, surprise, engagement, laugh | Feel | - |
| iHealth™ [ | Wireless Blood Pressure Monitor | Wristband | Oscillometer | BP, HR | Stress, surprise | iHealth™ | Approved |
| MOCACARE™ [ | MOCACuff | Wristband | HR sensor | BP, HR, SpO2 | Stress, surprise | Mocacare™ | - |
| Wavelet™ [ | Biostrap Wristband | Wristband | 3-axis accelerometer | HR, HRV, SpO2, RR, in-depth sleep tracking | Stress, relaxation, postures, surprise | Biostrap | - |
| WHOOP™ [ | WHOOP Wristband | Wristband | HR sensor | RR, HR, HRV, EDA, sleep | Stress, engagement, laugh | WHOOP™ | - |
Abbreviations: SpO2: Oxygen saturation; BP: Blood pressure; HR: Heart rate; HRV: Heart rate variability; EEG: Electroencephalogram; ECG: Electrocardiogram; PPG: Photoplethysmography; EMG: Electromyography; EDA: Electrodermal Activity; GSR: Galvanic Skin Response; RR: Respiratory rate; ST: Skin temperature.
Figure 4Incidence of Commercial Wearable Form Factors in the Literature.
Figure 5FDA Approval Status of Commercial Wearables for Engagement Detection.
Main Physiological Signals for Engagement Detection.
| Physiological Signal | FDA-Approved Devices | Non-FDA Devices | Total |
|---|---|---|---|
| Distress | 1 | 0 | 1 |
| Stress | 22 | 39 | 61 |
| Relaxation | 10 | 19 | 29 |
| Sleep disorders | 1 | 0 | 1 |
| Postures | 4 | 14 | 18 |
| Surprise | 23 | 25 | 48 |
| Concentration | 0 | 1 | 1 |
| Frustration | 0 | 1 | 1 |
| Interest | 11 | 6 | 12 |
| Laugh | 10 | 6 | 16 |
| Emotional arousal | 0 | 1 | 1 |
| Excitement | 0 | 1 | 1 |
Figure 6Correspondence between Commercial Wearables and Physiological Signals for Engagement Detection.
Non-Commercial Wearables for Engagement Detection.
| Aim | Device Type | Function | Sensors | Real-Time Monitoring Capabilities | Educational Environment |
|---|---|---|---|---|---|
| Physical exertion, health, and heart function monitoring; tracking an individual’s performance and exertion level [ | Patch | It consists of a patch that includes a sensor to measure a biochemical (lactate) and an electrophysiological (electrocardiogram) signal to monitor physical exertion, health, and heart. The patch can recognize emotions such as stress and anger. | Lactate sensor, electrophysiological sensors | Yes | Unstated |
| Skin temperature measuring [ | Patch | The patch detects multimodal biosignals, measures skin temperature with a sensitivity of 0.31 Ω/°C, skin conductance with a sensitivity of 0.28 μV/0.02 μS, and pulse wave with a response time of 70 msec. | ST, skin conductance, and pulse wave sensors | Yes | Unstated |
| Heart rate monitoring [ | Scarf | It helps users to reflect on their emotional state, modify their affective state, and interpret the emotional states of other people. The design of SWARM is based on a scarf so that people with different disabilities have access to this type of technology. SWARM can detect emotions such as stress, sadness, calm, happiness, and excitement. | Biosensors | Yes | Unstated |
| Heart rate and skin conductance monitoring [ | Scarf | It is a design of a wearable device based on a scarf form factor. The device features color-changing and olfactory properties to affect people’s emotional state. The wearable comprises two sensors: a heart rate sensor and a skin conductance sensor. When changing color and emitting an odor, the scarf potentiates positive emotions and reduces negative ones. | HR and EDA sensors | Yes | Unstated |
| Blood volume pulses and muscle contraction monitoring [ | Glove | It is an emotion recognition framework using machine learning of physiological patterns. The framework relies on a PPG sensor for heart rate monitoring, an EDA sensor, a skin temperature sensor, and an EMG sensor. The proposal focuses on the preprocessing of emotion recognition and supports the recognition of emotions such as happiness, anger, fear, disgust, and sadness. | PPG and EMG sensors | Yes | Unstated |
| Physiological arousal detection and monitoring [ | Gloves, bracelet | The device monitors the student’s psychological and physical condition using heart rate, skin conductivity, and respiration sensors. The data obtained are sent to an assistive host to process, analyze, and evaluate student moods and stress levels. | HR sensor, EDA sensor, respiratory rate sensor | Yes | Mobile |
| Detection of eye movements, eyes closed, and teeth clenching [ | Eyeglasses | AttentivU is a device that monitors physiological data to measure the engagement and enhance learning activities using silver electrodes. The data collected by the device can be processed in real time or sent to a separate computer. | EEG sensor or electrooculography (EOG) | Yes | Unstated |
| Monitoring of physiological characteristics related to heart rate, oximetry, skin temperature, and GSR [ | Patch | The proposal uses an Arduino board to obtain physiological signals from the user and connected sensors to acquire data on skin temperature, GSR, pulsometer, and a respiratory rate sensor. The data is processed using Matlab. | ST sensor, oximeter breath-flow rate sensor, HR sensor, GSR sensor | Yes | Unstated |
| Heart rate monitoring [ | Shirt | The prototype is based on an Arduino Uno board to which is connected a pulse sensor that uses infrared light to detect user heart rate. | Pulse sensor | Yes | Unstated |
| Human physical activity monitoring [ | Shirt | SensVest is a wearable prototype to monitor physical aspects. The device includes a series of sensors that allow the recording of different data related to human performance to improve the understanding of scientific concepts in students. | HR sensor, ST sensor, accelerometer | Yes | Unstated |
| Heart rate and breathing rate monitoring [ | Patch | The device obtains an ECG tracing using two electrodes in symmetrical positions on the user’s body and a third ground electrode placed next to one of the sensing electrodes. | Electrodes, ECG sensor | Yes | Computer video |
| Heart rate variability monitoring, skin temperature measuring [ | Wristband | n-Gage is a system that evaluates the engagement levels of behavioral, emotional, and cognitive students. The system detects the student’s physical and physiological signals and environmental changes in the educational environment. | EDA sensor, accelerometer, ST sensor | Unstated | Unstated |
| EDA and pulse rate monitoring [ | Patch | The proposal measured and recorded electrodermal activity, pulse rate, and facial recognition during an e-learning session to determine the level of student engagement. The data collected were analyzed using software developed with Matlab. | EDA sensor, HR sensor | Yes | E-learning |
| Upper body pressure distribution [ | Chair | While students perform e-learning reading activities, the student’s upper body pressure is recorded using a chair with a pressure mat. The data is processed using classifiers. | Pressure mat | No | E-learning |
| Feet posture and movement detection [ | Insole | This platform contains an insole with ground contact force (GCF) plantar pressure sensors. In addition, a microcontroller with WIFI technology collects the data and sends it to a database to be analyzed by a Human Activity Recognition classifier. | Accelerometer, gyroscope sensor, magnetometer, barometer, and range finder sensors | Yes | No specified |
Figure 7Classification of Non-Commercial Wearables by Form Factor.
Non-Commercial Wearables for Engagement Detection with Real-Time Monitoring.
| Real-Time Monitoring | No. of Devices | Percentage |
|---|---|---|
| Yes | 13 | 87% |
| No | 2 | 13% |
Physiological Parameters for Engagement Detection by Non-Commercial Wearables.
| Target Parameter | No. of Devices | Percentage |
|---|---|---|
| Heart Rate | 11 | 73% |
| Skin Temperature | 5 | 33% |
| Skin Conductance | 4 | 27% |
| Electrodermal activity | 4 | 27% |
| Respiratory Rate | 2 | 13% |
| Pulse Wave | 1 | 7% |
| Oxygen Saturation | 1 | 7% |