Accuracy of the Withings BPM Connect Device for Self-Blood Pressure Measurements in General Population - Validation According to the Association for the Advancement of Medical Instrumentation/European Society of Hypertension/International Organization for Standardization Universal Standard.

Background: As recommended by various authorities and scientific societies, electronic devices for blood pressure (BP) measurements must undergo independent clinical validations for accuracy assessment. Objective: To assess the accuracy of the Withings BPM Connect device in the general population according to the Association for the Advancement of Medical Instrumentation/European Society of Hypertension/International Organization for Standardization "AAMI/ESH/ISO Universal Standard (ISO 81060-2:2018)".
Methods: The Withings BPM Connect device measures BP at the brachial level using the oscillometric method. According to this protocol using the same-arm sequential BP measurement method, subjects (n ≥ 85) fulfilling the age, gender, BP and cuff distribution criteria must be included. Two criteria are used for the analysis: Criterion 1: differences between observers' mercury sphygmomanometer reference measurements and test device BP values (test versus reference) and their standard deviation (SD) must be ≤5 ± 8 mmHg for both systolic (SBP) and diastolic BP (DBP). Criterion 2: the SD of the mean BP differences between the test device and reference BP per subject must be ≤6.91/6.62 mmHg for SBP/DBP.
Results: Ninety-two subjects were selected, 85 of whom were included. For validation criterion 1, the mean difference ± SD between the reference and device BP values was 0.6 ± 5.3 mmHg for SBP and 2.1 ± 4.3 mmHg for DBP. For criterion 2, the SD of the mean BP differences between the test device and reference BP per subject was 4.2/3.6 mmHg (SBP/DBP). These results fulfilled the AAMI/ESH/ISO Universal Standard (ISO 81060-2:2018) protocol requirements.
Conclusion: The Withings BPM Connect oscillometric device for home BP measurement fulfilled all of the accuracy requirements of the AAMI/ESH/ISO Universal Standard (ISO 81060-2:2018) in general population and consequently can be recommended for home BP measurements.

Introduction

Blood pressure (BP) measurement is one of the most frequently performed acts in medical practice. It is performed by healthcare professionals either to assess cardiovascular hemodynamics or to define the BP level in individuals and its consequences in terms of hypertension and other health conditions.1,2 BP measurement is also performed by individuals to evaluate their cardiovascular condition as well as by patients using home BP measurements to monitor their hypertension and its therapeutic control as recommended by international guidelines.3 An accurate measurement of BP is thus crucial since erroneous values and misclassification can incur serious clinical consequences.3,4 Whomever is performing the BP measurement, whether practitioner, nurse, pharmacist, patient, etc., the latter’s major concern must be the reliability of the BP device and the accuracy of the observed BP values.5 Several automatic devices for BP measurement, based principally on the oscillometric method, have been developed. These devices must undergo clinical validation in independent centers as recommended by several scientific societies/authorities.1,2,5 Different protocols have been used to assess the accuracy of BP measuring devices such as the International protocol of the European Society of Hypertension (ESH),6 the British Hypertension Society (BHS) protocol,7 the American Advancement of Medical Instrumentation (AAMI) protocol,8 etc. In 2018, members of the AAMI, the ESH and the International Organization for Standardization (ISO) committees achieved a consensus on an optimal validation standard, the AAMI/ESH/ISO Universal Standard (ISO 81060–2:2018) which is now considered as the standard protocol for the validation of non-invasive blood pressure measuring devices.9–11 In certain instances, such independent clinical validation as well as publication of its results is needed for new BP devices to be listed as “recommended” in selected professional websites.12 Therefore, for each new developed device, it is recommended to proceed to its clinical validation clarifying its accuracy according to the established standard. Over the last ten years, several devices have been validated, mostly in the general population.12 The aim of the present study was to evaluate the accuracy of the Withings BPM Connect, an automatic upper-arm BP monitor for home BP measurement in the general population, according to the AAMI/ESH/ISO Universal Standard (ISO 81060–2:2018).10,11

Methods

Ethical Aspects

This study using a medical device (Type IIa) was approved by the ethics committee of the Institute of Cardiology named after Levon Hovhannisyan (Yerevan, Armenia). Written informed consent was obtained from all included subjects. The study was performed in compliance with the Declaration of Helsinki and ICH Good Clinical Practice (GCP) guidelines.

Study Design

This is a prospective non-interventional, non-randomized, study using a Type IIa medical device. The study was performed according to the AAMI/ESH/ISO Universal Standard (ISO 81060–2:2018). Details of this validation protocol have previously been published.10,11 The recommendations for performing and reporting validation studies according to this validation standard were strictly followed.

Study Population

The required sample size is defined by the AAMI/ESH/ISO validation protocol.10,11 A minimum of 85 subjects must be included and analyzed, usually with 3 pairs of BP measurements per subject, as well as a minimum of 255 pairs of BP measurements using the reference sphygmomanometer and the tested device should be obtained. Inclusion criteria were treated or untreated patients, age > 18 years, arm circumference between 22 cm and 42 cm, the latter being in accordance with the instructions for use of the Withings BPM Connect device. The exclusion criteria were arrhythmia, poor quality of Korotkoff sounds, patient unable to give consent or properly understand the protocol information, patient with an open wound and/or damaged skin in both upper arms, patient wearing an implantable electrical medical device, the latter being in accordance with the instructions for use of the Withings BPM Connect device.

Procedures and BP Measurements

Test Device

The WITHINGS BPM CONNECT device is a new digital automatic device for home BP measurement at the brachial level developed by Withings SA (Issy-Les-Moulineaux France) (Figure 1). The device measures BP using the oscillometric method during cuff inflation, with a pressure range of 0–285 mmHg (systolic: 60–230 mmHg, diastolic: 40–130 mmHg) and pulse rate range of 40–180 beats/min. At the end of BP determination, a fast cuff deflation is performed using a release valve. The device includes:

Figure 1

The Withings BPM Connect device.

A cuff assembly for BP measurement in arm circumference 22–42 cm including:

Soft cuff of 55 cm length and 15 cm width with inflatable bladder of 23 cm length and 12.5 cm width,

Velcro pads to secure the cuff around the subject’s arm.

A main unit mounted on the outer side of the cuff including:

Pneumatic circuit: a pump, a release valve, a pressure sensor, an analog-to-digital converter, memory, Wi-Fi and Bluetooth components,

Li-ion rechargeable battery and a micro-USB charging plug,

LED screen and a trigger button.

Devices used in this study were equipped with version 5c (electronic) and 4a (mechanical) hardware and FW1401 firmware. Further details of the device are reported in its User-Manual. For this study, three Withings BPM Connect devices were provided by WITHINGS SA, one of which was randomly chosen to conduct the corresponding study according to the manufacturer’s recommendations.

Reference Blood Pressure Measurements – Mercury Sphygmomanometer

The validation team consisted of three investigators, namely two observers and one supervisor trained in accurate BP measurements. BP was first measured by the two observers blinded to each other’s result using: 2 parallel connected mercury sphygmomanometers (KDM® Germany), calibrated prior to study initiation, and a “Y” connected teaching stethoscope (3MTM Littmann®- United States), after which BP was measured by the supervisor using the tested device. Agreement between the 2 observers was verified by the supervisor to ensure that the difference between their measurements was < 4 mmHg for SBP and DBP. In case of disagreement between the 2 observers, additional pairs of measurements were performed with a maximum of 8 pairs of BP determinations after which the subject was excluded. Korotkoff sound (K5) was used for reference diastolic BP.

The circumference of the arm was measured at mid-arm to ensure that the reference cuff-size being used was adequate for the subject. Three cuffs with inflatable bladder dimensions 9×18 cm, 12×24 cm and 15×32 cm, respectively, were used such that the length reached 75–100% of the participant’s midarm circumference and width at 37–50%.

Procedure of BP Measurements and Data Collection

The validation procedure began with the patient seated comfortably and relaxed for at least 5 minutes, the back and the arm supported with the middle of the upper arm at heart level, legs uncrossed and feet flat on the floor. BP measurements were performed according to the “same arm, sequential measurements” method on the left arm supported at heart level as described in the AAMI/ESH/ISO Universal standard.10,11 Measurements with the tested device were performed on the same arm supported at heart level as recommended by the manufacturer. As required, nine consecutive BP measurements were performed in each subject using the mercury sphygmomanometers (5 times: R0, R1, R2, R3, R4,) and the tested devices (4 times: T0, T1, T2, T3). All nine-sequential same-arm measurements were recorded at 1-minute intervals starting with the standard mercury sphygmomanometer. The first auscultatory and first device measurement represent the recruitment pressures (R0 and T0) and were not used in the accuracy assessment of the test device.

Statistical Analysis

Results were analyzed and presented according to the validation protocol requirements.10,11 Statistical analysis was performed using specific analysis software established by the International Society of Vascular Health (ISVH). Each reference BP measurements (R1, R2, R3, R4) represented the mean of the simultaneous readings of the two observers. Each of the test device measurements was compared against the mean of the previous and next reference BP readings (eg, T1 versus the mean of R1-R2). Differences were calculated by subtracting the reference BP measurements from the test device measurements. The mean BP differences (test versus reference device) and the SD were calculated, ie criterion 1 (protocol requirements < 5 ± 8 mmHg), criterion 2 being the SD of BP differences per subject as defined by the AAMI/ESH/ISO Universal Standard (ISO 81060–2:2018). The same criteria were used for SBP and DBP measurements. Standardized Bland-Altman scatterplots were used to illustrate device - observer differences versus mean device and observer values for all pairs of comParisons. More specifically, the BP differences between the reference measurements and the test device were calculated according to arm circumference and presented in a Bland-Altman scatterplot. The distribution of the population was calculated according to gender, BP values and arm circumference as recommended in the protocol.10,11

Results

In the present study, 92 subjects from the Preventive Cardiology Department of the Institute of Cardiology were selected, 85 of whom were recruited and analyzed; 7 subjects were excluded due to BP variability and completed ranges (n=5), arrhythmia n=1), and poor Korotkoff sounds (n=1).

The intention to treat and the per protocol populations were similar. The clinical characteristics of the participants are summarized in Table 1. Results showed that their characteristics are in accordance with the requirements of the validation protocol.

Table 1

Characteristics of Study Participants (n=85)

	Mean ± SD	Range
Age (years)	50.1 ± 14.2	20–79
Gender (male/female)	35/50	–
Arm circumference (cm)	32.1 ± 5.5	23–41
Entry SBP R0 (mm Hg)	130.3 ± 24.7	91–183
Entry DBP R0 (mm Hg)	80.7 ± 15.2	57–113

Abbreviations: DBP, diastolic blood pressure; SBP, systolic blood pressure.

BP Measurements

Table 2 depicts the distribution of BP levels measured using the reference method, the results of which are in accordance with the protocol requirements, namely:

Table 2

Distribution in Percentages of Reference BP Measurements (R1-R4)

SBP	≤100 mmHg	≥160 mmHg	≥140 mmHg
%	11.76%	14.41%	34.12%
DBP	≤60 mmHg	≥100 mmHg	≥85 mmHg
%	12.06%	11.76%	40.88%

Abbreviations: DBP, diastolic blood pressure; SBP, systolic blood pressure.

For SBP readings: ≥ 5% readings must be ≤ 100 mmHg, ≥ 5% readings must be ≥ 160 mmHg and ≥ 20% readings ≥ 140 mmHg.

For DBP readings: ≥ 5% readings must be ≤ 60 mmHg, ≥ 5% readings must be ≥ 100 mmHg and ≥ 20% readings ≥ 85 mmHg.

The mean BP differences between the simultaneous observers’ measurements were 0.1 ± 2.3/0.2 ± 2.2 mmHg (systolic/diastolic, range −4 to 4 mmHg).

The validation analysis is shown in Table 3. For criterion 1, the mean differences between the tested device and reference BP values (test device minus reference BP values) were 0.6 ± 5.3 mmHg for SBP and 2.1 ± 4.3 mmHg for DBP. –For criterion 2, (for individual subjects): the SD of 85 mean BP differences (test minus reference BP per subject) must be within a threshold defined by the mean test-reference BP difference and calculated to be ≤ 6.91/6.62 for SBP/DBP. Results showed that SD was = 4.2 mmHg for SBP and 3.6 mmHg for DBP (Table 4), thereby confirming that the tested device fulfilled the requirements of the AAMI/ESH/ISO - (ISO 81060–2:2018) Universal protocol10,11 for both criteria 1 and 2 and is thus qualified as having successfully “PASSED” the validation.

Table 3

Validation Study Results

	Pass Requirement	SBP	DBP
Criterion 1 (255 BP pairs)
Mean BP difference (mm Hg)	≤5	0.6	2.1
SD (mm Hg)	≤8	5.3	4.3
		Pass	Pass
Criterion 2 (85 subjects)
SD (mm Hg, SBP/DBP)	≤6.91/6.62	4.2	3.6
		Pass	Pass
Result		Pass

Abbreviations: DBP, diastolic blood pressure; SBP, systolic blood pressure; SD, standard deviation.

Table 4

BP Differences According to Cuff Size or Arm Circumference

Cuff Size or Arm Circumference	Participants n (%)	Mean SBP Difference ± SD (mmHg)	Mean DBP Difference ±SD (mmHg)
Cuff size
≥ 34 subjects ≥ 32 cm	46 (54)	0.24±6.1	1.93±4.7
≥ 34 subjects < 32 cm	39 (46)	1.09±4.3	2.37±3.9
≥ 17 subjects ≥ 37 cm	23 (27)	0.41±6.7	1.94±4.7
≥ 17 subjects ≤ 27 cm	22 (26)	0.92±4.7	2.10±4.0
Arm circumference
≥ 9 subjects ≥ 39.5 cm cm	12 (14)	0.25±7.5	2.06±5.3
≥ 9 subjects ≤ 24.5 cm	10 (12)	0.35±5.2	1.45±3.9

Abbreviations: DBP, diastolic blood pressure; SBP, systolic blood pressure; SD, standard deviation.

Assessment of the accuracy of the tested device according to arm circumference is shown in Table 4. Arm circumference distribution was in accordance with the requirements of the Universal validation protocol, namely: ≥ 40% of the subjects must have arm circumference within the upper half of the specified range of use of the cuff, ≥ 40% within the lower half, ≥ 20% within the higher quarter, ≥ 20% within the lower quarter, ≥ 10% within the higher octal, and ≥ 10% within the lower octal of the specified range of use of the cuff (22–42 cm).The differences between BP values (test device minus reference) according to arm circumference are shown in Table 4 for both SBP and DBP.

Standardized Bland-Altman scatter plots of the test-reference BP differences against their mean are shown in Figure 2 for SBP (panel A) and DBP (panel B). The mean BP differences appeared equally distributed along the low, medium, and high BP ranges, with no significant correlation or trend observed between BP differences and BP levels. The Bland-Altman scatterplots showing the relationships between test-reference BP differences according to arm circumference are illustrated in Figure 3 for SBP (panel A) and DBP (panel B). The BP differences appeared equally distributed in the different arm circumferences, with no significant correlation or trend observed between BP differences and arm circumferences.

Figure 2

Standardized Bland-Altman scatter plots of test-reference BP differences against their mean values. Panel (A) SBP, systolic blood pressure; panel (B) DBP, diastolic blood pressure.

Figure 3

Bland-Altman scatter plots showing the differences between test-reference BP according to arm circumference. Panel (A) SBP, systolic blood pressure; panel (B) DBP, diastolic blood pressure.

Discussion

This is the first study assessing the accuracy of the newly developed WITHINGS BPM CONNECT device from WITHINGS SA for BP measurement at the arm (brachial) level in patients with an arm circumference 22–42 cm in the general population. The results of the study show that the WITHINGS BPM CONNECT device successfully fulfilled both criteria 1 and 2 required by the AAMI/ESH/ISO - (ISO 81060–2:2018) Universal validation protocol. The BP differences observed between reference and test device were homogeneously distributed in the low, medium and high BP values as well as in small, medium and large arm circumferences.

Certain points related to the tested device or the validation protocol warrant further consideration:

Oscillometric devices have the advantage of overcoming many biases related to the auscultatory method. Despite these advantages, there are several persistent concerns with these devices including: 1) the accuracy of BP measurements: this is of concern in the general population including hypotensive, normotensive and hypertensive individuals and even of greater concern in special populations; 2) the inter-individual variability of oscillometric BP measurements: in some patients, the oscillometric methods may show high variability among the repeated measurements in comparison to the auscultatory method. Reasons for such discrepancy and variability remain unclear. It is therefore recommended to verify the accuracy of the automatic oscillometric BP measurements at the individual level before its clinical application.4

Special Populations

The AAMI/ESH/ISO Universal standard (ISO 81060–2:2018)10,11 recommends clinical validation of automatic devices for BP measurements in the general population but also in other special populations. In the present study, the WITHINGS BPM CONNECT was exclusively validated in the general population (arm circumference 22–42 cm); hence the results presented herein cannot be extrapolated to other more specific populations such as arrhythmia, children, pregnant women, etc. Any extrapolation of these results to other populations would be incorrect and arbitrary. In this regard, another validation study of the same device has been performed in pregnancy and preeclampsia,13 the results of which corroborate the present analysis study and in which the Withings BPM Connect device fulfilled the standard requirements.10,11,13

Validation Protocol

The present study was performed according to the latest AAMI/ESH/ISO - (ISO 81060–2:2018) Universal validation protocol. Nevertheless, this standard does not specify the number of validation studies needed to approve a device’s accuracy. Due to the questionable reproducibility of validation studies, some experts agree that a device should be validated separately in at least two different centers (2 studies). For instance, one clinical validation, if successful would seemingly be sufficient to approve the device and consider its registration on the list of “approved” devices of several institutions and scientific societies.12

Conclusion

The Withings BPM Connect from Withings SA, an automatic oscillometric upper-arm BP monitor for home BP measurement, fulfils the accuracy validation requirements of the AAMI/ESH/ISO Universal Standard (ISO 81060–2:2018) for both SBP and DBP measurements in the general population with 22–42 cm arm circumference. These results are critical to ensure the accuracy of its BP measurement before its use by patients for home BP monitoring Consequently, this device can be recommended for home BP measurements in the general population. Whether this device is also appropriate for special populations such as children, pregnancy, arrhythmia, etc. requires further specific validation.