Clinical accuracy of the Omron M3 Comfort® and the Omron Evolv® for self-blood pressure measurements in pregnancy and pre-eclampsia - validation according to the Universal Standard Protocol.

BACKGROUND: Electronic devices for blood pressure (BP) measurements need to go through independent clinical validation as recommended by different authorities, both in general and in special populations such as pregnancy.
OBJECTIVE: To evaluate the accuracy of the Omron Evolv® (HEM-7600T-E) and the Omron M3 Comfort® (HEM-7134-E) devices in pregnancy and pre-eclampsia according to the Universal Standard Validation Protocol.
METHODS: Both devices, the Evolv and the M3 Comfort, measure BP at the brachial level using the oscillometric method. The study was performed according to the recently published protocol, the so-called "modified Advancement of Medical Instrumentation (AAMI)/British Hypertension Society (BHS)/European Society of Hypertension (ESH) protocol" or the "Universal Standard Protocol." Validation of each device included 45 pregnant women in the second and third gestational trimester of whom 15 had pre-eclampsia, 15 had gestational hypertension and 15 were normotensives. BP differences between the observer and the device BP values were classified into three categories (≤5, ≤10, and ≤15 mmHg) and the mean BP differences (test vs reference) and its SD were calculated.
RESULTS: Both devices, the Evolv and the M3 Comfort, achieved a grade A/A in both pregnancy and pre-eclampsia. The mean difference (SD) between the mercury standard and the device BP values in pregnancy were: 1) for the Evolv of -0.7±2.3 mmHg for systolic blood pressure (SBP) and -0.1±1.8 mmHg for diastolic blood pressure (DBP); 2) for the M3 Comfort of -1.6±2.8 mmHg for SBP and -0.1±2.3 mmHg for DBP.
CONCLUSION: Both devices, the Evolv and the M3 Comfort, achieved a grade A/A for both SBP and DBP and fulfill the validation protocol criteria in pregnancy and pre-eclampsia. Consequently, these two devices can be recommended for home BP measurements in this specific population.

Introduction

For many years, the gold standard instrument for blood pressure (BP) measurement was a mercury sphygmomanometer and a stethoscope, but this technique is being progressively removed from clinical practice because of the mercury toxicity and the number of errors that may taint this method.1,2 Several non-mercury techniques have been developed to gradually supplant the mercury-auscultatory method,3,4 such as the electronic devices using algorithms based on the oscillometric technique.5,6 It is evident that these devices need to go through a process of validation by experts in independent centers as recommended by guidelines and societies.4 Different protocols are used to validate the accuracy of BP measuring devices such as the international protocol published by the working group on BP monitoring of the European Society of Hypertension (ESH),7 the British Hypertension Society (BHS) protocol and the Association for the Advancement of Medical Instrumentation (AAMI) protocol.8,9 Over the last ten years, hundreds of devices have been successfully validated using established protocols,10 mostly on the general population. However, few studies have tested the accuracy of automated BP monitors in specific populations such as diabetic patients,11 pregnant women,12 obese,13,14 elderly,15 and in arrhythmic patients.16,17 All the above-mentioned organizations agreed to consider pregnancy and pre-eclampsia as a special population.12

Pregnancy is a specific condition where vascular hemodynamic and arterial function and structure changes influence the arterial signals and therefore the BP determination. These hemodynamic changes are more pronounced in pre-eclampsia suggesting that automatic devices for BP measurements tend to underestimate BP in these conditions.18 Considering that hypertension in pregnancy is common and has been reported to occur in up to 10% of pregnancies; accurate BP measurements is of paramount importance in diagnosing and monitoring high-risk pregnant women.12 In this regard, the validation protocols recommend, for BP devices designed to be used in pregnancy, to go through specific validation of their accuracy in this specific population. Only few devices have been shown to be accurate in pregnant women with and without pre-eclampsia.12,19,20 In a recent review,19 among the 36 validations of home BP monitors, 25 validations were performed according to the BHS or a modified BHS protocol and 11 according to the AAMI protocol whereas the most popular and most used validation protocol in the general population is the ESH protocol. However, experts agreed that accuracy must be evaluated according to a single validation protocol that had global acceptance. Therefore, an international initiative was taken by the AAMI, ESH, and the International Organization for Standardization (ISO) experts to develop a universal standard for device validation. This Universal Standard Protocol for the validation of BP devices has been recently published.21,22

The objective of this study was to assess the accuracy of the automatic oscillometric BP measurements, the Omron Evolv® (HEM-7600T-E) and the Omron M3 Comfort® (HEM −7134-E) devices in pregnant women including pre-eclampsia according to the Universal Standard Protocol.”21,22

Methods

Ethical committee

This prospective study using a medical device (Type IIA) was approved by the ethical committee of the National Institute of Health, Ministry of Health, Armenia. Written informed consent was obtained from each woman included in the study.

Study design

The present study was performed using the so-called “modified AAMI/BHS/ESH protocol” or the “Universal protocol.” Details of this validation protocol have been published previously elsewhere.21,22 Briefly, this protocol is based on the previous BHS, AAMI, and ESH protocols with results expressed using mixed criteria (C.F. hereafter).

The validations of the two devices were performed in parallel but not necessarily in the same subjects. Each device validation study was assessed separately from the other device.

Study population

According to the validation protocol, 45 women in the second and third trimesters of pregnancy were included; of whom 15 with pre-eclampsia defined as elevated systolic blood pressure (SBP) at least 140 mmHg and/or diastolic blood pressure (DBP) at least 90 mmHg with proteinuria, 15 with gestational hypertension (new onset in pregnancy with BP ≥140 mmHg and/or ≥90 mmHg without proteinuria), and 15 normotensives. The exclusion criteria were: arrhythmia, atrial fibrillation, etc.; poor quality of Korotkov sounds; absence of Korotkov phase five sounds; arm circumference >42 cm.

Procedures and measurements

Tested devices

Three devices of each model were provided by OMRON Healthcare Europe BV, one of them has been randomly chosen to perform the corresponding study. Each of the chosen devices for the study was used according to the manufacturer’s recommendations.

The HEM-7600T-E device is a digital automatic device for home BP measurements at the arm level with no tubes and no wires, the device using the inflationary oscillometric method is fixed directly to the arm cuff. The device can be connected via Bluetooth to another smart device to transfer the data. It uses inflation by Fuzzy-Logic controlled by electric pump and an automatic rapid pressure release valve for the deflation. The monitor weight is ~ 240 g (not including the 4 “AAA” needed batteries); its cuff allows BP measurements for an arm circumference of 22–42 cm. The device measures BP and pulse rate with a pressure range of 40–260 mmHg and pulse rate range of 40–180 beats/min. SBP, DBP and pulse rate are displayed on a liquid crystal digital (LCD) screen.

The HEM-7134-E device is a digital automatic device for home BP measurements at the arm level. The monitor uses inflation by Fuzzy-Logic controlled by electric pump and an automatic rapid pressure release valve for the deflation. The monitor weight is ~ 300 g (not including the 4 “AA” needed batteries); its cuff allows BP measurements of an arm circumference of 22–42 cm. The device measures BP and pulse rate with a pressure range of 0–299 mmHg and pulse rate range of 40–180 beats/min. SBP, DBP and pulse rate are displayed on a LCD screen.

Mercury sphygmomanometer BP measurements

The validation team consisted of three persons, two observers and one supervisor trained in accurate BP measurements. A total of 45 pregnant women had their BP measured by the two observers using parallel connected mercury sphygmoma nometers and a “Y” connected teaching stethoscope; they were blinded from each other’s result. The tested devices were used by the supervisor. The agreement between the two observers was checked all over the evaluation period by the supervisor to make sure that the difference between the two observers is not >4 mmHg for SBP and DBP values. Otherwise, the measurement should be repeated. Korotkov K5 sound was used for reference DBP.

Two standard mercury sphygmomanometers were used by the two observers as a reference standard. The circumference of the arm was measured to ensure that the reference cuff-size being used is adequate for the subject. Three cuff sizes were used according to the arm circumference: the small cuff for arm circumference of 17–22 cm; the standard cuff (medium) for 22–32 cm, and the large cuff for arm circumference of 32–42 cm.

BP data collection

The validation procedure started with the patient seated comfortably and relaxed for at least 5 minutes, the back and the arm being supported with the middle of the upper-arm at heart level, legs uncrossed and feet flat on the floor. Measurements were performed according to the “same arm, sequential measurements” method on the left arm supported at heart level. Measurements by the tested device was performed on the same arm supported at the heart level as recommended by the manufacturer. As described in the protocol, nine consecutive BP measurements were performed in each woman using the mercury sphygmomanometers (five times) and the tested devices (four times). All nine-sequential same-arm measurements were recorded as shown in Table 1, starting with the standard mercury sphygmomanometer. The first auscultatory and the first device measurement represent the recruitment pressures (BPA and BPB).

Table 1

Procedure for reference and test device BP measurements in the same arm sequential validation protocol

BPA	Entry BP, observers 1 and 2 each with the mercury standard
BPB	Device detection BP, supervisor
BP1	Observers 1 and 2 with mercury standard
BP2	Supervisor with the test instrument
BP3	Observers 1 and 2 with mercury standard
BP4	Supervisor with the test instrument
BP5	Observers 1 and 2 with mercury standard
BP6	Supervisor with the test instrument
BP7	Observers 1 and 2 with mercury standard

Statistical analysis

Results were analyzed and expressed according to the protocol to conclude if the device passes or fails to pass the validation protocol requirements. The statistical analysis was performed using specific analysis software. The device measurement was compared to each individual observer’s mean readings taken before and after, resulting in two sets of three differences (one set for comparisons “before” and one set for comparison “after” for SBP and DBP values separately. The set of differences that is more favorable to the test device was retained for the final statistical analysis. The numbers of “As,” “Bs,” and “Cs” were used to calculate the number of device-observer differences within 5, 10 and 15 mmHg respectively. To be recommended for clinical use, a device must achieve a grade A or B for both SBP and DBP. An additional evaluation was also performed based on the AAMI criteria, requiring a mean difference between the device and the observers of ≤5±8 mmHg for both SBP and DBP. The Bland–Altman graphs are used to show the device–observer differences vs average device and observer values for all pairs of comparisons.

Results

Omron Evolv® (HEM-7600T-E)

Study population

A total of 45 women in the second (n=11) and third (n=34) trimesters of pregnancy were included; of whom 15 presented with pre-eclampsia, defined as elevated SBP at least 140 mmHg and/or DBP at least 90 mmHg with proteinuria, 15 presented with gestational hypertension (new onset in pregnancy with BP ≥140 mmHg and/or ≥90 mmHg without proteinuria), and 15 normotensives. Their mean age (±SD) was 30±5 years (20–43 years), mean gestational age was 33±4 weeks, mean arm circumference 28±3 cm. The mean recruitment SBP was 133±17 mmHg (98–156 mmHg) and the mean recruitment DBP was 85±14 mmHg (60–110 mmHg).

BP measurements

The difference between the two observers was 0.1±1.1 and −0.1±1.3 mmHg for SBP and DBP respectively (−4 to +4 mmHg). The mean differences between the observers and the tested device were −0.7±2.3 mmHg for SBP and −0.1±1.8 mmHg for DBP. The numbers of measurements differing from the mercury standard by 5, 10, and 15 mmHg or less are shown in Table 2. Indeed, a total of 131 out of 135 (97%) comparisons for SBP showed an absolute difference within 5 mmHg and 134 out of 135 (99%) for DBP. In addition, a total of 135 out of 135 comparisons for both SBP and DBP showed an absolute difference within 10 mmHg. Therefore, according to this distribution criteria required by the validation protocol, the HEM-7600T-E achieved the grade A/A for accuracy in all pregnant women. An additional evaluation was also performed based on the AAMI protocol criteria, requiring a mean difference (SD) of <5±8 mm Hg. The results showed a mean difference of −0.7±2.3 mmHg for SBP and −0.1±1.8 mmHg for DBP respectively, therefore, fulfilling the AAMI criteria.

Table 2

Grading criteria achieved with the Omron Evolv® device in pregnancy and pre-eclampsia

	GRADE A, B, C
	≤5 mm	≤10 mm	≤15 mm	GRADE
SBP, %	97%	100%	100%	A
DBP, %	99%	100%	100%	A
SBP, n	131	135	135
DBP, n	134	135	135

Notes: Grades represent the cumulative percentage of readings with differences between device and observer falling within 5, 10 and 15 mmHg respectively.

Since the device must achieve a grade A or B for both SBP and DBP and fulfills the AAMI criteria to pass the validation, the HEM-7600T-E passed these criteria and therefore can be recommended for self-BP measurements and clinical use.

Bland–Altman plots of the differences between BP measurements obtained with the HEM-7600T-E device and the sphygmomanometer are shown for SBP (Figure 1A) and for DBP (Figure 1B).

Figure 1

Bland–Altman plots of systolic and diastolic BP differences between the Omron Evolv® (HEM-7600T-E) device and the observers’ readings.

Notes: (A) SBP. (B) DBP.

Abbreviations: BP, blood pressure; SBP, systolic blood pressure; DBP, diastolic blood pressure.

As required by the validation protocol, data from pre-eclampsia shall be reported separately. Analysis of the 15 women with pre-eclampsia showed a mean age of 32±5 years; mean arm circumference of 30±2 cm; mean SBP of 140±9 mmHg and DBP of 95±7 mmHg. The difference between the two observers was 0.1±1.2 and −0.1±1.7 mmHg for SBP and DBP, respectively. The mean differences between the observers and the tested device were −1.1±2.8 mmHg for SBP and −0.4±1.7 mmHg for DBP. According to the numbers of measurements differing from the mercury standard by 5, 10, and 15 mmHg or less; the absolute differences were 93% for SBP and 100% for DBP less than 5 mmHg difference; 100% for both SBP and DBP less than 10 mmHg difference. In this subgroup, the device maintains an overall grade A/A for both the SBP and DBP.

Omron M3 Comfort® (HEM-7134-E)

A total of 45 women in the second (n=10) and third trimesters (n=35) of pregnancy were included; of whom 15 presented with pre-eclampsia, 15 with gestational hypertension, and 15 were normotensives. Their mean age (± SD) was 29±4 years (18–38 years), mean gestational age was 33±5 weeks, mean arm circumference 28±3 cm. The mean recruitment SBP was 130±19 mmHg (90–152 mmHg) and the mean recruitment DBP was 84±14 mmHg (53–104 mmHg).

The difference between the two observers was −0.1±1.0 and 0.1±1.1 mmHg for SBP and DBP, respectively (−4 to +4 mmHg). The mean differences between the observers and the tested device were −1.6±2.8 mmHg for SBP and −0.1±2.3 mmHg for DBP. The numbers of measurements differing from the mercury standard by 5, 10, and 15 mmHg or less are shown in Table 3. Indeed, a total of 121 out of 135 (90%) comparisons for SBP showed an absolute difference within 5 mmHg and 131 out of 135 (97%) for DBP. In addition, a total of 135 out of 135 comparisons for both SBP and DBP showed an absolute difference within 10 mmHg. Therefore, according to this distribution criteria required by the validation protocol, the HEM-7134-E achieved the grade A/A for accuracy in all pregnant women. An additional evaluation was also performed based on the AAMI protocol criteria; results showed a mean difference of −1.6±2.8 mmHg for SBP and −0.1±2.3 mmHg for DBP respectively, fulfilling the AAMI criteria.

Table 3

Grading criteria achieved with the Omron M3 Comfort® device in pregnancy and pre-eclampsia

	GRADE A, B, C
	≤5 mm	≤10 mm	≤15 mm	GRADE
SBP, %	90%	100%	100%	A
DBP, %	97%	100%	100%	A
SBP, n	121	135	135
DBP, n	131	135	135

Notes: Grades represent the cumulative percentage of readings with differences between device and observer falling within 5, 10 and 15 mmHg respectively.

Since the device must achieve a grade A or B for both SBP and DBP and fulfill the AAMI criteria to pass the validation, the HEM-7134-E passed these criteria and therefore can be recommended for home BP measurements.

Bland–Altman plots of the differences between BP measurements obtained with the HEM-7134-E device and the sphygmomanometer are shown for SBP (Figure 2A) and for DBP (Figure 2B).

Figure 2

Bland–Altman plots of systolic and diastolic BP differences between the Omron M3 Comfort® (HEM-7134-E) device and the observers’ readings.

Notes: (A) SBP. (B) DBP.

Abbreviations: BP, blood pressure; SBP, systolic blood pressure; DBP, diastolic blood pressure.

Since the data from the pre-eclampsia group will be reported separately, analysis of the 15 women with pre-eclampsia showed a mean age of 29±3 years; mean arm circumference of 29±2 cm; mean SBP of 142±7 mmHg and DBP of 94±7 mmHg. The difference between the two observers was −0.3±1.1 and 0.1±1.0 mmHg for SBP and DBP, respectively. The mean differences between the observers and the tested device were −2.0±3.1 mmHg for SBP and −0.9±2.5 mmHg for DBP. According to the numbers of measurements differing from the mercury standard by 5, 10, and 15 mmHg or less; the absolute differences were 89% for SBP and 93% for DBP <5 mmHg difference; 100% for both SBP and DBP < 10 mmHg difference. In this subgroup, the device maintains an overall grade A/A for both the SBP and DBP.

Discussion

This study evaluates the accuracy of two automatic oscillometric devices, the HEM-7600T-E and the HEM-7134-E for brachial BP measurements in pregnancy and pre-eclampsia. The validations of these devices were performed according to the recent Universal Standard Protocol.21,22 The evaluation criteria considered in this protocol are double: first, the mean BP difference (test vs reference device) and its SD which is the criterion 1 of the present AAMI protocol9; second, the number of absolute BP differences (test BP minus reference BP readings) within 5, 10, and 15 mmHg which is the criteria of both the BHS and the ESH validation protocols.7,8 The results showed that both devices, the HEM-7600T-E and the HEM-7134-E, achieved grade A/A for SBP and DBP and fulfill the AAMI criteria in pregnancy and pre-eclampsia. For better understanding, the interpretation of several points of the present study need to be discussed.

Oscillometric devices

Oscillometric devices have the advantage of overcoming any bias related to the auscultatory method and its linked aneroid or mercury sphygmomanometer. Many of the new generation of these devices have added benefits such as automatic repeated BP measurements, storage and communicability with other devices and/or software. Despite these advantages, there are several persistent concerns with these devices: 1) The accuracy of BP measurements: this is of concern in the general population including hypotensive, normotensive, and hypertensive and even more in special populations such as children, elderly, obese, etc. and under certain clinical conditions such as arrhythmia, pregnancy, pre-eclampsia, vascular diseases, etc. where questionable accuracy of some of these devices have been reported.19,21,23 2) The inter-individual variability of oscillometric BP measurements: In some patients, BP measurements obtained using the oscillometric methods show important variability among the repeated measures and/or high difference by comparison to the auscultatory method. The reasons for such discrepancy and variability remain unclear and debatable. This phenomenon has been considered in the evaluation criteria of the validation protocols with limited accepted numbers21; even though, it is important and recommended to check the accuracy of the automatic oscillometric BP measurements at the individual level before its clinical application.

Special populations

Most of the validation protocols (AAMI, ESH, BHS)7–9 recommend the need for independent clinical validation of automatic devices for BP measurements in the general population but also, in other more special and problematic populations. According to theses protocols, the followings are regarded as special populations: children, pregnancy, pre-eclampsia, obesity or arm circumference >42 cm and arrhythmia. Diabetes and elderly have been also considered by some experts as special populations. This study focused on women in the second and third trimesters of pregnancy who are either normotensives, with gestational hypertension, or with pre-eclampsia. In fact, substantial hemodynamic changes occur during pregnancy including increased blood volume, stroke volume, heart rate, cardiac output alone with a decrease in peripheral vascular resistance and arterial compliance. Some of these modifications/alterations modify the pulse wave characteristics and consequently the oscillogram, thereby ultimately affects the accuracy of the device.23,24 Considering these hemodynamic changes, the questionable accuracy of automatic devices, the limited number of validated devices in this population and given the importance of measuring BP accurately in pregnancy, this study was undertaken according to strict and established validation protocol. Any extrapolation of these results to other populations would be incorrect and arbitrary.

Validation protocol

During the last decades and up to January 2018 several protocols7–9 have been used to assess the accuracy of automatic BP measurements devices. In a recent review, authors showed that the ESH protocol was the most popular and used protocol during the last decade.19 Despite the differences, all these protocols have major similarities and a common objective, the standardization of the validation procedures to establish minimum standards of accuracy and performance. To establish an agreed single validation protocol that had global acceptance, an international initiative was taken by the AAMI, ESH and ISO experts who agreed to develop a “Universal” validation protocol. This very recently published (January 2018) Universal Standard Protocol, is a better mixture, based on the previous protocols and including parameters and criteria taken from one or other protocol.21,22 For these reasons, our study was performed according to this latest universal validation protocol which differ from the others in terms of BP ranges, arm circumference, and other parameters which are not applied. In a recent systematic review12 of the accuracy of BP measurement devices in pregnancy, the authors analyzed 18 studies in which brachial home BP devices were examined. They reported that the devices passed validation in 13 studies, three of which had no protocol violation, seven had at least one minor violation, one had at least one major violation and two had major and minor violations. In another recent publication,19 the authors reported that violations of the ESH validation protocol were identified in 33% of the studies, some with two or more violations. The most frequent violations involved BP recruitment ranges, BP distribution, age, inadequate measurement procedure, etc. In the present study, special attention was made to perform the study according to the validation protocol requirements without any major or minor violations. Moreover, in the present report, the test device BP measurements are compared against the most suitable BP reference measurements (pre- or post-device measurements). Additional analysis, comparing each of the test device BP measurements against the average of the previous and next reference BP readings was performed and showed identical results.

Previous studies

The above-mentioned systematic review12 which focused on brachial BP measurement devices in pregnancy showed 18 studies (nine devices) for home BP devices in which the devices passed the validation test in 13 studies. As expected, some of the devices passed the validation in one study and failed in another one, even if performed according to the same validation protocol. The other review19 reported 25 devices for home BP measurements with validation studies in pregnancy and pre-eclampsia. Most of these studies were performed according to the BHS protocol followed by the AAMI protocol. According to the literature and to our knowledge, this is the first study assessing the accuracy of the Omron Evolv and the Omron M3 Comfort in pregnancy and pre-eclampsia. Moreover, this is the first validation study performed according to the very recent Universal Standard Protocol for the validation of BP devices jointly developed by the AAMI, ESH, and ISO authorities.21,22

Elsewhere, given some contradictory results of some validation studies of the same device and using the same validation protocol, we are entitled to raise the issue of the reproducibility of the validation studies. While waiting for the results of specific designed studies, it is advisable to duplicate validation studies in at least two different independent expert centers.

Conclusion

In conclusion, both Omron devices, the Evolv and the M3 Comfort achieved a grade A/A for both SBP and DBP and fulfill the AAMI criteria in pregnancy and pre-eclampsia. Consequently, these two devices can be recommended for home BP measurements and clinical use.