Comparison of cutaneous facial temperature using infrared thermography to standard temperature measurement in the critical care setting.

To assess the accuracy and precision of infrared cameras compared to traditional measures of temperature measurement in a temperature, humidity, and distance controlled intensive care unit (ICU) population. This was a prospective, observational methods comparison study in a single centre ICU in Metropolitan Melbourne, Australia. A convenience sample of 39 patients admitted to a single room equipped with two ceiling mounted thermal imaging cameras was assessed, comparing measured cutaneous facial temperature via thermal camera to clinical temperature standards. Uncorrected correlation of camera measurement to clinical standard in all cases was poor, with the maximum reported correlation 0.24 (Wide-angle Lens to Bladder temperature). Using the wide-angle lens, mean differences were - 11.1 °C (LoA  - 14.68 to  - 7.51),  - 11.1 °C ( - 14.3 to  - 7.9), and  - 11.2 °C ( - 15.23 to  - 7.19) for axillary, bladder, and oral comparisons respectively (Fig. 1a). With respect to the narrow-angle lens compared to the axillary, bladder and oral temperatures, mean differences were  - 7.6 °C ( - 11.2 to  - 4.0),  - 7.5 °C ( - 12.1 to  - 2.9), and  - 7.9 °C ( - 11.6 to  - 4.2) respectively. AUCs for the wide-angle lens and narrow-angle lens ranged from 0.53 to 0.70 and 0.59 to 0.79 respectively, with axillary temperature demonstrating the greatest values. Infrared thermography is a poor predictor of patient temperature as measured by existing clinical standards. It has a moderate ability to discriminate fever. It is unclear if this would be sensitive enough for infection screening purposes. Fig. 1 Bland-Altman plots for temperatures measured using clinical standards to infrared camera. a Wide-angle camera versus bladder temperature. b Narrow-angle camera versus bladder temperature.

Introduction

The Coronavirus 2019 (COVID-19) pandemic has increased the adoption of novel technologies for the screening of infectious disease. The adoption of thermal imaging cameras for rapid screening at public gathering points such as hospital entrances and airport terminals were first adopted during the 2003 severe acute respiratory syndrome (SARS) pandemic [1]. The worldwide proliferation of COVID-19, coupled with the reduced cost and increased availability of thermal imaging technology, has resulted in increased use of this technology.

Thermal imaging cameras work by rendering infrared radiation at wavelengths of 8–15 µm into visible light that can then be visualized. Modern thermal cameras utilize an uncooled microbolometer detector array where each pixel represents an individual sensor that changes electrical resistance in response to heat. The resolution of the image is based on how many sensors are contained on the array. The average infrared sensor on a smartphone has a resolution of 80X60 pixels [2], while a tripod mounted sensor is between 160X120 and 384X288 pixels [3] (0.019–0.081 megapixels). As of 2020, the maximal resolution available in an uncooled microbolometer detector is 1920X1200 pixels (2 megapixels) [4].

The reliability of thermal imaging cameras as a screening modality for fever has been questioned in the literature [5-8] and in the common press [9, 10]. There are concerns that many infectious COVID patients do not present with fevers [11, 12], with only 80% presenting with fever on hospital presentation [13] and as low as 50% 10 days from onset of symptoms [14]. Equally, concerns exist regarding sensor effectiveness. The conversion of heat data to a viewable image has historically been for the purposes of contrast between temperatures in relative terms for visualization, not towards accurate absolute temperature measurement [15]. When temperature is displayed, it is often unclear if the value is derived, or if correction is applied [16]. There are also concerns with regards to accuracy [17], drift of the reading [18], differences due to variation in patient distance from the sensor or patient positioning [19] and a poor correlation between surface temperature and core body temperature [20]. A recent review by Dell’Isola et al. highlights the numerous complexities and uncertainties that might affect an Infrared Thermography (IRT) derived temperature value [21].

Despite this, the use of IRT as a screening tool has continued to increase, including adoption in non-healthcare settings [9]. Some attempts have been made to validate the precision of thermal cameras, in the airport [7, 22] or laboratory [17, 23] setting but these have not validated against invasive measures of temperature, and had very low proportions of febrile and unwell patients. In airport [7] or Emergency Department [19] settings, sensitivities of approximately 85% were found, meaning up to 15% of febrile patients were potentially missed.

Questions remain about whether IRT can approximate core temperature and predict fever. While there is limited utility in the intensive care unit (ICU) for the use of temperature screening using IRT, it does provide a setting suited to validating precision and accuracy. Unlike screening settings, the incidence of infection is high, present in up to 50% of admissions [24]. Patients are monitored with multiple invasive and non-invasive methods of measuring core body temperature for long periods. Because minor changes in room temperature can have significant effects on the accuracy of the sensor [23], the climate controlled environ of the ICU is an ideal setting for this IRT validation. Lastly, due to the patient’s relative immobility the sensor can be placed at a fixed and constant distance, allowing data collection without active patient participation. To date, there have been no studies assessing the performance of IRT on patients receiving routine care in the ICU setting.

The primary questions of this study are:

In the ICU environment, what is the precision and accuracy of 2 separate thermal cameras with different focal lengths compared to bladder, axillary, and oral temperatures?

How well can thermal cameras predict fever at specific cut-offs of 37.5, 38, and 38.5 °C?

Methods

This was a prospective observational single centre study in the ICU of a major metropolitan hospital (Box Hill Hospital) between March 2019 and May 2020. All patients admitted into an ICU room equipped with a thermal camera were eligible. Subjects gave informed consent allowing the use of their thermal data and clinical information. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Eastern Health Research and Ethics Committee (LRR 033/2017). Prior to the study, it was decided that the maximum accepted bias between measurements would be according to the American Society for Testing and Materials (ASTM) standard for allowable errors of clinical thermometers. This is ± 0.1 °C in the temperature range 37–39 °C (0.26–0.27% error), ± 0.2 °C in the temperature range 36–37 °C and 39–41 °C (0.54–0.55% error and 0.49–0.51% error respectively), and ± 0.3 °C in the range below 36 °C (< 0.83% error) [25]. Allowing for a mean of 37.32 °C and a standard deviation of 0.96 °C based on previous studies [20] and to achieve a power of 80%, a total of 302 discrete measurements to detect a difference of 0.83%.

Demographics including age, sex, APACHE 3 score, reason for admission, use of ionotropic agents, presence of sepsis, and antipyretic use during admission were collected. Thermographic footage was obtained with Thermal Experts TE-Q1 narrow-angle and QE1PLUS wide-angle (Daejeon, Korea) cameras, each with 384X288 resolution, a field of view (FOV) 28.7º (H) × 21.7º (V)—35.3º (diagonal) and 56.3º (H) × 41.8º (V)—71.4º (diagonal) respectively, a sensitivity of < 50mK@F/1 and < 80mK@F/1.3, and a frame rate of 6 Hz. Cameras were connected to a Raspberry Pi 2 Model B (Cambridge, UK), streaming raw unprocessed files as Numpy arrays to portable storage. Complete specifications can be found in the supplemental information.

To facilitate continuous monitoring, the cameras were placed 2.2 m away from the patient on the ceiling of an ICU room measuring 5.2 m × 4.2 m × 2.7 m at a 30º angle. Room temperature was controlled to 22 °C. Patients were free to move in their beds and were clothed and covered. Both invasive and peripheral measures of temperature taken as part of routine patient monitoring were used as the gold standards, as they are both used as for clinical correlation against IRT monitoring in different settings. These measurements occurred on average every 6 h for peripheral measurements and hourly for bladder measurements. Measurements were taken via Covidien Mon-A-Therm bladder catheter (Dublin, Ireland), or via axillary/oral reading with a Welch-Allyn Suretemp Plus non-invasive probe (New York, USA). Readings were paired with the temperature value taken by the thermal camera at the same time point.

Sequential two-dimensional thermal frames were extracted at the time frames where nursing measurements were documented on the Electronic Medical Record. A region of interest 40 × 40 pixels was manually selected over the head of the patient, and the mean value was extracted from this region. Any captured values below 26 °C were considered to represent hair, surrounding pillows, sheets, or floor, and were excluded from analysis.

FDA guidelines suggest the use of a reference standard with known temperature and emissivity known as a thermal blackbody, to standardize measurements [26]. While this would have been ideally used in this study, the blackbody we obtained emitted a high-pitched noise that disrupted patient care. As an alternative, in the same frame, a 40 × 40-pixel region of floor was manually selected as an internal reference point, confirmed to be consistently 22 °C via manual measurement with a traditional thermometer. The floor temperature bias was then applied to the mean facial temperature.

Values were analysed independently and blinded from nurse-recorded values and patient clinical information. Each timeframe was represented as a single replicate and all replicates across multiple patients were aggregated for analysis.

Statistical analysis

Each temperature measurement modality was analysed separately. Differences between camera-measured temperatures and clinically recorded temperatures were analysed using the Bland–Altman approach of mean difference and limits of agreement (LoA). Precision of both the mean difference and LoA (95% CI) were calculated. Graphical display was employed to visually assess the relationship between the differences and the average. To statistically assess for heteroscedasticity, Pearson correlation of the means and differences was performed. In the event of a statistically significant correlation, data was log-transformed and reliability assessed with ratio statistics. A mean bias less than 0.5 was considered clinically acceptable.

Sensitivity and specificity analysis were performed by comparing the thermal reading to oral, axillary, and bladder temperatures as gold standard measures. Cut offs of 37.5, 38.0 and 38.5 °C were used with each clinically recorded measurement modality. Values were analysed via Receiver Operator Characteristic (ROC) analysis, with the optimal thermal camera threshold for each gold standard measure identified using Youden index (maximising sum of sensitivity and specificity). Positive and negative predictive values were calculated.

All data were analysed using Stata 16.1 (College Station, TX, USA).

Results

From March 2019 to May 2020, a total of 149 patients were admitted into the ICU bed in which the thermal camera was installed. Patients were admitted with a variety of medical and surgical conditions and acuity, from routine postoperative monitoring to end of life care. Consent was obtained and data successfully captured in 61 patients. The mean ± SD age was 67 ± 15.2. Complete demographic information can be found in Table 1.

Table 1

Demographic data

Patients, n	61
Age, mean (SD)	67 (15.2)
Male sex, n (%)	35 (57%)
APACHE III, mean (SD)	60.9 (24.1)
Mechanically ventilated > 8 h	17 (28%)
Delirium, n (%)	10 (16%)
Noradrenaline use, n (%)	18 (30%)
Adrenaline use, n (%)	1 (2%)
Milrinone use, n (%)	1 (2%)
Paracetamol use > 1 dose in 24 h	46 (75%)
Cardiac
Acute MI	2 (3%)
Cardiac arrest	4 (6%)
Other	2 (3%)
Respiratory
Pneumonia	7
COPD	2 (3%)
Other	3 (6%)
Sepsis	21 (34%)
Postoperative	28 (46%)
Drug overdose	1 (2%)
Endocrine	2 (3%)

During the study period, a short circuit in the circuit board of the wide-angle lens rendered the sensor inoperative for a period of approximately 6 months. During this time, the narrow-angle lens continued to collect data. In total, thirty-nine patients were recorded from the wide-angle lens, and 58 from the narrow-angle lens for a total of 1905 measurements, with 572, 1097, and 236 paired measurements with no replicates using oral, bladder, and axillary temperature probes. The range of temperature measurements collected via conventional means was between 34.6 and 39.6 °C. Sepsis was diagnosed in 21 of the 61 patients (28%), with a temperature greater than 38 °C in approximately 4% of oral temperature readings, 7% of axillary temperature readings, and 24% of bladder readings. The complete breakdown of paired measurements can be found in Table 2.

Table 2

Breakdown of measurements

	n	Number of measurements
Wide-angle camera
Oral	32	194
Bladder	11	500
Axillary	21	75
Total	39	792
Near-angle camera
Oral	52	378
Bladder	16	597
Axillary	32	161
Total	58	1212

Uncorrected correlation of camera measurement to clinical standard in all cases was poor, with the maximum reported correlation 0.24 (Wide-angle Lens to Bladder temperature). Using the wide-angle lens, mean differences were  − 11.1 °C (LoA  − 14.68 to  − 7.51),  − 11.1 °C ( − 14.3 to  − 7.9), and  − 11.2 °C ( − 15.23 to  − 7.19) for axillary, bladder, and oral comparisons respectively (Fig. 1a). With respect to the narrow-angle lens compared to the axillary, bladder and oral temperatures, mean differences were  − 7.6 °C ( − 11.2 to  − 4.0),  − 7.5 °C ( − 12.1 to  − 2.9), and  − 7.9 °C ( − 11.6 to  − 4.2) respectively (Fig. 1). A complete summary of LoA testing can be found in Table 3.

Fig. 1

Bland–Altman plots for temperatures measured using clinical standards to infrared camera. a Wide-angle camera versus bladder temperature. b Narrow-angle camera versus bladder temperature

Table 3

Limits of agreement tests

Method	Gold standard temperature min	Gold standard temperature max	N patients	N measurements	Correlation with camera measurement	Mean difference	Upper LoA	Lower LoA	Correlation between difference and mean	Bradley-Blackwood F test	Lin’s concordance correlation coefficient (95% CI)	Log transformation
												Mean difference	Upper LoA	Lower LoA	Lin’s concordance correlation coefficient (95% CI)	Bradley-Blackwood F test if log transformed
Wide-angle lens
Axillary	34.8	38.3	20	75	0.19	− 11.1	− 14.69	− 7.51	0.76	< 0.001	0	− 30.4	− 39.5	− 19.9	0	< 0.001
Bladder	32.6	39.4	10	500	0.24	− 11.08	− 14.29	− 7.87	0.58	< 0.001	0.01	− 29.9	− 37.89	− 20.9	0.01	< 0.001
Oral	35.5	38.4	32	194	− 0.04	− 11.22	− 15.24	− 7.2	0.88	< 0.001	0	− 30.7	− 40.91	− 18.8	0	< 0.001
Near-angle lens
Axillary	34.6	39.5	30	157	0.13	− 7.6	− 11.17	− 4.03	0.65	< 0.001	0.01	− 20.79	− 29.67	− 10.79	0.01	< 0.001
Bladder	32.6	39.4	16	597	0.1	− 7.52	− 12.15	− 2.89	0.8	< 0.001	0.01	− 20.38	− 31.74	− 7.13	0	< 0.001
Oral	35	38.8	50	348	0.03	− 7.87	− 11.58	− 4.17	0.82	< 0.001	0	− 21.54	− 30.91	− 10.91	0	< 0.001

There was significant negative bias between differences and means in all cameras compared to all clinical standards, with the bias being larger at lower temperatures. This was evidenced both by visual inspection (Fig. 1) and via comparison with the Bradley-Blackwood F test (p < 0.001). Log transformation did not remove this association (p < 0.001).

Using cut-offs of 37.5, 38, and 38.5 °C as measured by clinical standards, the optimal thermal camera cut off was established and AUC under a ROC curve established. AUCs for the wide-angle lens and narrow-angle lens ranged from 0.53 to 0.70 and 0.59 to 0.79 respectively, with axillary temperature demonstrating the greatest values. The overall sensitivities, specificities, positive and negative predictive values for the population sampled, and AUC of the ROC curve can be found in Table 4.

Table 4

Sensitivity analysis

Method	Cutoff for fever	Prevalence of fever	Optimal cutoff for thermal camera (max temperature)	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	AUC (95% CI)
Wide-angle lens
Axillary	37.5	13%	25	90.0 (55.5, 99.7)	35.4 (23.9, 48.2)	17.6 (8.4, 30.9)	95.8 (78.9, 99.9)	0.63 (0.51, 0.74)
Axillary	38	7%	29.5	40.0 (5.3, 85.3)	100.0 (94.9, 100.0)	100.0 (15.8, 100.0)	95.9 (88.5, 99.1)	0.70 (0.46, 0.94)
Axillary	38.5	0
Bladder	37.5	38.00%	27.4	26.8 (20.7, 33.7)	83.2 (78.6, 87.2)	49.5 (39.5, 59.5)	65.0 (60.1, 69.7)	0.55 (0.51, 0.59)
Bladder	38	23.80%	28.5	12.6 (7.2, 19.9)	97.1 (94.9, 98.6)	57.7 (36.9, 76.6)	78.1 (74.1, 81.7)	0.55 (0.52, 0.58)
Bladder	38.5	6.80%	28.2	17.6 (6.8, 34.5)	93.3 (90.7, 95.4)	16.2 (6.2, 32.0)	94.0 (91.4, 95.9)	0.55 (0.49, 0.62)
Oral	37.5	8.80%	24.6	76.5 (50.1, 93.2)	31.1 (24.3, 38.5)	9.6 (5.2, 15.9)	93.2 (83.5, 98.1)	0.54 (0.43, 0.65)
Oral	38	3.60%	22.3	100.0 (59.0, 100.0)	7.0 (3.8, 11.6)	3.9 (1.6, 7.8)	100.0 (75.3, 100.0)	0.53 (0.52, 0.55)
Oral	38.5	0
Near-angle lens
Axillary	37.5	14%	31.7	27.3 (10.7, 50.2)	95.6 (90.6, 98.4)	50.0 (21.1, 78.9)	89.0 (82.7, 93.6)	0.61 (0.52, 0.71)
Axillary	38	6%	29.2	100.0 (66.4, 100.0)	58.8 (50.4, 66.8)	12.9 (6.1, 23.0)	100.0 (95.8, 100.0)	0.79 (0.75, 0.83)
Axillary	38.5	2.50%	29.2	100.0 (39.8, 100.0)	56.9 (48.6, 64.8)	5.7 (1.6, 14.0)	100.0 (95.8, 100.0)	0.78 (0.74, 0.82)
Bladder	37.5	39.50%	32.1	25.0 (19.6, 31.0)	89.2 (85.5, 92.2)	60.2 (49.8, 70.0)	64.5 (60.2, 68.7)	0.57 (0.54, 0.60)
Bladder	38	21.90%	29.7	71.8 (63.2, 79.3)	58.2 (53.5, 62.7)	32.5 (27.2, 38.3)	88.0 (83.8, 91.4)	0.65 (0.60, 0.69)
Bladder	38.5	6.70%	29.9	70.0 (53.5, 83.4)	56.0 (51.8, 60.2)	10.3 (6.9, 14.5)	96.3 (93.6, 98.1)	0.63 (0.56, 0.70)
Oral	37.5	12.60%	28.8	75.0 (59.7, 86.8)	43.4 (37.8, 49.2)	16.1 (11.3, 21.9)	92.3 (86.7, 96.1)	0.59 (0.52, 0.66)
Oral	38	4.60%	29.9	62.5 (35.4, 84.8)	75.0 (70.0, 79.6)	10.8 (5.3, 18.9)	97.6 (94.9, 99.1)	0.69 (0.56, 0.81)
Oral	38.5	0.30%

Discussion

This study compared multiple modalities of standard temperature monitoring to two infrared cameras. The principal findings demonstrated a large discrepancy and poor correlation between clinical standard measurement and IRT. Uncorrected, this discrepancy was greater than the clinical accepted difference margin of error of 0.27–0.55% but appropriately equal across all three different temperature measuring modalities.

To control against the well-known limits of absolute temperature measurement from thermal cameras, the US FDA guidelines on the minimal accepted conditions for IRT use includes a climate-controlled room with minimal humidity, the absence of draft and reflective backgrounds, and a fixed distance from the camera [26]. Despite controlling for these factors, selecting a camera that has previously documented robust internal calibration [23] and normalizing against the internal drift of the camera with an internal reference, the thermal cameras demonstrated a bias significantly higher than the minimum acceptable difference for clinical measurement.

The narrow-angle lens average bias was 7 °C, while the wide-angle lens was 11 °C. This difference between the two is likely due to the wide-angle lens having a shorter focal length, increasing the effective distance away from the camera. This difference illustrates the point that an absolute temperature reading derived from IRT is very distance sensitive and is in keeping with conclusions from both medical [19] and non-medical thermographic studies [27]. Lenses that can focus on long distances are usually not available on cheaper IRT setups but would be necessary if an absolute temperature would be required. Future studies could assess the possibility of using range finders to manually correct the temperature value based on the patient distance away from the sensor.

On the outset, overall temperature values 7 or 11 °C from the gold standard seem exceedingly large. However, it must be emphasized that the purpose of IRT screening is to determine the absence or the presence of fever. The displayed temperature value by many setups is often a derived or approximated value rather than a true representation of temperature. Most non-invasive temperature measuring modalities possess a fixed offset that is generally software-corrected [28, 29], which was intentionally not employed over the course of this study. In many circumstances, despite values aligning closely at specific temperatures, they poorly predict the presence or absence of fever due to significant variability in the measurement [7, 30].

When applying discrete cut offs to the bias corrected values from each thermal camera, there was a moderate ability to discriminate a negative fever result when compared to axillary temperature using a narrow-angle lens and a cut-off of 38 °C, and confidence in a positive fever result with a wide-angle lens.

Both cameras demonstrated lower AUCs across all cut offs compared against bladder temperature rather than axillary temperature. As documented in previous studies, this is likely due to a temporal lag in core temperature changes being reflected in peripheral temperatures [31]. IRT, though performing well at discriminating axillary fever, may therefore not be effective at discriminating changes in core temperature. This is in keeping with previous airport and hospital lobby studies demonstrating very poor correlation but reasonable sensitivity and specificity for infrared thermography in diagnosing fever [5, 7], though this was at a rate equivalent or only slightly greater than self-reported temperature [22]. Equally concerning for the use of temperature as a screening tool for active infection is the fact that multiple long stay patients in our study with documented sepsis did not maintain a fever, and many infectious patients, both with COVID-19 and in previous Coronavirus outbreaks, present without fever [19, 32].

The specific influence of antipyretics, severity of illness, ionotropes and sepsis was not assessed in this study. A similar study by Moran comparing multiple modes of traditional temperature measurement in ICU found no effect of these variables. Dunn and Roberts [33] suggest a minimum of 200 datapoints in each category to assess confounding factors, and patient measurements in this case were not all independent. In our study, given some patients were measured longer than others with higher frequency, there is also the possibility of some patients contributing to the majority of datapoints in any one diagnostic group. If those values were somehow affected by a confounder, they might potentially have skewed the results. Assessment of the effect of these variables would need to be performed in a future, larger study.

We also acknowledge potential bias that could have resulted from the use of the floor as a reference standard instead of a thermal blackbody. While the floor does not possess an emissivity close to 1 the way a reference standard might, it was at a constant distance from the camera, and was confirmed to be at a constant temperature. A future study could potentially employ the use of a blackbody, provided it did not interrupt patient care.

Ultimately, the motivation of this study was to investigate and help validate the use of IRT as a screening modality. The strict conditions that are required to maintain equipment precision and accuracy including distance, ambient temperature control, and internal calibration are unlikely to be met in non-clinical settings. Despite maintaining these, the overall ability to predict either core or peripheral temperature remains very poor. Given the better performance when comparing to specific temperature cut offs, there is greater potential in using IRT as a fever screening tool. However, more work needs to be performed to ensure optimal calibration and thresholds or even if isolated fever is a sensitive enough marker of active infection. Without this, there is the potential of IRT deployments to create screening “theatre” rather than true infection screening.

Below is the link to the electronic supplementary material.

Supplementary file1 (TIF 19587 kb)

Supplementary file2 (TIF 19587 kb)

Supplementary file3 (TIF 19590 kb)

Supplementary file4 (TIF 19590 kb)

Supplementary file5 (PDF 907 kb)