Intrarater Reliability of Digital Thermography in Detecting Pin Site Infection: A Proof of Concept Study.

AIM AND
OBJECTIVE: The purpose of this study was to explore the capability and Intrarater reliability of thermography in detecting pin site infection.
MATERIALS AND METHODS: This is an explorative proof of concept study. Clinical assessment of pin sites was performed by one examiner with the Modified Gordon Pin Infection Classification from grade 0 to 6. Thermography of the pin sites was performed with a FLIR C3 camera. The analysis of the thermographic images was done in the software FLIR Tools. The maximum skin temperature around the pin site and the maximum temperature for the whole thermographic picture were measured. An Intrarater agreement was established and test-retests were performed with different camera angles.
RESULTS: Thirteen (four females, nine males) patients (age 9-72 years) were included. Indications for frames: Fracture (n=4), two deformity correction, one lengthening and six bone transport. Days from surgery to thermography ranged from 27 to 385 days. Overall, 231 pin sites were included. Eleven pin sites were diagnosed with early signs of infection: five grade 1, five grade 2 and one grade 3. Mean pin site temperature for each patient was calculated, varied between patients from 29.0°C to 35.4°C (mean 33.9°C). With 34°C as cut-off value for infection, sensitivity was 73%; specificity, 67%; positive predictive value, 10%; and negative predictive value, 98%. Intrarater agreement for thermography was ICC 0.85 (0.77-0.92). The temperature measured was influenced by the camera positioning in relation to the pin site with a variance of 0.2.
CONCLUSIONS: Measurements of pin site temperature using the hand-held FLIR C3 infrared camera was a reliable method and the temperature was related to infection grading. CLINICAL SIGNIFICANCE: This study demonstrated that digital thermography with a hand-held camera might be used for monitoring the pin sites after operations to detect early infection. HOW TO CITE THIS ARTICLE: Rahbek O, Husum HC, Fridberg M, et al. Intrarater Reliability of Digital Thermography in Detecting Pin Site Infection: A Proof of Concept Study. Strategies Trauma Limb Reconstr 2021;16(1):1-7.

Introduction

Pin site infections are common after external fixation in orthopaedic surgeries and impose a great burden to the patient and healthcare system.[1,2] Prompt treatment of pin site infections at an early stage is believed to reduce the risk of severe complications including loss of external frame stability and ostemyelitis.[3] However, the current pin site surveillance is based on clinical judgement, as there is no universally accepted definition for pin site infection.[4] Disagreements in the judgement of a possible pin site infection between different members of a healthcare team can lead to patient confusion and loss of confidence[2] and might delay proper treatment. Progression of infection represents a continuum from increased serous drainage over inflammatory redness to a stage with pus drainage or even fulminant osteomyelitis. However, validated scores describing this continuum are warranted. In the present study, we modified the Gordon score[5] by adding a grade only with the presence of serous drainage and we also did not include the subjective parameter of pain when objective scoring the pin sites by visual inspection (Table 1).

Table 1

Modified Gordon Pin Infection Classification

Grade	Description
0	Clean
1	Serous drainage, no erythema*
2	Erythema, no drainage
3	Erythema and serous drainage
4	Erythema and purulent drainage
5	Erythema, purulent drainage, radiographic osteolysis
6	Ring sequestrum or osteomyelitis

Erythema was judged as a clinical redness suspicious of inflammation in contrast to redness from scarring around the pin site that was not graded as erythema

Digital infrared thermography by generating images of the thermal patterns of the patient's skin surface provides qualitative and quantitative information on local inflammation.[6,7] In the field of wound assessment, this technique has been successfully applied to evaluate the healing process, prognosis and the risk of infection in burns,[8-11] in patients after surgeries[7,11-13] and in chronic ulcers.[14-16] Considerable progress has been made over the last few decades in the performance of equipment and standardization of technique for thermal imaging,[17] and small-scale lightweight imagers are raising new possibilities for the thermography to get involved in home-based surgical site monitoring.

However, since no study has explored the role of thermography in diagnosing pin site infection, many issues are left unclarified, and there is no agreement on a standard method and equipment. Besides, the reliability and validity of this method for pin site surveillance are in question. The aim of this study was to investigate the capability of digital infrared thermography in timely detection of pin site infection in patients with external fixators and to determine its reliability, while we are seeking to gain more knowledge about the variations in temperature measurements by this method and sampling data for a prospective cohort study.

Materials and Methods

Study design: This is an explorative proof of concept study. Reporting is performed according to the proposed guidelines for reporting reliability and agreement studies (GRRAS).[18] The study was conducted in the orthopaedic outpatient clinic at Aalborg University Hospital in a period of 4 weeks and serves as pilot study for a future larger cross-sectional study.

Ethics

The Danish national ethics committee on request agreed on proceeding with this explorative pilot study without an ethical approval procedure.

Subjects

A consecutive series of 13 orthopaedic patients with external fixators were included during their routine outpatient post-operative visits.

Preparation

At least 10 minutes before performing thermography, all dressings were removed and the patients rested on the bed in the examination room without direct sunlight. The room temperature was 20°C and stable.

Clinical grading

All the patients were scored by the same orthopaedic surgeon. The examiner classified all pin sites according to the Modified Gordon Pin Infection Classification (Table 1, Figs 1A and 2A),[19] and if purulent drainage was observed, radiography would be performed to distinguish between grade 4, grade 5 and grade 6.

Figs 1A and B

This pin site was grade 0 according to Modified Gordon Pin Infection Classification as the redness around the pin site was judged as scarring and not inflammation. To the right is the corresponding thermographic image. Measurements were performed in the left-sided image and maximum temperature was registered inside the region of interest marked by the circle

Figs 2A and B

This pin site was grade 3 according to Modified Gordon Pin Infection Classification: the redness was judged as inflammatory erythema and serous drainage was present. To the right is the corresponding thermographic image and white colour represents temperature around 36.5°C

Thermography

Thermal imaging was performed after clinical grading with a digital infrared camera FLIR C3 at a hand-held estimated distance of 25 cm and the images were stored labelled with pin site and patient ID (Figs 1B and 2B). Table 2 shows the adjusted parameters of the device for imaging.

Table 2

Applied settings for FLIR C3 camera in the present study

Camera settings (FLIR C3, 720181832)
Emissivity	0.95
Reflection temperature	20°C
Distance	25 cm
Atmospheric temperature	As room temperature
Relative humidity	50% by default
External optics temperature	As room temperature
External optics transmittance	1

Temperature analysis

Analysis was done with the FLIR Tools software (version 5.13.1931.2002). The FLIR C3 camera obtained a normal digital image (JPEG) and a thermographic image with the infrared detector for each pin site snapshot. The analysis was done on the JPEG digital image (Fig. 1A) not to be biased by the information visible on the temperature colour scale in the thermal images (Fig. 1B). The temperature analysis was performed blinded without knowledge of the Modified Gordon Pin Infection Classification of each pin site.

Exploring the influence of camera positioning using two different ways of temperature analysis

Between six to fifteen images were taken of forty-two pin sites from different random angles, resulting in 357 pictures. These images were used to evaluate the influence of camera position on the measured temperature around the pin site. The maximum temperature of the region of interest around the pin site (local method) and the maximum temperature within the whole image (whole image method) were measured in all images (Fig. 3).

Figs 3A to D

Four images of the same pin site imaged at different angles. The images illustrate that the camera position influences the temperature in the region of interest (white circle) (local method) and the maximum temperature measured in the region (Max) (whole image method). Maximum temperature for the whole image is given in the upper right corner in each of the images

Intrarater agreement analysis

Intrarater agreement was evaluated by repeating the temperature analysis with the local method on a random sample of 53 of the pin sites. Analysis was performed with one week between measurements.

Statistics

Descriptive statistics were performed in Stata version 16.1 (Stata Corp, College Station, Texas, USA). Variances in temperature measurements across camera positions were calculated for the two measurement methods (local and whole image) while controlling for patient ID and pin site location. Variation in variances across methods was calculated using a mixed model method. The mean pin site temperature for each patient is the sum of all maximum pin site temperatures evaluated with the local method divided with number of the total number of pin sites for the patient. Infection (defined as Gordon grade 1 or above) was tested against thermography measures in a contingency table. Intraclass correlation coefficient (ICC) with 95% confidence intervals was calculated using a two-way mixed-effects model with absolute agreement.

Results

Thirteen (four females, nine males) consecutive patients with circular external fixation were included (age 9–72 years). Indications for frames were fracture (n = 4), deformity correction (n = 2), bone lengthening (n = 1) and bone transport (n = 6). Days from surgery to thermography ranged from 27 to 385 days. Two patients (patient 1 and 7) had two separate examinations on different dates (Table 3). Patient 1 had a 1-day interval between examinations and patient 7 had 28 days.

Table 3

Patient demographics where patients 1 and 7 had two separate examinations

Patient ID	Gender	Age (years)	Examination time (days postoperative)	Frame indication	Half pins/wire sites per frame	Segments
1	M	72	172 and 173	Bone transport	6/12	Tibia/foot
2	F	28	317	Deformity correction	5/6	Tibia
3	M	21	385	Bone transport	4/8	Tibia/foot
4	M	52	176	Bone transport	6/12	Tibia/foot
5	M	68	64	Fracture	3/8	Tibia
6	M	14	53	Fracture	5/4	Tibia
7	M	20	69 and 97	Bone transport	6/4	Tibia/foot
8	F	28	27	Fracture	4/20	Tibia
9	M	21	59	Fracture	3/8	Tibia/foot
10	M	48	311	Bone transport	7/12	Tibia/foot
11	M	41	276	Bone transport	8/12	Tibia/foot
12	M	42	139	Deformity correction	7/8	Tibia/foot
13	F	9	96	Bone lengthening	1/14	Tibia

Variance across random camera positions for the local method was 0.22 and for the whole image method was 0.32 (Fig. 4). Variation in variances across methods, while controlling for patient and pin site factors, was 0.17. The differences between maximum temperatures for each of the 42 different pin sites imaged at different angles and positions was on average 1.1°C (range: 0.2°C–3.1°C) for local method analysis and 1.5°C (range: 0.3°C–5.0°C) for whole image analysis. Based on this, the local method was chosen for comparison with infection grading as the precision was slightly lower (nonsignificant).

Fig. 4

Box and whisker plot, distribution of absolute differences from mean temperature (°C) within the same pin location for local and whole image methods

Pin site temperature versus infection grade

Of 231 included pin sites, 11 were identified with initial signs of infection (five sites grade 1, five sites grade 2 and one site grade 3) (Table 3). Mean pin site temperature varied between patients from 29.0°C to 35.4°C (mean 33.9°C) (Fig. 5). Mean temperature and range for pin sites with infection grade 0, 1, 2 and 3 were 32.8°C (26.3–37.3), 34.3°C (33.2–35.4), 34.8°C (33.0–35.9) and 36.1°C, respectively. Based on the mean temperature of infection grade 1, a cut-off value for infection was set to 34°C. Applying 34°C as cut-off value for infection, the sensitivity was 73% and the specificity was 67%. At the same time, positive predictive value was 10% and negative predictive value, 98%. Box plot distributions of measured pin site temperatures stratified with Modified Gordon Pin Infection Classification of the pin site into grade 0 and above are shown in Figure 6.

Fig. 5

Scatter plot of mean temperatures across all pin sites per examination against patient ID (n = 13). The plot demonstrates huge variation between individuals

Fig. 6

Box plot of the distribution of measured pin site temperatures stratified into grade 0 and above grade 0 (Modified Gordon Pin Infection Classification of the pin site). N (class 0) = 220, N (class ≥ 1) = 11

Reliability of thermography analysis with the local method

Intraclass correlation coefficient (ICC) for Intrarater agreement for thermography was calculated as 0.85 (0.77–0.92).

Discussion

We performed this pilot study to investigate the capability and reliability of digital thermography. We found that thermographic measurement using the FLIR camera seems to be a reliable tool, with an ICC of 0.85 and there appears to be a relationship between the degree of elevated temperature and the infection grade.

Our study is limited by several factors. A major limitation was few numbers of patients, as it was an explorative study on only 13 patients. Besides, due to the lack of a universally accepted definition for pin site infection, it is difficult to define a true pin site infection. We are left with the presence of classic signs or symptoms of infection around a pin or wire which are interpreted as pin site infection but may only represent inflammation.[2] We believe that the development of pin site infection represents a continuum starting with mild inflammatory signs that may proceed to classical signs and symptoms of infection (Figs 4 to 6). Therefore, we treat early and mild signs of inflammation such as serous drainage or erythema by increasing the frequency of pin site care. With the aim of using a grading system that was capable of including these early signs of inflammation, we applied the modified Gordon grading system.[19] A pragmatic definition of pin site infection could be a pin site with classical signs or symptoms of infection treated with antibiotics, pin removal or debridement.[20] However, in our case series, we did not have any patients with pin site infections requiring antibiotic treatment or pin removal and the temperature of such infections remains unknown.

Although thermal imaging has been widely used in different clinical conditions, there are controversial reports regarding the accuracy and reliability of thermography in the literature.[21-30] The factors influencing the temperature measurement are not fully examined yet. Numerous factors may cause deviations in the measurement of temperatures, such as the characteristics of thermal imaging equipment, measurement environment and technique, and internal factors such as differences in normal population like variations in blood circulation and skin colour tone. Currently, no agreement exists on standard measuring method and equipment, and it is not clear whether thermography can predict surgical site infection, especially in a telemedicine setting.

Our study demonstrated that measuring the temperature within a region of interest of the pin site must be performed manually, due to the fact that automatically, the calculation of the maximum temperature within the whole image by the software increased the variation significantly. Moreover, our pilot data showed some variations when images were obtained at different angles. However, the clinical significance of this observation is unknown yet. High variations in measurements are demonstrated by different studies,[31,32] and this variability is shown to be higher while using infrared camera, rather than infrared thermometry.[33]

It has been established that alteration in the surgical site temperature after operation follows a typical pattern, and the temperature decreases gradually, as the inflammation subsides. Romano et al. revealed that the temperature pattern after surgery had a peak of 4°C on day 3, which was highly correlated with the peak of CRP and ESR values, and followed a slow decrease afterwards, until day 90. However, in the case of infection, the temperature remained elevated with a mean differential temperature of 1.6°C (range 1.1–2.5°C) compared to the nonoperated knee. We did an explorative investigation on a patient 1 day after fracture surgery demonstrating a relatively high mean pin site temperature of 36.4°C. The data were not included in the study since they were performed just one day after fracture surgery during the time of hospitalization and not in the outpatient clinic as protocolled. The high temperature was obviously due to inflammation caused by recent trauma and surgery.

Our results showed that the mean pin site temperature varied from patient to patient, and therefore, the cut-off point could vary between different individuals. If 34°C would have been determined as the point of care, a positive predictive value of 10% and a negative predictive value of 98% were calculated. A prospective cohort study on thermal profiles over the patella tendon of athletes concluded that changes of temperature over 1.2°C for the individual over a period may indicate pathology;[34] however, an absolute cut-off value for point of care may not be appropriate and the patients rather need to serve as their own controls. In our study, two patients (nr. 1 and 7) had two independent examinations at different time intervals. Patient 1, with only one-day gap between images, showed remarkably similar mean temperatures with a difference 0.2°C. In contrast, the difference in mean temperature was 2.5°C in patient 7, with an interval of 28 days between images. Whether each pin site can serve as its own control must be evaluated in longitudinal studies. In a recent study, Childs et al. investigated the practicality of thermography in the evaluation of surgical site after Caesarean section in obese women, and by applying the difference between the surgical site and an abdominal reference region temperature, instead of a predetermined cut-off point for the temperature of the surgical site, they found 70% to 79% correct predictions for wound infection.[11] We did not investigate a reference region in our study, but for further studies, the contralateral leg could be useful as a reference.

We employed a basic hand-held thermography camera rather than expensive advanced and complicated cameras. It was a very deliberate choice we made, not to use expensive and sophisticated cameras, but rather we aimed for a low-tech solution for utilizing thermography in home-based surveillance of pin site infection. A recent study for the detection of arterial perforators showed that for clinical purposes, the differences between high-cost versus low-cost thermal imaging were negligible.[35] New point-of-care (POC) technologies are emerging rapidly, and the goal of these technologies is to facilitate immediate evidence-based medical decisions, improve patient outcomes and reduce patients’ afflictions.[36] Thermography is presently being integrated into or can be used as an add-on to regular smartphones. This is opening up new possibilities for home-nurses, relatives or patients to perform pin site surveillance. In the future, algorithms for home-based monitoring for infection in postoperative patients may be developed. However, further studies are needed to investigate how the variability of pin site temperature measurement can be reduced while using infrared camera thermography.

Conclusion

Measurements of pin sites using the hand-held FLIR C3 infrared camera was reliable with an Intrarater ICC of 0.85, and the temperature measures were related to infection grading; however, any valid conclusions on specificity or sensitivity could not be drawn from the data. Measurements of maximum temperature did show some variations in relation to camera positioning. Further studies will show if thermography can serve as an adjunct tool in home-based pin site infection surveillance.

Clinical Significance

Based on the findings in the present study, we are encouraged to proceed with a clinical prospective study aiming to establish the concept of home-based monitoring with thermography for pin site infection.