Clinical validation of an artificial intelligence-enabled wound imaging mobile application in diabetic foot ulcers.

There is a lifetime risk of 15% to 25% of development of diabetic foot ulcers (DFUs) in patients with diabetes mellitus. DFUs need to be followed up on and assessed for development of complications and/or resolution, which was traditionally performed using manual measurement. Our study aims to compare the intra- and inter-rater reliability of an artificial intelligence-enabled wound imaging mobile application (CARES4WOUNDS [C4W] system, Tetsuyu, Singapore) with traditional measurement. This is a prospective cross-sectional study on 28 patients with DFUs from June 2020 to January 2021. The main wound parameters assessed were length and width. For traditional manual measurement, area was calculated by overlaying traced wound on graphical paper. Intra- and inter-rater reliability was analysed using intra-class correlation statistics. A value of <0.5, 0.5-0.75, 0.75-0.9, and >0.9 indicates poor, moderate, good, and excellent reliability, respectively. Seventy-five wound episodes from 28 patients were collected and a total of 547 wound images were analysed in this study. The median wound area during the first clinic consultation and all wound episodes was 3.75 cm2 (interquartile range [IQR] 1.40-16.50) and 3.10 cm2 (IQR 0.60-14.84), respectively. There is excellent intra-rater reliability of C4W on three different image captures of the same wound (intra-rater reliability ranging 0.933-0.994). There is also excellent inter-rater reliability between three C4W devices for length (0.947), width (0.923), and area (0.965). Good inter-rater reliability for length, width, and area (range 0.825-0.934) was obtained between wound nurse measurement and each of the C4W devices. In conclusion, we obtained good inter-rater and intra-rater reliability of C4W measurements against traditional wound measurement. The C4W is a useful adjunct in monitoring DFU wound progress.

INTRODUCTION

Diabetes mellitus (DM) is debilitating chronic disease with a prevalence of 6.28% of the world's population in 2017. Patients with DM have a 15% to 25% lifetime risk of developing diabetic foot ulcers (DFUs). DFU is associated with significant morbidity and mortality, with major amputation rates ranging between 19.9% and 39%. , , , , , It is therefore prudent for proper management of DFUs.

Management of DFUs is complex and requires a multidisciplinary team of doctors, nurses, and allied health professionals for successful management. Wound care is an important facet of the DFU “care bundle”, consisting of wound assessment, monitoring, and management. Traditionally, wound assessment and monitoring are performed by specialised and trained wound nurses. With advancements in technology, until now there are several commercially available wound assessment or monitoring systems available for monitoring of chronic wounds. We recently conducted a systematic review on wound imaging modalities and concluded the following: (a) paucity of existing studies evaluating the efficacy of wound assessment and imaging systems and (b) bias in existing studies evaluating the effectiveness of wound imaging systems with lack of sample size calculation. Though they are several commercially available wound assessment systems, a majority of them have not been reviewed in the literature on measurement accuracy, especially because DFU may occur on curves or angles on the foot. Hence, the aim of this study is to add on to existing literature on commercially available wound imaging systems, by clinically validating an artificial intelligence‐enabled wound imaging mobile application (CARES4WOUNDS [C4W] system, Tetsuyu, Singapore) against traditional wound assessment measurements by a trained specialist wound nurse, for patients with DFU.

MATERIALS AND METHODS

This is a prospective cross‐sectional study on patients with DFUs from June 2020 to January 2021 in a single‐centre tertiary hospital. Inclusion criteria were all patients aged 21 years and above with DFUs. Exclusion criteria were patients who were pregnant, breastfeeding, had non‐DFUs such as primarily venous ulcer or neuropathic ulcer, or did not have capacity to consent. This study was approved by a local institutional review board (National Healthcare Group Domain Specific Review Board Ref No: 2019/00837). Written consent was obtained for all patients who were included in our study with appropriate translations as required for non‐English speakers.

Tetsuyu C4W wound imaging solution

The C4W system is intended to be used for wound imaging, automatic measurement of wound dimensions, and tissue classification to aid in the assessment of pressure ulcers, diabetic ulcers, venous ulcers, and problem healing wounds. It is a non‐contact, digital wound assessment, and documentation tool aimed to provide a standardised approach to the monitoring of wound healing progress by medical professionals. It consists of a software application that is used in combination with commercial off‐the‐shelf (COTS) mobile computing platforms (iPad with 3D structure sensor and iPhone with dual camera) and a web‐based software application that is tailored to the mobile platforms. The mobile application uses locked software algorithms to automatically measure the dimensions of wounds, detect wound bed, and classify the tissue types based on epithelisation, granulation, necrotic, and slough. The data output is documented in real time in accordance with the current state‐of‐the‐art clinical workflow to assure consistency and accuracy in wound assessment. The patients' data are accessible remotely through a secured web‐based portal to improve overall patient management.

Study protocol

Our study protocol is shown in Figure 1. Eligible patients were identified in both inpatient and outpatient settings. Baseline demographics and clinical profile were collected prior to the commencement of the study. The approximate study duration for each patient is five visits or until complete resolution of the ulcer, whichever was earlier. All patients who were included in the study were subjected to a standardised DFU management pathway with standardised follow‐up; no additional clinic visits were required for the purpose of this study. During each clinic visit, wound measurements were recorded traditionally by a trained specialised wound nurse and electronically by a dedicated research coordinator using the C4W imaging system. An additional digital wound image capture with a reference ruler was also taken independently with a Canon (PowerShot G7X Mark II) digital camera. A wound episode was defined as any clinic consultation for this study. The total number of wound images is calculated by the number of images taken by each of the C4W imaging system (ie, each wound episode should result in nine wound images, with three wound images taken per device).

FIGURE 1

Study protocol for participation recruitment and standardisation of process for wound measurement

Standardisation of ulcer measurement

For patients with multiple DFUs, an index ulcer was identified for the purpose of the study during the first clinic visit, which will be monitored during subsequent clinic visits. Measurement of ulcer was conducted in a dedicated room with adequate lighting prior to clinic consultation with the doctor. All participants were positioned sitting on a chair with their feet overhanging. Wound measurements (length and width) were taken traditionally by a specialised trained wound nurse first (Figure 2A) and subsequently taken using the C4W application by a dedicated research coordinator. Method of wound measurement was standardised between the wound nurses prior to the conduct of the study: placement of tracing over wound, followed by the use of sterile marker pen to outline the wound on the tracing paper. The length (defined as the longest axis in the wound box) and width (defined as the longest axis perpendicular to the length in the wound box) were measured using the traced wound (Figure 2B). Area computation was conducted by overlaying the tracing paper on a graphical paper, in which squares were counted corresponding to the area covered by the wound.

FIGURE 2

Foot wound from a patient: A, manual measurement of the wound parameters by a trained wound nurse; B, schematic diagram for determination of length and width; C, application interface of the Tetsuyu CARES4WOUNDS system on an iPhone 11 Pro (C4Wi version 1, build 1)

Digital measurement of the wound was performed by a dedicated research coordinator with the C4W application (Version 1, build 1) installed on three different iPhone (Apple Inc., California) device models: device 1 was an iPhone 8 Plus, device 2 was an iPhone 11 Pro, and device 3 was an iPhone XS. All of the iPhone devices were running on the iOS 13.0 operating system. A novel feature of C4W is the ability to capture wound images and measure wound dimensions using software algorithms via an application on conventional Apple iPhone/iPad devices (Figure 2C). This widens the application of C4W to more clinical scenarios and removes the need for attachment cameras. An optical zoom of ×1.0 was used to capture the images at a distance of approximately 20 cm from the wound. Three images of the same wound were taken from each C4W device. Each repeat image of the same wound involved repositioning of the patient and the research coordinator. This was repeated across the three different devices. The parameters measured (length, width, and area) were automatically calculated based on the image boundaries determined by the imaging system. For images where automatic boundary detection was vastly different from the actual boundaries, manual adjustments were made to ensure accurate detection of the wound boundaries. These instances occurred when there was poor colour contrast with the patients' underlying skin tone, were too small (<1 cm), or were located in areas of large variation in contours (eg, bony prominences such as the malleolus).

Sample size calculation

Sample size was calculated as per number of wound images rather than the number of subjects as this is a cross‐sectional study on wound imaging. Based on Tetsuyu's internal validation, the baseline mean accuracy is 90%. Hence, assuming baseline correction (R 0) at 0 and alternative correlation (R 1) at 0.2, sample size required for one correlation test with power 90% and α 0.05 is 258 wound images. Intra‐rater reliability will be evaluated by comparing three different C4W measurements against the same wound with intra‐class correlation (ICC) statistics. Hence, assuming baseline correlation (R 0) at 0 with alterative correction (R 1) at 0.2, sample size required for ICC with power 90% and α 0.05 is 83. In our institution with an average of 20 and 30 DFU wound inspections in outpatient and inpatient setting, respectively, per week, our target sample size was 341 wound images.

Statistical analysis

All statistical analyses were performed with SPSS version 25.0 (SPSS Inc., Chicago, Illinois). Statistical significance was determined by P < .05. ICC statistics was used to analyse intra‐rater and inter‐rater reliability. Intra‐rater reliability between measurements taken by the same C4W device and inter‐rater reliability between the wound nurse measurements and each C4W device were analysed using two‐way mixed‐effects model, absolute agreement, and single measure. Inter‐rater reliability between the C4W devices was analysed using two‐way random‐effects model, absolute agreement, and single measure. Two‐way random‐effects model was used for inter‐rater reliability between the C4W devices for generalising our results for all existing C4W devices in the market.

Although there is no standard definition or cut‐offs available for ICC to determine the extent of reliability, we have defined ICC values as the following: <0.5 indicates poor reliability, between 0.5 and 0.75 indicates moderate reliability, between 0.75 and 0.9 indicates good reliability, and >0.9 indicates excellent reliability.

RESULTS

Patient baseline characteristics

The median age of the study population was 60 (IQR 52.5‐66) with a male predominance (n = 24, 85.7%). Common comorbidities include DM (n = 28/28, 100%), hypertension (n = 24/28, 85.7%), chronic kidney disease (n = 20/28, 71.4%), and peripheral vascular disease (n = 18/28, 64.3%). Table 1 summarises the overall patient demographics.

TABLE 1

Clinical profile of patients included in the study

	Number of patients (n = 28)
Age	60 (52.5–66)
Gender, male	24 (85.7)
Smoking, yes	7 (25)
Ethnicity
Chinese	17 (60.7)
Malay	2 (7.1)
Indian	9 (32.1)
Comorbidities
Diabetes mellitus	28 (100)
Hypertension	24 (85.7)
Coronary artery disease	3 (10.7)
Chronic heart failure	4 (14.3)
Chronic kidney disease	20 (71.4)
Cerebrovascular accident	4 (14.3)
Chronic obstructive pulmonary disease	0 (0)
Peripheral vascular disease	18 (64.3)
Previous ulcer	7 (25)
Previous surgical debridement	11 (39.3)
Previous revascularisation	7 (25)
Previous amputation (minor/major)	6 (21.4)
Location of ulcer
Heel	4 (14.3)
Dorsum	7 (25)
Sole	9 (32.1)
Toes	8 (28.6)
Wound parameters taken at the first visit a
Length, cm	3.00 (1.63–6.03)
Width, cm	2.35 (1.10–3.88)
Area, cm2	3.75 (1.40–16.50)

Note: All categorical variables are expressed as n (%) unless otherwise specified. All continuous variables are expressed as median (interquartile range) unless otherwise specified.

Wound area was calculated based on the manual measurements taken by the wound nurse.

Wound images

There was a total of 75 wound episodes from 28 patients. A total of 547 wound images were analysed in this study. One‐hundred twenty‐eight images (not included in the above count) were excluded in this analysis in view of inaccurate data: overestimated wound boundaries and inaccurate detection of wound due to unclear wound boundaries. Table 2 summarises the overall baseline characteristics of all included wound episodes. There is excellent intra‐rater reliability of C4W on three different image captures of the same wound for length (intra‐rater reliability ranging 0.956‐0.993 for three different devices), width (intra‐rater reliability ranging 0.933‐0.963 for three different devices), and area (intra‐rater reliability ranging 0.984‐0.994). Table 3 summarises the intra‐rater reliability of three different images taken from the same C4W device.

TABLE 2

Baseline characteristics of wound episodes

	Number of wound episodes (n = 75)
Age	61 (49–66)
Gender	70 (93.3)
Smoking, yes	23 (30.7)
Ethnicity
Chinese	51 (68)
Malay	4 (5.3)
Indian	20 (26.7)
Comorbidities
Diabetes mellitus	75 (100)
Hypertension	67 (89.3)
Coronary artery disease	10 (13.3)
Chronic heart failure	14 (18.7)
Chronic kidney disease	53 (70.7)
Cerebrovascular accident	12 (16)
Chronic obstructive pulmonary disease	0 (0)
Peripheral vascular disease	50 (66.7)
Previous ulcer	33 (44)
Previous surgical debridement	35 (46.7)
Previous revascularisation	26 (34.7)
Previous amputation (minor/major)	23 (30.7)
Location of ulcer
Heel	10 (13.3)
Dorsum	18 (24)
Sole	21 (28)
Toes	26 (34.7)
Wound parameters a
Length, cm	2.60 (1.00–5.70)
Width, cm	1.40 (0.50–3.30)
Area, cm2	3.10 (0.60–14.84)

Note: All categorical variables are expressed as n (%) unless otherwise specified. All continuous variables are expressed as median (interquartile range) unless otherwise specified.

Wound area was calculated based on the manual measurements taken by the wound nurse.

TABLE 3

Intra‐rater reliability of the same Testuyu device on three different images obtained from the same wound

	Measurements, mean ± SD
	Image 1			Image 2			Image 3			Intra‐rater reliability (95% CI)			P value
	Length	Width	Area	Length	Width	Area	Length	Width	Area	Length	Width	Area	Length	Width	Area
Device 1	4.12 ± 3.46	2.50 ± 1.86	9.30 ± 12.00	3.79 ± 3.25	2.37 ± 1.76	8.68 ± 10.90	3.77 ± 3.31	2.33 ± 1.77	8.48 ± 11.01	0.956 (0.931–0.973)	0.946 (0.917–0.967)	0.984 (0.974–0.990)	<.001	<.001	<.001
Device 2	4.15 ± 3.39	2.49 ± 1.76	9.81 ± 11.33	4.06 ± 3.35	2.48 ± 1.79	9.58 ± 11.07	4.07 ± 3.36	2.58 ± 2.00	9.72 ± 11.33	0.993 (0.989–0.995)	0.933 (0.901–0.957)	0.994 (0.991–0.996)	<.001	<.001	<.001
Device 3	4.32 ± 3.62	2.48 ± 1.73	10.09 ± 11.66	4.20 ± 3.40	2.44 ± 1.72	9.86 ± 11.42	4.20 ± 3.41	2.54 ± 1.87	9.86 ± 11.28	0.984 (0.976–0.990)	0.963 (0.944–0.977)	0.994 (0.991–0.996)	<.001	<.001	<.001

Note: Length and width were expressed in cm, while area was expressed in cm2.

Table 4 summarises the inter‐rater reliability between the three C4W devices; we obtained excellent inter‐rater reliability between the three C4W devices for length (inter‐rater reliability 0.947 [95% CI: 0.923–0.964, P < .001]), width (inter‐rater reliability 0.923 [95% CI: 0.890–0.948, P < .001]), and area (inter‐rater reliability 0.965 [95% CI: 0.949–0.977, P < .001]).

TABLE 4

Inter‐rater reliability of the three different Tetsuyu devices on the average measurements across all three images taken on each device

	Device 1			Device 2			Device 3			Inter‐rater reliability			P value
	Length	Width	Area	Length	Width	Area	Length	Width	Area	Length	Width	Area	Length	Breath	Area
Image	3.13 ± 3.21	1.95 ± 1.78	7.23 ± 10.35	3.40 ± 3.38	2.09 ± 1.88	7.98 ± 10.75	3.35 ± 3.47	2.01 ± 1.83	7.85 ± 10.81	0.947 (0.923–0.964)	0.923 (0.890–0.948)	0.965 (0.949–0.977)	<.001	<.001	<.001

Note: Length and width were expressed in cm, while area was expressed in cm2.

When comparing the manual measurements obtained by trained wound nurse with each of the C4W devices, we obtained good inter‐rater reliability for length (range 0.825–0.934), width (range 0.825–0.930), and area (range 0.872–0.932). The inter‐rater reliability values between the trained wound nurse and each of the C4W devices are summarised in Tables 5, 6, 7. When comparing the wound area calculated manually to each of the C4W devices, the areas are greater by 25.2%, 13.4%, and 15.3% for device 1, 2, and 3, respectively.

TABLE 5

Inter‐rater reliability between wound nurse and Tetsuyu device 1 for the corresponding image

	Measurements, mean ± SD			Inter‐rater reliability			P value
	Length	Width	Area	Length	Width	Area	Length	Width	Area
Device 1	3.13 ± 3.21	1.95 ± 1.78	7.23 ± 10.35	0.825 (0.714–0.892)	0.825 (0.737–0.886)	0.872 (0.794–0.920)	<.001	<.001	<.001
Wound nurse	3.85 ± 3.49	2.12 ± 1.86	9.05 ± 12.35

Note: Length and width were expressed in cm, while area was expressed in cm2.

TABLE 6

Inter‐rater reliability between wound nurse and Tetsuyu device 2 for the corresponding image

	Measurements, mean ± SD			Inter‐rater reliability			P value
	Length	Width	Area	Length	Width	Area	Length	Width	Area
Device 2	3.40 ± 3.38	2.09 ± 1.88	7.98 ± 10.75	0.934 (0.885–0.961)	0.930 (0.892–0.955)	0.932 (0.893–0.957)	<.001	<.001	<.001
Wound nurse	3.85 ± 3.49	2.12 ± 1.86	9.05 ± 12.35

Note: Length and width were expressed in cm, while area was expressed in cm2.

TABLE 7

Inter‐rater reliability between wound nurse and Tetsuyu device 3 for the corresponding image

	Measurements, mean ± SD			Inter‐rater reliability			P value
	Length	Width	Area	Length	Width	Area	Length	Width	Area
Device 3	3.35 ± 3.47	2.01 ± 1.83	7.85 ± 10.81	0.915 (0.857–0.948)	0.908 (0.858–0.941)	0.923 (0.878–0.952)	<.001	<.001	<.001
Wound nurse	3.85 ± 3.49	2.12 ± 1.86	9.05 ± 12.35

Note: Length and width were expressed in cm, while area was expressed in cm2.

DISCUSSION

Advancements in technology permit the use of adjuncts to improve efficiency of clinical care. Innovation is also ongoing to improve user experience and reduce difficulty in setting up of imaging systems for clinical use. For instance, prior version of the C4W imaging system required the use of a three‐dimensional (3D) sensor attachment on a tablet device (iPad) for measurement. With improvements in mobile technology and design, depth sensors are integrated into mobile devices, permitting 3D measurements with depth perception. Hence, this study aims to validate the new C4W imaging system. Our study demonstrated high intra‐rater reliability between devices and inter‐rater reliability between the C4W imaging system and traditional wound measurements.

Our recent systematic review until March 2020 summarised existing literature on the use of wound assessment, imaging, and monitoring systems in DFUs; of which, we identified 17 articles (5 on computer applications or handheld devices, 2 on mobile interfaces, 2 on optical imaging, 4 on spectroscopy or hyperspectral imaging, and 4 on artificial intelligence). The C4W imaging system allows easy use on mobile and/or handheld devices. In our study, this application was used on iPhone 11 Pro. Yap et al in 2018, who analysed the use of the application FootSnap (Manchester Metropolitan University, Manchester, UK) on DFUs, demonstrated high intra‐rater reliability for operator 1 (mean Jaccard Similarity Index [JSI] 0.89 [range 0.84–0.93]) and operator 2 (mean JSI 0.91 [range 0.84–0.95]). Inter‐rater reliability was also high (mean JSI 0.89 [range 0.83–0.94]). Comparatively, in our study, we demonstrated excellent (ICC > 0.900) intra‐rater reliability for each of the C4W imaging system and inter‐rater reliability between each of the C4W imaging systems.

Our systematic review also described the list of commercially available wound assessment products listed on WoundSource, a reference guide on wound care products, for a list of commercially available wound assessment and monitoring tools available for DFUs. Of which, six of the identified 18 wound assessment products have been described and/or validated in the literature: EPISCAN I‐200 (Longport, Inc., Pennsylvania), Scout (WoundVision, Indiana), Silhouette (ARANZ Medical Ltd., Christchurch, New Zealand), Swift Skin and Wound (Swift Medical, Ontario, Canada), WoundMatrix (WoundMatrix, Inc., Pennsylvania) and WoundZoom (WoundZoom, Inc., Wisconsin). , , , , , , , , Khong et al evaluated the use of WoundAide (Konica Minolta Business Solutions Asia Pte Ltd, Tokyo), an application that uses image segmentation and enhancement techniques. The use of C4W imaging system in our study was non‐inferior compared to their study (which assessed the use of WoundAide, WoundZoom (WoundZoom, Inc., Wisconsin) and Visitrak (Smith & Nephew, London, UK); we similarly demonstrated ICC of >0.900 for both intra‐rater and inter‐rater reliability. Unfortunately, we were unable to compare the ICC of the inter‐rater reliability of the manual measurements taken by a trained wound nurse compared to the wound imaging system with their study.

Previous literature has interestingly showed that wound measurements by standard ruler results in overestimation of wound area (calculated by length × width) by up to 41%. This is an expected finding as the simple multiplication of length and width is unable to account for wounds with irregular surface area. Our study similarly shows overestimation of the calculated wound area from manual measurements compared to the C4W devices (ranging from 13.4 to 25.2%). However, good inter‐rater reliability was demonstrated for device 1 (ICC 0.872 (95% CI: 0.794–0.920) and excellent inter‐rater reliability for device 2 (ICC 0.932 (95% CI: 0.893–0.957), and device 3 (ICC 0.923 [95% CI: 0.878–0.952]). This implies that despite the seemingly significant overestimation of wound area based on the multiplication of length and width, this overestimation is not significant when performed over a large number of wound images. An additional advantage conferred by the use of digital image capture is its ability to calculate the wound parameters in a shorter time. Unfortunately, we did not collect on time taken as our study was primarily focused on assessing the intra‐ and inter‐rater reliability of traditional wound measurement vs digital image capture with the C4W system.

The C4W imaging system is an adaptable software, which uses the native camera system of handheld device. Users do not have to physically install another 3D sensor to the handheld device prior to its use, providing convenience. The accuracy of the C4W imaging system is dependent on the camera of the handheld device itself. Though this implies that usage of older models of handheld devices with less advanced camera features will result in lower measurement accuracy, with advancements in camera systems and improvement in depth perception such as in the new iPhone 12 Pro Max, the C4W imaging system is able to utilise the improved camera system for better accuracy.

Our study exhibits the following strengths; we assessed the use of multiple parameters in wound assessment (length, width, and area) in the wound imaging system, which is not present in some of the existing reviews of wound imaging products. For instance, the studies by Jeffcoate et al in 2017 and Yap et al in 2018 assessed cross‐sectional area and perimeter/surface area only, respectively. , In addition, our study compared the use of the C4W imaging system with manual measurements taken by a trained wound nurse, the traditional method of wound measurement. We also standardised the method of manual measurement prior to the study to reduce bias. Similarly, only one research coordinator was involved in the digital image capture using the C4W devices, eliminating the risk of bias.

There are however, limitations to our study. Our study is limited to the quantitative characteristics of the C4W wound imaging system (length, width, and area). We were not able to evaluate other quantitative characteristics such as depth and undermining of wounds. Qualitative characteristics of the wound (such as the presence of exudates, eschar, and oedema) were only assessed by the wound nurse and not the C4W imaging system.

CONCLUSIONS

Our study demonstrates excellent inter‐rater and intra‐rater reliability of C4W measurements against traditional wound assessment by a trained specialist wound nurse, with C4W being a useful adjunct in monitoring DFU wound progress. There are technical limitations to the image capture resulting inaccurate detection of wound boundaries that require manual adjustment. Nevertheless, these limitations will improve with advancements in camera technology and machine learning, and they will serve as useful clinical adjuncts in monitoring of chronic DFUs. More studies are however required to assess the other commercially available wound imaging products.

CONFLICT OF INTEREST

The authors do not have any conflicts of interest.