Development and Usability Evaluation of a Telemedicine System for Management and Monitoring of Patients with Diabetic Foot.

OBJECTIVE: The aim of the present study was to develop and evaluate the usability of a telemedicine system for management and monitoring of patients with diabetic foot.
METHODS: This study was conducted in four phases. In the first phase, the information needs and characteristics required to design the telemedicine system were identified based on a literature review. Then, in a two-stage Delphi survey, 15 experts approved the identified information needs and characteristics. The prototype telemedicine system was then designed. In the third phase, system usability was evaluated through a semi-structured interview. In the fourth phase, users' satisfaction with the designed system was analyzed.
RESULTS: Out of 115 information needs and required characteristics, 95 were considered in the system design. Eight main pages for enabling patient-physician interactions and physician-physician interactions, monitoring the patient and controlling the disease process, providing medical consultation, and prescribing medications were considered. In the third phase, 26 distinct problems were identified. However, 75% of the participants were very satisfied with the system.
CONCLUSIONS: This study presents an attempt to design and evaluate a telemedicine system for the management and monitoring of patients with diabetic foot. In this system, patients receiving medical services or physicians who encounter rare cases can send the complete medical history, clinical test results, and videos and images related to the foot to specialist physicians. After examining the medical history or images and videos, the physician can provide the necessary medication prescriptions and laboratory tests or other recommendations.

I. Introduction

Diabetic foot, which is one of the most important complications of diabetes [1], affects about 15% to 25% of patients with diabetes [2]. This complication can be influenced by various factors, including neuropathy, peripheral artery disease, foot deformity infection, ulcers, and gangrene, and it is considered to be a major source of morbidity and mortality [3]. The diagnosis, management, and follow-up of these diabetic foot-related complications require clinical examinations, regular monitoring, and frequent contact with health care providers [4].

Telemedicine technology has been introduced as a potential method to provide follow-up care for patients with diabetic foot ulcers in recent years [5]. This technology maintains high standards of ulcer care, reduces the cost of ulcer healing, and improves the quality of life of patients with chronic ulcers [6]. It also removes some barriers to frequent screening and monitoring, such as long distances, time constraints, and referrals to medical centers. It has been proven that this technology can play a major role in reducing amputations in patients with diabetic foot [7]. Wilkins et al. [8] showed that 98.2% of patients with ulcers healed when care was provided through telemedicine, since telemedicine for ulcer care allows patients to receive faster counseling about the ulcer healing process.

In some countries, medical centers are located very far from rural areas [9], and visiting medical centers in person for people with diabetic foot is very difficult both in terms of distance and cost [10]. Furthermore, people living in different parts of these countries are faced with various health challenges, such as a limited number of centers and specialized physicians providing medical and health services to patients with diabetes and diabetic foot [11], difficult access to specialized physicians [12], low health literacy and knowledge [13], poorly equipped health facilities, and poor economic status [14]. These challenges adversely affect the quality of services. Considering these challenges, the aim of this study was to develop and evaluate the usability of a telemedicine system for the management and monitoring of patients with diabetic foot.

II. Methods

This study was conducted in the following four phases.

1. Phase 1: Identifying Information Needs

A comprehensive review was conducted in the IEEE, PubMed, Web of Science, and Scopus databases to identify information needs for diabetic foot telemedicine system design. The keywords and search strategies used in these databases are listed in Table 1.

Table 1

Keywords and search strategy

Keyword categories	Keywords
1	(diabetes* OR type2 diabetes OR type1 diabetes OR neuropathy OR diabetic neuropathies)
2	(diabetic foot OR diabetic feet OR diabetic neuropathies OR foot ulcer OR plantar ulcer OR neuropathic ulcers OR ischemia OR wounds OR osteomyelitis)
3	(telehealth OR telemedicine OR telemonitoring OR tele wound care OR telehomecare OR e-health OR smart phone OR mobile health OR wearable electronic devices)
Search strategy	[(1) AND (2) AND (3)]

Information needs were extracted from the studies using a data extraction form, and a questionnaire was developed based on these needs (5-point Likert scale). Then, to confirm these information needs, the questionnaire was distributed to experts in a two-round Delphi process. Since the participation of 15 to 20 experts is considered adequate in most Delphi studies [15], invitations were sent to 20 experts who were affiliated with Kerman University of Medical Sciences (KUMS). Fifteen experts accepted the invitation; the participants included specialists and subspecialists in endocrinology, physical medicine and rehabilitation, infectious diseases, and surgery, as well as general practitioners and medical informatics experts specializing in telemedicine.

The questionnaire was distributed and collected in person in two rounds at an interval of 1 month. The questionnaire consisted of two sections: the first section included participants’ demographic information and the second section included 105 questions related to information needs. The validity of the questionnaire was confirmed in accordance with the views and opinions of five experts in the fields of physical medicine and rehabilitation and surgery. The reliability of the questionnaire was evaluated by Cronbach’s alpha (0.952). To make decisions about each information need, thresholds regarding the agreement level were applied. Information needs with less than 50% agreement were removed; those with 50%–75% agreement were re-assessed in the second round of the Delphi survey, and information needs with at least 75% agreement were considered in the development of the telemedicine software without the need for re-assessment in the second round of the Delphi survey [16]. We used descriptive statistics and frequency distribution in SPSS version 23 (IBM Corp., Armonk, NY, USA).

2. Phase 2: Development of the Telemedicine System

Four 7-hour sessions were held electronically and in person among researchers to finalize the information needs. Then, a prototype of the telemedicine software was developed with the ASP programming language in Visual Studio 2019. A SQL database was designed to record the data of both patients and physicians. In order to develop this system, a team was formed comprising programmers (two software engineers fluent in ASP programming), system analysts, and a database designer. In order to ensure the security of the designed system, we used the Secure Sockets Layer (SSL) protocol [17,18].

3. Phase 3: System Usability and Troubleshooting

Five experts (experts in physical medicine and rehabilitation and medical informatics, as well as general practitioners) affiliated with KUMS were invited. Four of them agreed to participate.

The appropriate time and place for a one-on-one interview with each participant were coordinated. The software related to the designed system was installed on the participants’ computers. They were then asked to use the system in the roles of a patient with diabetes and a specialist in different scenarios. In the first scenario, participants registered as a patient in the system, and then entered information about their medical history, paid for the visit, selected a physician to continue the treatment process, and finally, sent the necessary information to the physician. In the second scenario, the physician retrieved and reviewed information about the patient’s medical history, prescribed the necessary medications, and provided appropriate treatment recommendations. In the next scenario, two physicians interacted with each other, shared patient information, and provided appropriate medical advice.

Semi-structured one-on-one interviews were then conducted and recorded. Each interview lasted 35 to 50 minutes. An interpretive approach was used to analyze the collected data after the end of each interview session; one of the researchers (KM) transcribed all the recorded conversations on paper after listening to the interview several times. The transcript of each interview was then carefully reviewed and coded. Finally, similar codes were classified as themes and sub-themes.

4. Phase 4: Satisfaction with the System

In this phase, the level of satisfaction with the system of the participants in the previous step was evaluated using a researcher-made questionnaire. The validity of this questionnaire was confirmed according to the opinion of two medical informatics specialists. Participants completed the questionnaire after using the system. The physical medicine and rehabilitation specialists and general practitioners visited an average of two patients using the telemedicine system. During these visits, patients presented their medical history to the physicians and uploaded images and videos related to their foot ulcers and the results of their laboratory tests. After receiving the information sent by the patients, the physicians prescribed medications or provided medical advice. A physician-physician relationship was also established between general practitioners for consultation. The results of this section were analyzed using SPSS version 23 (IBM SPSS, Armonk, NY, USA).

5. Ethical Considerations

This study was approved by the ethical committee of Kerman University of Medical Sciences (No. IR.KMU.REC.1399.038). All methods of the present study were performed in accordance with the relevant guidelines and regulations of the ethical committee of KUMS. All study participants signed a hard copy informed consent approved by the Student Research Committee prior to participation in the study.

III. Results

The results of the four phases of the study are presented below.

A total of 664 articles were retrieved from databases. After removing duplicate articles, another 501 articles were carefully screened. Finally, 89 articles were included in the review. Then, 115 information needs were identified from these articles.

The demographic characteristics of participants in the Delphi survey in this phase are presented in Table 2.

Table 2

Demographic characteristics of the participants for identifying information needs

Variable	Frequency (%)
Sex
Male	12 (80.0)
Female	3 (20.0)
Age (yr)
36–45	6 (40.0)
46–55	7 (46.7)
>55	2 (13.3)
Level of education
Specialist	9 (60.0)
Subspecialist	3 (20.0)
Specialty
General practitioners and PhD in medical informatics	3 (20.0)
Endocrinologist	2 (13.3)
Physical medicine and rehabilitation	3 (20.0)
Surgery	5 (33.3)
Infectious diseases	2 (13.3)
General practitioners and medical informatics	3 (20.0)
Work experience (yr)
1–10	7 (46.7)
11–21	6 (40.0)
>21	2 (13.3)

After removing irrelevant and similar items (10 items) from the 115 identified information needs, 105 items were reviewed by experts. Finally, 75 information needs were confirmed. We categorized these requirements along six axes: demographic information; medical history; present illness; laboratory tests, symptoms, and signs; treatment measures; and lifestyle. Table 3 presents the necessity of these elements in the first and second rounds of the Delphi survey.

Table 3

Information needs for designing the telemedicine system

Axis	Row	Name of information need	First round of Delphi	Second round of Delphi
Demographic information	1.	First name	×	*
	2.	Last name	×	*
	3.	Date of birth	×	*
	4.	Gender	×	*
	5.	Record number	×	*
	6.	Height	×	*
	7.	Weight	×	*
History	8.	Type of diabetes (type 1, type 2, and gestational diabetes)	*
	9.	Duration of suffering from diabetes	*
	10.	Duration of suffering from foot ulcers	*
	11.	Presence of foot ulcer	*
	12.	Duration of foot ulcer	*
	13.	Amputation	*
	14.	Laterality of amputation (left, right, or both)	*
	15.	Anatomical site of amputation	*
	16.	History of surgery	*
	17.	Leg undergoing surgery (left, right, or both)	*
	18.	Anatomical sites of surgery	*
	19.	Debridement	*
	20.	Gangrene	*
	21.	Critical ischemia	*
	22.	Peripheral neuropathy	*
	23.	Osteomyelitis	*
	24.	Calluses	*
	25.	Charcot foot	*
	26.	Underlying diseases (for example cardiovascular diseases, retinopathy, nephropathy, etc.)	*
	27.	Medications used	*
	28.	Smoking and alcohol consumption	*
	29.	Hospitalization history	*
	30.	Date of last hospitalization	*
	31.	Date of last foot examination	*
	32.	Number of foot examinations during the last month	*
Present illness	33.	Existence of ulcers on the feet	*
	34.	Laterality of foot ulcer (left, right or both)	*
	35.	Number of ulcers on the foot	*
	36.	Ulcer length	*
	37.	The exact site of the ulcers on the foot	*
	38.	Severity and depth of ulcer	*
	39.	Smell of ulcers	*
	40.	Numbness in the feet	*
	41.	The exact site of numbness in feet	*
	42.	Infection in the feet	*
	43.	Infected foot (left, right or both)	*
	44.	Exact sites of infection in the feet	*
	45.	Swelling, blisters and redness on the feet	*
	46.	Laterality of swelling, blistering, and redness of the foot (left, right, or both)	*
	47.	Exact sites for swelling, blisters and redness on the feet	*
	48.	Pressure on the sole of foot	*
	49.	Dry feet	*
	50.	Feet temperature	*
Laboratory tests, symptom and sign	51.	Fasting blood sugar (FBS)	*
	52.	Random blood sugar (RBS)	*
	53.	HbA1c (hemoglobin A1c)	*
	54.	Diastolic blood pressure	*
	55.	Systolic blood pressure	*
	56.	Respiratory rate	*
	57.	Heart rate	*
	58.	Oxygen saturation	*
	59.	Disorders and blood lipid levels	×	*
	60.	Pathological tests	*
Therapeutic measures	61.	Prescription drugs	*
	62.	Drug dosage	*
	63.	Laboratory tests	*
	64.	Washing and bandage of ulcer	*
	65.	Amputation	*
	66.	Surgery	*
	67.	Laser therapy	*
	68.	Rehabilitation	*
	69.	Educating patients	*
	70.	Providing medical recommendations to patients	*
	71.	Debridement	*
	72.	Prescribing medical shoes	*
	73.	Determining the date of the next examination	*
Life style	74.	Smoking and alcohol consumption	*
	75.	Nutrition	*
	76.	Level of physical activity during the day	*

accepted,

×: reviewing in the second round of Delphi.

We developed a telemedicine system that included eight pages, entitled: “main page” (Figure 1), “creating an account”, “login to personal account and panel”, “medical history record”, “presentation of patient medical history”, “reporting of patient medical history”, “physician communication with other physicians”, and “prescribing medication and medical consultations”.

Figure 1

Main page of the telemedicine system.

The patient first creates an account, logs in, and then performs the necessary processes in his/her personal panel to receive the appropriate treatment (Figure 2). Details about the doctor’s personal panel and patients’ treatment are displayed in Figure 3. For interactions between physicians, a page entitled “physician communication with other physicians” was designed. On this page, physicians who encounter suspected or rare cases of diabetes can contact other physicians, send patient information, and ask for second opinions (Figure 4). After sending this information, the consulting physician first retrieves the information from the personal panel. In the next step, the physician prescribes the appropriate medications and provides the necessary recommendations and treatment measures to the requesting physician.

Figure 2

Flowchart of panel of patients.

Figure 3

Flowchart of panel of doctors.

Figure 4

Physician-physician interaction page.

After sending the physician’s response to the patient or consultant physicians, the patient or physician requesting counseling receives a notification via text message and e-mail. Hence, the patient or physician requesting the consultation can receive the prescribed medication, consultation, or treatment measures in his or her personal panel.

Table 4 presents the demographic characteristics of participants in the system usability evaluation. Eighty-one problems were identified in various parts of the designed system. Similar problems were grouped together, and 26 unique problems were identified. These problems were classified into four main groups: “alignment of the needs of physicians and patients with the designed system” (nine problems), “accurate recording of data and information and ease of understanding their concepts” (11 problems), “correct uploading of required information (video files, images, and texts)” (three problems), and “time spent recording information” (three problems). Table 5 shows these problems and some of the participants’ statements. In order to fix these system problems, we sent a list of these problems and instructions on how to fix them to our programming team and system analysts.

Table 4

Demographic characteristics of the participants in the system evaluation step

Variable n	(%)
Sex, male	4 (100)
Age (yr)
36–45	2 (50)
46–55	2 (50)
Employment history (yr)
1–10	1 (25)
11–21	3 (75)
Type of specialty
Physics medicine	2 (50)
General practitioner and PhD in medical informatics	2 (50)

Table 5

Identified usability problems

Primary groups	Row	Subgroups	Number of participants’ opinions	Sample participant statements
Alignment of the needs of physicians and patients with the designed system	1.	Insufficient attention paid to the need to communicate with psychiatrists and counselors	1	“I think the designed system is appropriate for the needs of the physician and the patient, but you have forgotten an important dimension for the patient: providing consultations for diabetic foot patients, who are usually depressed after surgery or amputation.” (Participant 1)“The designed system is very good, but since patients with diabetic foot are more prone to amputation and ulcers in the foot area, I wish it would be possible to send medicine from the pharmacy to patients after prescribing the medicine.” (Participant 2)“Why did you forget the amputation field on the physician page and in the medication and treatment consultation section?” (Participant 3)
	2.	Insufficient attention paid to the possibility of sending the prescribed medicine to the patient’s home	1
	3.	Omission of the amputation field in the section of prescribing medicine and medical consultation	2
	4.	Omission of the possibility of uploading electromyography report in the information uploading section	2
	5.	Insufficient attention paid to the support of some insurance companies for patients	3
	6.	Absence of conditions for physician-physician interactions or physician-patient interactions through video conferencing	3
	7.	Insufficient attention paid to the needs of patients with visual impairment in the designed system	1
	8.	Absence of training videos and clips to prevent exacerbation of ulcers in the patient panel	2
	9.	Impossibility of holding group sessions among physicians on the physician-physician interaction page	2
Accurate recording of data and information and ease of understanding their concepts	1.	Ability to send information on some pages without completing some obligatory fields	2	“Some fields are shown with a star mark and are obligatory, but other data can be sent to the database without filling those fields out.” (Participant 4)“Why didn’t you define a numerical range for FBS tests. The patient can enter any number.” (Participant 3)“It is very difficult to understand the concepts of classifying the severity and depth of ulcers (based on Wagner) presented in the patient medical history section. It is better to use simple and understandable words.” (Participant 1)
	2.	No definition of the permissible numeric range for fasting blood sugar (FBS)	3
	3.	No specification of the left and right foot in the field for the number of foot ulcers	2
	4.	No specification of the left and right foot in the field for ulcer severity and depth	2
	5.	Difficult-to-understand concepts related to classifying the severity and depth of ulcers	3
	6.	English date and calendar, which are difficult to understand for people unfamiliar with English	2
Accurate recording of data and information and ease of understanding their concepts	7.	Not using the same language (Persian) in all presentations of some of the main concepts	2
	8.	Impossibility to connect some devices such as a glucometer to receive more accurate information and eliminate manual data entry	1
	9.	Ability to type numeric values instead of letters in some fields	2
	10.	Ability to type letters instead of numeric values in some fields	2
	11.	Mentioning type 1 diabetes twice in the field to select type of diabetes instead of type 2 diabetes	2
Correct uploading of required information	1.	Inability to upload multiple test results simultaneously	2	“Why is it not possible for me to upload multiple tests together – instead, I can upload only one test result?” (Participant 4)“The size of images and videos for uploading should be limited; otherwise, the system speed will slow down.” (Participant 3)“Any image can be uploaded with any quality.” (Participant 1)
	2.	Insufficient attention paid to the size of images and videos uploaded	2
	3.	Ability to upload any image with any quality	2
Time spent recording information	1.	Increased time spent uploading images and videos individually	2	“I think that the dose of medicine and description of the required surgeries on the physician page and in the prescription medicine sections should be designed in such a way that the physician selects the appropriate items and does not type them out, as it takes a lot of time.” (Participant 1)“I think that on the patient history page, the uploading images and videos section should be designed in such a way that several videos and images with small size and high quality can be uploaded, so that the patient’s time is not wasted and the work speed increases.” (Participant 3)
	2.	Impossibility of multi-period reporting process to save time for physicians	1
	3.	Time spent by duplicate typing of some fields by the physician and patient (typing the name of the province, city, and medicines used and prescribed)	3

The results of this phase are presented in Table 6.

Table 6

Frequency distribution of participants’ satisfaction with the designed system

Variable	n (%)	Satisfaction score
Level of satisfaction with system
Very high	3 (75)	3.75 ± 1.02
High	1 (25)	1.25 ± 0.87
Recommending it to other diabetic people
Yes	4 (100)	5.00 ± 1.82
Purchase of system
Yes	3 (75)	3.75 ± 0.92
No	1 (25)	1.25 ± 0.85

Values are presented as mean ± standard deviation.

IV. Discussion

The present study was conducted to develop and evaluate a telemedicine system for the management and monitoring of patients with diabetic foot. In this system, several features were developed for patient-physician communication, physician-physician communication, the possibility of monitoring patients’ conditions, controlling the disease process, providing medical consultation, and prescribing medications to patients by physicians. The system was then evaluated and 26 unique problems were identified in four main groups. The most frequently identified challenges were related to accurate recordings of the data and information and the ease of understanding their meaning. Finally, users’ satisfaction with the system was evaluated. Further details of the present study and other studies are presented below.

1. Needs Assessment

In the present study, various features were considered, such as the ability to record patients’ medical history, have patients provide a medical history, requesting second opinions from physicians, provide medical consultations, prescribe medications, and enable physician-physician interactions for consultation about rare cases. Rasmussen et al. [19] identified the factors influencing the implementation of tele-medical monitoring for patients with diabetic foot ulcers, which included management, financial considerations, and periods of absence from work and clinical care. Wilbright et al. [20] also investigated the management of foot ulcers through telemedicine consultations. This system had features such as the management of forefoot ulcers by a certified wound care nurse trained and supported by real-time interactive tele-medicine consultations. McGill et al. [21] also implemented teleconsultations to provide services to patients with diabetic foot. The system included routine consultations between an urban diabetes center and rural sites. To receive consultations, images were emailed to a hospital.

2. System Evaluation

We found 26 unique usability problems in the telemedicine system, and classified these problems into four main groups, including alignment of the needs of physicians and patients with the designed system, accurate recording of data and information and ease of understanding their meaning, correct uploading of required information, and time spent recording information. Most of the identified problems were related to accurate recording of data and information and ease of understanding their meaning. Wilkins et al. [8] evaluated the feasibility of a web-based telemedicine program to provide remote counseling consultation to patients with chronic wounds. Their study reported high levels of user satisfaction with the designed system and the process of sending wound images and receiving medical consultation and recommendations from the treatment staff. Overall, 98.2% of patients were satisfied with the care process. Providers also showed a high level of satisfaction with this system. Georgsson and Staggers [22] evaluated the usability of a diabetes mobile application and found 117 problems. The most important problems identified in their study, similar to the present study, related to the accurate recording of data and information. In particular, users could not accurately record their glucose and blood pressure levels. Lopez et al. [23] evaluated a web-based application for diabetes management and reported 46 problems. The most frequent problems identified by Lopez et al. [23], consistent with our study, were related to the incorrect recording of information in the system. Overall, the results of this study and the studies reviewed showed that usability problems could prevent the correct and easy use of telemedicine systems by users. Therefore, by understanding such problems, health system designers can design usable systems that fit the needs of users and prevent such problems from occurring in the design of similar systems. It is suggested that further research should provide standard methods for easily identifying these problems and how to solve them.

Overall, we found that the most important usability problems in telemedicine systems for foot ulcer reported in other studies were manipulation, cognitive aspects, and visual cues (for example deleting and entering glucose values) and information about how to save a file [22] and incorrect recording of information [22,23]. These problems may cause unsuccessful and inaccurate interactions of users with the system, recording of incorrect information by physicians and patients, fatigue and boredom, and increased errors by physicians in providing recommendations and prescribing medication [24]. These problems can also lead to medical errors, threats to the patient’s health [25], and reduced efficiency and effectiveness [24]. Regular system evaluations help to identify usability problems faster, plan to solve these problems, and prevent possible errors [26,27]. Regularly updating systems based on identified problems is another way to solve these problems. Updates should include the addition of new features and capabilities to the system and the resolution of existing problems to facilitate users’ interaction with the system and improve their understanding of the system [28].

One of the limitations of this study was the small number of participants. Therefore, it is recommended that this phase be carried out with a larger number of specialists. Furthermore, the perspectives of patients with diabetic foot regarding the usability of the system should also be evaluated.

In this study, a telemedicine system for managing and monitoring patients with diabetic foot was designed and evaluated. We developed various features, such as patient-recorded medical history and illness description, medical consultations and medication prescriptions by patients, and physician-physician interactions to obtain counseling related to rare cases of diabetic foot. We also found that the most commonly encountered problems included alignment of the needs of physicians and patients with the designed system, accurate recording of data and information and ease of understanding their meaning, correct uploading of required information, and increased time for recording data. These problems should be considered to develop a usable telemedicine system for foot ulcer care.