Assessment of Tibot® Artificial Intelligence Application in Prediction of Diagnosis in Dermatological Conditions: Results of a Single Centre Study.

OBJECTIVE: To analyze the accuracy of Tibot artificial intelligence (AI) application tool in predicting the diagnosis of dermatological conditions.
MATERIAL AND METHODS: In this prospective, observational study photographs of dermatological lesions with other details of patients having different skin conditions were fed in the AI application for the diagnosis. Predictions given by the Tibot AI application were compared with diagnosis done by the dermatologist. The performance of AI application was evaluated using accuracy, precision, and recall.
RESULTS: Data of 398 patients were included in the application of whom 159 (39.9%) had fungal infections. Other conditions included eczema 36 (9%), alopecia 28 (7%), infestations 27 (6.8%), acne 25 (6.3%), psoriasis 19 (4.8%), benign tumors 7 (1.8%), bacterial infection 19 (4.8%), viral infection 15 (3.8%), and pigmentary disorders 20 (5%). The prediction accuracy (ability to get diagnosis in top three conditions) for alopecia, fungal infections, and eczema was 100%, 95.6%, and 91.7%, respectively. Mean prediction accuracy for correct diagnosis in the predicted top three diagnoses was 85.2%, and for correct diagnosis was 60.7%. Sensitivity and specificity of the application were approximately 86% and 98%, respectively. The sensitivity and positive predictive value of the application to diagnose alopecia was 100% and for fungal infections it was 96.85% and 90.05%, respectively.
CONCLUSION: In the preliminary stages, AI application tool showed promising results in diagnosing skin conditions. The accuracy and predictive value of the test may improve with the expansion of the database. Copyright:

Introduction

Artificial intelligence (AI) is aimed at understanding and developing systems that can mimic or exhibit properties similar to human intelligence. Machine learning is a branch of AI in which computer programs are used to understand associations of prediction using examples in the data.[12] The main aim of AI is to mimic human thinking functions. The most popular AI techniques include machine learning methods for structured data, and the modern deep learning, as well as natural language processing for unstructured data.

AI can be applied to various types of structured and unstructured health care data. In the field of health care, AI is bringing a huge change with the assistance of ever-increasing availability of healthcare data and the rapid progress of various analytical techniques. There is rapid growth in research with AI in medicine.[3]

AI in the field of dermatology, a branch of medicine primarily focused on the evaluation and treatment of skin disorders, including hair and nails, is showing the potential to simplify the diagnosis of skin disorders.[4] With steady growth in popularity, particularly in dermatology, the AI tools have become more prevalent and are beginning to change how providers offer care on a day-to-day basis.

Tibot is an AI application developed to provide analysis of skin-related problems after uploading a photo of skin condition and answering a few questions related to the condition. Studies providing accuracy of diagnosis by such applications are necessary.

Objective

The objective of this study was to analyze the accuracy of Tibot AI tool in predicting the diagnosis of skin conditions in comparison with diagnosis made by the dermatologists.

Material and Methods

This prospective, non-interventional, observational study was conducted in outpatients presenting to the dermatology clinic of a tertiary care center.

Patients of all age groups consulting for different skin conditions for the first time irrespective of duration were included in the study after taking their informed consent. The skin conditions of enrolled patients included acne, psoriasis, alopecia, bacterial, viral, and fungal infections, eczema, infestations, and benign tumors. Complicated cases and those with prior treatments resulting in modification of diagnosis were excluded from the study. Patient not willing to give consent for uploading clinical photographs were excluded from the study.

The Tibot Android app was installed in the smartphone. After clinical history and physical examination, photographs of skin lesions were taken by OPD staff nurses with smartphones with at least 2MP camera and good internet connectivity. There was no restriction on the distance of the camera or lighting while taking the photograph. The picture of skin lesions was uploaded in the AI application. Demographic information and answers to a few important questions related to skin conditions were fed in the application. The diagnosis provided by the AI application was then compared with the diagnosis made by an expert dermatologist. The study was initiated after receiving approval from the institutional ethics committee. The study was conducted over a period of 4 weeks.

Statistical analysis

Descriptive analysis of different skin conditions is presented as frequency and percentages. The performance of the AI application was measured by assessing the possibility of getting top three diagnoses. Sensitivity, specificity, and positive predictive value of the application were also analyzed. Data analysis was performed using SPSS version 15 and Open Epi software.

Results

A total of 398 patients were included in the study of which 159 (39.9%) were of fungal infections. Other diseases included 36 (9%) eczema, 28 (7%) alopecia, 27 (6.8%) infestations, 25 (6.3%) acne, 19 (4.8%) psoriasis, 7 (1.8%) benign tumors, 19 (4.8%) bacterial infections, 15 (3.8%) viral infections, and 20 (5%) pigmentary disorders [Figure 1].

Figure 1

Distribution of different skin conditions included in the study

The mean prediction accuracy of the AI application to get the actual diagnosis in the predicted top three diagnoses was 85.2%, and to get the exact diagnosis is 60.7% [Figure 2]. The prediction accuracy, that is the ability to get the diagnosis in top three predictions was 100% for alopecia and 95.6%, 91.7%, 88.9%, 84%, 75%, 73.7%, 73.7%, 71.4%, 59.3%, and 26.7% for fungal infections, eczema, immunological disease, acne, pigmentary disorders, bacterial infections, psoriasis, benign tumors, for infestation, and viral infections, respectively.

Figure 2

Prediction accuracy of AI application

Sensitivity and specificity of the AI application was approximately 86% and 98%, respectively. Figure 3 shows the confusion matrix of predicted diagnosis versus actual diagnosis. Figure 4 shows pictures of five skin conditions predicted by the AI application and diagnosis made by the dermatologist.

Figure 3

Confusion matrix

Figure 4

Predicted diagnosis and actual diagnosis

The sensitivity and positive predictive value of the application to diagnose skin conditions are given in Table 1.

Table 1

Sensitivity and positive predictive value of Tibot in diagnosis of different skin conditions

Skin condition	Sensitivity	Positive predictive value
Acne	84%	87.5%
Alopecia	100%	100%
Bacterial infection	78.9%	78.9%
Benign tumour	71.4%	83.33%
Eczema	91.66%	94.3%
Fungal infection	96.85%	90.05%
Immunological	88.88%	42.10%
Infestation	62.96%	94.44%
Pigmentory disorders	75%	75%
Psoriasis	73.68%	82.35%
Viral infection	26.66%	80%

Discussion

With an objective to ease and increase early detection of skin conditions, researchers have come up with different AI applications. In this study, we analyzed the accuracy of Tibot AI tool in predicting the diagnosis of skin condition and compared it with the diagnosis made by the dermatologists. Our study included maximum patients with infections and eczema. The prediction accuracy of the application for correct diagnosis was highest for alopecia, that is, 100% followed by 95.6% and 91.7% for fungal infections and eczema, respectively. The positive predictive value i.e., the probability that subjects with a positive screening test truly have the disease was 100% for alopecia. For eczema, infestation, fungal infection, acne, psoriasis, and viral infection the positive predictive value was 94.3%, 94.4%, 90%, 87.5%, 82.35%, and 80%, respectively.

Prediction accuracy of the AI depends on the amount of data that is fed into the application. It is an active learning machine that enables the application to expand the database and diagnostic ability. The basic principle of AI in the diagnosis of skin condition can be understood by taking an example of a trainee dermatologist. The accuracy of the trainee is enhanced by looking at multiple cases with similar patterns. The AI works on the same principle, the convolutional neural network (CNN) of AI takes an image and converts it into numerical forms and retrieves from its own memory similar images from which it can generate a diagnosis.[5] AI can be of use to the people especially in developing countries like India where there is a shortage of experts. In India, for every 100,000 people, there are approximately only 0.49 dermatologists in India as compared to 3.2 in many states of the USA.[6]

The applications use machine learning to analyze images and to predict the probable skin condition by comparing the images uploaded with the mammoth data fed into the system. The Skin conditions are analyzed by uploading the images and answering few questions followed by the image analysis. Thus, the AI neural network learns to identify and distinguish images based on various characteristics of the dermatoses. For example, it automatically learns to distinguish between growth and rash. The difference in accuracy can be attributed to the varying visual manifestation of the condition in each patient. This might explain the differences in the accuracy of different skin conditions observed in our study.

Out of three diagnosed cancers, one is a skin cancer. Between two to three million non-melanoma skin cancers and 132,000 melanoma skin cancers occur annually. These data suggest a significant health burden contributed by skin cancers globally.[7] The five-year relative survival rate for melanoma detected in its earliest stage (stage 0) is 97%, which drops to about 10% if it is detected in stage IV.[8] This underlines the importance of early detection and treatment of skin cancer. There have been studies in the past showing the superiority of artificial intelligence in diagnosing dermatological conditions. A study by Marchetti and colleagues[9] comparing machine learning algorithms for characterizing pigmented lesions demonstrated superiority over eight dermatologists in diagnosing melanoma. Similarly Esteva and colleagues.[10] trained a Google-based CNN to diagnose melanomas and keratinocytic carcinomas with equal or superior success compared to 21 dermatologists, though not in a real-world clinical setting. In a study published in the leading cancer journal Annals of Oncology, researchers in Germany trained a CNN to identify skin cancer by showing it more than 100,000 images of malignant melanomas, as well as benign moles. They compared its performance with that of 58 international dermatologists and found that the CNN missed fewer melanomas and misdiagnosed benign moles as malignant less often than the group of dermatologists.[11] It is important to note, we did not include patients with skin cancer. The unique attribute of the current study was to apply AI technology to common skin problems presented in the clinic.

Apart from the skin analysis, these applications have information regarding whether to/when to visit a doctor, book an appointment with the doctor, order medicines, and skincare products, online consultation with dermatologists, providing reminders about the treatment dates and appointment and skin disease dictionary explaining causes, symptoms, and treatments for various skin diseases. Artificial intelligence (AI) and its application may become an important aid to the health professionals in image acquisition, processing, interpretation, reporting, and follow-up planning. Also, AI can provide certain additional benefits of data integration, data storage, and data mining.[12] AI can also help in optimizing time-consuming tasks and reducing inter-observer reliability issues.[13] Most of the current and emerging AI implementation in dermatology has been focused on analysis of the image (similar to visual examination). With improved accuracy, this can reduce the incidences of false positives in detecting skin diseases. It has been empirically observed that users present with skin conditions (e.g., fungal infection) at an advanced stage in dermatology clinics. Since skin problems can be ignored until it becomes a major inconvenience, there is a need for a tool that can be used to evaluate the urgency of skin issues. An AI tool, which a user can use to understand skin problems, can be effective in raising awareness to seek medical attention before it gets exacerbated.

The application used in this study is designed for the use by the patients. Keeping this in mind, photographs in the current study were taken by the OPD staff nurse without any restriction on the distance of camera or lighting. In real practice, photographs will be taken by the patients, but as the purpose of this study was to evaluate the performance of the AI tool, they were taken by the OPD staff nurse.

The current study suggests that AI tools can have great potential for practical usability as it is easy for use for the end-users to learn about their skin condition and seek clinical care for further analysis and treatment. It can be especially useful for people in the rural areas and smaller cities where patients who don't have direct access to a dermatologist and need urgent clinical consultation. It has to be noted that the AI tool is not a replacement for diagnosis by the dermatologist. It can complement dermatologists and patients to decide the plan of treatment. In short, AI in dermatology will not be “Man vs Machine” rather it will be “Man with Machine”.

The Tibot application protects patients’ data by multiple levels of security. All the personal data is encrypted both at-rest and in-flight. The users have complete control to delete their information permanently.

This is a single-center study with a limited sample size. Second, the skin type of patients was not recorded in this study. However, the skin type of the patients in this study was generally between Fitzpatrick types four to six. Studies in different types of skin are required to confirm our observations. Images take using smartphones can look very distinct from one another even when they are taken under the same conditions. Variations in angle, lighting, and focus need to be accounted. Unless a clinical history is fed into the application, it is difficult to accurately predict the diagnosis of skin conditions with similar morphology and clinical photographs. These are some of the limitations of AI in dermatology. Considering this, the results of our study should be carefully interpreted. Larger multicentric studies are required to confirm the findings of our observations.

Conclusion

In the preliminary stages of development of this application, it has shown promising results in diagnosing skin conditions. The accuracy and predictive value of the test may improve with the expansion of the database. Rather than succumbing to the fear of a futuristic robo-dermatologist, we should embrace AI and learn to integrate it into our patient care paradigms.

Financial support and sponsorship

Polyfins Technology Inc provided support in the conduct of study.

Conflicts of interest

Tibot is AI-enabled application of Polyfins Technology Inc. Faisal Basar is an employee of Polyfins Technology Inc. Salim Bate was an employee of Polyfins Technology Inc during the study period.