Assessing the impact of color blindness on the ability of identifying benign and malignant skin lesions by naked-eye examination.

BACKGROUND: Color vision deficiency describes the inability to distinguish certain shades of color. The aim of this study was to assess the impact of having color vision deficiency on the accuracy of distinguishing benign and malignant skin lesions by naked-eye examination.
METHODS: This was a cross-sectional study conducted during the period August 2020 to February 2021. We randomly selected a total of 20 nevi and 20 melanoma images from an open access image database. The 40 images were divided into four sets of images, each set contained 5 benign and 5 malignant skin lesion images simulated as if they were seen by a protanope physician, deuteranope physician, tritanope physician, and a set of images presented without simulation. In an online survey, students who were in their final year of medical school or had newly graduated were asked to diagnose each image as benign or malignant.
RESULTS: A total of 140 participants were included with a mean (SD) age of 24.88 (1.51). We found a significantly higher mean accuracy for non-simulated images compared to deuteranope simulated images (p< 0.001, mean difference = 11.07, 95% CI 8.40 to 13.74). We did not find a significant difference in accuracy classification for protanope simulated images (p = 0.066), nor for tritanope simulated images (p = 0.315). Classification accuracy for malignant lesions was higher than classification accuracy for benign lesions, with the highest difference belonging to deuteranope simulated images, with a difference in mean accuracy of classifying malignant lesions by 32.2 (95% CI 27.0 to 37.6).
CONCLUSION: Deuteranope participants (i.e., green color deficiency) had a significantly lower accuracy of distinguishing pigmented skin lesions as benign or malignant, an impact not found for other color vision deficiencies, which was mainly for misdiagnosing benign lesions as malignant.

Introduction

It has been almost 35 years since the introduction of the ABCD pneumonic to simplify and facilitate the detection and early recognition of malignant pigmented skin lesions [1]. The pneumonic originally described suspicious characteristics that may warrant referral to specialized dermatologists, including “A” for asymmetry, “B” for border irregularity, “C” for color variegation, and “D” for a diameter of more than 6 mm. Since then, several expert groups revisited the criteria, trying to increase its accuracy, by avoiding falsely classifying benign lesions as malignant (false positive), and more importantly, avoiding classifying malignant lesions as benign (false negative) [2-4]. They have also added the letter “E”, referring to evolution, a rapid change in shape, color, or size of the lesion [2]. Despite the need to put all lesion characteristics in the context, the color criterion was found to be among the top of the ABCDE criteria regarding accuracy of discriminating malignant lesions [4, 5]. Moreover, lesion pattern recognition, which mainly involved the color of the lesion, also played an important role in malignant lesion detection [6, 7].

Color vision deficiency originally described the inability to distinguish certain shades of color. On the other hand, the term "color blindness" is sometime used interchangeably [8]. Despite its relative high prevalence, people might be unaware of their color vision deficiency [9]. Physicians and healthcare professionals might also be affected by color vision deficiency without being aware of their condition, which might have a significant impact on their practice [10]. In a previous study, skin condition assessment was among the most difficult tasks for people with color vision deficiency [11].

Patients with skin lesions usually present to primary care physicians for evaluation, those lesions thought to be of malignant potential are usually referred to dermatology clinics for further evaluation and possible intervention. The diagnostic and referral accuracy of pigmented skin lesions were generally found to be low among primary care physicians compared to dermatologists [12-14]. The impact of having color vision deficiency on distinguishing benign and malignant skin lesions has never been studied, despite the potential impact on lesion color, an integral part of the ABCDE criteria. The aim of this study was to assess the impact of having color vision deficiency on the accuracy of distinguishing benign and malignant skin lesions by naked-eye examination. We conducted this study on participants who were in their final year in medical school or had newly graduated, whom we believe depend more on lesion color criteria in distinguishing benign and malignant lesions.

Methods and materials

Design and participants

This was a cross-sectional study conducted during the period between August 2020 and February 2021. The study was approved by the institutional review board (IRB) and research committee at Jordan university hospital and the University of Jordan, and was conducted in concordance of the latest declaration of Helsinki. The study included participants who were either in their final year of medical school or had newly graduated and were working as general practitioners. Participants who did not have a dermatology rotation during their medical school were excluded at the beginning of the survey.

Using an online form, we detailed the nature of the project. Participants then provided their consent for participation. The rater was first asked to take an online Ishihara test to confirm the absence of preexisting color vision deficiency, and participants who had a score below 12/13 were asked not to proceed with the questionnaire. After that, the rater was provided a short online presentation on Medscape medical platform that explains how to differentiate between benign and malignant tumors [15].

Assessment

The questionnaire started by asking the participant if he/she took a dermatology rotation during medical school, where all Jordanian universities usually have a 2-week long dermatology rotation with an exam at the end of rotation to assess acquired knowledge. This was followed by Ishihara testing, as described above. After that, the questionnaire asked about the participant’s demographic variables (i.e., age, gender, and occupation).

Regarding benign and malignant lesions used in the survey, we used randomizer.org to randomly select a total of 20 nevi and 20 melanoma images from MED-NODE database [16]. The 40 images were divided into four sets of images, each set contain five images for benign and five images for malignant skin lesion. The four sets were:

Five benign and five malignant skin lesions to be simulated as if were seen by a protanope physician;

Five benign and five malignant skin lesion images to be simulated as if were seen by a deuteranope physician;

Five benign and five malignant skin lesion images to be simulated as if were seen by a tritanope physician.

Five benign and five malignant skin lesion images presented without simulation.

To transform images into what a protanope, deuteranope, and tritanope participant would see, we used Vischeck color blindness simulator implemented in Fiji software [17], which has been proven to be highly accurate in simulating color-blind images [18]. We used Google forms for data collection. In order to correct for any variabilities in the screens of the device’s, we compared the accuracy of non-simulated images with simulated images. (Fig 1) shows a benign skin lesion (i.e., nevus) simulated into what a protanope, deuteranope, and tritanope participant would see, and (Fig 2) shows a malignant skin lesion (i.e., superficial spreading melanoma) simulated into what a protanope, deuteranope, and tritanope participant would see. Both figures were not part of the MED-NODE images used in this study’s survey. The 40 images used in this study’s survey were deposited openly in Harvard Dataverse [19]. It is important to note that both images were collected from our hospital setting after proper consent had been obtained, so they were not a part of the images shown in the survey originally collected for this study. The 40 images were shuffled and were presented sequentially and individually. Images were standardized in size and resolution. The rater was asked to rate each image as either benign or malignant.

Fig 1

A nevus presented as an example of a benign skin lesion (A) simulated into what a protanope (B), deuteranope (C), and tritanope (D) participant would see. Note that the simulation included the lesion and the background skin, affecting the distinction at lesion borders. The image presented here is not a part of the MED-NODE database.

Fig 2

A melanoma presented as an example of a malignant skin lesion (A) simulated into what a protanope (B), deuteranope (C), and tritanope (D) participant would see. The image presented here is not a part of the MED-NODE database. Figure adapted from Longo et al. study (non-simulated image) [20].

Statistical analysis

We used SPSS 26.0 in our statistical analysis. We used mean and standard deviation (SD) to describe continuous variables, and we used frequency (percentage) to describe categorical variables. The diagnosis for each image was scored as either correct (1) or incorrect (0). There were four sets of images, each containing images for five benign and five malignant lesions. Of the four sets, 3 were simulated into what a deuteranope, protanope, and tritanope participant would see, and the last set was not simulated. We calculated the accuracy score as a percentage for the benign and malignant images in each set, then the accuracy of image classification in the set (i.e., overall accuracy). We used Paired sample t-test to analyze the mean difference in accuracy between the four sets. We used independent sample t-test to analyze the accuracy score difference between men and women. We presented the data as mean difference and 95% confidence interval (CI). We used a p value threshold of 0.05 as our significance threshold.

Results

A total of 152 participants completed image classification. Among them, 12 participants did not have a dermatology rotation during their medical school, and thus were excluded from further analysis. For the 140 participants included in the analysis, the mean (SD) age was 24.88 (1.51). Among the participants were 86 men with a mean (SD) age of 25.2 (1.50), and 54 women with a mean (SD) age of 24.37 (1.40). Table 1 details the mean accuracy of classifying lesions presented in the images in each set (i.e., protanope, deuteranope, tritanope simulated images and non-simulated images) as either benign or malignant lesions. It also presents the mean difference in classification accuracy between images containing benign and malignant lesions.

Table 1

The mean accuracy of classifying lesions presented in the images in each set (i.e., protanope, deuteranope, tritanope simulated images and non-simulated images) as either benign or malignant lesions.

As well as, the mean difference in classification accuracy between images containing benign and malignant lesions.

		Mean %	Std. Deviation %	p value	Mean difference %	95% CI %
Protanope simulated images	Accuracy of benign lesion classification	73.86	24.68	<0.001	8.4	4.0 to 12.8
	Accuracy of malignant lesion classification	82.28	18.90
Deuteranope simulated images	Accuracy of benign lesion classification	48.28	22.5	<0.001	32.2	27.0 to 37.6
	Accuracy of malignant lesion classification	82.58	19.04
Tritanope simulated images	Accuracy of benign lesion classification	70.0	26.98	<0.001	8.4	3.0 to 14.0
	Accuracy of malignant lesion classification	78.42	21.6
Non-simulated images	Accuracy of benign lesion classification	67.14	26.06	<0.001	16.7	10.6 to 22.8
	Accuracy of malignant lesion classification	83.86	22.5

Upon comparing classification accuracy among protanope, deuteranope, and tritanope simulated images with non-simulated images, we found a significant difference in accuracy classification between deuteranope simulated images compared to non-simulated images (p< 0.001), with higher mean accuracy for non-simulated images (mean difference = 11.07, 95% CI 8.40 to 13.74). We did not find a significant difference in accuracy classification between protanope simulated images with non-simulated images (p = 0.066), nor between tritanope simulated images with non-simulated images (p = 0.315).

Upon comparing classification accuracy for benign and malignant lesions between each image simulation, we found significant differences for all images, where classification accuracy for malignant lesions was higher than classification accuracy for benign lesions. The highest difference was for deuteranope simulated images, where the accuracy of classifying malignant lesions was higher by a mean of 32.2 (95% CI 27.0 to 37.6), as shown in Table 1.

Upon comparing gender difference in regard to classification accuracy, there was no statistically significant difference between men and women for all groups, as shown in Table 2.

Table 2

Comparison in the accuracy of classifying lesions presented in the images in each set (i.e., protanope, deuteranope, tritanope simulated images and non-simulated images) between male and female participants.

	Gender	N	Mean %	Std. Deviation %	p value
Protanope simulated images	Male	86	79.42	18.68	0.257
	Female	54	75.93	15.96
Deuteranope simulated images	Male	86	63.49	14.61	0.301
	Female	54	65.93	11.58
Tritanope simulated images	Male	86	74.65	19.08	0.720
	Female	54	73.52	16.62
Non-simulated images	Male	86	73.49	16.36	0.058
	Female	54	78.70	14.54

Discussion

This study tested the accuracy of classifying benign and malignant pigmented skin lesions among images simulated to appear as if seen by participants with different types of color vision deficiency, and we compared their accuracy with images that were not simulated. We found that the only color vision deficiency that affected classification accuracy was deuteranopia (i.e., green color deficiency). Upon performing our sub-group analysis to analyze if there was a difference in classification accuracy for benign or malignant lesions, we found that the classification accuracy for benign lesions was significantly lower for all images, with the highest magnitude arising from deuteranopia simulated images. In fact, non-simulated images had lower accuracy for the classification of benign lesions compared to protanopia and tritanopia simulated images. It was noted that the borders between the lesion and the surrounding skin is less distinct for deuteranopia-simulated images, which might increase the rater’s classification bias toward malignant lesions. This effect was also observed, but with a lesser degree, for protanopia-simulated images, but not for tritanopia-simulated images. This can be explained by previous studies’ observations, where the distinction of image borders increases for higher wavelength colors (i.e., toward the red colors) [21, 22], and deuteranopia- and protanopia-simulated images have more low-wavelength colors than normal or tritanopia-simulated images.

Due to the shortage of specialized dermatology clinics to diagnose and manage skin lesions, primary care physicians may need to depend on their skills and knowledge to diagnose and possibly conduct a biopsy on skin lesions [23]. Ensuring that those physicians can distinguish such lesions is of great importance. Here, we found that physicians with color vision deficiency will have a significantly higher frequency of false positives in their diagnosis’, thus, referring and performing procedures more than people without color vision deficiency, particularly those with green color deficiency. It is important to note that green color deficiency is the most common form of color vision deficiency [8].

There are various medical specialties to which a patient with a skin lesion can be presented to, including dermatology and plastic surgery, but most patients end up presenting to primary care clinics [24]. Primary care physicians play an important role in distinguishing skin lesions that warrant referral to specialized clinics, or even lesions that might need clinic based interventions [25, 26]. Many efforts have been evaluated for improving the diagnostic accuracy of primary care physicians for diagnosing malignant melanoma such as short training courses [13, 27, 28]. The most recent reviews of literature indicate that primary care physicians had sensitivities, specificities, and diagnostic accuracies ranging from 0.25 to 0.88, 0.26 to 0.71, and 0.49 to 0.80, respectively in comparison to 0.74 to 1.00, 0.56 to 0.95, and 0.85 to 0.89 sensitivities, specificities, and diagnostic accuracies for dermatologists [12]. Early referral to specialized dermatology clinics will lead to a decrease in the time from first diagnosis to excision and final histopathological diagnosis [29, 30], which in turn has significant impact on survival [31]. While the accuracy of distinguishing benign and malignant lesions was lower with color vision deficiency, it resulted in more lesions categorized as malignant and required further evaluation, rather than missing malignant lesions.

Even though the study included a relatively high number of participants with 40 instances from the openly accessible Med-NODE database to be assessed by each participant [32], with color vision deficiency simulation using high fidelity validated software, it still has limitations that need to be considered upon result’s interpretation. While we provided a short course for each participant on how to distinguish benign and malignant features, clinical decision on whether a lesion is benign or malignant is usually also dependent on clinical data, which was not provided in our case. Moreover, this study was not performed on specialized dermatologists, whom would have more experience and confidence in diagnosing benign lesions as benign depending on more lesion characteristics, other than color. Further studies in this regard might show if including experienced dermatologist might abolish such accuracy difference for participants with color vision deficiency. Another limitation to keep in mind is that participants used their own devices to complete the questionnaire, which might result in an impact from screen color and brightness on the classification accuracy.

Conclusion

Deuteranopia (i.e., green color deficiency) led to a significant decrease in accuracy of distinguishing pigmented skin lesions as benign or malignant on simulated images, an impact that was not found for other color vision deficiencies. We also found that the poor distinguishing accuracy was mainly for misdiagnosing benign lesions as malignant, which was also most prominent for deuteranopia simulated images.

4 Aug 2021

PONE-D-21-20492

Assessing the impact of color blindness on the ability of identifying benign and malignant skin lesions

PLOS ONE

Dear Dr. AlRyalat,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

While understanding the impact of color blindness on the ability of diagnosing skin lesions, both clinically and dermoscopically, is certainly of major importance for the dermatology clinic, a study like this is only of use if (1) the right methodology is applied and (2) the data are interpreted without any bias. Thus, in accordance with the expert reviewers I have some conceptual and technical concerns that need to be addressed. Rather than repeat further points here, I refer you to the specific remarks (below) for details.

Please submit your revised manuscript by Sep 18 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see:  http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at  https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Nikolas K. Haass, MD/PhD

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that Figure 1 in your submission contain copyrighted images. All PLOS content is published under the Creative Commons Attribution License (CC BY 4.0), which means that the manuscript, images, and Supporting Information files will be freely available online, and any third party is permitted to access, download, copy, distribute, and use these materials in any way, even commercially, with proper attribution. For more information, see our copyright guidelines: http://journals.plos.org/plosone/s/licenses-and-copyright.

We require you to either (1) present written permission from the copyright holder to publish these figures specifically under the CC BY 4.0 license, or (2) remove the figures from your submission:

a) You may seek permission from the original copyright holder of Figure 1 to publish the content specifically under the CC BY 4.0 license.

We recommend that you contact the original copyright holder with the Content Permission Form (http://journals.plos.org/plosone/s/file?id=7c09/content-permission-form.pdf) and the following text:

“I request permission for the open-access journal PLOS ONE to publish XXX under the Creative Commons Attribution License (CCAL) CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). Please be aware that this license allows unrestricted use and distribution, even commercially, by third parties. Please reply and provide explicit written permission to publish XXX under a CC BY license and complete the attached form.”

Please upload the completed Content Permission Form or other proof of granted permissions as an "Other" file with your submission.

In the figure caption of the copyrighted figure, please include the following text: “Reprinted from [ref] under a CC BY license, with permission from [name of publisher], original copyright [original copyright year].”

b) If you are unable to obtain permission from the original copyright holder to publish these figures under the CC BY 4.0 license or if the copyright holder’s requirements are incompatible with the CC BY 4.0 license, please either i) remove the figure or ii) supply a replacement figure that complies with the CC BY 4.0 license. Please check copyright information on all replacement figures and update the figure caption with source information. If applicable, please specify in the figure caption text when a figure is similar but not identical to the original image and is therefore for illustrative purposes only.

Additional Editor Comments:

While understanding the impact of color blindness on the ability of diagnosing skin lesions, both clinically and dermoscopically, is certainly of major importance for the dermatology clinic, a study like this is only of use if (1) the right methodology is applied and (2) the data are interpreted without any bias. Thus, in accordance with the expert reviewers I have some conceptual and technical concerns that need to be addressed. Rather than repeat further points here, I refer you to the specific remarks (below) for details.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Disclaimer: I am not a statistician and this manuscript should be reviewed by a statistician

Title: This should include “…by naked-eye examination” because in some countries dermatoscopic evaluation is standard of care for any assessment of pigmented skin lesions by any practitioner

Design

What is the duration of a “dermatology rotation” and what proportion of that rotation involves assessing skin lesions with respect to melanoma diagnosis?

Were the readers asked if they were colour-vision impaired and excluded for that reason and were any discovered on Ishihara testing? I ask this because in a cohort of 152 it might be expected that some males at least would have an impairment. I know that does vary with ethnicity but it is worth commenting on.

Discussion

I suggest including the following observation in the discussion re Figure 1:

In C the lesion is green and so is the background and in D the lesion is red and the background pink, whereas in B (deuteranope) the surrounding skin is similar to that of the unadjusted image (A)

As a result the colour contrast between the lesion and the surrounding skin is greater for Figure 1B than for C and D which makes irregular structures at the border (right side) more visible (similar as in A). This makes the lesion more clearly asymmetrical which may lean the reader towards a malignant diagnosis.

(The Figure legend should include the appropriate letter for each image respectively)

It would be interesting to see more images. At least a collage similar to Figure 1 but with a malignant lesion. This may or may not support what I have stated about the contrast between lesion and surrounding skin colour. I suggest that the authors evaluate this with every lesion and include those results in the manuscript.

Additional Limitations:

The readers were all inexperienced and an undefined proportion were not practising clinicians. There was a selection bias for suspicious lesions which may have influenced the readers (all lesions had been excised). This could in part explain the lower overall accuracy for benign lesions.

Reviewer #2: Dear Authors,

thank you for your manuscript "Assessing the impact of color blindness on the ability of identifying benign and malignant skin lesions". The idea behind your study is appreciated. However, there are some important issues to be adressed.

1) please improve the writing, there are many errors to be corrected (example: Page 1. Background: Color vision deficiencydescribes the inability to distinguish certain

shades of color. We found a significantlyhigher mean accuracyasthe accuracy of classifying)

2) diagnostic precision. You cite several sources to emphasize that "The diagnosis and referral accuracy of pigmented skin lesions are generally low among primary care physicians compared to dermatologists [12-14]." Despite that fact your study was conducted with participant in their final year in medical school or newly graduated. Please describe your intention to choose this group of participants and discuss a potential impact on the results.

3) "we used Vischeck color blindness simulator in Fiji software [16], which has been proved to be highly accurate in simulating color-blind images [17]".

It remains unclear how the whole process was conducted. Please specify how the images were presented (probably online via a private account shown on an uncertified screen). Especially the presentation of colors may vary and is dependent from many factors (model and age of the screen, contrast, intensity, surroundings etc). This might be crucial regarding the results.

Best regards and good luck!

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: Review.docx

Click here for additional data file.

11 Sep 2021

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

Response: Thank you.

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: Yes

Response: The statistical analysis was performed by an expert biostatistician.

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

Response: The data used in this study is openly and freely accessible at http://www.cs.rug.nl/~imaging/databases/melanoma_naevi/ with no restriction.

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

Response: We performed further English language polishing to ensure the absence of any typographical or grammatical errors.

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Disclaimer: I am not a statistician and this manuscript should be reviewed by a statistician

Title: This should include “…by naked-eye examination” because in some countries dermatoscopic evaluation is standard of care for any assessment of pigmented skin lesions by any practitioner

Response: We agree with the reviewer, and we amended the title and further detailed this in the aim accordingly.

Design

What is the duration of a “dermatology rotation” and what proportion of that rotation involves assessing skin lesions with respect to melanoma diagnosis?

Response: We provided details on dermatology rotations at Jordanian universities: “all Jordanian universities usually have a 2-week long dermatology rotation with an exam at the end of rotation to assess acquired knowledge.”. Moreover, we provided a link for an optional educational slides on differentiating benign and malignant skin lesions:

https://reference.medscape.com/features/slideshow/suspicious-skin-lesions

Were the readers asked if they were colour-vision impaired and excluded for that reason and were any discovered on Ishihara testing? I ask this because in a cohort of 152 it might be expected that some males at least would have an impairment. I know that does vary with ethnicity but it is worth commenting on.

Response: We agree with the reviewer, we further explained this in the methodology, where any participant who had an Ishihara score below 12/13 should not proceed with the questionnaire.

Discussion

I suggest including the following observation in the discussion re Figure 1:

In C the lesion is green and so is the background and in D the lesion is red and the background pink, whereas in B (deuteranope) the surrounding skin is similar to that of the unadjusted image (A) As a result the colour contrast between the lesion and the surrounding skin is greater for Figure 1B than for C and D which makes irregular structures at the border (right side) more visible (similar as in A). This makes the lesion more clearly asymmetrical which may lean the reader towards a malignant diagnosis.

Response: We would like to than the reviewer for this important observation, which led us to more literature digging to find the basis to explain the observation. We found that the distinction of borders can be affected by colors at the border. However, the distinction is higher for high wavelength colors (i.e., red and green), and the distinction is lower for low-wavelength colors (i.e., blue). We provided this to the discussion to further explain this important point:

“It was noted that the borders between the lesion and the surrounding skin is less distinct for deuteranopia-simulated images, which might increase rater’s suspicion toward malignant lesion. This effect was also observed, but with a lesser degree, for protanopia-simulated images, but not for tritanopia-simulated images. This can be explained by previous studies’ observations, where the distinction of image borders increases for higher wavelength colors (i.e., toward the red colors) [19,20], and deuteranopia- and protanopia-simulated images have more low-wavelength colors than normal or tritanopia-simulated images.”.

(The Figure legend should include the appropriate letter for each image respectively)

Response: We added the letters referring to each simulation type.

It would be interesting to see more images. At least a collage similar to Figure 1 but with a malignant lesion. This may or may not support what I have stated about the contrast between lesion and surrounding skin colour. I suggest that the authors evaluate this with every lesion and include those results in the manuscript.

Response: We provided detailed discussion on the impact of contrast. As per journal’s request, we had to replace the image used from the database with a one from our own patients in order to publish the image under CC BY 4.0 license, so it was difficult to include an image for a malignant lesion.

Additional Limitations:

The readers were all inexperienced and an undefined proportion were not practising clinicians. There was a selection bias for suspicious lesions which may have influenced the readers (all lesions had been excised). This could in part explain the lower overall accuracy for benign lesions.

Response: We agree with the reviewer, and we explained this limitation at the end of discussion as follows:

“Moreover, this study was not performed on specialized dermatologist, whom would have more experience and confidence in diagnosing benign lesions as benign depending on more lesion characteristic, other than color. Further studies in this regard might show if including experienced dermatologist might abolish such accuracy difference for participants with color vision deficiency.”

Reviewer #2: Dear Authors,

thank you for your manuscript "Assessing the impact of color blindness on the ability of identifying benign and malignant skin lesions". The idea behind your study is appreciated. However, there are some important issues to be adressed.

1) please improve the writing, there are many errors to be corrected (example: Page 1. Background: Color vision deficiency describes the inability to distinguish certain

shades of color. We found a significantlyhigher mean accuracyasthe accuracy of classifying)

Response: We reviewed the manuscript for typographical or grammatical errors. The above-mentioned typographical error appeared due to pdf-HTML conversion by the editorial system.

2) diagnostic precision. You cite several sources to emphasize that "The diagnosis and referral accuracy of pigmented skin lesions are generally low among primary care physicians compared to dermatologists [12-14]." Despite that fact your study was conducted with participant in their final year in medical school or newly graduated. Please describe your intention to choose this group of participants and discuss a potential impact on the results.

Response: This is an important point to elaborate. We explained the rationale of including participant in their final year in medical school or newly graduated in the introduction as follows: “We conducted this study on participant in their final year in medical school or newly graduated, whom we believe depends more on lesion color criteria in distinguishing benign and malignant lesions. Moreover, it also enabled us to include a relatively larger sample size.”. However, including such sample resulted in a limitation compared to including experienced dermatologist, which we also detailed in study limitations as follows: “Moreover, this study was not performed on specialized dermatologist, whom would have more experience and confidence in diagnosing benign lesions as benign depending on more lesion characteristic, other than color. Further studies in this regard might show if including experienced dermatologist might abolish such accuracy difference for participants with color vision deficiency.”.

3) "we used Vischeck color blindness simulator in Fiji software [16], which has been proved to be highly accurate in simulating color-blind images [17]".

It remains unclear how the whole process was conducted. Please specify how the images were presented (probably online via a private account shown on an uncertified screen). Especially the presentation of colors may vary and is dependent from many factors (model and age of the screen, contrast, intensity, surroundings etc). This might be crucial regarding the results.

Response: We agree with the reviewer on the importance of elaboration on this part. We used Google forms for data collection. To account for device variability, we assessed data about skin lesion classification without simulation, in order to compare with classification accuracy for simulated images. However, as this does not eliminate the impact of device screen variability, we also added a limitation statement in this regard: “Another limitation to keep in mind is that participants used their own devices to complete the questionnaire, which might result in impact of screen color characteristic and brightness on classification accuracy.”

Best regards and good luck!

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: Response letter.docx

Click here for additional data file.

10 Jan 2022

Assessing the Impact of Color Blindness on the Ability of Identifying Benign and Malignant Skin Lesions by Naked-Eye Examination

PLOS ONE

Dear Dr. AlRyalat,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

In agreement with the expert reviewers, the authors have improved the manuscript in the revised version. However, there are still points that need to be addressed. Please see the reviewers' comments below for detail.

Please submit your revised manuscript by Feb 24 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Nikolas K. Haass, MD/PhD

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: See attachment for necessary formatting (bold italics re English grammar issues)

See attachment for necessary formatting (bold italics re English grammar issues)

Reviewer #2: Dear Authors,

thank you for improving your manuscript significantly. "We conducted this study on participant in their final year in medical school or newly graduated, whom we believe depends more on lesion color criteria in distinguishing benign and malignant lesions. Moreover, it also enabled us to include a relatively larger sample size." As far as I know the first statement is a hypothesis. The motivation to include a larger sample size might not justify to conduct such a study with graduating medical students instead of general practicioners or dermatolists. From a clinical stanpoint, conducting this study with general pracitioners would probably be ideal in order to simulate what you wanted to express.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Alexander F. Scheuerle, MD

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.