Current status of pathological image analysis technology in pharmaceutical companies: a questionnaire survey of the Japan Pharmaceutical Manufacturers Association.

The Japan Pharmaceutical Manufacturers Association (JPMA) has instituted a task force (TF) for the "development of image analysis technology for histopathological changes" as part of the collaboration for realizing cutting-edge drug development since 2016. In recent years, there has been progress in the digital pathology technology; however, few applications in nonclinical drug development studies have been observed. Therefore, TF performed a questionnaire survey to investigate the current status, needs, possibility, and development of image analysis. The subjects were 35 member companies of the JPMA. The questionnaire was set to assess the efficacy and/or safety of researchers engaged in pathological evaluations for each company. The questions focused on the experiences, implementation, and issues regarding histopathological examinations; the need for image analysis software; and future views. Valid responses were obtained from 26 companies. Most companies assumed that the beneficial aspect of image analysis is to gain objectivity and persuasiveness; however, challenges in the analysis conditions with regard to accuracy and without subjectivity persist. Additionally, there seems to be a need for image analysis software with advanced digital pathology technology, with most companies believing that, in the future, pathological evaluations will be partly performed by computers. In conclusion, in this questionnaire survey, TF extracted the current status of image analysis in nonclinical studies performed by pharmaceutical companies and collected opinions on future prospects regarding the development of image analysis software with advanced digital pathology technology. ©2020 The Japanese Society of Toxicologic Pathology.

We set up a task force (TF) as one of the collaborative activities among industries, toward realizing cutting-edge drug development at the Research and Development (R&D) Committee of the Japan Pharmaceutical Manufacturers Association (JPMA). This TF has been working on developing image analysis technology for histopathological changes since 2016.

Pathological evaluations using glass slides are a qualitative or semiquantitative morphological observation performed by pathologists using hematoxylin and eosin (H&E) staining and an optical microscope; however, for several decades, there have been attempts to quantify lesions by morphometry using image analysis software[1]. In recent years, there has been rapid progress in morphological measurement technology owing to the dissemination of digital technology, such as whole slide imaging, for digitizing the morphological information of glass slides. Furthermore, it is possible to utilize commercially available digital pathology applications (Visiopharma, Definiens’ Tissue Studio, and Indica Labs’ HALO) or open-source software programs such as QuPath or Image J[2]. In clinics, computerized pathology diagnosis support software has been developed and used for some types of cancer[3, 4]. In addition, artificial intelligence (AI), which has rapidly advanced in recent years, has excellent image recognition (e.g., human faces and objects). Accordingly, these applications can be employed in image recognition and quantification of histopathological changes in the evaluation of nonclinical efficacy or safety studies during drug development and are expected to provide useful information.

Meanwhile, there has been no tangible implementation of a computerized automated image analysis software for a wide variety of histopathological changes observed in nonclinical drug development studies, with the exception of a few studies[5, 6] with a detailed understanding of this situation still unknown. Therefore, the authors conducted a questionnaire survey on the R&D Committee members of the JPMA to understand the issues, possibilities, and the necessity for the development of image analysis of histopathological changes in nonclinical studies.

The questionnaire was prepared by a TF member, and the questionnaire survey was performed among 35 member companies belonging to the R&D Committee of the JPMA. There were differences in the purpose and research contents in nonclinical efficacy or safety studies; therefore, the questionnaire was established for researchers/technicians who were engaged in efficacy and/or safety pathological evaluations. One response was obtained from one company, including answers on efficacy or safety or both. Questionnaires including experiences, implementations, histopathological evaluations, issues, development, and future visions for image analysis software were prepared. The questionnaires consisted of questions numbered Q1 to Q26. Furthermore, the option “Other” with a free description was collected in case of some questions. The questionnaires and the results are shown in charts Q1–Q26. Valid responses were received from 26 out of 35 member companies. Six companies were not able to respond, and three companies did not answer.

The questionnaires were distributed on February 13, 2018, through the office of the JPMA, and collected by March 13, 2018. The answered questionnaires were anonymized through the JPMA secretariat and returned to the TF, and then the TF gathered and analyzed them.

The TF questioned, “Which industry does your company belong to?” Almost all valid responses were from the Japanese pharmaceutical companies (Table 1). Furthermore, among the companies that provided valid responses, responses were obtained from almost all efficacy and safety research divisions.

Table 1.

Companies that Responded on the Development of Image Analysis for Histopathological Changes in Nonclinical Studiesa

I. Current status of experiences and implementations on image analysis (Q1–Q11)

Experience (Q1, Q2):

All efficacy research divisions that provided responses had experience in performing image analysis (Q1). Overall, morphometry was performed more frequently in efficacy research divisions than in safety research divisions (Q2). Image analysis was performed several times a year in both divisions. More than two-thirds of the efficacy research divisions or half of the safety research divisions conducted image analysis several times a year or routinely.

Do you conduct image analysis using histopathological glass slides?

What is the frequency of image analysis using histopathological glass slides?

Target tissue and endpoint (Q3, Q4):

Notably, the tissue targets for image analysis were the liver, kidney, lung, brain, heart, intestine, spleen, and pancreas (in descending order) (Q3). The endpoints, quantified by image analysis, were the area, the number of cells or any objects, length/thickness, and staining intensity in immunohistochemistry (in descending order) (Q4). The image analysis of the spleen, kidney, brain, and intestinal tract tended to be performed more often in efficacy studies than in safety studies. All these items tended to be more frequent in efficacy studies than in safety studies. In particular, staining intensity in immunohistochemistry was mostly performed in efficacy studies.

Which organs or tissues are analyzed by the software? (Multiple answers allowed: M/A)

What is quantified by image analysis? (M/A)

Software (Q5):

In descending order, the types of image analysis software used included Image J (free software), Image-Pro (Media Cybernetics), Leica Biosystems software, Definiens Tissue Studio (Definiens AG), and HALO (Indica Labs). In terms of free description, WinROOF (Mitani Corporation) was highly mentioned in both the efficacy and safety research divisions (six and three companies, respectively).

What kind of image analysis software do you use? (M/A)

Staining method (Q6):

In image analysis, H&E staining, immunohistochemistry, and special staining methods were equally utilized. Furthermore, all staining methods were more frequently adopted in efficacy studies than in safety studies. A description of special staining in descending order is presented as follows: Masson Trichrome (six companies) and Sirius Red (six companies) in case of efficacy studies, and Sirius Red (four companies) and Masson Trichrome (three companies) in safety studies. Staining methods for collagen fibers were well adopted in both divisions.

What is the staining method used in image analysis? (M/A)

Animal species (Q7):

Q7 was requested only from safety research divisions. The animal species subjected to image analysis in safety studies were rats, mice, dogs, and monkeys in descending order. Rats are the most commonly used experimental animals in nonclinical safety studies.

What animal species do you use for image analysis? (M/A) This question was asked only for safety.

Target lesion (Q8):

Q8 was requested only from the safety research divisions. The most image analyzed lesion was fibrosis, followed by an increase/decrease in cells, atrophy/hypertrophy, degeneration/necrosis, and tumor/hyperplasia. The frequency of measuring inflammation morphology was relatively low.

What kind of lesion is quantified by image analysis? (M/A) This question was asked only for safety.

Apparatus (Q9, Q10):

Digital imaging performed with digital cameras was the most used method, followed by digital imaging with virtual slide scanners (Q9). Among companies that used virtual slide scanners (i.e., whole slide imaging [WSI]) in Q9, Leica Biosystems or Hamamatsu Photonics slide scanners were widely employed models in Japanese pharmaceutical companies (Q10). Most companies use either of these two digitizing methods, employed more frequently in efficacy research divisions than in safety research divisions. Reportedly, WSI represents a paradigm shift in pathology, playing a pivotal role in image analysis[7].

How do you obtain the digital images of histopathological glass slides?

To those who answered “virtual slide scanner” in Q9, which company’s digital slide scanners do you use? (M/A)

Image analysis in drug application study (Q11):

Image analysis for application studies was performed more frequently in efficacy research divisions than in safety research divisions. Although seven companies responded with “for application,” they comprised less than one-fourth of the nonapplication studies. This is partly due to the fact that key issues that require consensus for the adoption of digital pathology and image analysis have not been established. Furthermore, guidelines covering the use of digital pathology or image analysis in nonclinical studies, as in clinical trials, are lacking[2].

Which type of study did you conduct for image analysis?

II. Histopathological examinations, and issues and merits on image analysis (Q12–Q18)

Histopathological examination and peer review (Q12, Q13, Q14):

The histopathological examination of nonclinical safety studies is routinely conducted by qualified pathologists in almost all companies (Q12). However, approximately two-thirds of the safety researchers opined that bias was present in findings, terminologies, and grading criteria of the qualified pathologists. Similarly, among efficacy researchers, two-thirds of the companies assumed the presence of bias when conducting image analysis (Q13). Pathology peer review is commonly performed to improve the accuracy and quality of pathological diagnosis in nonclinical safety studies[8]. Therefore, the TF questioned the safety researchers regarding peer review. The results demonstrated that two-thirds of companies utilized pathologists from within the same company (Q14). The conventional, subjective, histopathological scores by experienced pathologists are the gold standard[9]. Even for each qualified pathologist, it was a formidable dilemma to be consistent in his/her findings among studies.

Do qualified pathologists conduct histopathological examinations in safety studies at your company? This question was asked only for safety.

(For Efficacy) Does bias pose a problem when conducting image analysis? (For Safety) Have you ever experienced bias among pathologists or studies regarding findings, terms, and grades obtained by pathologists?

What measures need to be employed to resolve bias among pathologists? (M/A) This question was asked only for safety.

Bias (Q15, Q16):

Most safety researchers provided the reply “partly eliminated” (Q15) and consider that the quantification of histopathological changes will reduce bias. Quantifying morphological changes is considered to lead to improvements in objectivity, reliability, and robustness[5].

The caveat of reducing bias, when conducting morphometric analysis, is that the alignment of the staining conditions for specimens is the most common answer for both efficacy and safety research divisions, followed by skilled persons/same persons performing blind analysis (Q16). Bias can affect scientific investigations and even damage a study’s validity depending upon its degree[9, 10]. To avoid preanalytical variables, the preparation of a suitable specimen to be used for the measurement of tissue morphology should be emphasized. Furthermore, as well known, morphometric analysis requires trained and skilled histotechnologists to produce pristine specimens[11].

Do you think that the bias in pathological findings will be eliminated by image analysis and quantification? This question was asked only for safety.

How do you reduce bias during image analysis? (M/A)

Merit of image analysis (Q17):

The efficacy and safety researchers of several companies believed that the advantage of quantifying lesions by morphometry was to obtain objectivity and persuasiveness. The term “objectivity” is the opposite of subjectivity. Objectivity is defined as a method, process, statement, or decision that is based on facts and is not influenced by opinions, perceptions, or emotions[12]. On the contrary, “persuasiveness” is the quality of being able to convince a person to take a course of action or believe a particular idea. Software is vital for digital pathological applications and plays an important role in providing objectivity and persuasiveness in the interpretation of tissue structures and analysis.

What is the merit of quantifying lesions by image analysis? (M/A)

Issues in conducting image analysis (Q18):

The issues were subdivided into four categories: specimen preparation, bias, software, and evaluation. Interestingly, there were no opinions regarding the captured digital images. WSI is widely utilized in research. Therefore, it is important to collect data that demonstrate a valid equivalent performance of digital imaging against glass slides[13].

What is the rationale for difficulties in image analysis?

III. Future visions and development of image analysis software (Q19–Q26)

Future visions (Q19, Q20):

If an image analysis software that could automatically quantify or accurately detect lesions was developed, most efficacy researchers positively agree to use this software (Q19). Furthermore, most safety researchers agree to use software capable of morphometric recognition similar to the pathologists (Q19). The comparable number of the answer was collected in each purpose: add objectivity to findings, quickly conduct image analysis, process a large amount of data in efficacy division, and support use in grading for safety division (Q20).

(For Efficacy) Do you want to use image analysis software that can automatically analyze lesions quantitatively for efficacy evaluation? (For Safety) Do you want to use image analysis software that has the same morphological recognition capability and ability to accurately obtain findings as a pathologist?

What kind of purpose do you have for software as described in Q19? (M/A)

Useful target tissue and lesion analyzed automatically (Q21, Q22):

The useful organs or tissues analyzed and quantified automatically by the software were the liver, kidney, lung, spleen (including the thymus and lymph node), brain, heart, and intestine (in descending order) (Q21). Both researchers considered that it would be most beneficial to possess the capability to automatically analyze the liver. Comparing the results of Q3 and Q21 (current status and future, respectively), the safety researchers tended to select all choices compared with Q3.

The useful lesions analyzed and quantified automatically by the software were a decrease/increase in cells, atrophy/hypertrophy, fibrosis, inflammation, degeneration/necrosis, tumor/hyperplasia, and regeneration (in descending order) (Q22). Researchers in both groups opined that it would be highly advantageous to be able to automatically analyze the changes in the number of cells, and it was considered useful if automatic measurement could be performed employing a computer. Comparing the results of Q8 and Q22 (current status and future, respectively), a tendency to select all choices was observed compared to Q8 (not asked for efficacy).

What kind of organ lesion would be useful to automatically analyze and quantify? (M/A)

What kind of lesion would be useful to automatically analyze and quantify? (M/A)

Software as a support tool (Q23, Q24, Q25):

All companies opined that automated image analysis software could be used as a support tool for efficacy evaluation (Q23). Interestingly, most of the responding pharmaceutical companies believed that morphometric recognition will be partly replaced by computers (Q24). Q25 indicated the rationale behind the Q24 answers. Approximately three reasons were indicated for the answers: affirmative, mutual, and negative. In case of the most selected, “partly replaced,” the respondents had a mixture of the above three positions. For “replaced,” the answers were apparently affirmative and included future forecasts. In contrast, for “not replaced,” the respondents replied that morphological recognition will not be replaced by computers owing to the differences in the roles between humans and computers. Automated image analysis has already been developed and partly implemented in human surgical pathology[3, 4]. Computer technology with AI and machine learning will increasingly become integrated into the evaluation workflow of nonclinical studies when objectivity and persuasiveness are established.

Do you think that automatic image analysis of pathological tissue slides can be a supportive tool for efficacy evaluation? This question was asked only for efficacy.

(For Efficacy) What do you think about the possibility of future efficacy evaluations being replaced by computers with the advancement of morphological recognition technology? (For Safety) What do you think about the possibility that, in the future, the ability to recognize morphology will utilize computers with the advancement of morphological recognition technology?

Regarding the previous question, tell us why you think so. Freely describe your reason.

Development of software with AI technology (Q26):

If image analysis software with highly advanced AI technology are developed, efficacy researchers primarily expect an increased data processing capacity followed by labor-saving, accuracy, reproducibility, and innovativeness. The safety researchers selected objectivity the most, with comparable numbers of other items, such as process large amount of data, labor-saving, accuracy, and ability to detect lesions.

Currently, the primary diagnosis by WSI has already been approved overseas[14, 15], and the next step is to implement software with a highly advanced image analysis algorithm in nonclinical studies on both efficacy and safety. Nonetheless, the nonclinical safety study in compliance with GLP requires the development of regulation, and the bars seem to be difficult at this time.

In conclusion, in this questionnaire survey, the TF extracted the current situation of image analysis in nonclinical studies performed by pharmaceutical companies and collected opinions on future prospects regarding the development of image analysis software with advanced digital pathology technology.

If pathology diagnosis software with an advanced morphological recognition function that features AI technology was developed, what is your highest expectation?

Disclosure of Potential Conflicts of Interest

The authors declare that they have no conflicts of interests.