In-process evaluation of culture errors using morphology-based image analysis.

INTRODUCTION: Advancing industrial-scale manufacture of cells as therapeutic products is an example of the wide applications of regenerative medicine. However, one bottleneck in establishing stable and efficient cell manufacture is quality control. Owing to the lack of effective in-process measurement technology, analyzing the time-consuming and complex cell culture process that essentially determines cellular quality is difficult and only performed by manual microscopic observation. Our group has been applying advanced image-processing and machine-learning modeling techniques to construct prediction models that support quality evaluations during cell culture. In this study, as a model of errors during the cell culture process, intentional errors were compared to the standard culture and analyzed based only on the time-course morphological information of the cells.
METHODS: Twenty-one lots of human mesenchymal stem cells (MSCs), including both bone-marrow-derived MSCs and adipose-derived MSCs, were cultured under 5 conditions (one standard and 4 types of intentional errors, such as clear failure of handlings and machinery malfunctions). Using time-course microscopic images, cell morphological profiles were quantitatively measured and utilized for visualization and prediction modeling. For visualization, modified principal component analysis (PCA) was used. For prediction modeling, linear regression analysis and the MT method were applied.
RESULTS: By modified PCA visualization, the differences in cellular lots and culture conditions were illustrated as traits on a morphological transition line plot and found to be effective descriptors for discriminating the deviated samples in a real-time manner. In prediction modeling, both the cell growth rate and error condition discrimination showed high accuracy (>80%), which required only 2 days of culture. Moreover, we demonstrated the applicability of different concepts of machine learning using the MT method, which is effective for manufacture processes that mostly collect standard data but not a large amount of failure data.
CONCLUSIONS: Morphological information that can be quantitatively acquired during cell culture has great potential as an in-process measurement tool for quality control in cell manufacturing processes.

Introduction

Human-derived mesenchymal stem cells (MSCs) are among the most promising cell sources for clinical applications of cell therapies in regenerative medicine. MSCs, which can be harvested relatively easily from patients, are widely studied somatic stem cells, and have been successfully used in clinical applications, leading to the introduction of commercial cellular products in the market [1], [2], [3], [4], [5], [6].

Industrial-scale manufacture technologies for producing MSCs are required to widely distribute established cell therapies [7], [8], [9], [10], [11]. However, one of the most difficult tasks in cell manufacture is controlling cell quality [7], [8], [9]. Numerous patient-derived variations exist in MSCs; these variations can trigger unexpected alterations in cell quality during their manufacture.

In contrast to other industrial product manufacturing processes, technological difficulties affect cell quality control in cell manufacturing processes. One of the most important issues is the lack of effective in-process measurement methods for monitoring the transition of cellular states during the long and complex cell culture processes. Therefore, the detailed culture process is not completely understood, and little information is available for understanding the “on-going” status of cultured cells. This can impact cell manufacturing facilities in two ways. The first is the possibility of failure of a large culture after a long and costly culture process. Because partial testing by sampling may not indicate the status of the total cellular population, the lack of an in-process measurement for monitoring the entire sample can result in unexpected quality alterations at the end of culture; particularly, the impact of an unexpected yield of cells following stable manufacture is costly. The other is the difficulty in providing feedback for troubleshooting of the process. Understanding the essential points of the process based on feedback information from in-process monitoring data is one of the most common and effective approaches to improving manufacturing processes. However, such data-driven process improvement is currently difficult for MSC culture.

Monitoring of cellular morphology has long been the most practical and effective in-process evaluation technique for cell culture. Determining irregularities in cellular morphology is an important quality criterion in cell culture and clinically applied cell culture protocols. However, cell morphology is typically evaluated in a non-quantitative manner; therefore, special training and skill is required for cell culture experts to stably control the cell culture. Although such expert manual skills are essential, there is an increasing need to mechanize the process using advancing technologies. Image processing and analysis by computational technology have attracted attention, and an increasing number of reports has described the successful application of such technology for cellular evaluations [1], [12], [13], [14], [15], [16], [17], [18]. Our group proposed using computational analysis techniques to quantitatively utilize information obtained from phase-contrast microscopic images as “morphology-based prediction techniques” for non-invasive cell quality evaluation in regenerative medicine [19], [20], [21], [22], [23], [24]. We suggested using a combination of advanced imaging instruments, image-processing, and the multiple parametric machine-learning technique based on morphological parameters to construct models for predicting the experimentally defined quality data using only images.

Although an increasing number of image-based analysis reports has reported the applicability of image-based analysis in non-invasive cell quality evaluations, few studies have focused on detecting “practical and trivial errors” that can occur during the actual cell culture process. In the actual cell manufacture process, the culture protocol is carefully determined and designed to reproduce an identical cellular status during culture. Application failure of such protocols are mostly caused by unexpected errors and are trivial. Therefore, it is more important that the performance of image-based analysis for in-process measurements be investigated under conditions where such variations (such as differences that can occur in the same medium) commonly occur rather than under conditions where the variations are too distinct (such as differences between completely different types of media). However, most culture error is considered as “immatureness” of the experiment and their actual differences from the normal state or their difficulty of detection has not been examined.

Therefore, we compared the differences caused by errors under standard culture conditions that have not been quantitatively examined and investigated the performance of “morphology-based image analysis” as an example of in-process measurement for monitoring cellular status. Practically, conditions under which intentional errors occur were designed and compared with the standard condition: Condition A, normal medium containing 2% dimethyl sulfoxide (DMSO), assuming insufficient removal of DMSO in a cryopreserved cell stock containing 10% DMSO; Condition B, damaged medium that had been repeatedly warmed to represent repeated mishandling; Condition C, normal medium culture under 0% CO2, assuming unexpected errors in the sensors and air supply in the incubator; and Condition D, normal medium culture under 10% CO2, also assuming incubator malfunction (Table S1). Most of these designed errors were exaggerated from the aspect of matured researchers; we consider it meaningful to quantitatively determine the detection performance of these errors. “Logically abnormal” conditions are rarely quantitatively examined and presented; therefore, it is unknown how extensively morphological features respond under such conditions. Understanding the limit of measurement data is important for designing measurement equipment, such as image-based monitoring software. Additionally, to advance cell manufacture, various factors cannot be judged based on the experience of the researcher. When manual processes are carried out using robotics, mis-operation or mis-programming in panel operation is difficult to eliminate. Thus, an automated detection process is needed. Moreover, in industrial manufacturing in which higher production requires increased human resources at a lower cost, not all operators are sensitive to errors. To advance cell manufacture, we determined the effectiveness of quantitative morphological analysis using in-process image data by comparing different conditions (Fig. 1). With visualization and prediction model construction, the intentionally designed “errors in the culture” were quantitatively compared. We found that continuous label-free quantification of morphological parameters effectively described the in-process measurements, supporting the cell manufacture processes for regenerative medicine products.

Fig. 1

Schematic illustration of morphology-based analysis in this study. The analysis step consists of (1) Image acquisition, (2) Image processing and morphology measurement, and (3) Visualization, or (4) Prediction modeling. In visualization, we used the modified PCA plot. In prediction modeling, we constructed a growth rate prediction regression model, culture condition discriminant regression model, and discriminant scoring model using the MT method.

Materials and methods

Cells and culture

Nine lots of human bone-marrow derived MSCs were purchased from Lonza Japan, Ltd. (Tokyo, Japan) and 1 lot was purchased from Lifeline Cell Technology (Frederick, MD, USA). Nine lots of human adipose derived stem cells (ADSCs) were purchased from Lonza Japan, Ltd. and 1 lot was purchased from KURABO (Osaka, Japan). Cells were maintained in MSCGM (Lonza Japan, Ltd.) supplemented with BulletKit (Lonza Japan, Ltd.) under conditions of 37 °C and 5% CO2 according to the companies’ protocols; these were designated as the “Standard” conditions. The antibiotics penicillin (100 U/mL) and streptomycin (0.1 g/mL) were added. The medium was stored at 4 °C protected from light for the Standard condition. Four irregular conditions mimicking the “errors” that can occur in standard conditions were prepared: (Condition A: 2% DMSO) The cells were cultured at 37 °C and 5% CO2, with a final concentration of 2% (v/v) DMSO added to the medium. This condition represented a situation in which DMSO in the cryopreservation stock (using 10% DMSO) was not sufficiently removed from the culture. (Condition B: Damaged medium) The cells were cultured at 37 °C and 5% CO2, but the used medium was warmed for 10 cycles of (37 °C for 2 h) before use. This condition was an exaggerated condition designed to assume that the pre-warming period of the medium was conducted in error. (Condition C: 0% CO2) The cells were cultured at 37 °C in an incubator with no CO2 supply. This condition was designed to mimic a malfunction of both the CO2 monitoring sensor and gas supply bulb. Because the CO2 supply is one of the most essential conditions for maintaining the medium pH, this condition was designed to induce fatal biological effects. (Condition D: 10% CO2) The cells were cultured at 37 °C with 10% CO2 to mimic a malfunction in the incubator. Compared to condition C, a higher CO2 level can maintain the pH of medium and therefore was expected to have minimal biological effects. These four conditions were designed to be over-exaggerated compared to normal cell culture conditions. However, for detecting “unexpected errors” that include mis-operation/mis-programming in the automatic cell culture or oversights by inexperienced operators, such error should be automatically detected for risk control.

Image acquisition

Phase-contrast microscopic images of MSCs were obtained in 6-well plates using the BioStation CT (Nikon Corporation, Tokyo, Japan) as described previously with some modifications [22]. Each condition was evaluated in three wells. For all conditions, images were acquired at 4× magnification (8 × 8 tiling per well, covering 16 mm2, 1000 pixels2/image). Time intervals were set to 6 h, which started at 12 h after seeding. Time points were designated as Time 1 (0.5 days/12 h after seeding) to Time 22 (5.5 days/132 h after seeding). For Conditions C and D, several wells contained non-cellular objects as noise in the image which were eliminated manually.

Image processing

All images were processed and quantified using CL-Quant software (Nikon Corporation) as previously described with some modifications [19], [22]. Five original filter processes were designed: (1) background adjustment, (2) enhancement of texture, (3) cell recognition, (4) noise removal, and (5) filling the blank regions in cell objects (Fig. S1). After recognizing the cellular areas through image processing, cell numbers were measured as recognized objects. Therefore, the cell growth ratio in this study was calculated using the following formula: Cell growth ratio = object number in Time 22/object number at time 1. For morphological profiling, 14 parameters (Table S2) were measured from each recognized cellular object and their statistical profiles were calculated by collecting all measured parameter profiles from 1000 to 3000 cells in the tiling image per well. The morphological parameters were summarized as the statistical values average and standard deviation. Therefore, each condition (= 1 well) was represented by a total of 28 parameters (= 14 parameters × (mean/standard deviation)). The practical morphological parameters used for further analysis are illustrated in Figs. S2–S4.

Morphological transition analysis

The morphological profiles and their time-course transitions were visualized by principal component analysis (PCA) (Fig. S2). First, all samples covering all time-course data with their total morphological parameters were analyzed by PCA. Next, using two determined axes representing PC1 and PC2, data (individual data separated into each time points) were plotted and connected as traits represented as color gradation, indicating their changes over time. All analyses and visualizations were performed using R (version 3.4.1) (R Development Core Team, https://www.r-project.org/).

Construction of prediction models

Two individual prediction models were constructed: growth rate prediction model and culture condition discrimination model. In the former model, all morphological parameters recorded at the time points served as inputs and the image-based growth rate was determined and used as the teacher signals; the linear regression model was applied to analyze this data. A total of 63 samples (21 lots × 3 wells) were modeled. In the latter model, all morphological parameters recorded at all time points served as the inputs, and 5 labels for the culture conditions (Standard and Conditions A–D) were used as teacher labels, to which the linear discriminant model was applied. A total of 315 samples (21 lots × 3 wells × 5 conditions) were modeled. All predictive performance of the models was evaluated by leave-lot-out cross-validation. The morphological parameters of both models were examined to reduce the earlier prediction model, and their time-course window effects were evaluated by plotting model performance (Fig. S3). All calculations were performed using R (version 3.4.1).

Mahalanobis-Taguchi (MT) method

The MT method is a machine-learning algorithm that is only trained using positive data, yielding the formula of the “standard state data” as the unit space and scores the newly applied data by measuring the Mahalanobis distance from the unit space (Fig. S4). Therefore, the MT method does not require both “regular” and “irregular” conventional machine-learning algorithms and provides a discriminative score by measuring the similarity to the regular samples based on information distance. For the unit space data, one-third of the data from Standard samples (sample number; m) were randomly selected for training, while the remaining Standard data and data for Conditions A–D (sample number; n’) were evaluated as test data (Fig. S4). To train the unit data, the “centering data matrix (C)” and its “distributed covariance matrix (∑)” were obtained from the test data. From the test data, the centering data matrix (C′) was calculated. Scores indicating the rate of “irregularity” were obtained using the following formula; (Score) = C∑−1C/m, or C′∑−1C′/n). For statistical evaluation, Student t-test with Bonferroni correction was applied.

Results

Morphology transition analysis for visualizing differences in lot performance

In the first analysis, we quantitatively compared cell growth performance between different cell lots (Fig. 2A) based on their morphological information, as low growth performance is a common failure encountered in cell culture. To visualize morphological transitions, time-course morphological profiles were separately plotted as time-course points in the PCA plot. A single plot of the morphological profile expresses the overall morphological character (Fig. S2), practically described using 28 parameters per image in this analysis, and their trails express their morphological transitions. Among the 21 lots, we observed a wide variety of cell growth rates and morphologies (Fig. S5), but it is difficult to quantitatively assess the morphological profile differences. However, when Lot 6 (which showed the highest growth rate) and Lot 5 (which showed the lowest growth rate) were compared, their traits in the PCA plot showed large differences (Fig. 2B). The direction and speed of their morphological transitions demonstrated that low-growth lots had relatively different morphological profiles and transitions compared to high-yield lots (Fig. S6). Such characteristics of traits were shared between Group A (high-growth lots) and Group B (low-growth lots) (Fig. 2C). Therefore, compared to manually evaluating differences in morphological characteristics, our visualization of their real-time morphological transition provides quantitative recordable data for monitoring the in-process cellular status, with the results reflecting cellular quality.

Fig. 2

Morphology-based analysis of growth rate performance. (A) Growth rate performance among the 21 lots of MSCs used in this study. (B) Representative phase contrast images of MSC lots (Lot 6 and 5) and images of morphological transition analysis. Scale bar = 200 μm. All images were prepared with the same scale. (C) Representative image of morphological transition analysis applied to group A (good growth lots) and group B (bad growth lots). (D) Prediction performance of growth rate-prediction models examining the time-course window effect.

Prediction of future yield from early morphological information

Because non-invasively obtained morphological parameters and their time-course transition profiles were shown to be useful descriptors, we further examined their applicability for supporting the cell culture process. By modifying previously reported concepts of morphology-based prediction models, we predicted the future growth rate of cells (Fig. 2D). “Morphological parameters” were used as explanatory variables, while “its experimentally determined final biological data” were used as objective variables as the dataset for training. When all morphological parameters (0.5–5.5 days) were collected during culture, the prediction accuracy of the growth rate on the last day was 100%. Although such prediction accuracy was achievable only within the morphological parameters containing no data on cell numbers, this approach cannot be used in culture processes because cell number prediction is conducted after harvest for the next process. To fix the errors and failures in the culture process, it is important to examine how early the prediction can be obtained. By comparing different types of prediction models by reducing the number of morphological parameters with a shorter window (Fig. S3), more than 80% accuracy was stably obtained in certain periods of time (Fig. 2D). Prediction accuracy was not stable when the morphological parameters were collected within only 2 days (48 h). However, prediction accuracy become stable and high when the time for morphological parameter accumulation exceeded 54 h. This result indicates that the in-process-measured morphology can detect errors occurring in future cultures at a very early stage. Additionally, high accuracy was achievable even for the variation of 21 lots, including hBMSC and hADSC cultures. This performance indicates that growth rate prediction based on morphological information can be applied robustly for culturing MSCs.

Morphology transition analysis for visualization of errors in cell culture

To further examine the applicability of morphological information to support error detection during the culture process, we investigated whether morphological profiles reveal the differences between “standard” and “deviation” culture statuses. Four deviation conditions with intentionally designed errors added to the standard culture were examined and visualized as time-course traits in the PCA plot (Fig. 3). Although some morphological characters were indicated in their images (Fig. 3A), their differences were compared quantitatively based on the type of trait (Fig. 3B). At 12 h after seeding, Conditions A, B, and D slightly differed from the Standard. However, Conditions B (damaged medium) and D (CO2 10%) gradually followed the traits of the Standard as culturing was continued, indicating the recovery of cell morphology. The morphological traits in Condition A (DMSO 2%) completely differed compared to those of the Standard. Initially, cells cultured under Condition C (CO2 0%) showed morphological similarities to the Standard (12 h); however, the traits gradually changed and differed from those of the Standard. Therefore, Conditions A and C represent examples of errors that accumulated over time and lead to deviations in cellular quality. As a result, visualization of morphological transition can be used to monitor unexpected errors that may occur during the culture process.

Fig. 3

Morphology-based analysis of errors in cell culture. (A) Representative phase contrast images of MSCs under 5 types of culture conditions (Standard and 4 types of intentional errors). Scale bar = 200 μm. All images were prepared with the same scale. (B) Representative visualization image of morphological transition analysis applied to the 5 culture conditions. (C) Prediction performances of culture condition discrimination models examining the time-course window effect.

Prediction of errors in culture based on early morphological information

Because visualization of the morphological transition involves only a relative comparison of the data, we further confirmed whether errors can be evaluated using morphological parameter information. Five discrimination analysis categories were used to examine the prediction of “errors” during culture (Fig. 3C). By comprehensively examining the earliest morphological parameter combination for such predictions (Fig. S3), we found that images collected over more than 36 h were sufficient to provide stable and high discrimination accuracy. Discrimination accuracy was high (>80%) even for initial results when using only four images at different times (12, 18, 24, and 30 h). These data indicate the possibility of detecting unexpected errors that may have escaped detection when using other methods, and the errors can be detected in early stages. Additionally, the suspected error type can be predicted based on morphological profiles.

Discrimination scoring by MT method to evaluate errors in the culture

Although computational machine-learning techniques have been effectively employed in practical industrial settings, two practical essential points must be considered in cell manufacture. First, a sufficient amount of data and variation in the data are lacking. Because cell manufacture is a developing field, there are no large datasets that can be used for computational training. Second, when cell manufacture is conducted, carefully designed standard protocols are typically used to stabilize the process. Therefore, “failure” occurs at a lower rate compared to “successful” data. Data-driven machine-learning approaches can fail to detect differences between the characteristics of “failure,” resulting in less robust models. Recently, more sophisticated algorithms have been applied in the machine-learning field for difficult solutions. However, as the algorithms become more sophisticated, data volume and calculation costs are increased. To compensate for this, the MT method has been used in the manufacturing industries, such as in automobile production, to estimate unexpected deviations. Briefly, the MT method is one of the simplest machine-learning concepts, which only trains the model using “standard data” as unit space and discriminates “irregular data” by measuring the informational distance of the sample from this unit space. Irregular errors are commonly caused by a variety of factors, making it difficult to detect these errors through experiments. In such cases, the MT method only requires standard data and does not require the collection of all types of irregular data for training. Moreover, the MT method uses an extremely simple formula for calculation and thus has extremely low calculation costs. This is the first application of the MT method for morphology-based evaluation of cell culture processes.

We randomly extracted one-third of the Standard samples and first trained the model as a unit space. The remaining Standard samples with error conditions (Conditions A–D) were used to determine the discrimination ability of the MT method (Fig. S4, Fig. 4). Condition A (2% DMSO) samples showed a large deviation score from the Standard and this score increased over time. Conditions B (damaged medium) and D (10% CO2) showed some deviation; however, most samples showed nearly the same score as the Standard. Condition C (0% CO2) did not initially show score deviation. However, its deviation score increased dramatically as cells started to die because of the lack of CO2. Thus, the MT-method scoring shows that machine learning can be used to evaluate the errors rather than using unfit data with standard machine-learning approaches. Furthermore, morphological information can is a useful descriptor in such cases.

Fig. 4

Morphology-based discriminant scoring of error samples using the MT method. (A) Schematic concept of the MT method using morphological information. (B) Time-course discriminant scoring by the MT method for detecting error samples from the 5 culture conditions. Each dot indicates an image converted into 28 morphological parameters (total 63 images per condition). The Y-axis indicates the discrimination score calculated from the MT method, which reflects the Mahalanobis information distance from the centroid of unit space to the sample. (C) Bar plot of discriminant scoring by the MT-method. **p < 0.01, ***p < 0.001. The Y-axis indicates the discrimination score calculated from the MT method.

Discussion

Owing to advances in regenerative medicine and cell culture technologies, developing methods for supporting industrial-scale cell manufacture has become very important. To overcome one of the most critical issues, quality control, in manufacturing cells for therapy, we proposed analyzing cell morphological information. By combining image processing technology for measuring several morphological parameters and computational machine-learning technology to determine the relationships between morphological information and cellular quality, non-invasive image-based cell quality evaluation can be used to predict and evaluate the performance of cell culture. However, few studies have evaluated the applicability of this method to support practical cell manufacture processes involving unexpected culture error detection. Therefore, in contrast to other morphology-based applications described in our previous reports, we applied two conceptual approaches for image analysis, PCA-based real-time visualization of morphological changes and prediction modeling introducing MT method, for detecting deviations during cell manufacture processes. We also quantitatively examined the limit of using morphological parameters by evaluating intentionally induced and exaggerated error conditions, as the limits of the method must be identified to ensure appropriate software design.

By applying PCA, time-course visualization of morphological transitions was used to show differences in lots or differences between standard and irregular samples during culture. Compared to end-point imaging analysis, such as fluorescent staining imaging, time-course visualization can indicate and record sample transitions. When intentional error conditions (Conditions A–D) were compared to Standard conditions, slow changes (Condition C) or rapid changes (Condition A) were observed. Moreover, change-like symptoms were observed in the early phase, which recovered during culture (Condition B and D). Such detailed differences are important in-process measurement data, not only for quality evaluation, but also as feedback information for process improvement. Additionally, by comparing the rather exaggerated error conditions, we determined the maximum value of each axis in our PCA-based time-course visualization. This was an important step in PCA visualization, as PCA is designed to maximize data deviation. The use of exaggerated data which show maximized parameter values is required, so that PCA can over-expand small differences in data.

The computational machine-learning approach is a well-established and widely supported approach for automatic evaluation of the culture process in a data-driven manner. Particularly, for image data, which includes a massive number of measured parameters, computational modeling is an effective application. Therefore, with a sufficient amount of high-quality data, this image analysis technology can be applied to the field of cell evaluation. However, for insufficient amounts or quality of data amount, the use of machine-learning algorithms considering only advantages may provide inaccurate results. In practical cell manufacturing, the accumulated data are not always suitable for conventional machine-learning strategies. An essential factor that is often overlooked in image-processing and machine-learning cell evaluation studies is the variation of data in cell manufacture. For example, if undifferentiation and differentiation media are compared as counterpart conditions, such conditions can have a large effect on cells and tend to show large morphological changes. In a computational training problem, such data with large differences are an ideal task. However, in practical cell manufacture processes, irregular data are commonly reported even in routine work. Therefore, the variation in data can be small. Moreover, even if the variation is low, the cause of variation can be completely different and may affect the morphology in a completely different manner. Therefore, a sophisticated evaluation system that can sensitively detect small differences without forcing the user to collect all possible failure cases is needed for cell manufacture processes. We demonstrated the use of both a conventional machine-learning model (regression model) and MT-method model, which require only standard data. Detection of irregular samples was possible using both models and their visualization plots. Therefore, multiple approaches can be used to provide a support system to cell manufacturers, which reveal deviations in the cell manufacture process, and users should combine the advantages of each technique.

Conclusions

In conclusion, we determined both the importance and the descriptive performance of multiple morphological parameters extracted from time-course images as in-process data measurement data and demonstrated its practical applicability for detecting unexpected errors that may impact standard culture operations.

Conflicts of interest

The authors state no conflict of interest.