Single-Image-Referenced Colorimetric Water Quality Detection Using a Smartphone.

In this paper, we present a smartphone platform for colorimetric water quality detection based on the use of built-in camera for capturing a single-use reference image. A custom-developed app processes this image for training and creates a reference model to be used later in real experimental conditions to calculate the concentration of the unknown solution. This platform has been tested on four different water quality colorimetric assays with various concentration levels, and results show that the presented platform provides approximately 100% accuracy for colorimetric assays with noticeable color difference. This portable, cost-effective, and user-friendly platform is promising for application in water quality monitoring.

Introduction

Colorimetry is a major technique used in physical and analytical chemistry that aims to quantify the concentration of colored solutions.[1] Colorimeters are instruments that quantify concentration via absorbance measurements using spectrophotometry and Beer–Lambert law,[2] and they are available for both laboratory and field use depending on the complexity of the experiment. The advances in smartphone technology, with the help of three-dimensional printing, helped to realize colorimetric measurements for a wide range of chemical and biological analytes.[3] Several smartphone colorimeter designs were demonstrated for water quality sensing for environment,[4,5] leaf color analysis for agriculture,[6] pH,[7,8] glucose sensing,[9] and peanut allergen detection.[10] Apart from colorimeters in the traditional sense, color data were obtained from paper-based sensors, where color change is quantified by parameters in various color spaces, that is RGB, HSV, and L*a*b*.[11−15] A representative study in this category is ref (15), where alcohol concentration was detected in saliva using paper-based test converting images from RGB to HSV color space. This methodology has the advantage of requiring only a smartphone for taking and processing an image with variety of colors. Further, nontraditional colorimeters were demonstrated on liquid samples in vials for chlorine detection[16] and fruits for ripeness estimation.[6] These systems estimate color by using analytical formulas extracted from color space parameters and are prone to disadvantages of JPEG images such as low bit depth and heavy postprocessing (white balance, contrast, and brightness adjustment).[17,18] However, we have previously shown that JPEG images could play a significant role in colorimetric detection when used as a part of training set for machine learning algorithms.[12,19] Using more advanced methods such as machine learning, classifiers require much larger data sets and high computational power for training.[19,20] Here, we show a new methodology for simple colorimetric detection of water quality using smartphone-embedded color matching algorithms on JPEG images under well-defined experimental conditions. This study differentiates itself from our previous work[12] by using simple color matching algorithms to detect the concentration in real time with rapid response necessary for lab-on-a-chip systems.

Recently, researchers have proposed lab-on-a-chip design for several distinct applications.[21,22] Salles et al.,[21] used paper-based test to detect explosive types using hierarchical clustering analysis and principal component analysis (PCA) regarding the color discrimination of the explosives. Images were captured in a closed chamber to eliminate ambient light conditions. Helfer et al.[22] employed linear correlation and PCA methods, respectively, for univariate and multivariate analysis. They stated that the results of univariate analysis were not statistically significant and ambient conditions might one of the reasons as images were not captured in a controlled environment. However, they reported promising results from multivariate analysis which used eight different color spaces together with PCA and clustering methods including red, green, blue, hue, saturation, value, lightness, and intensity. In this study, our proposed method is computationally cheaper than aforementioned studies,[21,22] which reduces computation time for output calculation. In addition, we aim to quantify the solution concentration level, which is more challenging compared to the discrimination of the distinct inputs.

Vashist et al.[23] proposed a smartphone-based colorimetric reader for enzyme-linked immunosorbent assays, where images were captured in a controlled environment illuminated from the bottom. Ozcan and his research group[24−29] have significantly contributed to this field, driven by applications such as food allergen testing,[24] urine[25] and blood[26] analysis, immunoassays,[27] microscopy,[28] and cytometry.[29] Here, for the first time, we have proposed using local database referenced with a single image for the quantification of concentration level of solutions. The proposed system is able to calibrate an entire set of concentrations for four water quality parameters (nitrite, phosphate, chromium, and phenol) using a single image to obtain the reference data, which is further used to predict concentration of an unknown solution with great accuracy. The main motivation is to design a portable, cost-effective, and user-friendly platform and develop the experimental methodology that can easily be applied to any lab bench without the need for expensive equipment and has the potential to transform colorimetry for both regular and citizen scientists.

Results and Discussion

In Figure a,b, the performance of the ΔE* and color-correlation (CC) methods in detecting phosphate concentration levels is presented with confusion matrices, where each row of the matrix represents the instances in an actual (true) class, whereas each column represents the instances in a predicted class. The confusion matrices were created based on selecting 50 random test areas from six different concentration levels of phosphate, nitrite, and phenol solutions and from seven different concentration levels of chromium solutions. Figure c shows the selection of random areas from the region of interest. Because of the space constraint, the results of other assays are provided in the Supporting Information. One can see from the confusion matrices in the Figure a,b that ΔE* method outperforms the CC method in correctly identifying the concentration levels of phosphate, where the maximum number of misclassified concentration is only one for each true label. Furthermore, misclassification only happens between neighboring concentration values. The overall performance of the two methods was compared by calculation detection accuracy, which is the ratio of sum of diagonal elements of confusion matrix to total number of data points. The detection accuracy of the CC method was obtained as 23.3, 33, 26.3, and 26.2%, whereas it was 100, 98.7, 100, and 76% for the ΔE* method for chromium, phosphate, nitrite, and phenol, respectively. Hence, ΔE* method should be preferred to run in the Android app because of its superior overall performance over the CC method. To further evaluate the performance of the ΔE* method in automatically identifying the concentration levels of the solutions, we use metrics of classification report, referred to as precision, recall, and f1-score. Precision is defined as the proportion of true positive events to the sum of true and false positive events in a classification task. Recall measures the proportion of true positive events to the sum of true positive and false positive events, whereas f1-score considers both the precision and the recall of the binary test to compute a total score, f1, which is equal to 1 for perfect precision and recall. We present the precision, recall, and f1-scores of the ΔE* method for automatically classifying phosphate concentrations in Table , where f1-score of 0.99 is reached. For classifying concentration levels of both chromium and nitrite, the proposed approach achieved even better results with a perfect confusion matrix and an f1-score of 1.

Figure 1

Confusion matrices using the ΔE* and the CC methods for phosphate are given in (a,b), respectively. Random area selection from the solutions for the calculation of performance metrics is shown in (c).

Table 1

Classification Report of Color Matching Algorithm (ΔE*) for Phosphatea

	solution
concentration	precision	recall	f1-score	support
0	1	0.98	0.99	50
0.25	0.98	1	0.99	50
0.50	0.98	0.98	0.98	50
0.75	0.96	0.98	0.97	50
1.00	1	0.98	0.99	50
3.00	1	1	1	50
avg/total	0.99	0.99	0.99	300

Support is the number of occurrences of each particular class.

However, the misclassification of the concentration levels of phenol is significantly higher compared to the misclassification of phosphate, nitrite, and chromium as the confusion matrix of phenol diverges from being diagonal (see Supporting Information). The precision, the recall, and the f1-scores for phenol are obtained as 0.77, 0.76, and 0.76, respectively.

The diminished classification performance is attributed to the less pronounced color transition between successive phenol solutions in Figure b. To analyze the effect of color transition on the performance, histogram representations of phosphate and phenol are given in Figure . The histogram representation of phosphate was shown with stairs for 0, 0.75, and 3 ppm, and the transition of R, G, and B channel by the concentration is very distinct. However, it is more similar for phenol shown with bar plots. Nevertheless, we believe that our proposed single-image referencing is a better approach to identify subtle color transitions between phenol concentrations (i.e., 0–0.20 ppm) compared to the human eye inspection.

Figure 3

Experimental setup is given in (a) and the four different colorimetric assays used in the experiments are shown in (b).

Figure 2

RGB histogram representation of phenol and phosphate.

In some applications,[20,23] there is a tendency to create a big data set, including images from various smartphones to make the system robust to variations in smartphone specifications, which causes increased computational complexity. One advantage of the proposed system is that the training data set needs to be created by the user for the smartphone to be used. As the training and test images are captured with the same smartphone, the issues caused by specifications of the smartphone are eliminated.

Despite the fact that our proposed approach is very promising for colorimetric applications, there are some constraints and limitations associated with it. First, the method relies on training discrete levels of concentration and the experimental phase assumes that the unknown solution is also at a discrete level. If an unknown solution with an untrained concentration is tested, our app will assign the closest concentration as the result. This is different from intuitive thinking in spectrophotometry, where an analytical formula is used to predict any concentration level. The detection accuracy depends on the color transition of the solutions, and poor color transition may result in reduced accuracy for smaller concentrations, as in the case of phenol experiment. Second, the experimental conditions need to be kept constant between training and experimental phases. To get rapid response, each training data set was created for the respective solution and experimental conditions. Therefore, the required computational power to detect the concentration level is minimized. The user needs to keep smartphone camera settings the same, while making sure the illumination conditions do not change. Any relocation of the imaging setup to another illumination setting will cause retraining of the colorimetric assays. It is worth mentioning that because it is supervised learning, the predictions for untrained samples are prone to poor results.

Conclusions

In this paper, we devote our endeavors to build a smartphone platform that is able to monitor water quality in a portable, low-cost, and user-friendly manner. Our platform runs on an Android app and has been tested with four different colorimetric assays with different color transitions between successive solutions. The main features can be summarized as follows:

the user can create reusable experimental models for respective assays;

multiple regions of interest can be accurately selected in real time; and

rapid and reliable results are calculated using pretrained models.

All pictures were captured by a smartphone in a controlled environment to satisfy the similar conditions for all pictures and also to eliminate the issues caused by illumination variations. A reference model was created for each assay to calculate the concentration level of unknown solutions with the help of color matching algorithms. The experimental results demonstrated that the discrete concentration level of the individual solutions could be reliably calculated. The detection accuracy can even reach 100% if the color transition in the reference model is more pronounced. The limitations associated with the proposed platform could be an interesting research area in the future.

Materials and Methods

Experimental Design

To test our methodology, a conceptual experimental setup was designed as illustrated in Figure a. A cardboard box (40 × 40 × 25 cm) painted white inside was used to provide a portable photo studio suitable for lab benches. A slim tube with surface-mounted white light-emitting diodes inside (LedTimes Co., LT-9960S) was placed inside on the box ceiling 15 cm over the samples for illumination purposes. The setup includes an optional sample holder platform at the same height as the camera. Images were taken using a smartphone (LG G4, 1/2.6 in. sensor size with 5312 × 2988 resolution, 1.12 μm pixel size) located 30 cm away from the samples. The smartphone camera was used in manual mode, and the shutter speed (1/30 s), ISO level (50), focus level (f1.8), and white balance settings were all kept constant during imaging.

Assays

Four different colorimetric assays of nitrite (NO2–), phosphate (PO43–), hexavalent chromium (Cr(VI)), and phenol were prepared according to the colorimetric standard methods (4500-NO2–-E, 4500-P-E, 3500-Cr-B, and ISO 6439).[30,31] The preparations of the assays are provided in the Supporting Information. Figure b shows the photographic images of the solutions with different concentration levels ranging from 0 to 3.00 ppm. It can be clearly observed that as the concentration of each analyte increases, the NO2–, PO43– and Cr(VI) solutions get noticeably darker, whereas phenol assay color difference is not substantial.

ChemTrainerSIR Mobile App

To test the colorimetric assays, we upgraded our custom mobile application ChemTrainer[19] for single-image reference (SIR) named as ChemTrainerSIR. Screenshots of the app given in Figure explain the flow of training (from b to g) and experiment procedures (from h to m). Users can train models for multiple assay types and later use these models to measure concentration levels in newly acquired photographs. To train a model with the app, the user first enters the name of the chemical compound as the name of the model (e.g., phosphate), the units of measurement (e.g., ppm), and the number of samples with known concentration levels (e.g., 0–3) that their photographs will have. Then, the user either captures the photograph of the collection of samples or loads it from the smartphone gallery. The user proceeds to mark each sample with a rectangle (preferably inside the vial) and enters the known concentration level for that sample. The photograph, marked areas, and their concentration levels are stored in a designated folder in the devices internal storage.

Figure 4

(a) Steps of the colorimetric detection with a single image. Training steps are presented at the top row (b–g) while the user utilizes the trained model during the experimental phase as in the bottom row (h–m).

Digital Image Processing

When the user wants to measure a solution of unknown concentration on a new image, the previously trained model that is appropriate for the solution to be tested first needs to be selected. Then, the user either captures a photograph of the solution or loads it from the gallery and marks the colored area on the photograph. The app then uses the trained model along with color matching algorithm to calculate the concentration level of the solution. The photograph, marked area, and calculated concentration level are stored in another designated folder in the devices internal storage.

In this demonstration, an unknown solution was centered alone for testing, whereas reference solutions were imaged as a group in a single shot during training. Hence, some of the reference solutions were not located at the center, although this has a negligible effect on the color reproducibility due to controlled illumination conditions.

One challenge in running colorimetric tests in a smartphone is to be able to finalize the result in a short time. Because the processing power of a smartphone is usually limited, a simple and efficient method is preferred rather than sophisticated methods, as they usually require high computational power. Therefore, we employ two different color matching algorithms with low computational complexity and compare them to select the best method to be run on a smartphone. The first one is CC method,[32] which calculates correlation coefficients between test image and trained images. The value of the coefficient varies between 0 and 1, where the highest similarity is indicated by 1 and the lowest similarity is represented by 0. The second method is based on measuring color differences between two images using deltaE (ΔE*) distance metric obtained from the CIE 1976 L*a*b* color difference formula[33]Here, L*, a*, and b* are dimensions of the CIE L*a*b* color space, where L* axis represents lightness in the range of black (0) to white (100), whereas the a* axis varies over red (+a*) to green (−a*) and the b* axis describes yellow (+b*) to blue (−b*). Contrary to the CC method, ΔE* calculates a lower score for the similar images as the distance between similar images is smaller than the distance between dissimilar images. In this study, all images are converted to L*a*b* color space as it is more robust to illumination variations.[34]

To test the Android app and the color matching algorithms, the colorimetric assays in Figure b were first processed from the captured images and saved with their corresponding concentration values. During the experimental test phase, the individual photograph of each concentration in every assay was taken and the marked area was cropped and sent to CC and ΔE* methods to detect the corresponding solution. Here, the image processing algorithm works as follows. First, the cropped patch from the test assay was converted to L*a*b* color space as all reference patches were in that format. For ΔE* score calculation, ΔL*, Δa*, and Δb* were calculated by subtracting L*, a*, and b* channels of test and reference images, respectively. Then, ΔE* for each pixel was calculated using eq . By averaging ΔE* scores of the pixels, final ΔE* scores between test and reference image were obtained. This process was repeated for all reference images. Therefore, one test image had several ΔE* scores, and the smallest one points to the most similar matching. Similar process was repeated for the CC method and correlation coefficients between test image and reference images were calculated, and the highest coefficient indicates the highest similarity. These two algorithms were compared in terms of performance metrics regularly used in machine learning for multiclass classifiers. These metrics are confusion matrix, detection accuracy, precision, recall, and f1-score.