Automated waste-sorting and recycling classification using artificial neural network and features fusion: a digital-enabled circular economy vision for smart cities.

Waste generation in smart cities is a critical issue, and the interim steps towards its management were not that effective. But at present, the challenge of meeting recycling requirements due to the practical difficulty involved in waste sorting decelerates smart city CE vision. In this paper, a digital model that automatically sorts the generated waste and classifies the type of waste as per the recycling requirements based on an artificial neural network (ANN) and features fusion techniques is proposed. In the proposed model, various features extracted using image processing are combined to develop a sophisticated classifier. Based on the different features, different models are built, and each model produces a single decision. Besides, the kind of class is determined using machine learning. The model is validated by extracting relevant information from the dataset containing 2400 images of possible waste types recycled across three categories. Based on the analysis, it is observed that the proposed model achieved an accuracy of 91.7%, proving its ability to sort and classify the waste as per the recycling requirements automatically. Overall, this analysis suggests that a digital-enabled CE vision could improve the waste sorting services and recycling decisions across the value chain in smart cities.

Introduction

Waste generation is continuously increasing in cities, and the measures related to its management are interim. In many cities, even in the so-called smart cities, mostly, a linear approach, i.e., “take-make-use-dispose” is followed. In this approach, the waste handling authorities primarily rely on disposal option, especially in the landfills. But these landfills eventually fill up with the waste, and one day, there won't be enough space to manage continually growing amounts of waste in the smart city. Such growing scenarios of waste generation would question the sustainability of smart cities in the long run. As stated earlier, the interim practices of waste handling in smart cities contrast with the actual smart city definition [9, 14, 23].

Smart cities should be developed considering the effective waste management practices focusing on the strategic actions that ensure long-term sustainability. To ensure sustainability, a circular economy (CE) for smart cities is suggested. The supportive R frameworks are briefly presented in Fig. 1. But most of the generated waste does not meet the recycling requirement under CE standards [1, 18]. Also, the smart city’s CE vision faces critical challenges related to waste collection and segregation. On the other side, for waste materials to be appropriately disposed of, it is crucial to have models and techniques to help individuals sort garbage. The existence of a wide range of recycling categories makes it challenging for people to decide the most appropriate waste bin to dump each trash [17].

Fig. 1

The transition from a linear model to a circular model and the support of R frameworks [19]

In developing nations, it is crucial to treat this issue uniquely because waste management may have a profound bearing on economic development and urbanization [31]. Digital technologies could be critical enablers of CE vision in the smart city to improve the waste sorting services and recycling decisions across the value chain.

Machine learning techniques are being used successfully in a variety of fields, including autonomous vehicles, medical imaging, and a variety of industrial applications, with impressive effects on object recognition challenges [3, 13, 21, 25]. Implementing these waste separation techniques will maximize the amount of recycled material and, as a result, make everyday living simpler for the average consumer. Additionally, the industry would benefit from increased effectiveness [29].

This work's motivation lies in finding a method through which waste can be automatically sorted to minimize the pollution caused by waste. This can be a great benefit to the economy and environment. The proposed method has a great community appeal as it adds the value of knowledge and the social stimulus in the separation and disposal of garbage. Thus, the artificial neural network (ANN) is investigated to enable the classification of waste into three classes: metal, cardboard, and trash. In the proposed work, the most appropriate trash can for the waste to be disposed of is selected according to each waste category. In this work, many experiments are carried out using machine learning. This way, problems like weak computing, insufficient data, and universalization of models in the proposed garbage identification and classification method can be solved so that their advantages and disadvantages can be summarized.

The contributions and objectives of our study can be summarized in the following points:

A model of classifying and identifying waste is proposed based on the lightweight neural network, that can help a user sort trash based on the image of the material obtained using a digital camera. In other word, classify garbage as degradable and non-degradable at the domestic level with low cost and good accuracy to develop perceptive garbage segregation.

Different waste images features such as color, LBP, HOG, Uniform LBP features that are fused to generate a new features vector based on majority voting.

Carry out an analysis of the different extant methods of garbage segregation;

The remaining paper is organized into the following sections: Section 2 contains a review of related work. In Section 3, the experimental methodology, which includes the models, image database, and methods, is described. Section 4 presents the results of the experiments. The study is concluded in Section 5.

Literature review

In the past years, many researchers have carried out studies to reduce the impact of uncontrolled waste disposal. Researchers have achieved the optimization of waste management systems through the use of technologies like Sensor Network (SN) and Radio Frequency Identification (RFID) [15]. With such technologies, authors have been able to store the data of owners and the information about their bins; the data is then used in identifying and tracking the selective collection of garbage. According to [15], the approach was to enable monitoring the flow of waste in a city that was proposed to enhance the efficiency of selective collection. In their method, when waste is being processed, the information stored in an RFID tag is used to detect each waste's origin. This way, relevant information about the waste can be obtained. Another study by [10] proposed an open container, where the contents of the open container were analyzed using image analysis so as to enable the estimation of the volume of household waste. With the image analysis, the label can be used to match each bin with its originating address.

Many works have adopted artificial neural network (ANN) and different machine learning model for waste management purpose. In the study [20], authors have developed artificial neural network (ANN), support vector machine (SVM) and random forest (RF) have been developed and compared for the prediction of plastic waste generation rate. ANN model had best prediction accuracy. In the study [7], authors have emphasized on the prediction of SW generation in the city of Dhaka and finding sustainable pathways for minimizing the gaps in the existing system. The data of SW generation, for few years of each month, in the city of Dhaka were collected to develop a model named Artificial Neural Network (ANN). The ANN model was used for the accurate prediction of SW generation. In [32] authors proposed ANNs in recognising and classifying different medical waste items in the image form. The networks were trained on a large number of medical waste items. The testing confirmed that ANNs are capable of recognising rigid objects by their shape.

In this review of related literature, the studies have been presented according to the problems associated with the identification, tracking, and analyzing deposited waste so that the users can be stimulated to behave responsibly, thereby enabling selective collection. Nevertheless, none of the authors focused on guiding the population on how to dispose of garbage correctly. Based on the review of literature, the recycling of waste is influenced by some significant obstacles. The key obstacles to waste recycling include the following [5, 8, 11, 16, 21, 22, 26, 27]

Government policies and regulations, which is inclusive of inadequate planning, regulations, and budgeting for the management of solid waste;

Household level of education: the ignorance of households regarding the relevance of self-waste recycling also affects the proper disposal of waste;

Technology: the absence of effective technology for recycling; and

Management expense: the management of waste, mostly manual waste classification, is accompanied by the high cost.

To this end, an Automated Sorting Approach and Trash Recycling Classification System are proposed so that all the issues mentioned earlier can be addressed. In the proposed system, the Artificial Neural Network is used, and different features are combined.

Methodology

Dataset used

The author acquired the data manually because of the unavailability of publicly accessible datasets containing garbage-related materials. Initially, the images used were obtained from Google Images and Flickr Material Database [31]. However, the source of images was changed because the recycled products' state was not represented accurately by these images. This was discovered after investigating the true state of recycled goods and recycling plants. For instance, the Flickr Material Database's waste materials appear in the original and undamaged state, unlike recycled waste as waste materials are often crumpled and dirty. Thus, the dataset used in this study was manually produced, and it is hoped that the dataset will be made publicly available after thorough preparation. The dataset is made up of 2400 images of recycled objects that fall under six categories, with each category having approximately 400–500 images (this is excluding the “trash” class in which just 100 images are contained). During data collection, a white poster-board was used as a background, and pictures of trash around Stanford were captured. The garbage images were also captured from trash generated in the homes of the authors and that of their relatives. There is variation in each picture's orientation and lighting, thereby resulting in disparities in the images contained in the dataset. In other words, the pictures contained in the dataset are not the same in terms of appearance. Examples of some of the images from the three classes are presented in Fig. 2.

Fig. 2

Waste Dataset samples

In this study, the techniques used in processing include rotating the image randomly, controlling the random brightness of the image, translating the image randomly, and random shearing of the image. The images' transformations were selected so that the various orientations of recycled material can be accounted for, while the size of the dataset is maximized. Besides, the processes of mean subtraction and normalization were also applied to the images.

Data pre-processing

This work aims to develop a machine learning model that can be used for the classification of isolated garbage elements. Here, the use of Yang Trash dataset was employed [31]. Also, all the aforementioned ANN architectures were used. Nevertheless, there was a need for all the images to be resized for all the models due to computational problems. Apart from that, the brightness values were normalized between 0 and 1.

Moreover, the number of images available for the models' training is small since. For that reason, the use of data augmentation was employed in generating a pseudo-infinite on-demand number of training samples. During the model's training, new images were produced through the application of different transformations on the original data. The transformations' selection was made randomly between 0° and 40°, width changes between 0 and 20%, height changes between 0 and 20%, shear between 0 and 20%, zoom between 0 and 20% and horizontal flips.

Automated sorting model

It is suggested that one of the ways through which garbage could be efficiently processed is the use of a computer method that is capable of classifying waste into different categories of recycling. As mentioned before because lack of public image dataset is available only small number of images are included in the training, validation, and testing of the proposed system.Such a system aims to take images of a single piece of garbage and classify it into three categories: metal, trash, and cardboard, as contained in Fig. 3.

Fig. 3

Automated Sorting of garbage Model

The classification of garbage into different classes can be challenging because most people use one waste bag to dispose of different waste types. Thus, in this work, a model that can automatically identify the types of waste is proposed. In the proposed method, the initial stage involves image processing to enable the extraction of relevant information from the original images. Here, the classification of the garbage into the different classes is achieved using machine learning. The proposed model's major contribution is that a wide range of features is combined to produce a robust and sophisticated classifier. The different features have been combined to build a model, and the individual models will generate their individual decisions. In other words, a single decision is produced by each model. In order to obtain a single decision, the use of majority voting has been employed.

The proposed model involves the following steps:

Training stage: the training stage involves the following actions:

Some samples were selected from the dataset, and they were tagged as cardboard, metal, and trash.

The most relevant information was extracted from the image. With this vital information, the model can be trained. More so, four kinds of features have been used in constructing the model:

Hog feature;

Colour features;

LBP; and

Uniform LBP.

The ANN was used as a classifier. The input at this stage is each of the features which were fed into the ANN classifier. For each feature, there was a single trained model.

Testing stage: at this stage, the newly tested image is used as the input for the trained models:

The testing image is used as input for the texture feature extraction approach so that the fourth vectors can be extracted.

Then the fourth trained model is used in testing the previously extracted features.

Fusion stage: This stage involves getting a decision by using majority voting.

Training stage

The training samples' acquisition for classification requires no efforts, as it can be done by simply adjusting the premier training dataset. One of the advantages of the proposed model is eminent in the training phase because it allows the use of a large amount of training dataset, which is inclusive of metal, cardboard, and trash image. In theory, one unique datum is obtained for each pixel in every training. Consequently, it is not valuable to certainly utilize the common of these samples as regions that are close to each other, and thus are very matching. For this reason, the pick-up of data involving these areas is of no significance. In addition, if a metal, cardboard, and image area are produced for each pixel, a larger size of data can be obtained; the module may be unable to handle the pixel. Nevertheless, in the proposed approach, there is a step in which the proposed model is trained. This is a crucial step in the extraction of color features subsequent to training and for the segmentation of pixel as metal, cardboard, and trash. Subsequently, a decrease occurs in the overlapping of two nearby as cardboard, metal, and trash samples to , where denotes the volume of every as cardboard, metal, and trash part. Based on the experiments' results, training = 10 is the right default pointer for the training step.

Testing stage

In this phase, the waste images is used as input. At the initial stage, the extraction of descriptors involves building a square region about each point inner to waste images. The region is further divided into L × L square sub-regions. This action is of great importance to spatial data. For each sub-region, a few straightforward descriptions at recurrently spaced sample points are calculated. The blocks’ size, estimation of features, and quality of segmentation are all related. Where the block size is directly proportionate to the estimation of the features and inversely proportionate to the quality of segmentation, if the block is small-sized, texture blocks will enclose fewer information points from which the feature was derived, and the nonconformities at feature extraction from the block of comparable texture will be greater—these process consequences at greater variation at the line with perhaps comparable texture. In the same way, the extractions of features from big-block results in the creations of fewer disparities in the distance among possibly analogous texture. The input for the trained models is the new test image, which is used as follows:

The fourth vectors should be extracted by using the testing image as input in the texture feature extraction technique.

Test the previously extracted features using the fourth trained model.

The fusion method

With the use of a single classifier for identifying garbage images with the numerous cases of garbage texture feature based vectors, a moderate to a high level of precision was achieved, particularly by ANN, but the level of accuracy accomplished is distant underneath the specified precision for such garbage that affected on the environment. It is a common thing to see multi-schemes pattern recognition being combined so as to improve accuracy. The fusion of the scores of classifications for all the individual feature scheme is done based on the given rule so that a merged score which will be utilized for the final classification can be deduced. At the level score, the model fuses numerous combinations that are put together for the same three individual feature vectors (F1, F2, F3), and the model makes use of the ANN-related rules to determine the levels of confidence for each combination [2].

Initially, the ANN classifier's application is made to each feature space differently, and then the score of each feature is signed. The three signed classification scores obtained for the three features are averaged. The average score's sign determines the final class, and then the averaged score value is mapped to a level of confidence based on the rules presented. The workflow of the fusion methods is presented in Fig. 4.

Fig. 4

Workflow of the fusion method

Feature extraction

The image process task often involves a large corpus, which takes a good amount of time, and it is less practical for use in the efficient classification of objects from the background during segmentation. One of the best strategies that can be used for the reduction of computational time is to transform the input data by reducing the number of feature vectors. The process through which input is transformed is referred to as feature extraction. Usually, feature vectors contain related information, and they are explored as input vectors in the classification tasks. Features can be classified according to color, texture, and shape [6, 30].

Artificial neural network (ANN)

ANN is a classifier that is based on Artificial Intelligence concepts. With this algorithm, the human brain uses the process to carry out learning and identify patterns. The architecture of the algorithm is made up of input, output, and one of the different hidden layers, whereby there are neurons contained in the layer (s)—the neurons from the most basic structure of a neural network [24]. For the purpose of document classification, a variety of neural network methods have been proposed. However, the most acceptable technique among scholars is that which makes use of one layer perceptron, which is made up of a single input layer and single output layer attributed to implementation simplicity. On the other hand, the architecture of a multi-layer perceptron, which scholars favored during the classification of documents, is more advanced and made up of one input layer with either one or multiple hidden layers and a single output layer, see in Fig. 5.

Fig. 5

Structure of an Artificial Neural Network

The major benefit of ANN implementation lies in the fact that it is superior to scale to document with large features dimensionality. In addition to that, it is capable of addressing the noises present in the document and abnormal data. More so, a computing architecture that is naturally parallel is provided by linear speed-up in the linear speed-up in the matching process. Here, each element can contrast its input value against the value of stored cases independently from others.The ANN model in this paper is implemented from the scratch and no pretrained model is adopted.

Results and discussion

Generally, machine learning algorithms function best in huge datasets that cannot be easily mined through the use of basic data analysis techniques, and as such, different parameters are used to measure the performance of such techniques. The performance of any machine learning model in terms of its ability to analyze data can be evaluated using sensitivity, accuracy, ROC, and specificity. The use of the parameters, as mentioned above, is dependent on the garbage issue to be assessed.

In the first step, the data is divided into testing and training data, where the model is trained using the training data with correct classification outputs, and then the efficiency of the model is assessed using the test data. Afterward, a Confusion matrix is produced for each model, which carries the correct number of predictions or classification that can be made by a model based on positive and negative scenarios. This matrix can be combined with other statistical parameters to assess the model’s performance.

Three classes (cardboard, metal, and trash) are chosen in this study for the testing of the proposed model. The proposed model lies in the idea of combining features so that a sophisticated model can be produced and used for the detection of the test image as cardboard, trash, or metal. There are two main stages involved in the proposed model. The first stage involves the preparation of some images that were used in training the system. Those images have been selected randomly, and then the features were extracted and classified so that the trained model can be built and used in the subsequent testing stage. The experimental results showed that a good prediction result could not be obtained when only one feature is utilized to train and test the automated garbage classification model. Table 1 presents the results obtained from using different single features with accuracy.

Table 1

Accuracy based on different features

Features type	Color features	LBP feature	HOG features	Uniform LBP features
Accuracy	69.1	81.4	84	83.2

In terms of the accuracy metric for multiple models presented in Table 1, the results revealed that among all the features, the Uniform LBP features was the only one that achieved the highest accuracy of 83.20%. However, given the small number of images used in the experimental investigation, it will be important to consider HOG Features as the best algorithm because it achieved an accuracy of 84% in this study. Color Features continuously work tolerantly with little samples, and it can be expected to yield high accuracy when it is trained with a small case of garbage data. Table 1 shows that there are variations in the prediction accuracy of each feature.

The best performance in terms of accuracy was achieved by the Uniform LBP and Hog features. All the features are leveraged in this study by using the fusion based on majority voting. The central idea of the fusion technique is to make use of each model's prediction as output, and the prediction of all models presents the final decision of the proposed model. The efficiency of the proposed model can be seen in Fig. 6.

Fig. 6

Predication of the fusion model

It can be observed that the proposed model improved the accuracy from 84 to 91.7%. The proposed model was used to enhance the result because each feature is able to capture unique information that can enable the detection of some images. Different information can be identified by combining the fourth features. The accuracy has been greatly improved through this idea. Table 2 shows the accuracy of the three classes with fusion features.

Table 2

Accuracy of the three classes with fusion features

Parameter	Cardboard	Metal	Trash
Accuracy	91.1%	87.7%	91.7%

The receiver operating characteristics (ROC) graph is one method through which the proposed model was evaluated (McClish, 1989). The ROC is a method through which the classifiers can be visualized, organized, and selected according to their performance. The use of ROC graphs is mostly employed in image decision-classification and has in recent times, seen increased usage in data mining and machine learning research. ROC graphs are two-dimensional graphs as they contain the X and Y-axis. The TP rate is plotted on the Y-axis, whereas the FP rate is plotted on the X-axis. A ROC graph represents the comparative tradeoffs between benefits (true positives) and costs (false positives). A ROC graph with five classifiers labeled A to E is represented in Fig. 7a. A discrete classifier refers to the kind of classifier which produces just one class label as the outputs. Each discrete classifier produces an (fp rate, tp rate) pair that corresponds to a single point in ROC space. It can be observed that all the classifiers presented in Fig. 7a are discrete classifiers. Many points should be noted about ROC space. First is that the lower-left point (0, 0) represents the strategy of never issuing a positive classification; with this classifier, the error of false positive cannot occur, yet, it does not produce true positives. On the other hand, the upper right point (1, 1) represents the opposite strategy of unconditionally issuing positive classifications. The perfect classification is represented by the point (0, 1). It can be seen the Ds achieved a perfect performance. Informally, if a point in ROC space is positioned in the direction of the Northwest (FP rate is lower, the TP rate is higher or both), it means that the point is better than another one point in ROC space is better than another. The classifiers that appear on the left side of the ROC graph close to the X-axis can be regarded as ‘‘conservative’’. Such classifiers need powerful proof in order to be able to make positive classifications, and this way, their false positive errors are few, but they sometimes produce low trues positive rates. The classifiers that are positioned on the right side of the ROC graph may be regarded as ‘‘liberal’’. With these classifiers, the weak proof is required when positive classifications are made, and for this reason, almost all the positives are correctly classified, but they often have high rates of false positives. It can be observed from Fig. 7b that A is more conservative than B.

Fig. 7

a A basic ROC graph showing five discreet classifiers; b ROC metric for the proposed automated garbage system with fusion features

In several real-world applications, a large number of negative instances are dominant. For this reason, the performance on the left side of the ROC graph becomes more interesting. It can be seen that the ROC for the proposed model has been produced in Fig. 7b. Based on the ROC figure, the proposed model is able to identify the class. Computer vision algorithms and Machine learning allow trash to be classified into different recycling classes. One of the main benefits of this is that a wide range of possible data can be made available. This means that any object can be placed under one category of waste. Thus, a huge and constantly growing data source is required to create a system with higher accuracy.

Benchmarking is the foremost main task that should be considered in evaluating the scientific study on common images processing and classification domains. Benchmarking can be utilized to comparing the reliability and efficiency quality of recently created techniques and current methods. Benchmarking is more often than not conducted either through the utilize of a standard dataset or distinctive techniques for the same issue application or domain. Additionally, benchmarking is accomplished by using the most excellent and advanced strategies for waste classification based on current machine learning and AI methods and features selection techniques. Table 3 presented the distinctive benchmarking methods for waste classification problem.

Table 3

Accuracy of the of state-of-the-art techniques versus with our proposed method for waste classification problem

Author(s)/year	Method	Accuracy (%)
Adedeji and Wang (2019) [4]	Deep neural network	87
Ruiz et al. (2019) [28]	Inception-ResNet model	88.6
Chu et al. (2018) [12]	Multilayer hybrid deep-learning system	90
Our proposed method	Automated waste-sorting and recycling classification using artificial neural network and features fusion	91.7

The limitations of our study can be briefed as follows:

Automated waste-sorting and recycling classification using artificial neural network and features fusion have been evaluated by one classifier and must be tested to supply a more comprehensive assessment of the outcomes.

The proposed Automated waste-sorting and recycling classification depends on the fusion features method, its fundamental limitations are time complexity and high computational cost.

The proposed Automated waste-sorting and recycling classification depends on the fusion features method required more classes such Waste COVID-19 and other Waste medical classes.

Conclusion

In this work, an automated system is proposed based on Artificial Neural Network and Fusion of Different Features so as to reduce the effect of improper disposal of waste, especially domestic waste such as glass, paper, plastic, and organic waste. With the proposed automated system, waste can be separated correctly into their different categories. In this work, three categories were considered: paper, glass, plastic, and metal. Based on the results, the Artificial Neural Network and Fusion of Different Features is one of the methods that can efficiently solve this problem, as it was able to achieve an accuracy of 91.7%. Nevertheless, it appears that Artificial Neural Network and Fusion of Different Features are more computationally costly as compared to other traditional methods. More so, in order to improve the accuracy of ANN methods, the use of some techniques like fine-tuning and augmentation can be employed in the future. Additionally, the availability of more data helps the ANN to yield better results. Overall, this analysis suggests that a digital-enabled CE vision could improve the waste sorting services and recycling decisions across the value chain in smart cities.

In the future, using the genetic algorithm with deep learning approaches will be investigated since humans can interpret the offer rules. Apart from that, the capabilities of models that have been implemented on a totally functional resourceful system would be explored in future work. Such a system can be used to test the models against stress while their robustness and performance remain intact. Furthermore, it is suggested that the trash classification model be implemented for a multi-label image because this is more complicated and needs more resources in terms of effort and time.