Monitoring System of Drowsiness and Lost Focused Driver Using Raspberry Pi.

BACKGROUND: Drowsiness condition is one of the significant factors often encountered when an accident occurs. We aimed to detect a method to prevent accidents caused by drowsiness and lost a focused driver.
METHODS: The image processing technique has been capable of detecting the characteristic of drowsiness and lost focus driver in real-time using Raspberry Pi. Video samples were processed using the Haar Cascade Classifier method to identify areas of the face, eyes, and mouth so that drowsy conditions. The methods can be determined based on the bject detected.
RESULTS: Two parameters were determined, the lost focused and drowsiness driver. The highest accuracy value for driver lost focused detection was 88.00%, while the highest accuracy value for drowsiness driver detection was 90.40%.
CONCLUSION: In general, a system developed with image processing methods has been able to monitor the drowsiness and lost focused drivers with high accuracy. This system still needs improvements to increase performance.

Introduction

One of the biggest causes of death in Indonesia is traffic accidents. Driving a vehicle with drowsiness is a dangerous condition and results in an accident. Several previous researches on the detection of drowsiness was applied to determine the trends of night workers and adaptation to night shifts in hospital staff. It is reported the importance of application for drowsiness detection as an effort to improve work safety (1). Then a lot of research on drowsiness have been turned around for the detection of drowsiness for drivers with a variety of methods. Research on drowsiness uses image processing with eye-tracking methods, including blink frequency, and PERCLOS, which are used to confirm results (2). Then another study to detect drowsiness by using the neural network method and Viola-Jones to detect facial characteristics (3). Besides, the condition of drivers who are drowsy or alert based on images taken during driving is conducted by analyzing the state of the driver’s eyes: opened, half-open, and closed using image processing and ANN (4). Image processing using the human visual system model was used, and changes in energy levels in the frame were utilized (5).

Other researches were development the drowsiness detection using the parameters of the physical conditions of the sleepy driver. In this system, the parameters used were recording brain activity on EEG (Electroencephalography) (6,7), heart rate variability, and pulse (8). However, the level of sensitivity in this system is still low compared to the visual approach because the needed devices must be attached to the driver’s body. It can interfere with concentration and driving activity, so this system is less practical to use in some situations.

The next detection system is road monitoring, which is the most commonly used technique. The detection system is such as the Attention Assist by Mercedes, the Fatigue Detection System by Volkswagon, the Driver Alert by Ford, the Driver Alert Control by Volvo. All of these techniques monitor street characteristics and driver behavior to detect sleepiness. Some parameters used include whether the driver follows the path rules, uses the right indicators, and others. If there are irregularities in this parameter, the system concludes that the driver is sleepy. This system can be said to be defective. Because monitoring the road to detect sleepiness is an indirect approach and also has no accuracy (9).

The driver’s fatigue can be monitored using a video camera by analyzing the condition of the driver’s face so that it becomes was more practical. Many results indicate that facial expressions provide important information about driver fatigue, and if visual behavior can be detected, it can increase driver awareness (10).

The development of methods for detection of drowsiness is needed as an effort to prevent accidents caused by the negligence of the driver. Fatigue in the driver of the car does not only affect the concentration of the driver but also can create potential in a car accident. From existing accident data, between 10% and 20%, each year, accidents occur due to one of them due to fatigue and lack of alertness level (11).

Therefore it is necessary to make a system that can detect sleepiness in the driver of the vehicle to reduce accidents. The Haar-Like Feature method was often referred to as the Haar Cascade Classifier. This method has the advantage that the computation is high-speed because it only depends on the number of pixels in a square instead of every pixel value of an image (12). This research is a development of previous research using the total pixel algorithm (13).

The image represents the spatial distribution of physical quantities such as light intensity and spatial frequency of an object. The information was represented by components such as brightness, color, and edges (13,14). The digital image is an array containing real and complex values represented by specific rows of bits (15,16).

The image was divided into three types, namely color image or RGB (Red, Green, Blue) image, grayscale image, and binary image. RGB image is an image that has red, green, and blue colors as the primary color of the arrangement. Grayscale images have the possibility of a color between black and white (17). While the binary image is an image with each pixel only expressed by a value of two opportunities (i.e., 0 and 1). A value of 0 represents black, and a value of 1 denotes white (18).

Digital image processing states a two-dimensional image processing using a digital computer (13), consistsing of several objects processed using a transducer to represent an object. The digital image is carried out image processing to produce a new image that can be used for analysis.

The face detection process can be seen as a problem that has a classification pattern. The input is the image, and the output will be determined in the form of a class label from that image. There are two types of classes of images, namely, face and non-face. Face recognition techniques that have been carried out so far use the assumption that available face data are of the same size and similar background. But in the real world, this assumption does not always apply because faces can appear in various sizes and positions in the image and with different backgrounds. Face detection is one of the critical initial stages before the face recognition process (19).

The condition when the human body needs rest, or the tendency to sleep is called sleepy. Drowsiness can be caused by several factors, including fatigue doing repetitive work such as staring at the monitor screen for a long time or driving a vehicle with a long trip. Sleepiness and fatigue have many of the same effects. When the eyelid starts to feel heavy and slowly closes, then the view begins to blur, and suddenly, the eyelid has shut 100%, even though in mind, it still feels awake. This is a sign that someone is sleepy.

We aimed to detect a method to prevent accidents caused by drowsiness and lost a focused driver.

Materials and Methods

In this system, there are two parameters in this detection that is when the driver object is out of focus, and the driver object is drowsy. The characteristics of an unfocused driver object were marked by the movement of the head sideways and down. The characteristics of a sleepy driver object are the condition of the eyes closed and evaporated. The characteristics of the detection are processed in Raspberry Pi using Python and OpenCV software. When the camera captures objects in which there is a face, the monitor will be displayed the results of recording the camera and the detected area of the face given a blue square box. After the face was found, then the system looks for the eyes and mouth. Results that indicate the presence of eyes are indicated by a green square box in the eye area, and if the mouth is detected, the mouth area was given a red square box.

Haar-Like Feature and Cascade Classifier

The method commonly used in object detection is the Haar Like Feature, which is a rectangular (square) feature, and gives a specific indication of an image or image. Haar-like features recognize objects based on simple values of features but are not pixel values of the image on that object. This method has the advantage that the computation is high-speed because it only depends on the number of pixels in a square instead of every pixel value of an image (12).

Haar algorithm uses statistical methods to detect faces. This method uses sample haar like features. This classifier uses a fixed-size image (generally 24×24). The workings of haar in detecting faces are by using a 24×24 sliding window technique on the entire image and looking for whether there are parts of the image that are shaped like faces or not. Haar also has the ability to scaling so that it can detect faces that are larger or smaller than the images in the classifier (19). Haar Feature is a feature based on Haar Wavelet (12). Haar Wavelet is a single square wave. For two dimensions, one light and one dark. Furthermore, box combinations are used for better visual object detection. Each Haar-like feature consists of a combination of black and white boxes.

The Haar feature was determined by reducing the average pixel in the dark area from the average pixel in the bright area. If the value of the difference is above the threshold value or threshold, then it can be said that the feature exists. The value of the Haar-like feature is the difference between the number of gray level pixel values in the black box area and the white box area (20).

Integral Image is used to determine the presence or absence of hundreds of Haar features in an image and on a different scale efficiently. In general, this integration means adding small units simultaneously. In this case, the small units are pixel values. The integral value for each pixel is the sum of all pixels from top to bottom. The whole image can be summed up with several integer operations per pixel starting from the top left to the lower right,

In Fig. 1 (a), the value at the pixel location (x, y) contains the sum of all pixels in the rectangular area from the top left to the location (x, y) or shaded area. To get the average pixel value in the square area (shaded area), this can be done simply by dividing the value in (x, y) by the rectangular area, as shown in the following formulation: Where ii (x, y) is an integral image, and i (x, y) is the original image.

Fig. 1:

An integral image

To know the pixel value for several other rectangles, such as the rectangle D in Fig. 1 (b), it can be done by combining the number of pixels in the rectangular area A + B + C + D, minus the number in the rectangle A + B and A + C, plus the number of pixels in A. With, A + B + C + D is the value of the integral image at location 4, A + B is the value in location 2, A + C is the value at location 3, and A at location 1. Therefore, the results of D can be computed, and formulated in the following equation.

Cascade classifier is a chain stage classifier, where each stage classifier is used to detect whether, in the sub-window image, there is an object of interest. The stage classifier is built using the adaptive-boost algorithm (AdaBoost) (20). The algorithm combines the performance of many weak classifiers to produce a robust classifier. The weak classifier, in this case, is the value of the haar-like feature (21).

Drowsiness Detection System

The system is processed to be able to detect the characteristics of the driver object that is not focused and sleepy. If the system recognizes the characteristics of the driver object that is out of focus, then the monitor appears “Driver Lost Focus.” Whereas when the system detects the characteristics of a drowsy driver object on the monitor screen appears “Drowsiness Detected.” From Fig. 2, the first poses import the camera. The camera captures the image that records the detected object. The image results changed to a size of 300×300 pixels. Then the image is converted into the grayscale or grayscale format. After being converted, the image will experience histogram equalization, so it helps to adjust the contrast of the entire image.

Fig. 2:

Block Diagram of Drowsiness Detection System

Driver Lost Focus and Drowsiness Detection

From the Haar Classifier method, it has been trained to detect facial areas in the image. This data training can be done in OpenCV. To be able to do data training, positive samples and negative samples are needed. Positive samples contain detected object image data, while negative samples contain random image data that does not contain detected objects. OpenCV helps to create a vector file that contains a mixture of positive sample data and negative samples. So that the results of the data training operation are XML files that can be used to detect the area of the object in the image. But the data is already in the OpenCV software, so it only needs to call the file into the program.

The next process is detection that starts from detecting the face, then the eyes and mouth. On face detection, an XML file that has been called into the program is marked with a face area with a blue square. If the face is not detected, it is said that the driver lacks concentration. So that the words “Driver Lost Focus” appeared.

In this eye, detection marked the eye area with a green square. Ratios for eye parts are determined. If the eyes are open, a green box appears, but when the eyes begin to close so that the box does not appear, it says that the driver is sleepy, and the words “Drowsiness Detected” appear.

This mouth detection is to detect the driver is yawning. The value of the ratio for the mouth evaporates has been determined. If the height/width value in the mouth area is close or equal to the value of the evaporating ratio and the eye closes, it is said that the driver is sleepy. So that “Drowsiness Detected” appears.

In the classification, the stage has been determined that if the face object is not found, it is said that the driver is less focused on driving, then the words “Driver Lost Focus” appears. Whereas for drowsiness, if the driver starts to close his eyes and evaporate, it is said that the driver is sleepy, so the words “Drowsiness Detected” appear.

Results

Image Acquisition

The results of the system design that has been installed in the car are shown in Fig. 3. The image consists of a camera, monitor, and a connected Raspberry Pi placed above the car dashboard in front of the driver’s seat.

Fig. 3:

Driver Drowsiness and Lost Focus Detection System

The testing of this system was carried out in real time on four drivers’ objects, namely women with glasses, women without glasses, and men without glasses. This process was carried out perpendicular between the camera and the driver object (Table 1).

Table 1:

Image acquisition in vehicle

No	Name of Data	Image Display	Camera Resolution
1	Video 1		8 MP
2	Video 2		8 MP
3	Video 3		8 MP
4	Video 4		8 MP

From the results of the process, the results of the number of frames, and the duration of each video were obtained (Table 2).

Table 2:

Data of video duration and number of frames

No	Name of Data	Video Duration (s)	Number of frame per secon
1	Video 1	140	140
2	Video 2	105	105
3	Video 3	97	97
4	Video 4	99	99

Detection Not Focused

The result of this unfocused driver detection is marked by the appearance of the words “Driver Lost Focus” (Fig. 4).

Fig. 4:

Result Driver Lost Focus Detection

Drowsiness Detection

Detection of drowsiness is divided into two, namely the condition when the eyes are closed and the condition when they yawned. Result of drowsiness detection with eyes closed (Fig. 5). Then the detection results are drowsy with yawned conditions, as shown in (Fig. 6).

Fig. 5:

Result driver drowsiness detection with eyes closed

Fig. 6:

Result driver drowsiness detection with yawned conditions

Accuracy calculation

The calculation results for accuracy and error rate values in each video for lost drivers focused are shown in Table 3.

Table 3:

Value of Accuracy and Error Rate of Drivers Lost Focused

Name of Data	Driver Condition	Accuracy (%)	Error Rate (%)
Video 1	Without glasses	78.12	21.88
Video 2	Without glasses	88.00	12.00
Video 3	Without glasses	72.70	27.30
Video 4	With Glasses	86.90	13.10

The results of calculating the accuracy for the driver’s drowsiness in the two conditions are shown in Table 4.

Table 4:

Value of accuracy and error rate of drowsiness

Name of Data	Driver Condition	Accuracy (%)	Error Rate (%)
Video 1	Without glasses	44.00	56.00
Video 2	Without glasses	80.60	19.40
Video 3	Without glasses	90.40	9.60
Video 4	With Glasses	82.75	17.25

Discussion

This system can detect the characteristics of the driver who is not focused, which is marked by the movement of the head down or looking down and turning sideways. If these features are identified, the words “Driver Lost Focus” appear. Besides that, it can also detect the characteristics of a drowsiness driver that is characterized by a closed and yawning eye condition. This yawning condition is characterized by eyes closed and mouth wide open. If one of the two features is identified, the words “Drowsiness Detection” appear. System testing is carried out in the morning, afternoon, and evening with different driver objects.

Based on Table 3, it appears that the accuracy for drivers without glasses has accuracy ranging from 72.00 to 88.00%. As for drivers, using glasses has an accuracy of 86.90%. Accuracy for lost focused drivers is not affected by the presence of glasses or not, because what is detected is the movement of the driver’s head. The effect of accuracy on the detection of lost focused drivers is more on lighting effects, so it will cause different pixel values when there is light or not. This result is in line with research (22), which states that if the lighting is not suitable, the performance of the machine vision will not be optimal. Then lighting effects will determine image quality (23).

In Table 4, the results of accuracy for drowsiness detection in both conditions give an accuracy of 44.00 to 90.00%. The lowest accuracy in the driver without glasses, while the driver with glasses provides a good value, which is in the range of 82.00%. Factors that affect the low value of this accuracy are the driver’s mouth width when yawning together with the condition of the driver’s head slightly lowered so that the system does not detect the mouth.

The accuracy of this system, when compared with previous research (5) the system developed, has better accuracy, which is equal to 90.40%. The difference between the two methods is 0.40%, but overall the performance of the two methods is almost the same. Conversely, when compared with other studies (2), the method offered has a lower accuracy of 3%. However, the method offered has the ability to detect 2 things, namely the detection of drowsiness and lost focus on the driver simultaneously with sufficiently promising accuracy. In addition, the developed method has been tested in the actual system environment, which has been installed in the car to be able to detect drowsiness and lost focus on the driver. Overall, this system has been running well but still needs some improvements to the method so that it has high accuracy.

Conclusion

The system can be used to detect objects with two conditions, namely the driver who is lacking/unfocused, which is characterized by downward or sideways head movements for a long time and drowsiness characterized by closed eyes and yawn. Using the Haar Cascade Classifier method, the highest accuracy of the drowsiness driver condition is obtained with 90.4% with the driver object being detected, such as a woman with glasses, a female driver without glasses, a male driver with glasses and male drivers without glasses. The highest accuracy of driver detection is not focused at 88.00%.

Ethical considerations

Ethical issues (Including plagiarism, informed consent, misconduct, data fabrication and/or falsification, double publication and/or submission, redundancy, etc.) have been completely observed by the authors.