Real time data analysis of face mask detection and social distance measurement using Matlab.

This paper describes mask detection using Matlab when complex images in the dataset. Matlab specified the Faster R-CNN algorithm and Dataset allotment for mask detection. This paper manages complex pictures using facial recognition packages. The Faster R-CNN methodology used in the security system and the medical system. The proposed work balanced face restriction, color changes, brightness changes, and contrast changes. Segmentation and feature extraction used in face restriction of the person image. We chose RCNN, Fast RCNN, and Faster RCNN algorithm for detecting Mask detection and Social distance. Regions with Convolutional neural network Based on Mixing pictures, pixel prediction, and specific enhancements. The main objective was to solving multiple and multitask picture detection problems with speed rates. The Methodology used for face detection and detection of Unmask person in a dataset of face database.

Introduction

Coronavirus (COVID-19) is a type of coronavirus family. This pandemic locks down the world and causes a global recession. This virus spreads mostly because of the droplets produced by sneezing and coughing. Regular hand washing, wearing a face mask, and maintaining the social distance in a public place can be preventive measures to stop the spread of the virus. First, the face detection method is applied to extract the face region from the entire image. Color based [] face detection algorithm applied for face detection. Extraction of the face is a challenging task because of the change of face structures and their skin color variations. Pose variation, lighting conditions on the face, edges of output camera will also affect the accurate face localization. After face localization, face mask detection process applied which will show that a face wearing a mask or not. An alarm implemented which can inform that a person does not wear a mask. Social distancing is also a crucial step to control the spread of the virus. This paper proposed a combined approach for the detection of the mask and monitoring social distance also (see Fig. 1 ).

Fig. 1

workflow of Mask Detection using Deep Learning.

Face features

More quality images required for detecting face with mask and without detection. It trains this model using a classification in the dataset. Different faces may be confused about the analysis of the manual feature extraction and it cannot predict the mask or Unmask pictures. Biometrics sensors used for face capturing and face detection, which faces determined from the nose, lips, eyebrow size, and eyes. Multiface issues reduced using the R-CNN algorithm. In multimedia application, video streaming possible to driving fast detection of unmasking using Faster R-CNN with 97% accuracy. Face limitation and facial communication focused on security purposes. Therefore, Framework shorted for facial detecting in the disable proof of dataset. Human face color changed when skin concealing and changing skin structure. Person counts based on lighting appearance and edge of yield. Picture pixels examined to serving or merging face features using image segmentation when feature extraction required.

Face surfaces

Unmistakable pictures based on good contrast, good brightness, and facial recognition for generating face surfaces. Color of Skin, the color of hair, size of the nose, and length of the eyebrow, and black of eyes majority utilized to generating accuracy of face detection. Color data stored in the dataset and identified face surface [3].

Skin color

Skin color varied depends upon the brightness of the photo and contrast of photos. When increased contrast, color decreased to white. When decreased contrast of the picture, color id increased to black. Skin color affected to change facial conditions, the total size of the face, relocation of facial and finding age, or sex of pictures.

Skin discovery

After accurate face detection, face contains some dark portions like eyes, eyebrows, and some light portion of skin color like cheeks, forehead, etc. It is likely to validate face skin color and characterize each single-pixel shade in a picture by restrictions of the space.

Color and classification selection

In the field of colorimetry, imaging, and picture preparation, different color spaces have different properties. Each model has different skin color subspaces and structures as well. Characterization techniques can be different for different color subspace. RGB subspace contains red green and blue colors by the combinations of these three colors, the color image formed. YCbCr color subspace represents the JPEG picture format. Y contains the luminance part Cb, and Cr represents the color difference.

Skin color detection

Face area divided into squares of skin like pixels and non-skin like pixels as the detected face contains some dark and skin like portions. First, the skin color area characterized. It has skin color like pixels. It is better to characterize skin like pixels based on examples in color space. It is the tentative distribution by examination of distinctive designs. It is simple in use and gives better results. It decided tentatively the nature of results relies on model and exactness of rules.

Non-parametric models of skin-color selection

First skin like color pixels extracted and form a set, which contains skin-like color subspace, is characterized based on the characterization rule. The same pixel is similar skin color subspace form a setting that will be different from the other non-skin color subspace pixels.

Parametric models remaining portion, which contains the non-skin region, form another set. Skin color subspace and non-skin color subspace boundaries formed and separated each other. Non-skin region contains eyebrows eyes and other portions that are not similar to skin color subspace.

Binary pattern

Dataset assigned an adequate number of various skin picture designs when used 400 times trailed and trained. Minimum 1000 faces must store in the database for higher accuracy using R-CNN. The 256x256x256 framework made for representing the database in the color estimation, when storing RGB pictures and distance of the image. It mentioned Skin color point = 1 and skin is not color point = 0. The contrast value stored pixel range in the picture with the database. This guide harmonizes of pictures stored in the color zone. The Correlation scientific methodology used in the database for detecting facial coloring area. The correlation method is better than other methods to overcome facial recognition problems. Color spaces YCbCr, HSV, and RG utilized from RGB color. YCbCr is one of the ideal color space for facial identification when the identification of RGB color.

Stochastic pattern

Face detection is a crucial step in this paper. After accurate face detection, skin color subspace, and non-skin color subspace which contain the pixels values similar to the skin portion and non-skin portion. Skin color like subspace contains the pixel values to the skin color regions, non-skin color subspace regions contain different pixel values that do not resemble the skin color pixel values. To find the skin and non-skin pixel values, we proposed a 3x3 pixel square & it will run on the whole face region. Mean of all the pixel values estimated and this 3x3 pixel square runs on all the face image. After taking the mean, the new value compared with all the other square values, and on behalf of that, we will distinguish the skin color region and non-skin color region. The mean of the pixels in square gives high values for the skin color region as compared to non-skin regions. We can also assign different values for squares like one for (high mean worth, low disperse-square which are related to skin color regions) and zero for (low mean worth, high disperse which are related to non-skin regions). This 3 × 3 pixel square can also run horizontally as well as vertically to better filtering of skin color and non-skin color regions.

Wavelet transform

The stochastic framework distinguishes the skin like Color and non-skin like the one Colors separately based on the mean of 3 × 3 pixels square. However, in a non-stochastic framework, another concept luminance segments of the picture to wavelet change. In this paper Daubeschi, wavelet type 2-wavelet transform method is used. There is expressive progress in the vertical direction as there are eye regions, lips, etc. In a vertical direction, a point to point coefficient and the first degree of disintegration is taken. A low pass filter is utilized as the size of the face is bigger. There is an assumption considered to widen the coefficients more vertically. We utilize uncommon arrived at the midpoint out of the fiction of sinusoid. We change the field of coefficients on a two-fold guide measure and duplicate them with this guide.

Test case I: Mask detection using deep learning

Coronavirus (COVID-19) pandemic locks down the world and causes a global recession. This virus spreads mostly because of the droplets produced by sneezing and coughing. Regular hand washing, wearing a face mask, and maintaining the social distance in public places can be preventive measures to stop the spread of the virus. In this paper, a CORONA veil location made with a thoughtful learning model. It includes self-loader information marking, model preparing, and GPU code age for the constant deduction. Some test information added for the Sample Mask Data, which shows a person, is wearing a mask or not.

In the figure, Prepare data collected from data access and pre-processing ground-truth labeling, and simulation-based data generation. After prepare data, the Train model generated from model design, hyperactive parameter tuning, model exchange across frameworks, and hardware-accelerated training. After the trained model generated, deploy system set in iteration and refinement, which includes multiplatform code generation, embedded deployment, and enterprise deployment. Label image based on automated labeling with a pre-trained model for facial recognition. Train Object detection model based on Single-Shot Multibox Detector: SSD and You Only Look Once v2: YOLO. Generate the CUDA mix for inference speed acceleration.

Part1 - prepare data

See Fig. 2

Fig. 2

Mask detection bounding boxed in picture.

Labelled mask automation mlx file

This file used for ground truth labelling with a semi-automated and trained model with a dataset. This file interfaced from original data to labeled data. It is one of the Matlab live Xml (mlx) code.

Trained mask detection mlx file

This file has a training model for getting from evaluation, framework architecture creation, and data segmentation. API utilized in SSD and YoloV3 files analysis. After finishing the training model, the next test for still image and live streaming data.

Tested still image file

After completion of trained image, Test trained model for facial recognition.

Video running Matlab file

After test the still image, Test trained model for existing video.

Live webcam mask Matlab file

This file used for testing a trained model for live image. Here required supported packages like a USB webcam package to installation. When connected USB webcam, System installed automatically.

Deploy system

See Fig. 3 .

Fig. 3

Accuracy of Mask detection using Faster CNN.

Train mask detection Matlab file

This file have simple concept. This code can be used for image conversion to detecting unmask image.

I = imread(‘picture.png’);

Masked = [1 0;0–1];

Masky = [0–1;1 0];

X = imfilter(I,maskx);

Y = imfilter(I,masky);

Z = X + Y;

Test case II: Social distance measurement using Matlab

The strain of illness person affected by COVID 19 before 14 days testing. SARS COV and MERS –COV respiratory syndrome give feedback to lungs after this virus affection, Social distance of people monitored heartbeat, the distance between people, and mask detection to reduce spreading. In the isolation period, some people escape from hospitals when the head of pressure. At that time, this system Identify the persons and who spread to the virus. Matlab software handle analysis of the detection of the people and where they are spreading the virus in which place (see Fig. 4 ).

Fig. 4

detection of pedestrian walking with social distance.

Workflow of social distancing

You Only Look Once v2 – Yolov2 file identifies pedestrians' foot recognition after the trained model using the classification ion of daBirdseyerds eye concept used to detection of masking persons using faster R-CNN algorithm social Social distance measured between point so the point (x, birds in birds-eye position (top-down) and ground plane option. It found movement birds of birds' eye coverage images and estimated passer-by’s (x, y) area in the bird's-eye Detect Pedestrians in the perspective view (see Fig. 5, Fig. 6 ).

Fig. 5

Workflow of social distancing.

Fig. 6

rounded marked for social distancing for distance measurement.

People extract GT mlx file

This file extracted different datasets and provide ground information. This is written in Matlab supported file. It can be connected or interfaced trained file and tested file for executing final social distance.

Train people YOLOv3 mlx file

This file trained for detecting pedestrian detection model from GTD with extracted people extract GT mlx file. It convert normal view to Bird's-eye and detect (x,y) distance in between (see Fig. 7 ).

Fig. 7

Feature extracted of video streaming image.

Social distancing script Matlab file

After extracted and trained model in dataset, the social distancing Script Matlab file had script with pretrained model. It can be covered calibrated tasks from bird's-eye view, find extract images and its location distance.

After running the Social distancing script file, it executed to measuring distance between nearest two point. This file can be find social distance using Faster R-CNN when video streaming (see Fig. 8 ).

Fig. 8

Social distance measurement in between two points.

Social distancing mlapp file

This file detect social distance between two images, when running lightweight mlapp file. After set distance of two persons in this file manually, this file detect social distance while it running (see Fig. 9, Fig. 10 ).

Fig. 9

Social distancing measurement using Faster - RCNN.

Fig. 10

Feature Extraction of Social distancing.

Conclusion

When built an unmasked detecting file and social distance-measuring file using the Faster R-CNN algorithm, It automatically shown who is Unmasked person and Who is not keep a social distance. Here Social distance set 3 m in this video streaming Matlab file. In benchmarking, Social- distance measured 50 times for getting accuracy. The distance measured from two object movements. High Contrasted and High brightness image adjusted using the Faster R-CNN algorithm, when capture complex image. Face turning, wearing classes, beard faces, and scarf images easily detected using Faster CNN with 93.4% accuracy, when various image and different color image occurred. Both Applications of Mask detection and Social distance measurements detected using the CNN algorithm when it extracted its features in Matlab Toolbox.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.