1. Introduction

Object recognition is the computer recognition of the objects in images captured in real-world scenes. Object recognition is a compound word of classification and localization, meaning the classification of objects and the identification of location information simultaneously. Object recognition technology is widely applied in medical, autonomous driving, and military fields and research on it is being actively conducted. Until recently, visually impaired people have suffered considerable inconvenience in their daily life, and accidents are frequent. Therefore, it is time to find an alternative solution to these problems. This paper proposes a dangerous object recognition application to solve this problem through image recognition using a mobile phone. Joseph Redmon et al. proposed the you only look once (YOLO) algorithm for a one-step method of these object recognition algorithms ^[1]. Wei Liu et al. studied the Single Shot Multi-Box Detector algorithm (SSD) ^[2]. Regarding the two-step method, Shaoqing Ren et al. researched the faster region-based convolutional neural network (R-CNN) algorithm ^[3]. On the other hand, the faster R-CNN method is unsuitable for real-time object detection because it is a two-step method, and its speed is relatively slow. Sanika Dosi et al. applied object recognition to mobile phones ^[4]. In addition, Sumitra A. Jakhet et al. studied object detection applications for the visually impaired ^[5]. Object recognition networks are generally divided into a one-step method and a two-step method. In the one-stage method, localization and classification are performed simultaneously, whereas they are performed separately in the two-stage method. The two-step method is more accurate but slower because it takes one more step than the one-step method. This paper proposes a system that recognizes objects in front of the user with a camera and informs them in real-time using a smartphone for the visually impaired. A mobile-based real-time dangerous object recognition system is proposed using an SSD network using a first-stage detector suitable for real-time object recognition among two representative image object recognition methods.

2. Design of Risk Object Recognition System for the Visually Impaired

This paper presents a system for real-time risk object recognition for the visually impaired. For the visually impaired, even objects that are not hazardous to non-disabled people may be at risk. Therefore, this study designed a mobile application that recognizes objects in everyday life, identifies the objects in front of them, and informs the visually impaired. Mobile phones are easy to use and light, which makes them very convenient for users. Using this point, an application for the visually impaired on a mobile phone can help the visually impaired to solve the inconvenience of life. The system proposed in this paper is based on dangerous object recognition.

After training the dataset in advance, it was converted to a pb-file suitable for Android applications. Subsequently, it produces an Android application to which the model is applied. When a specific object is detected ^[6], it notifies the user of what kind of object is in front of them by sounding an alarm. The highest priority class, i.e., the object, is processed first using the priority queue among the recognized objects. In this structure, the user is notified of the recognized object according to the priority. The user can hear the voice and know the object in front.

Fig. 2 shows the structure of the one-stage detector. As mentioned earlier, unlike the two-stage method in which Regional Proposal and Classification are performed sequentially, it is performed simultaneously in the one-stage method. Therefore, it is relatively fast, which is much more advantageous for real-time object detection ^[7]. Recently, as the accuracy of the one-stage detector is improved, it has an accuracy close to that of the two-stage detector.

Fig. 1. Overall Object Recognition System.

Fig. 2. 1-Stage Detector.

3. Object Recognition Network Training

Although various networks exist for object detection, in this paper, a suitable one-stage detector is selected for real-time object detection. This paper considers the representative SSD and YOLO networks among the one-stage detector methods. YOLO and SSD networks have the advantage of high speed, so they are suitable for real-time object detection. On the other hand, the YOLO network has difficulty distinguishing overlapping objects, and the SSD network is more difficult to use than YOLO. In this study, a real-time object recognition system ^[8] was designed by selecting an SSD network.

3.1 SSD Network

The SSD model is a one-stage detector like YOLO. The entire network uses the pretrained VGG16 as a base. After that, it has a structure to add a secondary network. The secondary network is composed of a general convolutional network connected to the secondary network by changing the fully connected layer to a convolutional network. The detection speed is improved in this process, and the overall network is similar to a general convolutional network.

Fig. 3 presents the rough structure of the SSD network. The main idea of the SSD model is to use feature maps of various scales. The existing model used only feature maps of the same size. When a feature map of a constant scale is used, it may be difficult to detect objects of various sizes. The SSD model is used for detection by extracting feature maps of convolutional layers in the middle.

Fig. 3. SSD Network.

4. Experimental Result

This paper proposes the application of real-time risk object recognition for the visually impaired to Android applications. Therefore, identifying an object that can threaten the visually impaired in daily life is essential. This paper proposes a mobile application by selecting the most easily conceivable objects.

The four items in Table 1 were selected as they were judged to be the most common dangerous objects that visually impaired people can encounter outdoors daily.

4.1 Hardware, Software Configuration

The hardware used in this experiment was an LG V30 smartphone. The specifications of the proposed hardware are as follows. The experiment results may change depending on the camera performance of the hardware.

As shown in the table above, the camera used for object recognition has a specification of 16 megapixels. It is lightweight, even if the visually impaired use it only for object recognition. In addition, there are many possibilities that it can be used in more advanced parts using built-in sensors and GPS. When developing an Android application, it is connected to a computer using a USB port. Application design and creation were carried out through Android Studio. In addition, a function to give an alarm to the user when the highest priority class is a specific object using a priority queue has been added.

Fig. 4 shows the code that executes the sound file only when the class is 'person' among the objects recognized by the smartphone. By varying the sound according to a specific object, it can recognize what kind of object it is by sound.

The dataset used for object recognition can recognize approximately 90 types of objects using the coco dataset. Among them, a dangerous object is selected, and an alarm function is added separately. The system provides sound by prioritizing high-priority objects and can be adjusted easily in Android Studio according to convenience.

Fig. 4. Add Sound Code on Android Studio.

4.2 Experimental Result

This paper proposes a notification system through dangerous object recognition for the visually impaired. Four objects were selected as experiment conditions: person, car, bicycle, and traffic light. In the experiment method, the above four objects were photographed randomly, and the recognition rate was calculated based on the total number of recognition and notification success times. The formula for calculating the recognition rate is as follows (4).

(4)

$ Recognition\,\,Rate=\frac{Number\,\,of\,\,Recognized\,\,Objects}{Total\,\,Number\,\,of\,\,Object} $

Because this paper focuses on the visually impaired, eventually converting to sound is also essential. Therefore, the number of recognized objects reflected in the recognition rate in Eq. (4) is the number of times the user hears the correct notification. Real-time object recognition was carried out by shooting based on the designed application.

Fig. 5 above is a picture captured while shooting with a smartphone to which the object recognition system is applied. In addition to filming in real life, additional photographs and videos were used. For all four items, object recognition was tested approximately 50 times. Table 3 lists the results of the recognition rate derived through Eq. (4).

Each of the four types of the experiment was performed 50 times, and the recognition rate was 96% for Person, 96% for Car, 98% for Bicycle, and 94% for Traffic light. On the other hand, these results may vary depending on the degree of light reflection or weather.

Fig. 5. Object Recognition Screen.

Table 3. Recognition Rate.

Object	Recognition Rate (%)
Person	96
Car	96
Bicycle	98
Traffic light	94

5. Conclusion

This paper proposed an application for real-time dangerous object detection for the visually impaired. Among the one-stage and two-stage detector methods, a one-stage detector more suitable for real-time object detection was selected, and a model trained through an SSD network was used for real-time object detection. Even a trivial object for the visually impaired can be considered a dangerous object in life. Therefore, this paper proposes a system that detects objects by selecting four objects: people, cars, bicycles, and traffic lights. The system notifies the visually impaired person with a sound when the camera detects the object. After applying the learned model to the application, experiments were performed approximately 50 times for each item. The experiment was for the visually impaired, so it was filmed using a camera in everyday life. The recognition rate for the experiment was calculated as the number of times the application made a sound after object detection. The experiment results were 96% for people, 96% for cars, 98% for bicycles, and 94% for traffic lights. In view of these results, the proposed system can guarantee a safer life for the visually impaired who have difficulty in life and can further improve the quality of life. In the future, more in-depth studies are needed to detect objects in environments with poor shooting conditions, such as light reflection and weather. If further research is conducted, a system useful for non-disabled people will come out when it is necessary to secure a dark space or a field of vision.