1. Introduction

The convolutional neural network (CNN) has proved outstanding in the field of computer vision and has become an essential technology in various fields, such as image classification ^[1,2], object detection ^[3-5], and image segmentation ^[6,7]. In addition, the CNN has dramatically improved the performance of object detection, which is highly useful, to a level that can be applied to actual applications. Accordingly, recent object detection algorithm research has been extended to not only improving performance but reducing costs through efficient structures ^[8-11] or new learning methods ^[12,13] for real-time operation on edge devices.

Most object detection studies focus on improving performance with benchmark datasets such as PASCAL VOC ^[14] and MS COCO ^[15]. In this case, a common approach is to use the same configuration on all datasets to make the comparison as fair as possible. However, the problem with that approach is not considering the characteristics of the objects in the datasets. In addition, analysis for real-world environments is rarely conducted, and may seem impractical to researchers and developers who apply object detectors to their applications.

This paper analyzes performance change according to the configuration and learning strategy in a basic object detection algorithm. The analysis confirms that a configuration considering data distribution is a variable that significantly influences performance improvement, rather than the complexity of the algorithm, e.g., knowledge distillation ^[12]. For example, it was shown that performance can be improved by up to 2.6\% AP with the KITTI dataset ^[16] only by modifying the configuration. We expect that the results of this study will provide meaningful insights for researchers who introduce object detection into applications.

2. Related Work

Object detection algorithms are classified into the two-stage object detector (e.g., Faster-RCNN ^[17], Mask-RCNN ^[18]) and the one-stage object detector (e.g., SSD ^[3] and Bi-FPN ^[8]). In this paper, we analyze the one-stage object detector, which is used more frequently, in practice, due to its lightness.

The first one-stage object detector, the SSD, senses several feature maps in a backbone CNN architecture. Then, classification and localization are performed by feeding them to the detection heads. Here, for efficient detection, the SSD utilizes several default boxes. After that, the SSD flow-up techniques improve performance by fusing and improving the feature map while maintaining this framework. Representatively, the FPN ^[4] fuses feature maps of different sizes by adding top-down and bottom-up pathways (so-called necks) before feeding them to the detection heads. This framework improves semantic information and location information in large feature maps and small feature maps, respectively. The Bi-FPN ^[8] goes further and adds a residual connection to the neck to enable aggregation of higher-level feature information. Recently, a method of directly injecting semantic information into an image was proposed ^[19]. The necks proposed so far can successfully improve the performance of SSDs. However, because they require complex feature aggregation, they incur burdensome memory costs. Therefore, when applying the object detection algorithm to a real-time application, it is necessary to select an appropriate neck considering the cost.

Another way to improve the performance of the object detector is to utilize external information. For example, there are ways to improve the backbone network to provide a better representation ^[20,21] via self-supervision. One way is to train large datasets, such as ImageNet, through self-supervision tasks to obtain better visual representations. Another example is knowledge distillation ^[12]. Knowledge distillation is a technique to improve performance by transferring information from a more extensive network to a smaller target network. Since they do not change the architecture, they have the advantage of no additional cost for inference. However, knowledge distillation utilizes two networks, which incurs a high cost. In addition, if there is a great difference between the source domain and the target domain of the external information, the expected improvement may not be achieved.

3. Method

In this section, we introduce various methods to improve the performance of the object detectors analyzed experimentally in this paper.

The neck type is one of the essential factors determining the performance of one-stage object detectors. Therefore, it is necessary to decide which neck to apply, carefully staying within the available cost budget. We used the most basic structures of the SSD and the FPN for experiments with the brief structures shown in Fig. 1. Since performance differences have already been discussed in previous papers, we analyze the differences in feature maps with two necks through knowledge distillation.

Knowledge distillation is a technique to improve performance by transferring information from one network to another. We selected and analyzed feature transfer techniques, i.e., AT ^[22], FT ^[23], SP ^[24], and SKD ^[25] to apply knowledge distillation to the object detector. The feature sensing points and structure diagram for knowledge distillation are shown in Fig. 2. To distill knowledge, we sense feature maps F for input to the detection heads in the SSD and the FPN and input them to each detection module, D. Then, the distilled knowledge from the teacher and student network is trained through objective function O. This process is expressed as follows:

For example, AT normalizes the teacher's and the student's feature maps, and minimizes their L2 distance. Therefore each component can be expressed as $D(x)=x /|x|, O(x, y)=\|x-y\| 2^{2}$. For other techniques, please refer to previous papers.

For the datasets, we adopted PASCAL VOC ^[14] and MS COCO ^[15], which are widely used in the object detection field. KITTI ^[16], which has a similar environment to real-world data, was also used. The KITTI dataset consists of road images at 1224x370 that were captured by a single camera. Therefore, we can say that KITTI is closer to a real environment than the existing benchmark datasets. In addition, since there is a big difference between KITTI and the PASCAL VOC and MS COCO data distributions, we checked the performance differences based on training strategies through KITTI.

Fig. 1. One-stage object detector frameworks.

Fig. 2. Examples of knowledge distillation frameworks.

4. Experiments

In this section, we present the results of each comparison experiment. First, we checked the effect of knowledge distillation based on the architecture and neck type for PASCAL VOC. The experimental results in Table 1 show that most combinations failed to improve performance when the architecture and neck type were different. Especially when the architecture was different, there was little or no performance improvement, regardless of the neck. Therefore, if there is no teacher network with the same structure, knowledge distillation is a bad choice for performance improvement.

Next, to further observe the effect of knowledge distillation, we show experimental results from MS COCO. In this experiment, we adopted the FPN, which was the most sensitive in previous experiments. Our results are in Table 2. First, by replacing the teacher network with ResNet-101, we checked the tendency when the size difference between teacher and student increased. Experimental results showed significant performance improvement, even when there was a big difference in the size of the architecture, which is contrary to results from the classification task ^[27]. This phenomenon is presumed to occur because over-constraints are prevented by mitigating the feature distribution difference in backbone networks with large-capacity differences owing to the neck. Also, in the ResNet-50 and MobileNet-v2 pairing, there was improvement in all techniques, unlike with PASCAL VOC. In particular, it is noteworthy that SKD showed the highest degree of improvement. SKD is a knowledge distillation technique designed to prevent over- and mis-constraints. This characteristic provides relatively high performance by suppressing mis-constraints caused by architecture differences. Nevertheless, it is difficult to find a clear trend in the results of each experiment, unlike the classification task. These results suggest that existing knowledge distillation techniques focus on classification, and even though they are generalized in classification datasets, they might not be in detection datasets.

Next, the transfer learning effect based on the dataset was verified. ResNet-50-FPN was used for the experiments. We used ImageNet and MS COCO as the source datasets, with KITTI as the target dataset. As a method of transferring external information, the most generally effective AT (-AT) among knowledge distillation techniques and fine-tuning was used. In the pre-trained networks, we used a public ImageNet pre-trained model that had only semi-weakly supervised learning (ImageNet- SWSL) ^[28] and an MS COCO pre-trained model.

The experimental results are shown in Table 3. Compared to the general benchmark dataset, the objects of the KITTI dataset exist sparsely. For this reason, we can see that the ImageNet pre-trained model had difficulty learning. Therefore, to obtain high performance, a pre-trained network is required as a detection dataset, such as MS COCO. However, because the KITTI dataset has wide images, each object has a very narrow form when resized to a rectangle. Therefore, although performance was improved when it was used as a pre-trained network, it is difficult to expect additional performance improvement through knowledge distillation. Therefore, when transferring knowledge, this difference in data distribution must be considered.

Fig. 2. Pseudo code for default box configurations for the MS COCO and KITTI datasets. Note that KITTI uses only tall default boxes.

We also found that when training datasets with unusual data distributions, such as KITTI, adjusting the configuration is more effective than applying special training strategies like knowledge distillation. In order to fit object shapes in KITTI, we modified default boxes as shown in Fig. 3. Those experimental results are the last row of Table 3. Even though only the default boxes changed, it gave higher performance than knowledge distillation. Examples of the detection results are in Fig. 4, showing that a detector with modified default boxes detects narrow objects quite well.

The experimental results in this paper show that widely used techniques to improve classification performance are not generalized well to detection tasks. On the other hand, the more commonly used data considering the configuration setup showed effective performance improvement. Therefore, in order to improve the performance of the detection algorithm, a parameter search based on analysis of the target dataset should be conducted, rather than using the general configuration as it is.

5. Conclusion

In this paper, we analyzed how to improve performance practically while adjusting various learning strategies and configurations. Experimental results showed that sometimes a heuristic and data-oriented approach is more effective than a complex learning strategy. This trend must be considered in the practical application stage. We expect this paper will provide meaningful insights to researchers or developers who want to apply object detectors to actual applications.