3.1 MW Security Imaging and Radiation Characterization of Objects

Objects made of different materials have different MW radiation properties. We generated different MW images based on the different radiation properties of human skin and typical materials. MWs are electromagnetic waves with a wavelength of 1-10$mm$ and a frequency of 30-300$GHz$. They are widely used in the field of civil aviation for scanning security screening equipment to accurately identify whether passengers are carrying contraband and to ensure the traffic safety of civil aviation ^[16,17]. The principle of MW imaging is that an external source emits a coherent MW in order to irradiate the target, the reflected wave carrying the phase and amplitude information of the target and the background is received, and then the temperature difference data of the echoes are then processed by an algorithm to obtain the scattered intensity distribution of the target (see Fig. 1).

Fig. 1. Diagram of Millimeter-wave imaging principle.

In Fig. 1, the millimeter wave scanning plane is parallel to the plane $\left(X,Y\right)$, and the vertical distance from the plane is $R_{0}$, and the coordinate of any point of the target is $\left(x,y,z\right)$, then the corresponding point on the millimeter wave scanning plane is $\left(x',y',z_{0}\right)$. Since thermal radiation is the inherent electromagnetic energy emitted by the thermal perturbation of electrons in the material of which an object is composed, any object with a physical temperature above absolute zero has thermal radiation properties. We analyzed the radiation properties of real-world radiators based on an ideal black-body radiator. The spectral brightness $B_{f}$ of the black-body emission amplitude can be expressed by Planck's law ^[18]:

In Eq. (1), $h$ and $k$ represent Planck's constant and Boltzmann's constant, and their units are $J.s$ and $J/K$, respectively. $T$ and $c$ represent the physical temperature and speed of light, and their units are $K$ and $m/s$, respectively. $e$ is the base of the natural logarithm, $f$ is the frequency, and its units are $Hz$.

All objects in the real world are grey bodies. A grey body is a non-ideal radiator compared to a black body. It radiates less energy than a black body at the same physical temperature. The spectral brightness of a grey body, $B'_{f}$, is given in Eq. (2).

$\lambda $ and $T_{R}$ in Eq. (2) represent the wavelength and radiation temperature of a grey body, respectively, and $T_{R}$ is expressed in Eq. (3).

$e_{f}$ is the grey body emissivity, and a uniform object’s $e_{f}$ is only a function of frequency. When the physical temperature of the radiating body is certain, the grey radiation energy is less than the black-body radiation energy, so $0\leq e\leq 1$.

An object in thermal equilibrium is irradiated by an incident wave with an incident flux density of $\mathrm{S}_{\mathrm{in}c}$. According to Eq. (4), the incident flux density is equal to the object's total reflected, transmitted, and absorbed flux densities.

In Eq. (4), $\alpha $, $\beta $, and $\chi $ indicate the reflectivity, transmittance, and absorbance of the object, respectively. According to Kirchhoff's law of radiation, the absorbed power of an object in thermal equilibrium is equal to the radiated power of the object:

$e$ is the object’s radiation power. Because the human body has a thickness and is an opaque object, the MW band transmittance can be neglected:

Objects of different materials have different reflectivity, emissivity, and transmittance, and their radiation properties in the $k_{a}$ band of MWs are shown in Table 1.

The transmissivity of human skin, metal, and plastic products in Table 1 is negligible, and the transmittance of clothing is 0.98. Because human bodies and the things they carry have different temperatures, an MW imager can detect temperature differences for metal and illegal plastic materials. Human skin, metal, plastic articles, and clothing also have varying reflectivity and emissivity.

3.2 Design of a Contraband Detection Algorithm based on MFD-WT Algorithm and CED Algorithm

MW images are different from optical images in that their spatial resolution and contrast are lower ^[19]. The contraband image target detection approach based on edge detection is primarily a mix of two image processing techniques: image denoising and edge extraction ^[20]. The mean filtered image $G\left(x,y\right)$ is obtained as:

In Eq. (7), $F\left(i,j\right)$ represents the original image, $mn$ is the size of the convolution kernel, and $R_{xy}$ is the center point at $\left(x,y\right)$. The process of mean filtering is shown in Fig. 2.

In Fig. 2, the grayscale image $F\left(i,j\right)$ is processed by MFD with a convolutional kernel of size $3\times 3$ and the convolution values are all set to 1. The convolutional kernel goes over the original image during MFD processing, and the processed grayscale value is equal to the convolutional value of the kernel and the corresponding window. WT is able to extract the features of the signal in different frequency bands and retain the time-domain features of the signal at each scale.

Fig. 2. Diagram of MFD treatment process.

Fig. 3. Process flow of CED algorithm.

After using the MFD-WT algorithm to denoise the image, the CED algorithm is used to extract the edge of the image. The CED algorithm is a common edge detection algorithm with stable generalization capability and a simple implementation process. It satisfies the three edge detection criteria of low error rate, accurate edge localization, and single edge marking. Therefore, the CED algorithm was chosen as the edge detection algorithm for MW contraband image targets.

As shown in Fig. 3, the image is smoothed using a Gaussian filter to remove the noise in the image. Next, the gradient intensity and direction of the image's pixel points are calculated so that the non-maximum suppression process can be used to remove the detected weak edges. The true edges are determined using double threshold detection, and finally, the edge detection of the image is finished by suppressing the detected weak edges. The calculation of the Gaussian convolution kernel is shown in Eq. (8).

In Eq. (8), $\sigma $ is the standard deviation, and $\left(2b+1\right)\ast \left(2b+1\right)$ is the convolutional kernel size. The Canny algorithm mainly uses four edge detection operators to detect vertical, horizontal, and diagonal edges in the image and calculates the horizontal gradient, vertical gradient, and direction of each pixel using edge detection operators, as shown in Eqs. (9) and (10).

In Eq. (9), ${\nabla }_{\left(x,y\right)}^{h}$ and ${\nabla }_{\left(x,y\right)}^{v}$ represent the horizontal and vertical edge detection operators of size $\left(2b+1\right)\ast \left(2b+1\right)$, and ${G}_{\left(x,y\right)}^{h}$ and ${G}_{\left(x,y\right)}^{v}$ represent the horizontal and vertical gradients of pixel $\left(x,y\right)$, respectively. $W$ is the detection window, and $\Theta $ refers to the convolution calculator. The expression for the direction $\theta \left(x,y\right)$ of pixel $\left(x,y\right)$ is given in Eq. (10).

3.3 Design of Contraband Recognition Algorithm based on Mask R-CNN Algorithm

After using the target detection algorithm based on the MFD-WT algorithm and CED algorithm to complete the edge detection of security images, it is necessary to use a deep learning model to identify contraband targets in the detection area. A Convolution Neural Network (CNN) model is a deep learning model that can process image data through supervised learning. The parameters of the convolutional layer are expressed in Eq. (11).

In Eq. (11), ${x}_{j}^{l}$ is the $j$th neuron in the $l$th layer. $Kernel$, *, and $f\left(\cdot \right)$ represent the convolution kernel, convolution operation, and non-linear excitation function, and $a$ and $M_{j}$ represent the bias term and the number of inputs of the $j$th neuron, respectively. The common pooling operations are mean pooling and maximum pooling, as shown in Fig. 4.

Fig. 4. Diagram of pooling.

Fig. 4(a) shows a diagram of maximum pooling, and Fig. 4(b) shows a diagram of mean pooling. Maximum pooling retains the maximum value of each sliding window, and mean pooling retains all values of each sliding window by averaging them. The excitation layer introduces non-linear features to the neural network, allowing it to approximate arbitrary non-linear functions. A common activation function is shown in Eq. (12).

The sigmoid function, tanh function, and ReLu function are represented in Eq. (12) by $f_{S}\left(x\right)$, $f_{T}\left(x\right)$, and $f_{R}\left(x\right)$, respectively. The sigmoid function and tanh function have saturation zones, and the gradient is prone to vanishing during backpropagation. This phenomenon becomes more pronounced as the number of network layers increases.

The ReLu function exists to solve thisproblem with a positive activation value derivative of 1, and it can effectively avoid the phenomenon of gradient disappearance. The function of the fully connected layer is to integrate input image features and map the feature maps generated by the convolutional layer into fixed-length feature vectors, thus completing the image classification task, as shown in Eq. (13).

In Eq. (13), $w_{i,j}$ and $b_{j}$ represent the weights and offsets between neurons, respectively. The R-CNN algorithm is a CNN series method and is primarily made up of three modules. The R-CNN algorithm uses a selective search algorithm when generating candidate regions and trains the candidate boxes searched by the selective search algorithm using a CNN model to output element vectors to describe image content. The vectors are then input into a linear support vector machine for classification. However, too many candidate regions for the R-CNN algorithm make it run slowly, and it is not suitable for target recognition of MW contraband images. In response to this issue, the Mask R-CNN algorithm has been proposed, as shown in Fig. 5.

Fig. 5. Diagram of Mask R-CNN algorithm structure.

The Mask R-CNN algorithm in Fig. 5 mainly consists of a feature extraction network, a candidate region generation network, a mask prediction branching network, and a classification network. The feature extraction network is the backbone architecture of the Mask R-CNN algorithm, which uses ResNet-101 as the feature extractor. In order to accurately extract the semantic information of the low-level and high-level features, the feature extraction network is designed with a two-way structure. A one way structure is a bottom-up network forward process, where the feature map size is changed at some layers. The layers that do not change size are grouped into one order, and the last layer of one order is extracted as the output each time.

Another type of feature extraction network uses a top-down 2-fold up-sampling process. The results of the up-sampling are fused with a feature map of the same size generated from the bottom-up process, and then each fused result is convolved by a $3\times 3$ convolution kernel. A mask prediction branching network is a convolutional network with an input target candidate region and generates a mask for the target candidate region. The resolution of the generated mask is 28*28, which is small and helps to keep the mask branching network lightweight. In the Mask R-CNN algorithm, the loss function has classification error, detection error, and segmentation error due to the introduction of a new mask branch:

In Eq. (14), $L_{cls}$, $L_{box}$, and $L_{mask}$ represent the classification regression loss value, detection regression loss value, and mask loss function, respectively.

4.1 Performance Analysis of MFD-WT Algorithm and CED Algorithm

To verify the feasibility of the MFD-WT algorithm, CED algorithm, and Mask R-CNN algorithm for MW contraband detection and identification, we conducted comparative experiments using several algorithms. In order to verify the effectiveness of the target detection algorithm, the MFD-WT algorithm and CED algorithm were tested in a Matlab simulation. The control group for the MFD-WT algorithm was the MFD, WT, and median filter denoising algorithms, and the control group for the CED algorithm was the Sobel Edge Detection (SED) algorithm.

The Peak Signal-to-Noise Ratio (PSNR) and Mean Square Error (MSE) were the performance measurements. The higher the PSNR is and the lower the MSE is, the better the algorithm performs. The experimental samples selected in this study were image data of a human body carrying contraband obtained by an MW real-time imager, as shown in Table 2.

Table 2 shows the radiation effects of temperature, imaging speed, and clothing thickness on the detection target without the frequency band. In order to ensure the detection accuracy and generalization ability of the model, the MW image dataset was divided into a training set, test set, and verification set, and the total number of samples was set to 800, 300, and 400 samples.

The PSNR and MSE results for the four image denoising methods are shown in Table 3. The PSNR and MSE values for the MFD-WT algorithm were 25.439 dB and 65.4781, respectively. The WT algorithm’s PSNR and MSE values were 22.375 dB and 87.8943, respectively. The median denoising algorithm's PSNR value was 22.157 dB, and its MSE value was 99.4522. The MFD algorithm's PSNR and MSE values were 21.515 dB and 126.4586, respectively. The combined data in the table shows that the MFD-WT algorithm has superior performance and performs well in the denoising of MW contraband images.

Fig. 6 displays the PSNR findings for the two techniques for various noise categories. As the noise density rises, so do the PSNR values for both algorithms. The PSNR values for the two algorithms in the category of salt-and-pepper noise are displayed in Fig. 6(a). When the salt-and-pepper noise density was 1%, the PSNR values for the CED and SED algorithms were 27.987 dB and 26.115 dB, respectively. When the salt-and-pepper noise density was 13%, the PSNR values of the CED and SED methods were 26.710 dB and 25.187 dB, respectively.

Fig. 6. Results of two algorithms' PSNRs for various categories of noise density.

Fig. 7. Results of two methods' MSEs for various noise classes.

The PSNR findings for the two techniques in the Gaussian noise category are displayed in Fig. 6(b). When the Gaussian noise density was 1.0%, the PSNR value of the CED algorithm was 26.701 dB, and the SED algorithm’s value was 25.001 dB. When the Gaussian noise density was 4.0%, the PSNR values of the CED and SED algorithms were 24.871 and 22.996 dB, respectively.

Fig. 7 displays the MSE outcomes for both algorithms for various noise categories. Fig. 7(a) displays the MSE findings for the two methods in the category of salt-and-pepper noise. At 3% salt-and-pepper noise, the MSE values for the CED algorithm and the SED algorithm were 59.8412 and 123.5678, respectively. At 10% salt-and-pepper noise, the MSE values of the CED and SED algorithms were 74.5662 and 154.2314, respectively.

Fig. 7(b) shows the MSE results of the two algorithms for the Gaussian noise category. At 5% Gaussian noise, the MSE value of the CED algorithm was 69.8777, and the MSE value of the SED algorithm was 140.2312. At 15% Gaussian noise, the MSE values of the CED and SED algorithms were 98.7452 and 174.5782, respectively. Based on the data in Figs. 6 and 7, it can be seen that the average PSNR value of the CED algorithm was larger than that of the SED algorithm, and the average MSE value of the CED algorithm was smaller than that of the SED algorithm. This shows that the performance of the CED algorithm was better than that of the SED algorithm, and it can effectively and stably realize the real-time detection of MW contraband images.

4.2 Performance and Application Analysis of Mask R-CNN Algorithm

A Faster Regional Convolutional Neural Network (Faster R-CNN)-based algorithm, the R-CNN algorithm, and the Mask R-CNN algorithm were used in comparison tests to evaluate the performance of the Mask R-CNN method. The algorithm model was built based on the Tensorflow open-source framework, cuda10 graphics architecture, CPU model i7-7700, and GPU model Quadro P6000. The performance metrics were the Precision Recall (PR) curves, Mean Average Precision (MAP), F1 values, and accuracy rates.

The PR curves and F1 value curves of the three algorithms are shown in Fig. 8. Fig. 8(a) shows the PR curves of the three algorithms. The larger the area of the PR curve is, the better the performance of the algorithm will be. The area of the PR curve of the Mask R-CNN algorithm was 0.94, and the areas of the PR curve of the FasterR-CNN and R-CNN algorithms were 0.82 and 0.73, respectively.

Fig. 8(b) shows the F1 value curve. With the increase of the number of iterations, the F1 value of all three algorithms was improved. When the number of iterations was 25 epochs, the F1 value of the Mask R-CNN algorithm was 70.54%, the F1 value of the Faster R-CNN algorithm was 60.42%, and the F1 value of the R-CNN algorithm was 52.13%. When the number of iterations was 150 epochs, the F1 values of the Mask R-CNN, Faster R-CNN, and R-CNN algorithms were 94.14%, 82.33%, and 73.15%, respectively.

Fig. 8. PR curve and F1 results of three algorithms.

Fig. 9. MAP results of three algorithms.

Fig. 10. Classification accuracy and calibration frame accuracy results of three algorithms for different contraband.

The three algorithms' MAP results are displayed in Fig. 9. The MAP values of the three algorithms exhibit a quick increasing trend as the number of iterations rises, but after a certain number of iterations, the MAP values of the three methods fluctuate. When the number of iterations was 30 epochs, the MAP values of the Mask R-CNN, Faster R-CNN, and R-CNN algorithms were 82.13%, 71.88%, and 55.44%, respectively. The MAP value of the Mask R-CNN algorithm was 91.15%, that of the Faster R-CNN algorithm was 82.41%, and that of the R-CNN algorithm was 73.20% when the number of iterations was 150 epochs.

To verify the feasibility of the Mask R-CNN algorithm in MW contraband image recognition, we set up a folding pocket knife, gun, and large knife for contraband recognition experiments. Fig. 10 displays the classification accuracy and calibration frame accuracy for the three algorithms for various contraband. The categorization accuracy results for the three algorithms for various categories of contraband are displayed in Fig. 10(a).

For the folding pocket knife, the classification accuracy of the Mask R-CNN algorithm was 93.65%, that of the Faster R-CNN algorithm was 82.19%, and that of the R-CNN algorithm was 73.54%. For the gun, the classification accuracy of the Mask R-CNN algorithm was 89.94%, which was higher than that of the Faster R-CNN algorithm and R-CNN algorithm (75.48% and 69.87%). For the machete, the classification accuracies of the Mask R-CNN, Faster R-CNN, and R-CNN algorithms were 91.25%, 81.14%, and 71.57%, respectively.

The calibration frame accuracy results for the three algorithms for the various types of contraband are displayed in Fig. 10(b). The calibration frame accuracy of Mask R-CNN for the folding pocket knife, gun, and large knife were 94.83%, 91.24%, and 93.56%, respectively. The calibration frame accuracy of the Faster R-CNN and R-CNN algorithms for the folding pocket knife were 84.15% and 73.20%, while they were 79.88% and 68.79% for the gun and 81.61% and 71.27% for the large knife, respectively. In summary, the performance of the Mask R-CNN algorithm proposed in this study was robust and effective in identifying contraband in the MW domain.