2.1 Convolutional Neural Networks

The CNN ^[12] is one of the most typical and frequently used approaches in deep learning. Its features, such as rotation, translation, and size scaling, make it highly suitable for processing image data. Here is a brief introduction to the most important parts of the CNN approach. The convolutional layer is responsible for extracting features from the input and is the most important part of the CNN approach. It is executed multiple times with different convolutional kernels each time due to variations in extracted data. Its calculation is as follows:

where $x_{m+i，n+j}$ represents the image pixel value of point (m+i, n+j), $w_{ij}$ represents the weight of the convolutional kernel scale at point (i, j), b represents the size of the bias in the layer, and f is the network activation function.

The pooling layer is mainly responsible for subsampling the dance movement feature map obtained in the convolutional layer. It can compress a large amount of image data while maintaining the scale feature. The specific calculation is:

where $x_{m*{S_{1+i}}，n*{S_{2}}+j}$ represents the pixel value of point $\left(m*S_{1+i},\,\,\,n*S_{2+j}\right)$, and $y_{mn}$ represents the output value after the pooling operation.

The fully connected layer connects the extracted feature map and classifies input images based on the training data using the features. Every neuron in this layer is connected to the neuron of the last layer. The input of this layer is the vector output from the last layer, and the output of this layer is the final output of the CNN.

The activation function is a crucial component of a CNN. If the activation function is missing, the network input and output will be linear. For a nonlinear structure, the activation function should be added as a nonlinear unit. Commonly used activation functions include sigmoid, tanh, and ReLU ^[13]. All values of the ReLU function are compared with 0, and the largest value is selected. Therefore, when the input value is less than or equal to 0, the output value is also 0. The function only needs to judge whether input value x of the function falls within a positive interval, resulting in a small computation amount, i.e., its performance in network convergence speed surpasses that of the aforementioned two functions. This paper uses ReLU not only because it reduces network fit issues; it also improves network convergence speed. Its formula is

When training the network, if the model predicts unknown data, error in the test set for the model will be larger than in the training set, indicating poor prediction performance. To prevent this kind of network fit phenomenon, this paper adopts the dropout method ^[14] to optimize the model. The dropout principle is as follows. In the process of network training via deep learning, some neurons are removed while the remaining neurons continue to participate in network training without any changes. The removed neurons are randomly selected and their previous values are restored in subsequent network training sessions. This operation reduces the correlation between neurons, and prevents local features from fitting into the network. To prevent network fit issues, a dropout parameter of 0.5 was set.

2.2 Improving Movement Recognition

The above CNN method is able to recognize ballet movements, but the traditional CNN easily falls into the problem of overfitting during the training process. To improve recognition performance from the CNN, the PSO algorithm replaces the backward adjustment of weight parameters in the traditional CNN method. The forward calculation of the improved CNN method in the training process is consistent with the traditional CNN, and when the error obtained from the forward calculation does not converge within the set range, the iterative formula of the PSO algorithm is used to iterate the position and velocity of the particle swarm. The coordinates of each particle in the swarm represent a parameter scheme, and the iterative formula of the PSO algorithm is

where $v_{i}(t+1)$ and $x_{i}(t+1)$ are the speed and position of particle $i$ after one iteration, $v_{i}(t)$ and $x_{i}(t)$ are the speed and position of particle $i$ before the iteration, $\varpi $ is the inertia weight of the particle, $c_{1}$ and $c_{2}$ are learning factors, $r_{1}$ and $r_{2}$ are random numbers between 0 and 1, $P_{i}(t)$ denotes the optimal position experienced by particle~$i$ (excluding particles exceeding the limit), and $G_{g}(t)$ is the best position experienced by the particle population after excluding particles that exceed the limit. After the iteration, the parameter scheme represented by the particle is substituted into the CNN method, which performs forward calculation again, repeating the above process until the error converges to within a preset range.

The improved CNN approach autonomously extracts features from input images of real dance movements, eliminating the need for manual feature input. The specific process of dance movement recognition using the CNN approach is as follows. (1) A sufficient number of dance action videos are collected to form an initial dataset. (2) In data pre-processing after segmentation and denoising, image frames are extracted from the collected dance action data videos and are modified in terms of size, color, etc. to meet the standard requirements for image input network models. (3) The network model is trained, the processed image frames are divided into two parts, one of which is used as the training set, constantly adjusting parameters for the image data in that set until optimal performance is achieved. (4) After obtaining the optimal network model, the remaining image frames are used as input for the test set, and the dance movements are recognized by the network model.

3.1 Image Pre-processing

The pre-processing of images in dance action videos is an important link in the recognition system because it determines the effect of CNN training and testing. To ensure the validity of the experiment results, this paper randomly selected 1,000 ballet training video samples as a dataset. The dance movements in the videos were divided into five main categories: drawing circles with legs (Fig. 1(1)), small kicks (Fig. 1(2)), two-position mid-jumps (Fig. 1(3)), single-leg squats (Fig. 1(4)), and large squats (Fig. 1(5)). The number of videos for each specific type is shown in Table 1. The reason for choosing ballet movements is that different types of dances have their own characteristics, and conducting comprehensive training and recognition would result in an overwhelming workload. Therefore, ballet was chosen as the focus of recognition. Ballet is a graceful art form originating from the Italian Renaissance which flourished in France throughout its development and perfection. It combines elements of dance, music, and drama while highlighting dancers' body posture techniques as well as expressions. Being a classical dance style, it not only showcases elegance with artistic beauty but carries on a historical and cultural heritage. At the same time, the captured dance video images underwent pre-processing that mainly involved noise removal, contrast adjustment, grayscale transformation, video length reduction, and uniform resizing to 224 ${\times}$ 224. Additionally, image processing operations such as Gaussian blur were applied to prevent image blur after cropping. These image pre-processing operations ensured that the dance videos in the dataset had similar resolutions, durations, contrast, and sizes for faster training of the CNN model.

Fig. 1. Five types of dance movement.

3.2 Experiment Design

In order to verify the performance of the optimized CNN method in recognizing ballet movements, traditional CNN and SVM methods were also tested. The optimized CNN was obtained by adding PSO to the traditional CNN, so both methods were compared to test the improvement from PSO on the recognition performance of the CNN. The SVM method was a traditional machine learning classification algorithm. The recognition of ballet movements in this paper can also be considered recognition of dance movement types, and recognition was used to verify the performance of the optimized CNN method compared to other recognition algorithms ^[15]. The 1,000 ballet videos in the initial dataset were divided into a training set and a test set at an 8:2 ratio. The parameters of the traditional CNN are shown in Table 2. The parameters of the optimized CNN were the same as the traditional CNN. The parameters of the PSO are as follows: particle swarm size: 20, learning factor: 1.5, and inertia weight: 0.8. The parameters of the SVM method are as follows. The kernel used a sigmoid function, and the penalty factor was set to 1.

3.3 Analysis of Results

The SVM method directly computed the support vector hyperplane based on the data in the training set during the training process, which was different from the step-by-step iterative process of the traditional and improved CNN methods. Fig. 2 shows the error convergence curves of the traditional and optimized CNN methods during the training process. We can see that the recognition error in both algorithms decreased and converged to stability with an increase in the number of iterations. The improved CNN was the fastest, converging to stability after about five iterations, and the traditional method converged to stability after about 20 iterations. After convergence stability, the error of the optimized CNN method was smaller than the traditional CNN.

Fig. 2. The convergence curves of the traditional and optimized CNN methods.

The impact from the number of consecutive frames on network recognition accuracy was initially determined. The 1,000 dance videos were converted to images. Six sets of frames at five-frame intervals (from 5 frames up to 30 frames) were extracted for experimentation. We see from Fig. 3 that as the number of consecutive frames increased, recognition accuracy also increased and gradually tended to be stable, and the increase in the accuracy was very little after the number of frames exceeded 25. It is known that more consecutive frames require more computations and more time. Therefore, considering the experiment’s length, the final number of frames was set at 25.

Fig. 3. The influence of the number of consecutive frames from an image on recognition accuracy.

The recognition accuracy and speed of the three algorithms are in Table 3. From the comparison, we can see that the SVM method had the lowest accuracy and the least efficiency. The traditional CNN method had greater accuracy and efficiency, and the optimized CNN method had the highest accuracy and efficiency.