3.1. Generator Network Structure Design

The purpose of researching and designing AMS Cycle GAN is to address the limitations of existing image style transfer algorithms and meet their various potential needs in practical applications. Traditional style transfer models lack consistency in both content and style when generating images, and their generation efficiency is low. To address these issues, AMS Cycle GAN was designed and studied. In addition, when designing AMS Cycle GAN, the potential needs of image style transfer in different fields such as digital art creation, advertising design, film and animation production were considered to improve the visual quality of images and ensure consistency in image style while efficiently generating images.

The network structure of AMS-cycle-GAN image style migration algorithm generator is designed to provide improved image style migration effect. In order to achieve this goal, the network optimizes the model through three aspects. Firstly, PONO-MS is alternately used in the encoder decoder part, which can not only effectively retain the characteristic information of the input image, but also improve the optimization and convergence of the network. Secondly, the loss function is the same as the Cycle-DPN-GAN model, including the generation confrontation loss, cycle consistency loss, etc. Finally, a channel based attention mechanism is introduced into the discriminator to better focus on important content in the process of migration. The whole network structure has two generators and two discriminators, which realizes bidirectional data generation. The PONO-MS module in the generator is a variant of jump connection. It processes the average and standard deviation extracted by the encoder and injects them directly into the corresponding decoder layer, so that the characteristic map generated by the decoder has similar statistics with the corresponding layer of the encoder. Fig. 1 shows the overall structure.

Fig. 1, the $x$ and $y$ regions represent the input trends of the image, where $G$ and F both represent generators. The image is transmitted from the $x$ region to the $y$ region through generator $G$, and then from the $y$ region to the $x$ region through generator F. A PONO-DMS module is introduced between the encoders of the two generators to effectively save the style information of the image. The generator network structure includes encoder, converter, and decoder. The encoder captures the feature information from the input image by position normalization and directly injects it into the moment shortcut network layer of the decoder to effectively improve network optimization and convergence. The converter is responsible for keeping the size of the feature image unchanged, effectively transferring the features of the art image, and using reflection to fill the boundary of the input tensor. The generator network structure is shown in Fig. 2.

The moment shortcut network layer of the decoder receives the data information normalized by PONO, converts the size of the feature image from 64*64*256 to 256*256*64 through two layers of inverse transposition convolution, and maps the content and style information of the image to pixels. Finally, the convolution kernel size is $7\times7$ network layer outputs 256*256 three channel generated images. The role of PONO-MS module of position normalization and moment shortcut is to capture the feature mean and standard deviation extracted from the input image, and directly inject them into the moment shortcut network layer of the decoder, so as to improve the convergence of network training. Fig. 3 shows PONO-MS module schematic diagram.

All calculation formulas are derived based on the GAN algorithm and the network structure discriminator of the Patch GAN model. The calculation method of MS layer is shown in Eq. (1).

In Eq. (1), $F(x)$ represents the middle layer modeling, $\beta $ and $\gamma $ represents the mean and standard deviation parameters. The extracted mean and standard deviation are taken as new parameters respectively, as shown in Eq. (2).

3.2. Discriminator Network Structure Design

This discriminator network framework refers to the basic model of patch GAN, with a size of $70\times70$, and the attention mechanism is introduced in the fourth and fifth convolution layers. This mechanism helps to focus the key pixel areas in the image while ignoring or filtering the irrelevant parts. The application of attention mechanism can further improve the fine-tuning ability of generating confrontation model. The discrimination network structure is shown in Fig. 4.

The attention mechanism is realized through the convolutional block attention module (CBAM), which is a method to improve the performance of convolutional network, and mainly deals with the relationship between characteristic channels. When the image down sampling feature size is $1\times31\times31\times512$, the intermediate feature map obtains spatial information through global average pooling (GAP) and global maximum pooling (GMP) operations, generates two different spatial feature maps, and transmits them to the shared network. The characteristic diagram is shown in Eq. (3).

In formula (3), $R$ represents vector elements, $C$ represents the number of channels, $H$ represents the height of the feature map, and $W$ represents the width of the feature map.

The shared network is composed of multi-layer perceptrons with hidden layers, and the importance of each feature map channel direction is calculated. After obtaining the weight of each channel, they will excite J to the corresponding channel of the intermediate characteristic graph. This process is shown in Eq. (4).

Then Eq. (5) can be obtained.

In Eqs. (4) and (5), $F_{gap} $ represents the global average pooling characteristic graph, $F_{gmp} $ represents the global maximum pooling characteristic graph. $W_{0} $ and $W_{1} $ represents the shared weight. $GlobalAvgPool(F)$ represents global average pooling, $GlobalMaxPool(F)$ represents global maximum pooling, $MLP$ represents multi-layer perceptron, and $concat$ represents concatenation of feature maps both meeting the conditions as shown in Eq. (6).

In formula (6), $\frac{C}{r} $ represents the scaling factor, and $r$ represents the scaling ratio.

After this operation is completed, a GAP characteristic graph and a GMP characteristic graph are generated, both of which have weights. Then, the two feature maps are summarized. After concat operation, the number of feature map channels is doubled. Finally, a network layer with a $1\times1$ convolutional kernel is used to reduce the number of channels to 512. Then, a leaky rectified linear unit (Leaky ReLU) activation function is added. This completes the construction of the channel-based attention mechanism module. In the discriminator of the AMS-Cycle GAN model, a network layer with a $4\times4$ convolutional kernel is selected for output generation. Finally, a single-channel prediction map of size $1\times30\times30\times1$ is generated. The internal structure of the discriminator is shown below.

To make the model training more stable and meet the 1-lipschitz condition, spectral normalization (SN) is introduced into multiple convolution layers of the discriminator during the training process. AMS-Cycle-GAN model is shown in Eq. (7).

In formula (7), $L_{Generator} $ represents the total loss function of the generator, $L_{lsgan\_ Generato} $ represents the loss function of the generated adversarial network, $\lambda _{1} L_{identity} (G,F)$ represents the weight of identity loss, $\lambda _{2} L_{cycle} (G,F,X,Y)$ represents the weight of cyclic consistency loss, $\lambda _{3} L_{MS-SSIM} (G,F)$ represents the weight of MS-SSIM loss, and (G) and (F) represent the generator. Then there is Eq. (8).

In formula (8), $L_{discriminators} $ represents the loss function of the discriminator, and ${\mathop{\min }\limits_{D_{Y} }} L_{lsgan} (G,D_{y} ,X,Y)+\min L_{lsgan} (F,D_{x} ,Y,X)$ represents the game process between the generator and the discriminator. On this basis, there can be Eq. (9).

In Eq. (9), $(G,F)$ represents generator, $(D_{Y} ,D_{X} )$ represents discriminator, $L{}_{lsgan} $ represents generation confrontation process, $L_{Generator} $ represents generator model loss function and $L_{discriminators} $ represents discriminator model loss function. $L(G,F,D)$ represents a loss function that represents the minimization of all objectives. $MS\text{-}SSIM(x,y)$ is shown in Eq. (10).

In Eq. (10), $M$ is set as 5, $MS\text{-}SSIM(x,y)$ represents the multi-scale structural similarity index, $l_{m} (x,y)$ represents the brightness comparison function, $(x,y)$ represents the contrast comparison function, $s_{j} (x,y)$ represents the structural comparison function, $\alpha $, $\beta $, $\gamma $ represents the weights of each comparison function, and other parameters are set as Eq. (11).

$\alpha $ parameter setting is shown in Eq. (12).

Thus, the loss function $MS-SSIM$ can be obtained as Eq. (13).

$L_{identity} (G,F)$ in Eq. (7) is shown in Eq. (14).

In formula (14), $E_{x\sim P_{data(x)} } [\| F(x)-x\| _{1} ]$ represents the loss of transitioning from domain $X$ to domain $Y$ and then returning $X$, and $E_{y\sim P_{data(y)} } [\| G(y)-y\| _{1} ]$ represents the loss of transitioning from domain $Y$ to domain $X$ and then returning $Y$. $L_{cycle} (G,F,X,Y)$is show in Eq. (15).

AMS-Cycle-GAN model first randomly extracts an image from the natural image domain and inputs it into the generator. At the same time, position normalization is applied to extract the mean and standard deviation from the image $x$, and these information are transmitted to the MS network layer. Similarly, an image is randomly extracted from the art image domain and input into the generator to obtain the output. This process has also been processed by PONO-MS module. Then the generated image and the original art image are input into the discriminator. After passing through the attention mechanism module, the importance weights of different channels are obtained. Then these weights are applied to the corresponding channels of the intermediate feature graph. By minimizing the lsgan loss of the discriminator, the discriminator is optimized and the parameters are updated. Similarly, the generated image and the original natural image are input into the discriminator to minimize the lsgan loss of the discriminator, optimize the discriminator, and update the parameters. Next, the reconstructed image is obtained and the cycle loss is calculated. Then it is necessary to traverse and update the iterations in the training process. Finally, after completing the training process, the model establishes an image conversion model between the natural image domain and the artistic image domain, which can generate images with artistic style. During the training process, if the number of iterations exceeds a preset threshold, the learning rate will gradually decline.

Import tensorflow as tf

From tensorflow.keras import layers

Sef PONO_MS(x):

Mean, var $=$ tf.nn.moments(x, axes$=[1$, $2]$, keepdims$=$True)

Return (x $-$ mean) $/$ tf.sqrt(var $+$ 1e$-8$)

Def build_generator(input_shape):

Inputs $=$ tf.keras.Input(shape=input_shape)

X $=$ layers.Conv2D(64,(7,7),padding$=$`same')(inputs)

X $=$ layers.ReLU()(PONO_MS(x))

X $=$ layers.Conv2D(256,(3,3), padding$=$`same')(x)

Outputs $=$ layers.Conv2D(3,(7,7),padding$=$`same', Activation$=$`tanh')(layers.ReLU()(PONO_MS(x)))

Return tf.keras.Model(inputs, outputs)

Def build_discriminator(input_shape):

Inputs $=$ tf.keras.Input(shape$=$input_shape)

X $=$ layers.LeakyReLU(alpha$=$0.2)(layers.Conv2D(64,(4,4), strides$=$2, padding$=$`same')(inputs))

X $=$ layers.LeakyReLU(alpha$=$0.2)(layers.Conv2D(128,(4,4), strides$=$2, padding$=$`same')(x))

Outputs $=$ layers.Conv2D(1, (4, 4), padding$=$`same', activation$=$`sigmoid')(x)

Return tf.keras.Model(inputs, outputs)

4.1. Model quantitative analysis

First of all, the parameters used in the training phase were sorted out in detail. The beta value was set to $(0.5$, $0.999)$, which allowed the optimization strategy to set the trade-off between the first-order moment estimation (momentum) and the second-order moment estimation (RMSprop) at a suitable proportion, so as to converge faster in the training process. The batach size provided is 1. The research expects to focus on a single sample to update the parameters each time. The total number of epoch is set to 200, and the research expects that the model can fully learn and reduce the error to a reasonable range within this period of time. Parameter settings are shown in Table 1. The dataset settings are shown in Table 2.

The experimental environment of this case was deployed on Ubuntu 18.04.1 system, equipped with Intel (R) Xeon (R) platinum 8255C CPU and 47GB memory, which further improved the computing performance. In particular, the NVIDIA Ge Force RTX 3090 graphics card with 24G video memory was also used in this experiment, which had ideal execution efficiency and stability in large-scale data dependent deep learning tasks. The change of loss function is shown in Fig. 6.

As shown in Fig. 6, the line of the loss function in Fig. 6(a) rapidly decreases in the first 10 iterations and fluctuates frequently between 10 and 80 cycles, stabilizing after 80 iterations. The loss function line in Fig. 6(b) also rapidly decreases in the first 10 iterations, fluctuates frequently between 10 and 60 cycles, and tends to stabilize after 60 iterations. Figs. 6(c) and 6(d) show the situation after 100 iterations, indicating that it is still in a stationary state. Overall, during the training process of GAN, the total loss fluctuates greatly in the first 60 iterations. However, as the training cycle increases, the fluctuations gradually stabilize. This indicates that the proposed network model can be effectively trained. This indicates that the training effect is relatively good. With the training, the discriminator of the Cycle-DPN-GAN network was gradually difficult to distinguish the authenticity of the generated image.

4.2. Ablation test and performance analysis

In this ablation experiment, AMS-Cycle-GAN model was used for five groups of experiments. To evaluate the performance of the model, three kinds of losses were used: MS-SSIM loss, cyclic consistency loss, and identity mapping loss. The AMS-Cycle-GAN model was analyzed using the experimental data. The ablation test results are shown in Fig. 7.

As shown in Fig. 7, in this round of experiments, the MS-SSIM loss was 7.2, the cycle consistency loss was 3.1, and the identity mapping loss was 0.6. Compared with other experimental results, the benchmark experiment had lower values in MS-SSIM loss and cycle consistency loss, indicating better performance. In these five groups of experiments, making MS-SSIM loss, cycle consistency loss, and identity mapping loss reached a low and balanced weight combination was of great significance to improve the model performance, and AMS-Cycle-GAN had better performance. Removing the attention mechanism and changing the loss weight of identity mapping may reduce the performance of the model, which was enough to prove the effectiveness and robustness of AMS-Cycle-GAN. The image conversion effects on photo2vangogh and vangogh2photo datasets are shown in Table 3.

As shown in Table 3, if the attention mechanism module of AMS-Cycle-GAN was removed, the performance of IS and FID was better than that of the previous model, and the IS score of AMS-Cycle-GAN on the vangogh2photo dataset reached the highest $4.38 \pm 0.32$. It further explained the importance of attention mechanism in GAN model. Finally, AMS-Cycle-GAN model performed best in all settings. On the photo2vangogh dataset, the scores of IS and FID were $5.33 \pm 0.76$ and 113.28, respectively. The scores of vangogh2photo were $4.95 \pm 0.49$ and 149.68, significantly better than other models. The generalization test is shown in Fig. 8.

As shown in Fig. 8, the quantitative evaluation results of summer2winter, orange2apple and photo2flower datasets were analyzed. The evaluation indexes of these three data sets were mainly IS and FID scores. The IS score of summer2winter dataset was low, indicating that the generated image scene was relatively simple, but the FID score was low, indicating that the visual quality was good. The orange2apple dataset had a high is score, which indicated that the image diversity was high, but the FID score was high, and the visual quality needed to be improved. The IS and FID scores of photo2flower dataset were relatively low, indicating that it was relatively balanced in diversity and visual quality.

Although the algorithm designed for research has more performance and efficiency advantages than traditional methods, this method increases complexity, requires richer training resources, and also requires more tuning steps, which is an inevitable limitation of high-performance models.

Comparative analysis is shown in Table 4.

As shown in Table 4, AMS-cycle-GAN was used for comparison and Analysis on photo2vangogh and photo2monet datasets. On the photo2vangogh dataset, the AMS-Cycle-GAN's IS value was $5.3 \pm 0.8$, and the FID value was 114.1. Its IS value as relatively high, which meant that the image quality was relatively good. At the same time, the FID value of AMS-Cycle-GAN was relatively low, which indicated that the generated image was closer to the target image. Compared with other methods, AMS-Cycle-GAN had certain advantages in the performance of photo2vangogh dataset. On the photo2monet dataset, the IS value of AMS-Cycle-GAN was $6.0 \pm 0.9$, and the FID value was 75.5. These two indicators showed that AMS-cycle-GAN performed better than other methods in photo2monet. AMS-Cycle-GAN performed better on both photo2vangogh and photo2monet datasets. The IS value reflected the quality of the generated image, while the FID value indicated the similarity between the generated image and the actual image. AMS-Cycle-GAN had obvious advantages in IS value, and its FID value was relatively low. These data clearly showed that AMS-Cycle-GAN performed well on these two datasets, and the generated images were of high quality and close to the actual images. AMS-Cycle-GAN was quantitatively compared with other methods in vangogh2photo and monet2photo datasets. The comparative analysis of vangogh2photo and monet2photo datasets is shown in Table 5.

As shown in Table 5, on the vangogh2photo dataset, AMS-Cycle-GAN's IS score was $4.94 \pm 0.45$, which was much higher than MUNIT and UNIT's $2.35 \pm 0.18$ and $2.80 \pm 0.32$, and slightly higher than UGATIT's $3.92 \pm 0.14$. This showed that AMS-Cycle-GAN had high image quality and consistency when converting Van Gogh style into photo style. Similarly, in terms of FID score, AMS-Cycle-GAN scored 148.23, which was the lowest among all models. It meant that AMS-Cycle-GAN can better maintain the structural characteristics of the original image than other algorithms when moving from Van Gogh style to photo style. AMS-Cycle-GAN also performed well on monet2photo dataset. Its IS score was $3.51 \pm 0.32$, only lower than that of CycleGAN (IS $=3.38 \pm 0.34$) and Cycle-DPN-GAN (IS $=3.18 \pm 0.29$). In terms of FID similarity score, AMS-Cycle-GAN won with a score of 113.80, which meant that AMS-Cycle-GAN can retain the characteristics of the original image better than other models when transferring Monet's painting style to the photo style. Thus, AMS-Cycle-GAN had significant advantages in image style transfer.

As shown in Table 6, in terms of algorithm complexity, the number of parameters for AMS Cycle GAN is 12M, which is the lowest, followed by UNIT, UGatiT, and finally MUNIT. In terms of training time, AMS-Cycle GAN only takes 10 hours, which is the fastest among all algorithms. It can be seen that the complexity of the AMS Cycle GAN designed for research is the lowest. From the implementation time of shang.l, the implementation time of AMS Cycle GAN is 3600s, which is the shortest among all algorithms, and the longest is UNIT, which requires 4700 seconds. From this, it can be seen that the AMS Cycle GAN algorithm designed in this study has superiority in terms of comprehensive algorithm complexity and implementation time.

Table 7 shows the comparison of algorithm performance. It can be seen that the AMS Cycle GAN algorithm designed in the study has the best performance indicators, with the highest PSNR value of 27.5 and the highest SSIM value of 0.90,. Its PL value is the smallest, but this indicator is a reverse indicator, and the smaller the value, the better the effect. The LPIPS value of the AMS Cycle GAN algorithm is also the smallest, and the smaller the value, the better the effect. Overall, compared with advanced algorithms, the AMS Cycle GAN algorithm designed in research also has advantages.