| Title |
Generative Stylization using Non-linear Style Mixing via Layer-wise Transformers |
| Authors |
이준형(Junhyoung Lee) ; 김준우(Junwoo Kim) ; 오희석(Heeseok Oh) ; 지준(Jun Ji) |
| DOI |
https://doi.org/10.5573/ieie.2023.60.5.60 |
| Keywords |
Transformer; Style mixing; Stylization |
| Abstract |
Stylization is the method of transforming a given image into a specific desired style. In early studies, studies using gradients were conducted, but recently, stylization using generative networks have been developed along with the development of a generative model capable of generating high-quality facial images. Among them, JoJoGAN introduced a learning process that enables fine-tuning through the weighted sum of the reference style latent vector and random noise found through GAN inversion and showed that excellent style transformis possible using only one reference image. However, the existing weighted sum-based style combination has a problem in that the manifold expressing the style code is limited as a simple linear combination form. Therefore, the result of stylization using the potential space does not reflect the fine expression effectively and at the same time has a limit in diversity. To solve this problem, this paper proposes two methods. First, it is a method to broadly compose a style latent space by combining non-linear stylization to increase the diversity and quality of creation. This allows for different sampling in an expanded style space, improving the quality and variety of the output. Second, we used a hierarchical Transformer structure to identify spatial long-term dependencies in style combinations to generate meaningful combinations of style features. In other words, style codes are encoded by separate independent neural networks for coarse, medium and fine layers. In the proposed method, improved stylization performance was confirmed through quantitative/qualitative/subjective experiments by inducing the overall appearance to follow the original appearance and to reflect the reference style for fine details. |