| Title |
Generative Image Inpainting Method Reflecting Human Perceptual Characteristic |
| Authors |
오희석(Heeseok Oh) ; 최원석(Wonsuk Choi) |
| DOI |
https://doi.org/10.5573/ieie.2021.58.6.41 |
| Keywords |
Image inpainting; perceptual characteristic; MSCN coefficient; generative adversarial network |
| Abstract |
Image inpainting represents a technique of recovering the content in missed spatial area, and which is widely being employed to discard an unwanted object in terms of image adjustment. In recent, beyond the conventional diffusion- and patch-based approaches, various deep learning-based schemes have been introduced as the other computer vision tasks. However, inpainting methods with utilizing a CNN (convolutional neural network) and GAN (generative adversarial network) generally resulted in the blurred image or local artifacts. That is, previous methods cannot effectively decode the embedded features for abstracting global semantic representation to RGB image domain implying contextual structure and fine-details simultaneously. To cope with this problem, we propose a two-stage inpainting scheme reflecting the human’s perceptual characteristic. The proposed method consists of two GANs. The first generative network aims for generation of MSCN (mean subtracted contrast normalized) coefficients which resembles a visual conveyance process from the eyes to the primary visual cortex (area V1) in the brain. The second generative network completes RGB image which refers the generated MSCN coefficients as a prior structural information. Non-shared two generative networks with each corresponding discriminator are trained in end-to-end manner, and the result argues that the proposed two stage method can sufficiently recover both structure and high-frequencies of the missed regions. The experiments are performed on CelebA dataset, and the quantitative results show that the proposed generative inpainting scheme reflecting perceptual characteristic outperforms the previous methods regarding visual quality even in a large irregular hole scenario. |