Two-stage occlusion giant panda image inpainting based on partial convolutions, multi-scale contextual attention and a new PatchGAN with two discriminators

Image-based individual recognition of giant pandas in real wild scenes suffers from difficulties such as occlusion and multiple postures. Image painting is an important preprocessing step in solving the occlusion of giant panda images. A two-stage inpainting model of giant panda images based on partial convolutions and multi-scale features is proposed. In the coarse inpainting, the structural in-formation of the occluded part is restored, while the fine inpainting focuses on re-storing details such as textures and edges based on the coarse inpainting. A new PatchGAN with two discriminators in the coarse inpainting balances two-scale information guided by the WGAN-GP loss and L1 norm. The generator of the new PatchGAN uses partial-convolutions to avoid the propagation and influence of misinformation in the occluded area. In the fine inpainting, a new proposed module fusing multi-scale feature by contextual attention is added to the PatchGAN. The fine inpainting model learns and enhances the texture and details of the image output by the coarse inpainting through the global searching for mul-ti-scale similar image patches by multi-scale context attention. Thus, it can strengthen the semantic connection between occluded and real image regions. In order to achieve better multi-scale inpainting results, the perceived loss and style loss are added to the adversarial loss. Compared with state-of-art methods, the proposed method can effectively restore image textures and details while sup-pressing noise and artifacts from visual effects. The PSNR, SSIM and FIN of proposed method can achieve 35.69, 0.971 and 5.22 respectively, indicating that proposed method can obtain satisfied inpainting results.