Search Results (17)

Tang Dynasty female terracotta figurines, as important relics of ceramics art, have commonly suffered from natural and man-made damages, among which facial damage is severe. Image inpainting is widely used in cultural heritage fields such as murals and paintings, where rich datasets are available. However, its application in the restoration of Tang Dynasty terracotta figurines remains limited. This study first evaluates the extent of facial damage in Tang Dynasty female terracotta figurines, and then uses the Global and Local Consistency Image Completion (GLCIC) algorithm to restore the original appearance of female terracotta figurines, ensuring that the restored area is globally and locally consistent with the original image. To address the issues of scarce data and blurred facial features of the figurines, the study optimized the algorithm through data augmentation, guided filtering, and local enhancement techniques. The experimental results show that the improved algorithm has higher accuracy in restoring the shape features of the female figurines’ faces, but there is still room for improvement in terms of color and texture features. This study provides a new technical path for the protection and inpainting of Tang Dynasty terracotta figurines, and proposes an effective strategy for image inpainting with data scarcity. Full article

Advances in visual odometry (VO) systems have benefited from the widespread use of affordable RGB-D cameras, improving indoor localization and mapping accuracy. However, older sensors like the Kinect v1 face challenges due to depth inaccuracies and incomplete data. This study compares indoor VO systems that use RGB-D images, exploring methods to enhance depth information. We examine conventional image inpainting techniques and a deep learning approach, utilizing newer depth data from devices like the Kinect v2. Our research highlights the importance of refining data from lower-quality sensors, which is crucial for cost-effective VO applications. By integrating deep learning models with richer context from RGB images and more comprehensive depth references, we demonstrate improved trajectory estimation compared to standard methods. This work advances budget-friendly RGB-D VO systems for indoor mobile robots, emphasizing deep learning’s role in leveraging connections between image appearance and depth data. Full article

Generative models are used as an alternative data augmentation technique to alleviate the data scarcity problem faced in the medical imaging field. Diffusion models have gathered special attention due to their innovative generation approach, the high quality of the generated images, and their relatively less complex training process compared with Generative Adversarial Networks. Still, the implementation of such models in the medical domain remains at an early stage. In this work, we propose exploring the use of diffusion models for the generation of high-quality, full-field digital mammograms using state-of-the-art conditional diffusion pipelines. Additionally, we propose using stable diffusion models for the inpainting of synthetic mass-like lesions on healthy mammograms. We introduce MAM-E, a pipeline of generative models for high-quality mammography synthesis controlled by a text prompt and capable of generating synthetic mass-like lesions on specific regions of the breast. Finally, we provide quantitative and qualitative assessment of the generated images and easy-to-use graphical user interfaces for mammography synthesis. Full article

In this paper, we investigate the inpainting of normal maps that were captured from a lightstage. Occlusion of parts of the face during performance capture can be caused by the movement of, e.g., arms, hair, or props. Inpainting is the process of interpolating missing areas of an image with plausible data. We build on previous works about general image inpainting that use generative adversarial networks (GANs). We extend our previous work on normal map inpainting to use a U-Net structured generator network. Our method takes into account the nature of the normal map data and so requires modification of the loss function. We use a cosine loss rather than the more common mean squared error loss when training the generator. Due to the small amount of training data available, even when using synthetic datasets, we require significant augmentation, which also needs to take account of the particular nature of the input data. Image flipping and inplane rotations need to properly flip and rotate the normal vectors. During training, we monitor key performance metrics including the average loss, structural similarity index measure (SSIM), and peak signal-to-noise ratio (PSNR) of the generator, alongside the average loss and accuracy of the discriminator. Our analysis reveals that the proposed model generates high-quality, realistic inpainted normal maps, demonstrating the potential for application to performance capture. The results of this investigation provide a baseline on which future researchers can build with more advanced networks and comparison with inpainting of the source images used to generate the normal maps. Full article

Diffusion models specialized in image-to-image translation tasks, like inpainting and colorization, have outperformed the state of the art, yet their computational requirements are exceptionally demanding. This study analyzes different strategies to train image-to-image diffusion models in a low-resource setting. The studied strategies include incremental learning and task/domain transfer learning. First, a base model for human face inpainting is trained from scratch with an incremental learning strategy. The resulting model achieves an FID score almost equivalent to that of its batch learning equivalent while significantly reducing the training time. Second, the base model is fine-tuned to perform a different task, image colorization, and, in a different domain, landscape images. The resulting colorization models showcase exceptional performances with a minimal number of training epochs. We examine the impact of different configurations and provide insights into the ability of image-to-image diffusion models for transfer learning across tasks and domains. Full article

Palmprint recognition (PPR) has recently garnered attention due to its robustness and accuracy. Many PPR methods rely on preprocessing the region of interest (ROI). However, the emergence of ROI attacks capable of generating synthetic ROI images poses a significant threat to PPR systems. Despite this, ROI attacks are less practical since PPR systems typically take hand images as input rather than just the ROI. Therefore, there is a pressing need for a method that specifically targets the system by composing hand images. The intuitive approach involves embedding an ROI into a hand image, a comparatively simpler process requiring less data than generating entirely synthetic images. However, embedding faces challenges, as the composited hand image must maintain a consistent color and texture. To overcome these challenges, we propose a training-free, end-to-end hand image composition method incorporating ROI harmonization and palm blending. The ROI harmonization process iteratively adjusts the ROI to seamlessly integrate with the hand using a modified style transfer method. Simultaneously, palm blending employs a pretrained inpainting model to composite a hand image with a continuous transition. Our results demonstrate that the proposed method achieves a high attack performance on the IITD and Tongji datasets, with the composited hand images exhibiting realistic visual quality. Full article

Occlusion in facial photos poses a significant challenge for machine detection and recognition. Consequently, occluded face recognition for camera-captured images has emerged as a prominent and widely discussed topic in computer vision. The present standard face recognition methods have achieved remarkable performance in unoccluded face recognition but performed poorly when directly applied to occluded face datasets. The main reason lies in the absence of identity cues caused by occlusions. Therefore, a direct idea of recovering the occluded areas through an inpainting model has been proposed. However, existing inpainting models based on an encoder-decoder structure are limited in preserving inherent identity information. To solve the problem, we propose ID-Inpainter, an identity-guided face inpainting model, which preserves the identity information to the greatest extent through a more accurate identity sampling strategy and a GAN-like fusing network. We conduct recognition experiments on the occluded face photographs from the LFW, CFP-FP, and AgeDB-30 datasets, and the results indicate that our method achieves state-of-the-art performance in identity-preserving inpainting, and dramatically improves the accuracy of normal recognizers in occluded face recognition. Full article

Deep-learning-based image inpainting methods have made remarkable advancements, particularly in object removal tasks. The removal of face masks has gained significant attention, especially in the wake of the COVID-19 pandemic, and while numerous methods have successfully addressed the removal of small objects, removing large and complex masks from faces remains demanding. This paper presents a novel two-stage network for unmasking faces considering the intricate facial features typically concealed by masks, such as noses, mouths, and chins. Additionally, the scarcity of paired datasets comprising masked and unmasked face images poses an additional challenge. In the first stage of our proposed model, we employ an autoencoder-based network for binary segmentation of the face mask. Subsequently, in the second stage, we introduce a generative adversarial network (GAN)-based network enhanced with attention and Masked–Unmasked Region Fusion (MURF) mechanisms to focus on the masked region. Our network generates realistic and accurate unmasked faces that resemble the original faces. We train our model on paired unmasked and masked face images sourced from CelebA, a large public dataset, and evaluate its performance on multi-scale masked faces. The experimental results illustrate that the proposed method surpasses the current state-of-the-art techniques in both qualitative and quantitative metrics. It achieves a Peak Signal-to-Noise Ratio (PSNR) improvement of 4.18 dB over the second-best method, with the PSNR reaching 30.96. Additionally, it exhibits a 1% increase in the Structural Similarity Index Measure (SSIM), achieving a value of 0.95. Full article

In the image restoration task, the generative adversarial network (GAN) demonstrates excellent performance. However, there remain significant challenges concerning the task of generative face region inpainting. Traditional model approaches are ineffective in maintaining global consistency among facial components and recovering fine facial details. To address this challenge, this study proposes a facial restoration generation network combined a transformer module and GAN to accurately detect the missing feature parts of the face and perform effective and fine-grained restoration generation. We validate the proposed model using different image quality evaluation methods and several open-source face datasets and experimentally demonstrate that our model outperforms other current state-of-the-art network models in terms of generated image quality and the coherent naturalness of facial features in face image restoration generation tasks. Full article

Highlight removal is a critical and challenging problem. In view of the complex highlight phenomenon on the surface of smooth liquor bottles in natural scenes, the traditional highlight removal algorithms cannot semantically disambiguate between all-white or near-white materials and highlights, and the recent highlight removal algorithms based on deep learning lack flexibility in network architecture, have network training difficulties and have insufficient object applicability. As a result, they cannot accurately locate and remove highlights in the face of some small sample highlight datasets with strong pertinence, which reduces the performance of some tasks. Therefore, this paper proposes a fast highlight removal method combining U²-Net and LaMa. The method consists of two stages. In the first stage, the U²-Net network is used to detect the specular reflection component in the liquor bottle input image and generate the mask map for the highlight area in batches. In the second stage, the liquor bottle input image and the mask map generated by the U²-Net are input to the LaMa network, and the surface highlights of the smooth liquor bottle are removed by relying on the powerful image inpainting performance of LaMa. Experiments on our self-made liquor bottle surface highlight dataset showed that this method outperformed other advanced methods in highlight detection and removal. Full article

Face images are often used in social and entertainment activities to interact with information. However, during the transmission of digital images, there are factors that may destroy or obscure the key elements of the image, which may hinder the understanding of the image’s content. Therefore, the study of image completion of human faces has become an important research branch in the field of computer image processing. Compared with traditional image inpainting methods, deep-learning-based inpainting methods have significantly improved the results on face images, but in the case of complex semantic information and large missing areas, the completion results are still blurred, and the color of the boundary is inconsistent and does not match human visual perception. To solve this problem, this paper proposes a face completion method based on GAN priori to guide the network to complete face images by directly using the rich and diverse a priori information in the pre-trained GAN. The network model is a coarse-to-fine structure, where the damaged face images and the corresponding masks are first input to the coarse network to obtain the coarse results, and then the coarse results are input to the fine network with multi-resolution skip connections. The fine network uses the a priori information from the pre-trained GAN to guide the network to generate the face images, and finally uses the SN-PatchGAN discriminator to evaluate the completion results. The experiment is performed on the CelebA-HQ dataset. Compared with the latest three completion methods, the qualitative and quantitative experimental analysis shows that our method has obvious improvement in texture and fidelity. Full article

In face image recognition and other related applications, incomplete facial imagery due to obscuring factors during acquisition represents an issue that requires solving. Aimed at tackling this issue, the research surrounding face image completion has become an important topic in the field of image processing. Face image completion methods require the capability of capturing the semantics of facial expression. A deep learning network has been widely shown to bear this ability. However, for high-resolution face image completion, the network training of high-resolution image inpainting is difficult to converge, thus rendering high-resolution face image completion a difficult problem. Based on the study of the deep learning model of high-resolution face image generation, this paper proposes a high-resolution face inpainting method. First, our method extracts the latent vector of the face image to be repaired through ResNet, then inputs the latent vector to the pre-trained StyleGAN model to generate the face image. Next, it calculates the loss between the known part of the face image to be repaired and the corresponding part of the generated face imagery. Afterward, the latent vector is cut to generate a new face image iteratively until the number of iterations is reached. Finally, the Poisson fusion method is employed to process the last generated face image and the face image to be repaired in order to eliminate the difference in boundary color information of the repaired image. Through the comparison and analysis between two classical face completion methods in recent years on the CelebA-HQ data set, we discovered our method can achieve better completion results of $256*256$ resolution face image completion. For $1024*1024$ resolution face image restoration, we have also conducted a large number of experiments, which prove the effectiveness of our method. Our method can obtain a variety of repair results by editing the latent vector. In addition, our method can be successfully applied to face image editing, face image watermark clearing and other applications without the network training process of different masks in these applications. Full article

The last few years have witnessed the great success of generative adversarial networks (GANs) in synthesizing high-quality photorealistic face images. Many recent 3D facial texture reconstruction works often pursue higher resolutions and ignore occlusion. We study the problem of detailed 3D facial reconstruction under occluded scenes. This is a challenging problem; currently, the collection of such a large scale high resolution 3D face dataset is still very costly. In this work, we propose a deep learning based approach for detailed 3D face reconstruction that does not require large-scale 3D datasets. Motivated by generative face image inpainting and weakly-supervised 3D deep reconstruction, we propose a complete 3D face model generation method guided by the contour. In our work, the 3D reconstruction framework based on weak supervision can generate convincing 3D models. We further test our method on the MICC, Florence and LFW datasets, showing its strong generalization capacity and superior performance. Full article

This paper considers the use of a machine learning system for the reconstruction and recognition of distorted or damaged patterns, in particular, images of faces partially covered with masks. The most up-to-date image reconstruction structures are based on constrained optimization algorithms and suitable regularizers. In contrast with the above-mentioned image processing methods, the machine learning system presented in this paper employs the superposition of system vectors setting up asymptotic centers of attraction. The structure of the system is implemented using Hopfield-type neural network-based biorthogonal transformations. The reconstruction property gives rise to a superposition processor and reversible computations. Moreover, this paper’s distorted image reconstruction sets up associative memories where images stored in memory are retrieved by distorted/inpainted key images. Full article

In a computer-aided system for skin cancer diagnosis, hair removal is one of the main challenges to face before applying a process of automatic skin lesion segmentation and classification. In this paper, we propose a straightforward method to detect and remove hair from dermoscopic images. Preliminarily, the regions to consider as candidate hair regions and the border/corner components located on the image frame are automatically detected. Then, the hair regions are determined using information regarding the saliency, shape and image colors. Finally, the detected hair regions are restored by a simple inpainting method. The method is evaluated on a publicly available dataset, comprising 340 images in total, extracted from two commonly used public databases, and on an available specific dataset including 13 images already used by other authors for evaluation and comparison purposes. We propose also a method for qualitative and quantitative evaluation of a hair removal method. The results of the evaluation are promising as the detection of the hair regions is accurate, and the performance results are satisfactory in comparison to other existing hair removal methods. Full article