Enhancing Mesh Deformation Realism for Synthesizing Wrinkles

A series of solutions were proposed by [Graham et al. 2013; Nagano et al. 2015], addressing wrinkle simulations and deformation visual cues in terms of shading and rendering. Some solutions require accessing specialized hardware, such as light stages [Hawkins and Debevec 2000] or complex camera and light setups to capture geometry and textures. These hardware setups, while effective, may be out of reach for the user due to technical and finacial demands. Art direction is constrained to the acquired data and large changes require new data acquisition. The captured data might be too demanding in terms of performance, making them infeasible for some applications, such as video or mobile games, VR and other real-time applications. Other solutions that rely on captured data use it as a base for static texture synthesis. Once generated, they are tied to the geometry. Dynamic texture synthesis would need to be regenerated following the underlying deformations of the mesh to respond in a fundamental way for deformations.

Several approaches [Li and Kry 2014; Turchet et al. 2015] have been explored to synthesize wrinkles and visual cues beneath the geometry with certain limitations. One common drawback is the occurrence of self-intersections which can reduce the quality of the results. These approaches heavily rely on the tessellation of the geometry to provide fine details. Achieving finer details necessitates denser geometry tessellation, leading to increased storage requirements and computational costs during rendering [Nagano et al. 2015]. Alternatively, certain techniques rely on manual intervention from artists to paint wrinkle and tension maps or deformation areas [Turchet et al. 2015]. These maps act as textures that control the occurrence and distribution of deformation on a model. In turn, they modulate a heightmap that describe additional spatial variations on the surface. Tension maps are popular as a means of characterizing mesh deformation but they are lacking information such as principal deformation directions. Additionally, some physics-based simulations such as [Li and Kry 2014] generate wrinkle maps that refine the details. However, the art-directability of simulations may be limited in certain cases. The approach by [Zanni et al. 2012] uses anisotropic Gabor noise to model long term growth patterns in implicit surfaces but doesn’t address the geometry response for ongoing deformation at finer scales.

We see an opportunity to enhance the dynamic deformation patterns for wrinkles and other effects, using procedural methods. By avoiding the need for texture regeneration or reliance on captured data, a procedural approach provides greater creative potential and control over different wrinkle types, densities and responses to deformation. Additionally, the inclusion of static procedural patterns that simulate microstructure detail contributes to overall realism of the results. Our goal is to create a procedural heightmap pattern system suitable for modeling the appearance deformable bodies including human skin. The system is capable of taking into account the mesh deformations and flexible enough to be art-directable without any captured data or extensive user intervention, while maintaining a physically plausible appearance at multiple scales.

Our solution operates on a shading context by analyzing the mesh deformations and uses it to generate procedural heightmap patterns to simulate wrinkles and other deformation visual cues at the mesostructure level. A second static heightmap is also synthesized and compared to existing skin data from the publicly available databases using structural perceptual texture metrics.

We are given a mesh \(\mathcal {M}\) embedded in \(\mathbb {R}^3\). The mesh is given a fixed connection with parameter space \(\Omega \subset \mathbb {R}^2\) via a bijective C¹ mapping Φ, assigning to each point of the surface of the mesh a unique pair of (u, v) coordinates, hence without overlapping regions nor inversions. The mesh undergoes a potentially unknown transformation from a reference configuration \(\mathcal {M}_{ref}\) into a deformed configuration \(\mathcal {M}_{cur}\). Although we operate on the deformed configuration, we have simultaneous access to the reference configuration at any time.

Let \(\mathbf {p} \in \mathcal {M}_{\text{ref}}\) and \(\mathbf {q} \in \mathcal {M}_{\text{cur}}\) denote points in the reference and deformed configurations respectively. To quantify the deformation, we compute the Jacobians of the Φ mapping at p and q, J_p, J_q, augmenting them with the normals n_p and n_q to form the augmented differential matrices Dp and Dq. The rate of change of the deformed mesh in regard to the reference mesh can be computed by finding the matrix F taking Dq into Dp, akin to the deformation gradient F = ∂x/∂X in continuum mechanics, with x being the coordinates in the deformed configuration and X in the reference configuration.

We eliminate shear and rotation by symmetrizing the matrix F via the analogue of the left Cauchy-Green strain tensor B = F(F)^T. For large deformations, where significant non-linear effects might be present, the Euler-Almansi strain tensor \(\mathbf {e} = \frac{1}{2}\left(\mathbf {I} - \mathbf {B}^{-1}\right)\) can be used to capture the strains in the deformed configuration more accurately. The eigendecomposition of B or e provides the principal directions and magnitudes of stretching and compression, under the viewpoint of the deformed configuration. Using this approach directly results in a formulation in terms of 3 × 3 matrices whose eigendecomposition becomes more tractable only due to B or e being real, symmetric, positive semi-definite, with real eigenvalues.

To further analyze the deformation, we use the metric tensors of the map Φ at the deformed and reference configurations respectively, via the first fundamental form coefficients, shown in Eq. 1.G_q is computed similarly for the metric tensor of the reference configuration. The metric tensors provide intrinsic measures of distortion of the fixed map and are able to capture in-plane deformations. Out-of-plane deformations are captured indirectly, since they influence the local neighbourhood of the points p and q, changing their lengths and angles with tensile or compressive strains. Since the surface is locally linear and we’re interested primarily in stretch and compression, we can approximate the deformation gradient taking the intrinsic distortions that are contained in the metric tensor G_q of the Φ map, to the deformed configuration g_p. This is not necessarily symmetric, but we can remove shear and rotational components in an analogue of the left Cauchy-Green strain tensor, building the Euler-Almansi strain tensor if appropriate, as shown in Eq. 2.The eigendecomposition of B or e, simpler in \(\mathbb {R}^{2 \times 2}\), provides the principal directions and amounts of stretching and compression. The eigenvalues correspond to the squared singular values σ of F. As F is a measure of the change of distortions in terms of squared lengths and angles that are contained in the metric tensors of Φ, the engineering strain ϵ and a color mapping are given in Eq. 3 in terms of fourth roots, and similarly for the engineering strains given from the eigenvalues of e, shown in Eq. 4.

Since both B and e are symmetric, the eigenvalues are real, and the eigenvectors are orthogonal, though these are in parameter space relative to the deformed configuration \(\mathcal {M}_{cur}\). These can be used directly, or they can be transformed into the ambient space as linear combinations of the basis vectors that are shown in Eq. 5. Using the basis at the reference configuration will ensure these are anchored to the reference mesh when using solid noise.

We used the principal deformation directions with solid anisotropic Gabor noise [Lagae and Drettakis 2011; Lagae et al. 2009] to simulate the physically plausible generation of wrinkles during deformation. Alternatively, they can be used directly with 2D noise. Artistic control is given through an amplitude factor ρ, and frequency factor ω, analogous to material properties. The amplitude factor corresponds the reciprocal of Young’s modulus, representing material stiffness, while the frequency factor corresponds to Poisson’s ratio, reflecting lateral deformations and finer wrinkle characteristics. When using solid noise, the input manifold uses p in order to keep the texture pattern stable as the mesh undergoes motion and deformation. The resulting scalar quantity can be used with displacement or bump mapping to change the surface appearance at the mesostructure level. Because we’re operating on a shading context, we’re able to capture information that isn’t present at the geometric level, such as the contribution of other displacement maps.

A procedural pattern was developed to mimic the microstructure of skin using the ontogenetic modeling approach suggested by [Ebert et al. 2003]. This approach involves decomposing real-world examples into structural components which are then replicated procedurally. Using the skin samples captured by [Nagano et al. 2015] as exemplars, a sample of which is shown in the right image of the second row in Fig. 3, we segmented the images at different continuous tone levels via adaptive image thresholding, to more easily identify the underlying structures. Subsequently, we performed Delaunay triangulation on the centers of mass of the most prominent visual features to acquire information on the density and distribution of the cells for the Worley noise. These were replicated with procedural noise functions and multi-fractals as shown in Fig. 3. The Worley noise function was given domain distortion by a signed, fixed gain, vector form of fBm noise, shown in the left image of the first row in Fig 3. This, together with a choice of metrics, was done to break the cell boundaries’ regularity. The differences between the distances to the nearest four Voronoi cell centers were computed and thresholded using an antialiased step function s(σ_t, x), as shown in Eq. 6. The parameter σ_t tresholds x, and antialiasing is computed automatically using the pixel footprint. The result is displayed in the center image of the first row in Fig. 3.

The orientation of the anisotropic Gabor noise can be determined by inspection or using a filter bank of Gabor kernels with varying orientations and frequencies on a sample image. The resulting pattern is adjusted to match the visual structure of the exemplar shown in the right image of the first row of Fig. 3. A smoothstep function S₁ modifies the contrast via a τ_s parameter and is combined with the three differences as shown in Eq. 7. The results of this expression and variations in σ_t and τ_s are shown in the left and center image of the second row of Fig. 3.

Finally, Hery’s bump-to-roughness method [Hery et al. 2014] was employed. The height data was processed to generate the specular roughness, principal directions and amounts of anisotropy. The simulated dynamic wrinkles form a height base upon which the microstructure height and auxiliary channels are layered, resulting in combined height, normal, anisotropy directions and amounts, accounting for a combination of static and dynamic data.

Our work was implemented as two OSL shaders developed for Pixar RenderMan 24.5, integrated with Maya. One shader focused on deformation analysis and wrinkle synthesis, while the other handled the synthesis of microstructure patterns. Operating within a shading context provides to generate potentially high spatial frequency wrinkles at the mesostructure level without the need for dense mesh pre-tessellations. Additionally, we modeled various microstructure appearances, such as skin, and utilized them with dynamic mesostructure details. Localized stochastic micro-movements can also be depicted using high spatial frequency, time-varying fluctuations of the procedural wrinkles.

We compared the synthesized patterns to exemplars using various image quality assessment (IQA) metrics, as shown in Fig. 5. The patterns were baked into textures. Both local and global image structures were analyzed. A control group of images helped to determine the most suitable IQA metrics for our work. We accurately parameterized the anisotropic specular response of the material by preprocessing the optimal candidate into a bump-to-roughness texture and layering it with dynamic wrinkles. Due to energy conservation, this also impacted the response of the BSSRDF when modeling the appearance of, for example, skin, as shown in Fig. 1. More results can also be seen in the supplemental material.

We introduce a novel method to enhance animations by simulating the visual characteristics of deforming models at the mesostructure level. Our method involves a compact deformation analysis, facilitates the generation of high-frequency dynamic displacement and bump maps integrated with procedural anisotropic Gabor noise [Lagae et al. 2009].

Traditional techniques often incur significant computational costs when handling such high-frequency content [Nagano et al. 2015]. Our method, however, provides a flexible parameterization that accurately captures the physical plausibility of deformations while offering artistic control and versatility. It effectively represents various surfaces with diverse material properties, as shown in Fig. 4. Furthermore, we enhance mesostructure detail by incorporating static procedural patterns that emulate cell growth, primarily focusing on skin patterns but not limited to them.

Authors would like to thank Prof. Francisco Calheiros for the technical discussions. This work is supported by FCT, I.P. by project reference UIDB/50008/2020 with doi.org/10.54499/UIDB/50008/2020 and FCT Position (CEECIND/03766/2018/CP1579/CT0002).