Toward Richer Material Generation via Procedural Data Enhancement
1Cornell University, 2NVIDIA
SIGGRAPH 2026 Conference Proceedings
Creating complex materials at scale is challenging. We introduce a material enhancement procedure that uplifts PBR materials into rich appearances compactly represented as neural materials. By adding surface effects, we can make an object appear waxy, peach-fuzzed, or sugar-coated (top row). Similarly, we can introduce subtle spatial specular breakup, as well as large-scale changes like clearcoat or dust layers (bottom row). Insets show the textures used by each model. Normal maps are applied unchanged to the neural material.
Abstract
Generative models for material creation are fundamentally limited by the quality and expressivity of available training data. Simple physically based rendering (PBR) materials, which combine a diffuse term with a single-lobe specular component, are commonly used for training but are insufficient to capture many important visual effects present in real materials.
We present a method that enhances simple PBR materials to more expressive ones, by augmenting the single GGX specular lobe into a layered model that captures a broader range of non-diffuse effects. Starting from a simple material, we procedurally construct a corresponding multi-lobe non-diffuse component guided by physical priors, enabling effects such as dust, clearcoat, and layered scattering. To provide a compact representation for downstream applications, we encode this component as a neural material with a shared 6D latent space, where each material instance is represented by two latent textures and decoded by a pretrained universal MLP. We further regularize the latent space to support material generation.
The resulting neural material dataset enables training generative models for richer material creation. To demonstrate this application, we finetune a video diffusion model to produce neural latent textures that encode our multi-lobe material, and present generative results as proof of feasibility. Our procedural data enhancement approach is an important step toward improving expressivity in material generation.
Fast Forward Video
Examples of Enhanced Materials
In each image group below, the first row shows the original PBR materials, the second row shows enhanced versions reconstructed as neural materials, and the third row shows differences between the original and enhanced materials.
Type 0: Haze
Type 1: Dust
Type 2: Clearcoat
Type 3: Dust + Clearcoat
Type 4: Fuzz + Inner Scatter + Subcutaneous Scatter
Type 5: Fuzz + Clearcoat + Inner Scatter + Subcutaneous Scatter
Generative Results
We first present our generated neural materials for all the 3D models in our test set. Intended enhancement types of these assets' reference neural materials are haze (a), dust (b, c), clearcoat (d, e, f, g), dust + clearcoat (h, i), fuzz + scatter (j, k, l, m), all lobes (n, o, p).
We compare our generation with TRELLIS.2 and render the test set materials in four variants: the reference, hand-crafted neural materials, PBR materials generated from TRELLIS.2, base color from TRELLIS.2 combined with neural, non-diffuse materials generated from our method, and known base color combined with our generated neural materials.
Reference
TRELLIS.2
Ourtrellis
Ourref
Test Set Objects (a)--(d)
Reference
TRELLIS.2
Ourtrellis
Ourref
Test Set Objects (e)--(h)
Reference
TRELLIS.2
Ourtrellis
Ourref
Test Set Objects (i)--(l)
Reference
TRELLIS.2
Ourtrellis
Ourref
