UltraTex: Unleashing 2K Multi-View Diffusion for 3D Texturing

Highlights and Results: More Detail, Less Wasted Compute

Original paper: ​UltraTex: Unleashing 2K Multi-View Diffusion for 3D Texturing

UltraTex generates detailed textures for an existing 3D mesh using a reference image. Presented as a SIGGRAPH Asia 2026 project, it tackles a common bottleneck: multi-view texturing at 512 or 768 resolution loses fine patterns and surface detail. UltraTex raises the generated views to 2048 × 2048 by concentrating computation on the object foreground. Its central contribution is making high-resolution multi-view diffusion practical through coordinated changes to token processing, attention, and decoding. Official project.

On 100 held-out TexVerse objects, UltraTex reports ​unshaded LPIPS of 0.082 versus LumiTex’s 0.138​, where lower indicates closer perceptual agreement. On a single H200, the authors report 20.6–91.1× training and 22.3–74.6× inference speedups over the dense baseline for 10–30% foreground occupancy with 20% attention-block retention. These are configuration-dependent results, not a fixed acceleration for every asset or complete production workflow. Paper, experiments.

How It Works: From Reference Image to Textured Surface

The pipeline first renders six normal maps from the input mesh: four side views, a top view, and a bottom view. A VAE encodes geometry guidance and the reference image into latents. A pretrained FLUX diffusion transformer then generates six corresponding texture views through flow matching; the paper’s full pipeline reprojects them onto the mesh. At 2K, six target views, six geometry views, and one reference contribute ​212,992 image tokens​, explaining why dense processing becomes expensive. Pipeline; token-cost explanation.

Background Token Dropping removes image tokens outside the object before the transformer blocks. Masks identify retained positions, while the corresponding original rotary positional embeddings (RoPE) remain attached. The sequence becomes shorter without losing its spatial coordinates. This saves work throughout the backbone, including feed-forward layers. In the released sampler, Euler updates modify only selected foreground positions; background noise is left untouched until decoding preparation. Token selection; sampling implementation.

Block-Sparse Attention reduces the cost of interactions among the remaining tokens. The implementation averages query and key features within blocks, estimates their relevance, and selects the highest-scoring key blocks for each query block. Attention then uses the original token features inside those selected blocks. The FLUX.1 configuration retains 20% of key blocks: it avoids many interactions without reducing the image resolution. Block selection; attention configuration.

Foreground-Aware ​VAE​​ Decoding addresses the consequence of skipping background denoising. Combining a clean foreground with a noisy background can corrupt reconstruction. UltraTex replaces the background with a latent encoded from a solid black image, then uses a decoder adapted to this composite. The encoder stays frozen; decoder training minimizes reconstruction error only on foreground pixels. After decoding, the released inference scripts save multi-view images. Integrating those outputs into a mesh asset still requires the reprojection or baking step. Decoder training; inference outputs.

The data foundation is ​G-buffer TexVerse​. Its public release covers 351,847 BSDF assets, while UltraTex uses a filtered 268,365-asset training subset. Renderings provide normals, coordinates, albedo, and available material channels at up to 4096 × 4096. This data resolution should be distinguished from UltraTex’s demonstrated 2K generation. Dataset description.

Applications, Availability, and Future Directions

UltraTex targets the appearance stage of asset creation: supplying detailed surface textures for existing character, prop, and object geometry. The default output is ​albedo​, representing surface color without baked-in lighting. The current repository also supplies a FLUX.2 metallic–roughness branch with separate LoRA weights. This available extension should be distinguished from the main paper’s albedo benchmark, which renders UltraTex with fixed roughness and metallic values. Available material outputs; evaluation settings.

As checked on November 6, 2026, training and inference code and weight download instructions are public. The repository supports FLUX.1-dev and FLUX.2-Klein variants, each requiring matching backbone, LoRA, and decoder files. Its tested environment uses Python 3.12, CUDA 12.4, and PyTorch 2.5.1. This article reviews the paper and implementation; it does not claim an independent quality or speed reproduction. Official repository.

The authors identify difficulty with highly repetitive textures and detail loss from the pretrained VAE’s compressed representation. Their stated direction for scaling to 4K or 8K is to redesign DiT–VAE foundation models that natively support those resolutions, since the existing FLUX backbone is better matched to roughly 1K–2K inputs. These remain limitations and research directions, not released high-resolution capabilities. Paper, limitations.