Fact-checked by Grok 2 weeks ago

Texture synthesis

Texture synthesis is a computational technique in computer graphics used to generate large, realistic images or patterns from a small input sample texture, ensuring the output appears statistically similar to the input by replicating its structural and visual properties without visible repetition or artifacts.^[1] The process typically involves analyzing the sample's local neighborhoods—such as pixel patterns or patches—and iteratively synthesizing new content that matches these characteristics, often producing seamless, tileable results suitable for arbitrary sizes.^[2] This method draws from stochastic modeling to mimic natural texture formation, distinguishing it from procedural generation by relying on example-based guidance rather than predefined rules.^[1] Developed primarily in the field of computer graphics since the 1980s, texture synthesis addresses the challenge of creating diverse, realistic surface details for virtual environments, evolving from early statistical and syntactic models to more efficient example-based algorithms in the 2000s, and further advanced by deep learning techniques since the 2010s.^[3]^[4] Pioneering work in the mid-1980s surveyed approaches like growth models and stochastic simulations, highlighting their potential for enhancing visual realism despite initial computational limitations.^[3] The technique gained prominence in the late 1990s with Markov random field-based methods, which simplified synthesis by growing textures pixel-by-pixel from a seed, followed by breakthroughs in the early 2000s that enabled faster, higher-quality outputs through optimized neighborhood matching.^[2] Key approaches to texture synthesis include pixel-based methods, which build images sequentially by selecting pixels with compatible neighborhoods; patch-based techniques, which copy and blend larger texture fragments for efficiency; parametric models, which extract statistical features like color histograms or filter responses to guide generation; and deep learning-based methods using convolutional neural networks and generative adversarial networks that have gained prominence since the 2010s.^[2]^[4] These methods, exemplified by algorithms such as Efros-Leung (1999) for pixel growth and Wei-Levoy (2000) for vector quantization, balance quality and speed while handling both stochastic and near-regular textures.^[1] More advanced variants incorporate optimization frameworks to minimize inconsistencies or adapt to surfaces like 3D models.^[2] In applications, texture synthesis is widely used for rendering lifelike materials in animations and games—such as terrain, fabrics, or skin—and extends to image editing tasks like inpainting holes, super-resolution enhancement, and generating dynamic video textures.^[1] It also supports geometry-aware synthesis for texturing complex 3D objects and has influenced tools in content creation software for efficient, non-repetitive visuals.^[2]

Fundamentals

Definition and Overview

Texture synthesis is the process of generating large, seamless digital images that mimic the visual appearance of a given small sample texture by analyzing and replicating its statistical properties. This technique enables the creation of arbitrarily sized textures that are tileable without visible seams or artifacts, making it a fundamental tool in computer graphics for enhancing realism in rendered scenes. Early approaches, such as pyramid-based methods, focus on matching the frequency content across multiple scales to preserve the perceptual qualities of the input sample.^[5] More recent example-based methods emphasize capturing the local structure to produce outputs that appear to be generated by the same underlying process as the exemplar.^[6] At its core, texture synthesis relies on non-parametric sampling and statistical matching of local patterns, assuming textures as stationary processes where neighborhood statistics remain consistent across positions. This involves modeling textures using frameworks like Markov Random Fields (MRFs), where the goal is to ensure that each pixel in the output has a spatial neighborhood similar to at least one in the input, thereby producing visually plausible but novel variations. Such principles allow for the replication of repetitive patterns and randomness without requiring explicit parametric models of the texture's generation process.^[7]^[6] The basic workflow of texture synthesis begins with analyzing the input sample to extract relevant features, such as local neighborhoods or frequency distributions, followed by iterative output generation through sampling or optimization techniques that enforce spatial continuity. This process grows the synthesized texture outward from an initial seed, prioritizing coherence at boundaries to avoid discontinuities. The result is a larger image that maintains the input's perceptual texture while allowing for extension beyond the original sample's dimensions.^[6] Unlike general image generation, which often aims to produce semantically meaningful scenes with global structure, texture synthesis specifically targets repetitive, pattern-based content lacking inherent semantics, focusing instead on local stationarity and perceptual similarity. This distinction ensures that synthesized outputs exhibit uniformity in statistical properties across the image, suitable for applications like surface detailing rather than composing complex visual narratives.^[6]

Textures in Computer Graphics

In computer graphics, textures are digital images or patterns applied to surfaces to simulate material properties and enhance visual realism. They are broadly classified into three types: stochastic textures, which exhibit random variations such as noise or clouds; structured textures, featuring regular, repeating patterns like bricks or tiles; and hybrid textures, which combine elements of both, such as a patterned fabric with subtle random imperfections.^[5]^[8]^[9] Textures play a foundational role in rendering by being mapped onto three-dimensional surfaces, allowing complex details to be added efficiently without increasing geometric complexity. This mapping process projects the 2D texture coordinates onto 3D models, enabling realistic depictions of surfaces like wood grain or stone. To handle variations in viewing distance and resolution, mipmapping precomputes scaled versions of the texture in a pyramid structure, selecting appropriate levels during rendering to maintain quality across scales. Additionally, filtering techniques, such as bilinear or trilinear interpolation, are applied to smooth transitions between texels and prevent artifacts like aliasing or moiré patterns.^[10]^[11]^[12] Key properties of textures in graphics include periodicity, which refers to the repeating nature of patterns in structured types; isotropy, where the texture appears uniform in all directions, as in many stochastic examples; and anisotropy, where directional variations occur, such as in brushed metal or hair. Scale invariance is another important attribute, ensuring the texture maintains perceptual consistency when resized or viewed from different distances, which is crucial for effective mipmapping. These properties influence how textures interact with lighting and geometry in rendering pipelines.^[13]^[14]^[15] Mathematically, textures are represented as 2D functions f(x, y) that assign color or intensity values to points on a surface, often parameterized over UV coordinates. Autocorrelation analysis of these functions quantifies the regularity and repetition within the texture, aiding in the evaluation of pattern coherence and aiding perceptual analysis in graphics applications.^[16]^[17]

Contrast with Procedural Textures

Procedural textures are generated algorithmically using mathematical functions to simulate natural phenomena, such as Perlin noise for creating cloud-like or marble patterns, allowing for parametric control and infinite scalability without requiring input samples.^[18] These textures are rule-based and data-independent, enabling compact storage and rapid evaluation at any resolution, which makes them suitable for real-time applications in computer graphics.^[19] In contrast, texture synthesis employs non-parametric methods that learn statistical properties from an input sample image to generate larger textures, capturing complex, irregular patterns that are difficult to define through explicit rules.^[2] While procedural approaches rely on predefined functions for consistency and editability, synthesis is data-driven, prioritizing fidelity to real-world exemplars over manual parameterization.^[2] A primary advantage of texture synthesis lies in its ability to replicate intricate, non-repetitive details like the organic variations in wood grain, which procedural methods often simplify or fail to render realistically due to their reliance on generic noise functions that introduce unnatural symmetries.^[20] This makes synthesis particularly effective for photorealistic natural textures that evade straightforward procedural modeling.^[19] However, texture synthesis can suffer from visible seams at boundaries or high computational demands during generation, whereas procedural textures offer faster performance and seamless tiling for simpler, structured cases without preprocessing.^[19] For instance, a fractal-based procedural texture might efficiently generate uniform terrain patterns, but synthesizing a sample-based grass texture better preserves the stochastic irregularity and directional growth seen in real foliage.^[2]

Goals and Challenges

The primary goals of texture synthesis in computer graphics are to produce new textures from a small input exemplar that are visually indistinguishable from the original, enabling the creation of larger, seamless, and scalable images without visible repetitions or boundaries.^[2] This process aims to match the statistical properties of the input sample, such as pixel distributions and spatial relationships, while preserving both local details—like individual patterns or "textons"—and global structures, such as overall coherence and orientation.^[7] In contrast to procedural textures, which rely on predefined parametric models, example-based synthesis uses non-parametric estimation from the sample to generate infinite variations suitable for tiling or real-time rendering.^[2] Key challenges in achieving these goals include avoiding artifacts such as seams, blurring, or unnatural repetitions that arise during extension of the input sample, particularly in structured textures with distinct orientations or periodic elements.^[7] Handling variations in scale, orientation, and illumination across the input exemplar is difficult, as natural textures often exhibit non-stationary properties that require robust neighborhood matching to prevent "slipping" into incorrect visual subspaces.^[2] Additionally, ensuring computational efficiency is critical for real-time applications, as synthesis algorithms must balance quality with speed, often demanding large input samples to capture sufficient statistics without excessive processing time.^[7] Evaluation of synthesized textures typically focuses on visual and perceptual similarity to the input, assessed through qualitative inspection or quantitative metrics like neighborhood matching, where local pixel contexts are compared using measures such as sum-of-squared differences (SSD) to ensure coherence.^[2] Metrics emphasizing seamlessness, such as boundary continuity checks, help quantify the absence of visible artifacts, though subjective human assessments remain essential due to the perceptual nature of texture quality.^[7] A fundamental prerequisite for effective texture synthesis methods is the analysis of the input sample to extract relevant features, such as local histograms or Markov random field (MRF) models, which inform the generation process and ensure statistical fidelity.^[2] This initial step allows algorithms to approximate the underlying generative process of the texture, setting the foundation for scalable output without prior assumptions about its parametric form.^[7]

Methods

Tiling and Mosaic Techniques

Tiling techniques represent one of the earliest and simplest approaches to texture synthesis, involving the periodic repetition of a small input sample to generate larger textures. In basic tiling, the output texture is created by directly replicating the input sample across a grid, where the value at each position (x, y) in the output is determined by the corresponding modular position in the input:

\text{output}(x, y) = \text{input}(x \mod w, y \mod h),

with w and h denoting the width and height of the input sample, respectively.^[6] This method preserves the local structure of the sample but often results in visible seams at tile boundaries due to abrupt discontinuities. To mitigate these artifacts, edge blending techniques such as Poisson blending are applied, which solve the Poisson equation \Delta f = \text{div}(\nabla g) within the overlapping regions to smoothly interpolate gradients and ensure seamless continuity across edges.^[21] Mosaic techniques extend tiling by fragmenting the input sample into smaller patches and rearranging them randomly to reduce periodicity while approximating the original texture's appearance. A prominent example is the chaos mosaic method, which begins with an initial tiling of the input sample and then applies a chaotic transformation, such as the cat map iteration, to redistribute blocks within the tiles, followed by cross-edge filtering over a few pixel layers to minimize boundary discontinuities.^[22] This random arrangement promotes a stochastic distribution of local features, enabling the synthesis of large textures, such as a 512×512 image, in under 0.03 seconds on mid-2000s hardware.^[22] More advanced algorithms within this category, such as those based on Wang tiles, address the limitations of periodicity by creating aperiodic tilings from a small set of precomputed tiles derived from the input sample. The input is divided into diamond-shaped portions aligned with edge colors, which are then combined into a finite set of Wang tiles (e.g., 8 to 18 tiles) ensuring matching colors on adjoining edges; these tiles are stochastically placed to cover the plane without repetition, using random selection among compatible tiles at each step.^[23] This reassembling process allows for efficient, non-periodic texture generation directly on demand, with minimal storage requirements.^[23] The primary strengths of tiling and mosaic techniques lie in their low computational demands and simplicity, making them suitable for real-time applications and procedural texturing of vast areas, such as virtual 100,000×100,000 textures.^[22] However, basic tiling suffers from obvious periodic repetition, while mosaic methods like chaos mosaics can introduce randomness that disrupts structured patterns, potentially leading to blurred or mismatched features in complex inputs.^[22] Wang tile approaches mitigate some periodicity but may still exhibit quilting artifacts if the tile set is too small or if features span tile corners inadequately.^[23]

Stochastic and Structured Methods

Stochastic texture synthesis approaches model textures as realizations of Markov random fields (MRFs), where the probability of a pixel's color depends on its local neighborhood, capturing spatial dependencies in a probabilistic manner.^[24] In these methods, the synthesis process builds the output image sequentially, pixel by pixel, starting from a seed region copied from the input sample. For each new pixel, the algorithm examines its neighborhood in the growing output and searches the input sample for matching neighborhoods to select candidate colors, ensuring local consistency while introducing variability through random selection.^[7] A key formulation in stochastic synthesis is the conditional probability P(c \mid n), where c is the color of the pixel and n is its neighborhood context; this probability is estimated non-parametrically by counting occurrences of matching neighborhoods in the input sample.^[7] The seminal Efros-Leung algorithm exemplifies this by growing the texture outward from an initial patch, using an exhaustive search to find all pixels in the sample whose neighborhoods match the current output context, then selecting a candidate color randomly from those matches—effectively a k-nearest neighbors approach where k equals the number of exact matches.^[7] This non-parametric sampling avoids explicit parameter estimation, making it adaptable to diverse textures without assuming a specific distribution.^[25] Structured methods, in contrast, employ rule-based systems to enforce specific patterns, particularly for textures with repetitive or hierarchical elements, blending procedural generation with sample guidance. These approaches use formal grammars or rewriting systems to define production rules that replicate structural motifs from the input, such as tilings or branching patterns, while allowing controlled variation. For instance, stochastic tree grammars can infer hierarchical rules from the sample to synthesize textured patterns by recursively applying productions to generate tree-like structures.^[26] L-systems, extended for texture synthesis as TSL-systems (which incorporate parametric, stochastic, and context-sensitive rules), provide a procedural yet sample-informed framework for generating structured textures like fabrics or natural aggregates.^[27] In these systems, an initial axiom derived from the sample evolves through parallel rewriting rules, where probabilities guide stochastic choices to match the input's irregularity, producing outputs that preserve global organization without pixel-level sampling.^[27] Such methods are single-purpose, tailored to texture classes like quilt-like tilings, where rules enforce adjacency constraints akin to grammar productions for periodic arrangements.^[26] These techniques find applications in synthesizing natural, irregular textures such as sand or gravel, where stochastic methods excel at replicating fine-grained randomness and structured approaches handle underlying motifs like dune patterns.^[2] However, sequential pixel growth in stochastic algorithms poses challenges, rendering them computationally slow for large outputs—often requiring hours for modest resolutions due to repeated neighborhood searches—though they remain foundational for unconstrained synthesis tasks.^[7]^[28]

Pixel-Based Methods

Pixel-based methods in texture synthesis generate output images by sequentially determining the color or intensity value for each pixel, guided by matching local neighborhoods from an input sample texture. The foundational algorithm, developed by Efros and Leung, initializes the output with a single seed pixel randomly chosen from the sample and expands outward in a raster or spiral order. For each new output pixel, the method scans the input sample to find all neighborhoods that closely match the synthesized neighborhood surrounding the target pixel position, then selects one match—often the closest or randomly among the best—and copies its central pixel value to the output. This non-parametric sampling captures complex, non-stationary statistics without assuming a parametric model.^[7] The similarity between an output neighborhood n_o and a candidate sample neighborhood n_s is typically measured using the sum of squared differences over the neighborhood window N:

d(n_o, n_s) = \sum_{(i,j) \in N} \left( I_o(i,j) - I_s(i,j) \right)^2

The best match minimizes this distance, ensuring local consistency. Neighborhood sizes are usually small (e.g., 5x5 or 9x9 pixels) to balance detail capture and computational cost, though larger windows improve quality at the expense of speed.^[7] While effective for many textures, the greedy nature of this sequential assignment can propagate errors, leading to seams or artifacts in regions with sparse matches. To mitigate this, subsequent algorithms employ global optimization by formulating the synthesis as a Markov Random Field (MRF), where each output pixel is a node with labels drawn from the sample, and potentials enforce neighborhood similarity. Inference is approximated using loopy belief propagation or graph cuts to jointly optimize pixel assignments, reducing local inconsistencies. For instance, strong MRF models ensure Markovian properties across subneighborhoods for more robust texture modeling. These optimizations refine the entire output iteratively rather than pixel-by-pixel, improving coherence.^[2]^[29] Variants accelerate synthesis while preserving quality, such as jump maps, which preprocess the sample to create a data structure pointing each pixel to a similar one in the sample, allowing rapid propagation of values during output generation. Introduced by Zelinka and Garland, jump maps enable near-real-time performance by avoiding exhaustive searches, with preprocessing costs offset by repeated use. Other enhancements include tree-structured vector quantization for fast approximate matching and k-coherence methods that build candidate sets during analysis for constant-time lookups.^[30]^[2] These methods excel at reproducing intricate, aperiodic patterns and non-stationary textures that defy simple procedural rules, as they directly sample from the input's empirical distribution. However, they are computationally intensive without accelerations—original implementations can take minutes per image due to quadratic search complexity—and remain prone to error accumulation in low-redundancy samples, potentially yielding blurry or erroneous regions.^[2]

Patch-Based Methods

Patch-based methods in texture synthesis involve dividing an input sample texture into overlapping patches, which are then selected, placed, and blended to construct the output image, preserving both local structure and global consistency more efficiently than pixel-by-pixel approaches.^[31] This technique addresses limitations in earlier pixel-based synthesis, such as slow computation and potential boundary artifacts, by growing the output in larger chunks while ensuring seamless overlaps through specialized cutting algorithms.^[32] A foundational algorithm is Image Quilting, introduced by Efros and Freeman, which synthesizes textures by randomly selecting square patches from the input sample and stitching them in a raster-scan order, with overlaps typically set to about one-sixth of the patch size to maintain continuity.^[33] To achieve overlap consistency, the method employs minimum error boundary cuts, where the best seam is found using dynamic programming to minimize visual discontinuities along the overlap region; the boundary cost C(b) for a potential cut b is computed as the sum of squared differences between overlapping pixels:

C(b) = \sum_{(i,j) \in b} \| p_{ov1}(i,j) - p_{ov2}(i,j) \|^2,

with the optimal path selected to reduce seam visibility.^[31] This greedy placement can be extended for texture transfer by incorporating feature maps, such as luminance, to guide patch selection and apply the source texture to a target image while preserving geometric details.^[33] Building on this, Kwatra et al. advanced patch matching with graph cuts for optimal stitching, treating seam finding as a minimum-cost cut problem in a graph constructed over the overlap region, where edge weights represent pixel dissimilarities (e.g., color or gradient differences).^[34] This allows for more flexible, non-straight seams compared to dynamic programming, incorporating constraints from adjacent boundaries to avoid repetitive artifacts; patch candidates are chosen via approximate nearest-neighbor matching from the sample, enabling both greedy and globally optimal synthesis.^[32] The approach supports extensions to video synthesis by enforcing temporal consistency across frames.^[34] These methods offer significant benefits over pixel-based techniques, achieving synthesis speeds of seconds to minutes on standard hardware—up to orders of magnitude faster—while better preserving structural elements and minimizing visible seams through overlap optimization.^[31] They have been widely applied to texture enlargement, where patches are scaled and tiled to expand images without repetition, and to inpainting, filling missing regions by propagating surrounding patches with low-cost boundaries.^[32]

Deep Learning Methods

Deep learning methods have revolutionized texture synthesis by leveraging neural networks to capture and generate complex statistical patterns from exemplar images, enabling scalable and high-fidelity outputs that surpass traditional algorithmic approaches. Early neural techniques focused on convolutional neural networks (CNNs) for feature extraction, where pre-trained networks like VGG are used to define a perceptual loss that optimizes synthesized textures to match the Gram matrix statistics of input features, ensuring realistic appearance without explicit patch matching. This paradigm, introduced in seminal work on texture synthesis using CNNs, treats textures as stationary processes in feature space, allowing iterative optimization to produce seamless extensions of input samples. Adaptations of neural style transfer further extended this to texture transfer, preserving content structure while imposing stylistic textures via multi-scale feature correlations. Generative adversarial networks (GANs) advanced texture synthesis by employing adversarial training to align the distribution of generated samples with real textures, addressing limitations in optimization-based methods like mode collapse and lack of diversity. A notable example is TiPGAN, which incorporates intrinsic priors such as relative total variation (RTV) to enforce tileability during synthesis, particularly for cloth digitization applications, achieving superior seamlessness and quality metrics compared to prior GANs. This approach uses a generator-discriminator pair where the discriminator penalizes boundary discontinuities, enabling high-resolution, periodic textures suitable for 3D modeling and animation. Diffusion models represent a recent paradigm shift, offering stable training and high-quality generation through iterative denoising processes conditioned on input exemplars or geometries. In 3D texture synthesis, geometry-aware diffusion techniques, such as those in Hunyuan3D 2.0, enable zero-shot generation of consistent, high-resolution textures on meshes by integrating surface normals and UV mappings into the diffusion pipeline, producing photorealistic results with improved fidelity over GAN-based alternatives. These models excel in capturing fine-grained details and global consistency, often evaluated using Fréchet Inception Distance (FID) scores that highlight their superior realism, with reported FID values below 5 on standard texture benchmarks. Text-guided synthesis has emerged as a transformative capability, allowing control via natural language prompts to generate targeted textures on 3D assets. TexDreamer employs a multi-view diffusion framework to produce photorealistic, robust textures from text descriptions, optimizing for zero-shot applicability on human meshes and achieving consistent multi-view coherence without paired training data. Similarly, ProcTex facilitates part-consistent text-to-texture synthesis for procedural models, ensuring texture continuity across geometric variations in real-time interactive workflows, which is particularly valuable for parametric design in computer graphics. These methods leverage pre-trained large language models to condition diffusion samplers, yielding diverse outputs like "rusty metal" or "leather fabric" with semantic alignment. Additional innovations include orientation-aware CNNs that enhance synthesis for anisotropic textures by augmenting training data with rotated samples, mitigating directional biases and improving generalization to oriented patterns like fur or waves. In evaluation contexts, multi-modal fusion networks have been developed to detect AI-generated textures by combining texture statistics with frequency-domain features, revealing synthesis artifacts and aiding in authenticity verification with accuracies exceeding 95% on benchmark datasets. Despite these advances, challenges persist in requiring large-scale annotated datasets for training, which limits applicability to niche domains, and difficulties in generalizing to 3D surfaces where viewpoint inconsistencies arise; metrics like FID remain essential for quantifying realism but often overlook perceptual seamlessness.

Implementations and Applications

Software Tools and Libraries

Several open-source libraries provide implementations of texture synthesis algorithms, enabling researchers and developers to experiment with pixel-based, patch-based, and deep learning approaches. One prominent example is the Texture Synthesis library developed by Embark Studios, a Rust-based crate that implements multiresolution stochastic texture synthesis for generating images from example textures using non-parametric methods.^[35] This library supports both pixel and patch-based techniques, offering a lightweight API for CLI and programmatic use, with optimizations for performance in applications like game development. For deep learning variants, PyTorch implementations are widely available, such as the deep-textures repository, which reimplements convolutional neural network-based texture synthesis from the seminal Gatys et al. work, allowing optimization for style transfer and texture generation.^[36] More recent advancements include TiPGAN, a PyTorch-based model for high-quality tileable texture synthesis using GANs with intrinsic priors, particularly suited for cloth digitization, as detailed in its 2025 CAD journal publication.^[37] Diffusion model toolkits have also emerged in 2024-2025, exemplified by SceneTex, which leverages depth-to-image diffusion priors for style-consistent indoor scene textures, and Diffusion Texture Painting from NVIDIA, enabling interactive surface texturing on 3D meshes via 2D diffusion models.^[38]^[39] Commercial software integrates texture synthesis capabilities to streamline workflows in professional graphics pipelines. Adobe Substance Designer features procedural nodes and generators that facilitate texture creation and manipulation, allowing users to build complex materials through a node-based interface.^[40] In game engines, plugins extend real-time synthesis: Unreal Engine includes the MetaHuman Face Texture Synthesizer module for generating facial textures via synthesis algorithms integrated with its runtime features.^[41] Unity supports synthesis through asset store plugins and custom scripts for procedural generation, though specific real-time plugins often build on open-source libraries like the Rust crate for on-the-fly texture expansion during rendering. Key features across these tools emphasize efficiency and scalability, particularly GPU acceleration in deep learning implementations via CUDA-enabled frameworks like PyTorch, which speeds up training and inference for large-scale synthesis tasks. Accessibility is enhanced by educational resources, such as MATLAB implementations and tutorials for the Image Quilting algorithm from Efros and Freeman's 2001 SIGGRAPH paper, including benchmark code on MathWorks File Exchange for testing patch-based synthesis performance.^[42]^[31] Texture synthesis plays a crucial role in computer graphics by enabling efficient texture mapping for surfaces in video games and movies, where it generates seamless, large-scale textures from small exemplars to reduce manual artist effort. In procedural world generation for games, such as open-world environments, synthesis algorithms create varied terrains and surfaces by extending input samples, enhancing visual diversity without repetitive asset creation. For instance, generative adversarial networks have been applied to procedurally synthesize original textures in video games, achieving high-quality results that integrate seamlessly into dynamic scenes. In film production, texture movie synthesis facilitates special effects by generating animated textures for motion pictures and computer-generated animation, allowing realistic surface variations over time. Image inpainting, a key application of texture synthesis, supports photograph editing in graphics pipelines by filling missing regions with contextually matching textures derived from surrounding areas, preserving photorealism in post-production workflows. Tools like TextureShop combine texture synthesis with shape-from-shading to edit object textures directly in photographs, enabling artists to replace or extend surface details interactively. In 3D modeling and virtual reality (VR), geometry-aware texture synthesis ensures consistent texturing across meshes by incorporating spatial structure into the generation process, producing high-fidelity UV maps that align with object topology. For example, TextureDreamer employs geometry-aware diffusion models to transfer photorealistic textures from sparse input images (3-5 views) to arbitrary 3D shapes, facilitating efficient asset creation for VR environments. Similarly, TwinTex automates texture mapping for piece-wise planar 3D models, generating photorealistic results suitable for immersive applications. Dynamic omnidirectional texture synthesis further enhances photorealism in VR by real-time generation of 360-degree textures from RGB-D captures, supporting interactive virtual walkthroughs. Beyond graphics, texture synthesis aids medical imaging through anomaly detection by synthesizing normal tissue patterns to identify deviations in scans, such as in MRI brain images where generative models create shape- and texture-consistent samples for training detectors. In AI-generated content analysis, multi-modal texture fusion networks detect synthetic images by analyzing inconsistencies in texture features across modalities, providing robust forensics for 2025-era deepfakes. Recent advancements include text-guided 3D texture synthesis, which enables editing of mesh textures via natural language prompts using diffusion models, as demonstrated in TexGen's multi-view sampling framework for coherent 3D assets. For augmented reality (AR) and VR, these methods contribute to robust photorealism by synthesizing view-consistent textures that blend seamlessly with real-world views, improving immersion in mixed-reality applications.

Historical Development

Key Milestones

Early work on texture synthesis in computer graphics dates back to the 1980s, with initial techniques focusing on statistical and syntactic models to generate patterns. A 1985 survey outlined approaches such as growth models and stochastic simulations, laying the groundwork despite computational constraints.^[3] The field advanced in the early 1990s with foundational techniques focused on tiling and stochastic modeling to replicate visual patterns from sample images. Initial approaches, such as simple tiling methods, involved repeating small texture tiles to cover larger surfaces, though they often produced visible seams and lacked natural variation. A significant advancement came in 1995 with the pyramid-based texture analysis and synthesis algorithm by David J. Heeger and James R. Bergen, which introduced a stochastic model using multi-resolution pyramids to capture and resynthesize frequency content, enabling more realistic aperiodic textures by iteratively adjusting histograms at different scales.^[5] This work marked a shift toward parametric, signal-processing-inspired methods that modeled textures as stationary processes, influencing subsequent research on structured and stochastic synthesis. The late 1990s and early 2000s saw the rise of non-parametric, exemplar-based techniques that sampled directly from input textures to generate novel ones, addressing limitations in parametric models for complex, non-stationary patterns. In 1999, Alexei A. Efros and Thomas K. Leung proposed texture synthesis by non-parametric sampling, a pixel-based method that grows an output image pixel-by-pixel by finding the best-matching neighborhood from the sample, leveraging spatial locality assumptions to preserve local statistics and produce coherent results for a wide range of textures.^[7] Building on this, Efros and William T. Freeman introduced image quilting in 2001, a patch-based approach that overlaps and blends rectangular patches from the input sample using minimum-cut optimization, reducing boundary artifacts and enabling efficient synthesis of larger textures with improved scalability.^[31] These methods established the core paradigm of exemplar-driven synthesis, widely adopted in computer graphics for their simplicity and effectiveness. The 2010s brought deep learning into texture synthesis, adapting convolutional neural networks (CNNs) to capture hierarchical features and enable style transfer. A pivotal development was Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge's 2015 neural algorithm for artistic style, which, when applied to textures, optimizes images to match the Gram matrix statistics of a style texture while preserving content, effectively synthesizing textures through feature correlations in pre-trained CNNs like VGG.^[43] This optimization-based technique demonstrated high perceptual quality for texture generation and inspired feed-forward variants, bridging traditional synthesis with neural representations. In the 2020s, generative adversarial networks (GANs) and diffusion models have dominated, offering controllable, high-fidelity synthesis, including extensions to 3D and text-guided applications. Starting around 2018, GAN-based methods like TextureGAN by Wenqi Xian et al. integrated texture patches into conditional GANs for object-aware synthesis, allowing user-specified textures on sketches and achieving diverse, realistic outputs through adversarial training.^[44] More recently, diffusion models have advanced text-guided and 3D texture generation; for instance, TiPGAN in 2025 by Honghong He et al. employs a GAN-diffusion hybrid with intrinsic priors for seamless, tileable 2D textures, improving boundary consistency for applications like cloth digitization.^[45] Similarly, TexDreamer, introduced in 2024 by Yufei Liu et al., uses multi-view diffusion for zero-shot, photorealistic 3D human texture synthesis from text prompts, leveraging geometry-aware UV mapping to produce robust, high-resolution results on arbitrary meshes.^[46] These innovations address gaps in prior methods by enabling scalable, controllable synthesis in emerging domains like virtual reality and digital fabrication.

Influential Publications

One of the foundational works in texture synthesis is the 1999 paper by Efros and Leung, which introduced a non-parametric sampling approach for pixel-based texture generation. This method synthesizes textures by growing an output image from a seed pixel, sampling neighboring pixels from the input texture based on local similarity, enabling the preservation of structural details in a wide range of textures. The paper's innovation in avoiding parametric models marked a shift toward exemplar-based techniques and has been extended in numerous subsequent algorithms for image inpainting and completion.^[7] Building on pixel-based methods, Kwatra et al.'s 2003 SIGGRAPH paper advanced patch-based synthesis through graph-cut optimization, allowing seamless tiling of larger texture patches from an input exemplar. This approach minimizes boundary discontinuities by formulating overlaps as minimum-cut problems in a graph, producing visually coherent results for both images and videos, and influencing later works in video texture generation and seamless cloning. Its impact is evident in applications requiring scalable, non-repetitive textures, with extensions to 3D and dynamic scenes.^[34] The advent of deep learning brought new paradigms, exemplified by Gatys et al.'s 2016 CVPR paper on neural style transfer, which repurposed pre-trained convolutional neural networks (CNNs) to separate and recombine content and texture representations. By optimizing images to match Gram matrix statistics of texture features from VGG networks, it enabled high-fidelity texture synthesis without explicit training on textures, inspiring feed-forward variants and broader neural artistic rendering techniques. This work's separation of style (texture) from content has over 20,000 citations and catalyzed the neural era of texture modeling.^[47] Li and Wand's 2016 ECCV paper introduced Markovian Generative Adversarial Networks (MGANs) for precomputed real-time texture synthesis, training GANs on small input patches to generate infinite textures via autoregressive sampling. This method achieved efficient, high-quality outputs by leveraging Markov assumptions in the generator, bridging early GANs with traditional synthesis and enabling GPU-accelerated applications in graphics. It demonstrated superior perceptual quality over prior non-neural methods and paved the way for GAN-based texture extensions in real-time rendering.^[48] In recent developments, He et al.'s 2025 paper on TiPGAN proposed a GAN framework incorporating intrinsic priors for high-quality tileable texture synthesis, particularly suited for cloth digitization. By integrating patch-swapping and tiling modules with a novel relative total variation metric, it generates seamless, periodic textures that outperform baselines in boundary continuity and visual realism, addressing limitations in prior GANs for periodic applications like virtual fabrics. This work highlights the evolution toward domain-specific priors in neural synthesis.^[45] Xu et al.'s 2025 arXiv preprint on ProcTex advanced text-guided texture synthesis for procedural 3D models, enabling consistent, interactive generation across shape families using depth-aware diffusion inpainting. The system integrates part-level UV texturing with shape matching, supporting real-time edits in design workflows and extending 2D diffusion models to parametric 3D assets without retraining. Its focus on procedural consistency represents a key step in bridging text prompts with editable graphics pipelines.^[49] Li et al.'s 2025 conference paper on TexDreamer introduced a multi-view diffusion model for text-driven, photorealistic 3D texture synthesis on meshes, generating robust UV maps from prompts while handling complex geometries. By distilling stable diffusion into a multi-view framework, it achieves high-fidelity results with reduced artifacts compared to single-view methods, and supports extensions to human avatars via zero-shot adaptation. This contributes to scalable 3D content creation in VR and animation.^[50] The rise of diffusion models has further transformed texture synthesis, enabling probabilistic generation of diverse, high-resolution textures from noise via iterative denoising. These models extend earlier neural approaches by enabling unconditional and conditional synthesis (e.g., text-guided), with applications in infinite texture tiling and 3D extensions, though challenges in computational efficiency persist.

References

[1]
Texture Analysis and Synthesis
Texture synthesis creates new textures from a sample, avoiding repetition, and can produce tileable images. The goal is to create a new texture that appears to ...
[2]
[PDF] State of the Art in Example-based Texture Synthesis - GAMMA
Given a sample texture, the goal is to synthesize a new texture that looks like the input. The synthesized texture is tileable and can be of arbitrary size.
[3]
Texture synthesis techniques for computer graphics - ScienceDirect
Five primary approaches to texture synthesis for computer graphics are surveyed: a syntactic approach, a statistical approach, a growth model approach, ...
[4]
[PDF] Pyramid-Based Texture Analysis/Synthesis
This technique synthesizes textures by analyzing a sample, using parameters to create new images, and is automatic, requiring only the target texture.
[5]
[PDF] State of the Art in Example-based Texture Synthesis - Hal-Inria
Jul 7, 2011 · Texturing is a core process for computer graphics applica- tions. The texturing process can be divided into three com- ponents: (1) texture ...
[6]
[PDF] Texture Synthesis by Non-parametric Sampling - UC Berkeley EECS
In this paper we have chosen a statistical non-parametric model based on the assumption of spatial locality. The result is a very simple texture syn- thesis ...
[7]
A Three-Level Approach to Texture Mapping and Synthesis on 3D ...
May 4, 2020 · We demonstrate that the relevance of the three levels depends on the texture content and type (stochastic, structured, or anisotropic textures).
[8]
A hybrid-based texture synthesis approach | The Visual Computer
Mar 4, 2004 · A hybrid texture synthesis approach that combines the strength of Ashikhmin's [1] and Liang's [11] algorithms is presented in this paper.Missing: classification | Show results with:classification
[9]
(PDF) Survey Of Texture Mapping - ResearchGate
Aug 5, 2025 · This article surveys the fundamentals of texture mapping, which can be spilt into two topics: the geometric mapping that warps a texture onto a surface, and ...
[10]
MIP-Map Level Selection for Texture Mapping - ACM Digital Library
Texture mapping is a fundamental feature of computer graphics image generation. In current PC-based acceleration hardware, MIP-mapping with bilinear and ...
[11]
Dipmap: A new texture filtering method for texture mapping
Mipmap is a typical texture filtering method for texture mapping. Due to simplicity and efficiency, it has been popularly used in current graphics hardware.
[12]
Texture characterization of digital images which have a periodicity or ...
Mar 17, 2016 · In this paper we analyse digitally generated fractal images added to periodic patterns using the variance of grey levels with four ...
[13]
A Perceptually Based Spectral Model for Isotropic Textures
Aug 6, 2025 · We present a perceptually based texture synthesis model for isotropic textures. The model uses additive synthesis of band-limited noise in the ...
[14]
Lapped solid textures: filling a model with anisotropic textures ...
Aug 6, 2025 · We identify several texture types considering the amount of anisotropy and spatial variation and provide a tailored user interface for each.
[15]
(PDF) Analysis of texture characteristics associated with visual ...
Aug 10, 2025 · In this paper, a set of objective methods for measuring these characteristics is proposed: regularity is estimated by an autocorrelation ...
[16]
(PDF) Texture Analysis - ResearchGate
Feb 16, 2015 · This chapter reviews and discusses various aspects of texture analysis. The concentration is on the various methods of extracting textural features from images.
[17]
[PDF] SAN FRANCISCO JULY 22-26 Volume 19, Number 3, 1985 287
As with all procedural textures, the database is extremely small. Although it is not immediately obvious, this paradigm is a superset of conventional ...
[18]
[PDF] A Crash Course on Texturing - Microsoft
Texturing is a fundamental technique in Computer Graphics, allow- ing us to represent surface properties without modeling geometric or material details.
[19]
[PDF] Simulating the structure and texture of solid wood | CS@Cornell
Procedural 3D textures for color variation in wood have been demonstrated before, and very realistic wood appearance has been demonstrated using color and ...
[20]
[PDF] Poisson Image Editing
Poisson image editing uses the Poisson equation to achieve seamless local changes in image regions, enabling importation and modification of image appearance.
[21]
[PDF] Chaos Mosaic: Fast and Memory Efficient Texture Synthesis
Apr 20, 2000 · We present a procedural method for synthesizing large textures from an input texture sample. The basis of our algorithm is the chaos mosaic, ...
[22]
[PDF] Wang Tiles for Image and Texture Generation
We present a simple stochastic system for non-periodically tiling the plane with a small set of Wang Tiles. The tiles may be filled with texture, patterns ...Missing: seminal | Show results with:seminal
[23]
Markov Random Field Texture Models | IEEE Journals & Magazine
A texture model is a mathematical procedure capable of producing and describing a textured image. We explore the use of Markov random fields as texture models.
[24]
[PDF] Exemplar-Based Texture Synthesis: the Efros–Leung Algorithm
It relies on a Markov assumption and generates new textures in a non-parametric way, directly sampling new values from the input sample. In this paper, we ...
[25]
Stochastic tree grammar inference for texture synthesis and ...
This paper introduces a texture grammar inference procedure which employs a clustering algorithm and a stochastic regular grammar inference procedure.
[26]
Texture synthesis by L-systems - ScienceDirect.com
We present a method for texture synthesis based on the extended L-systems, called TSL-systems, which are defined as a combination of parametric, stochastic and ...
[27]
[PDF] Real-Time Texture Synthesis by Patch-Based Sampling
More recently, Efros and Leung [1999] demonstrated the power of sampling from a local PDF by presenting a nonparametric sampling algorithm that works well for a ...<|control11|><|separator|>
[28]
(PDF) Strong Markov Random Field Model - ResearchGate
PDF | The strong Markov random field (strong-MRF) model is a submodel of the more general MRF-Gibbs model. The strong-MRF model defines a system whose.
[29]
[PDF] Interactive Texture Synthesis on Surfaces Using Jump Maps
Jump maps were recently introduced12 as a means to accelerate image- based texture synthesis to near real-time speeds. The jump map stores for each pixel of a ...
[30]
[PDF] Image Quilting for Texture Synthesis and Transfer - People @EECS
We present a simple image-based method of generating novel vi- sual appearance in which a new image is synthesized by stitching.Missing: rule- | Show results with:rule-
[31]
[PDF] Graphcut Textures: Image and Video Synthesis Using Graph Cuts
In this paper we introduce a new algorithm for image and video texture synthesis. In our approach, patch regions from a sample.Missing: belief propagation
[32]
Image quilting for texture synthesis and transfer - ACM Digital Library
Image quilting for texture synthesis and transfer. Authors: Alexei A. Efros ... Freeman. William T. Freeman. Mitsubishi Electric Research Laboratories. View ...
[33]
Graphcut textures: image and video synthesis using graph cuts
Graphcut textures: image and video synthesis using graph cuts. Authors: Vivek Kwatra. Vivek Kwatra. Georgia Institute of Technology. View Profile. , Arno ...
[34]
Example-based texture synthesis written in Rust - GitHub
Oct 28, 2023 · A light Rust API for Multiresolution Stochastic Texture Synthesis [1], a non-parametric example-based algorithm for image generation.
[35]
Deep Textures - GitHub
Unofficial Reimplementation of "Texture Synthesis Using Convolutional Neural Networks" in PyTorch [Gatys et al. 2015](http://arxiv.org/abs/1505.07376).
[36]
TiPGAN: High-Quality Tileable Textures Synthesis with Intrinsic ...
High-quality tileable textures synthesis with intrinsic priors for cloth digitization applications. This is the official implementation of TiPGAN (CAD 2025).Missing: 2024 | Show results with:2024
[37]
High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors
We propose SceneTex, a novel method for effectively generating high-quality and style-consistent textures for indoor scenes using depth-to-image diffusion ...
[38]
[SIGGRAPH 2024] Diffusion Texture Painting - GitHub
We present a technique that leverages 2D generative diffusion models (DMs) for interactive texture painting on the surface of 3D meshes.
[39]
Texture generators | Substance 3D Designer - Adobe Help Center
Feb 14, 2025 · Build complex, procedural textures and patterns from scratch using a powerful node-based workflow. Open the app.
[40]
FMetaHumanFaceTextureSynthe...
UE API for creating MH Face textures based on the Texture Synthesis module feature set developed in titan.
[41]
[SigGraph2002] Image Quilting/Texture Synthesize - File Exchange
Matlab code to do Image Quilting as presented in the SIGGRAPH 2002 paper by Efros & Freeman. Note: this isn't Efros & Freeman's code, just an implementation ...Missing: tutorial | Show results with:tutorial
[42]
[1508.06576] A Neural Algorithm of Artistic Style - arXiv
Aug 26, 2015 · Here we introduce an artificial system based on a Deep Neural Network that creates artistic images of high perceptual quality.
[43]
[PDF] TextureGAN: Controlling Deep Image Synthesis With Texture Patches
TextureGAN is a deep image synthesis method where users control object texture by placing texture patches on sketches, allowing for novel image generation.Missing: seminal 2018 onwards
[44]
TiPGAN: High-quality tileable textures synthesis with intrinsic priors ...
We introduce an intuitive metric, Relative Total Variation (RTV), to quantify texture pattern tileability. •. Extensive qualitative and quantitative experiments ...
[45]
Towards Zero-Shot High-Fidelity 3D Human Texture Generation
Mar 19, 2024 · We present TexDreamer, the first zero-shot multimodal high-fidelity 3D human texture generation model. Utilizing an efficient texture adaptation ...Missing: synthesis | Show results with:synthesis
[46]
[PDF] Image Style Transfer Using Convolutional Neural Networks
In texture trans- fer the goal is to synthesise a texture from a source image while constraining the texture synthesis in order to preserve the semantic content ...
[47]
Precomputed Real-Time Texture Synthesis with Markovian ... - arXiv
Apr 15, 2016 · This paper proposes Markovian Generative Adversarial Networks (MGANs), a method for training generative neural networks for efficient texture synthesis.
[48]
ProcTex: Consistent and Interactive Text-to-texture Synthesis for Part ...
ProcTex enables consistent and real-time text-guided texture synthesis for families of shapes, which integrates seamlessly with the interactive design flow of ...Missing: ProTex | Show results with:ProTex
[49]
TexDreamer: Text-driven Photorealistic and Robust Texture ...
Jun 30, 2025 · We introduce TexDreamer, a novel method that generates high-quality 3D textures on meshes using text prompts to address these issues.