Most 3D artists spend hours hunting for the right texture or tweaking Substance Designer graphs to get a material that tiles cleanly. AI changes that equation. You can generate seamless PBR textures – base color, normal, roughness, displacement – from a text prompt in under a minute. Here’s how to set that up with Python.

1

pip install diffusers transformers accelerate torch pillow numpy

Generate Tileable Textures with Stable Diffusion

The trick to seamless textures is patching the UNet’s convolution layers so they wrap around at the edges. The diffusers library makes this straightforward – you modify the padding mode on every Conv2d layer from zeros to circular, and the model produces output that tiles perfectly in both directions.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

import torch
from diffusers import StableDiffusionPipeline

def enable_tiling(pipe):
    """Patch all Conv2d layers to use circular padding for seamless tiling."""
    for module in pipe.unet.modules():
        if isinstance(module, torch.nn.Conv2d):
            module.padding_mode = "circular"
    for module in pipe.vae.modules():
        if isinstance(module, torch.nn.Conv2d):
            module.padding_mode = "circular"

pipe = StableDiffusionPipeline.from_pretrained(
    "stabilityai/stable-diffusion-2-1",
    torch_dtype=torch.float16,
)
pipe = pipe.to("cuda")

# Enable seamless tiling
enable_tiling(pipe)

prompt = "weathered red brick wall, PBR texture, top-down view, even lighting, 4k"
negative_prompt = "text, watermark, logo, uneven lighting, perspective distortion"

image = pipe(
    prompt,
    negative_prompt=negative_prompt,
    num_inference_steps=30,
    guidance_scale=7.5,
    height=512,
    width=512,
).images[0]

image.save("brick_base_color.png")
print("Saved tileable base color texture")

That enable_tiling function is the entire secret. Without it, you get visible seams when the texture repeats across a mesh. With it, the edges wrap seamlessly. The prompt matters too – always include “top-down view” and “even lighting” to avoid perspective distortion that breaks tiling in a 3D renderer.

For best results, use prompts that describe flat surface materials rather than scenes. “Mossy stone floor, PBR texture, flat lighting” works. “A beautiful stone pathway in a garden” does not – you’ll get depth and perspective that kills the tile.

Generate Normal and Roughness Maps from a Base Color

Once you have a base color texture, you need the rest of the PBR stack: normal map, roughness map, and displacement map. You can derive decent approximations directly from the base color using image processing and depth estimation.

For the normal map, convert the base color to grayscale, treat pixel intensity as height, and compute the surface gradient with Sobel filters:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

import numpy as np
from PIL import Image, ImageFilter

def generate_normal_map(base_color_path, strength=2.0):
    """Generate a normal map from a base color texture using Sobel gradients."""
    img = Image.open(base_color_path).convert("L")
    img_array = np.array(img, dtype=np.float32) / 255.0

    # Sobel kernels for x and y gradients
    sobel_x = np.array([[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]], dtype=np.float32)
    sobel_y = np.array([[-1, -2, -1], [0, 0, 0], [1, 2, 1]], dtype=np.float32)

    from scipy.signal import convolve2d
    grad_x = convolve2d(img_array, sobel_x, mode="same", boundary="wrap") * strength
    grad_y = convolve2d(img_array, sobel_y, mode="same", boundary="wrap") * strength

    # Normal vector: (-dx, -dy, 1), then normalize
    normal = np.zeros((*img_array.shape, 3), dtype=np.float32)
    normal[:, :, 0] = -grad_x
    normal[:, :, 1] = -grad_y
    normal[:, :, 2] = 1.0

    norm = np.sqrt(np.sum(normal ** 2, axis=2, keepdims=True))
    normal = normal / norm

    # Map from [-1, 1] to [0, 255]
    normal_img = ((normal + 1.0) * 0.5 * 255).astype(np.uint8)
    return Image.fromarray(normal_img)


def generate_roughness_map(base_color_path):
    """Generate a roughness map by desaturating and inverting contrast."""
    img = Image.open(base_color_path).convert("L")
    # Blur to smooth out fine detail -- roughness is a macro property
    img = img.filter(ImageFilter.GaussianBlur(radius=2))
    img_array = np.array(img, dtype=np.float32)
    # Normalize to full range
    img_array = (img_array - img_array.min()) / (img_array.max() - img_array.min() + 1e-8)
    roughness = (img_array * 255).astype(np.uint8)
    return Image.fromarray(roughness)


# Generate maps from the base color we created earlier
normal_map = generate_normal_map("brick_base_color.png", strength=3.0)
normal_map.save("brick_normal.png")

roughness_map = generate_roughness_map("brick_base_color.png")
roughness_map.save("brick_roughness.png")

print("Saved normal and roughness maps")

You’ll need scipy for the convolution (pip install scipy). The boundary="wrap" parameter is critical – it ensures the Sobel filter wraps at the edges, preserving the seamless tiling from the base color.

The strength parameter on the normal map controls how pronounced the surface detail appears. For brick and stone, values between 2.0 and 4.0 work well. For smooth surfaces like metal or plastic, keep it below 1.0.

Generate Displacement Maps with Depth Anything

The Sobel-based approach works for normal maps, but for displacement (height) maps you get much better results from a monocular depth estimation model. Depth Anything V2 is the current best option – it produces high-quality relative depth from a single image, which maps directly to displacement.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

import torch
import numpy as np
from PIL import Image
from transformers import pipeline

# Load the depth estimation pipeline
depth_estimator = pipeline(
    "depth-estimation",
    model="depth-anything/Depth-Anything-V2-Small-hf",
    device=0,
    torch_dtype=torch.float16,
)

# Estimate depth from the base color texture
base_color = Image.open("brick_base_color.png")
result = depth_estimator(base_color)
depth_map = result["depth"]  # PIL Image

# Convert to grayscale displacement map
depth_array = np.array(depth_map.convert("L"), dtype=np.float32)
# Normalize to 0-255 range
depth_array = (depth_array - depth_array.min()) / (depth_array.max() - depth_array.min() + 1e-8)
displacement = Image.fromarray((depth_array * 255).astype(np.uint8))
displacement.save("brick_displacement.png")

print("Saved displacement map")

This gives you a physically plausible height map that captures the mortar lines between bricks, the surface roughness of individual bricks, and the overall relief pattern. The Depth Anything V2 Small variant runs on 2GB of VRAM and processes a 512x512 image in under a second.

For a full PBR set, you now have four maps: base color, normal, roughness, and displacement. Most game engines (Unreal, Unity, Godot) and 3D tools (Blender) accept exactly this set.

Batch Generate Texture Variations

When you’re building a game level or populating a 3D scene, you need variations – not one brick texture, but six. Here’s how to batch-generate a full material library from a list of prompts:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94

import torch
import os
import numpy as np
from PIL import Image, ImageFilter
from diffusers import StableDiffusionPipeline
from scipy.signal import convolve2d
from transformers import pipeline as hf_pipeline


def enable_tiling(pipe):
    for module in pipe.unet.modules():
        if isinstance(module, torch.nn.Conv2d):
            module.padding_mode = "circular"
    for module in pipe.vae.modules():
        if isinstance(module, torch.nn.Conv2d):
            module.padding_mode = "circular"


def make_normal(img_path, strength=2.5):
    img = np.array(Image.open(img_path).convert("L"), dtype=np.float32) / 255.0
    sobel_x = np.array([[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]], dtype=np.float32)
    sobel_y = np.array([[-1, -2, -1], [0, 0, 0], [1, 2, 1]], dtype=np.float32)
    gx = convolve2d(img, sobel_x, mode="same", boundary="wrap") * strength
    gy = convolve2d(img, sobel_y, mode="same", boundary="wrap") * strength
    normal = np.stack([-gx, -gy, np.ones_like(gx)], axis=-1)
    normal /= np.sqrt(np.sum(normal ** 2, axis=2, keepdims=True))
    return Image.fromarray(((normal + 1) * 0.5 * 255).astype(np.uint8))


def make_roughness(img_path):
    img = Image.open(img_path).convert("L").filter(ImageFilter.GaussianBlur(2))
    arr = np.array(img, dtype=np.float32)
    arr = (arr - arr.min()) / (arr.max() - arr.min() + 1e-8)
    return Image.fromarray((arr * 255).astype(np.uint8))


# Set up pipelines
sd_pipe = StableDiffusionPipeline.from_pretrained(
    "stabilityai/stable-diffusion-2-1",
    torch_dtype=torch.float16,
)
sd_pipe = sd_pipe.to("cuda")
enable_tiling(sd_pipe)

depth_pipe = hf_pipeline(
    "depth-estimation",
    model="depth-anything/Depth-Anything-V2-Small-hf",
    device=0,
    torch_dtype=torch.float16,
)

materials = [
    {"name": "brick_red", "prompt": "red brick wall, PBR texture, top-down view, even lighting, 4k"},
    {"name": "stone_grey", "prompt": "grey limestone floor tiles, PBR texture, top-down view, even lighting, 4k"},
    {"name": "wood_oak", "prompt": "oak wood planks, PBR texture, top-down view, even lighting, 4k"},
    {"name": "metal_rust", "prompt": "rusted steel plate, PBR texture, top-down view, even lighting, 4k"},
    {"name": "concrete_worn", "prompt": "worn concrete surface, PBR texture, top-down view, even lighting, 4k"},
]

output_dir = "texture_library"
os.makedirs(output_dir, exist_ok=True)

for mat in materials:
    name = mat["name"]
    mat_dir = os.path.join(output_dir, name)
    os.makedirs(mat_dir, exist_ok=True)

    # Generate base color
    base_color = sd_pipe(
        mat["prompt"],
        negative_prompt="text, watermark, logo, perspective, uneven lighting",
        num_inference_steps=30,
        guidance_scale=7.5,
        height=512,
        width=512,
    ).images[0]

    base_path = os.path.join(mat_dir, "base_color.png")
    base_color.save(base_path)

    # Generate PBR maps
    make_normal(base_path, strength=2.5).save(os.path.join(mat_dir, "normal.png"))
    make_roughness(base_path).save(os.path.join(mat_dir, "roughness.png"))

    depth_result = depth_pipe(base_color)
    depth_arr = np.array(depth_result["depth"].convert("L"), dtype=np.float32)
    depth_arr = (depth_arr - depth_arr.min()) / (depth_arr.max() - depth_arr.min() + 1e-8)
    Image.fromarray((depth_arr * 255).astype(np.uint8)).save(
        os.path.join(mat_dir, "displacement.png")
    )

    print(f"Generated full PBR set for {name}")

print(f"All materials saved to {output_dir}/")

This produces a clean folder structure: texture_library/brick_red/base_color.png, texture_library/brick_red/normal.png, etc. Each material gets four maps, ready to drag into Blender’s Principled BSDF or Unreal’s Material Editor.

Want multiple variations of the same material? Add a seed loop. Replace the generation call with a loop over different generator=torch.Generator("cuda").manual_seed(seed) values and you’ll get visually distinct textures from the same prompt.

Common Errors and Fixes

torch.cuda.OutOfMemoryError during generation Stable Diffusion 2.1 at 512x512 needs about 6GB of VRAM. If you’re running out, enable attention slicing before generation:

1

sd_pipe.enable_attention_slicing()

This trades speed for memory. If that’s still not enough, drop to height=384, width=384 – textures get upscaled in the engine anyway.

Visible seams when tiling the texture You forgot to call enable_tiling(), or you called it after running inference once. The tiling patch must be applied before the first forward pass. Also check your prompt – any perspective or directional lighting in the generated image creates visible seams even with circular padding.

Normal map looks flat or washed out Increase the strength parameter. The default of 2.0 is conservative. For materials with deep grooves (brick, bark, cobblestone), use 3.0 to 5.0. Also make sure you’re using the grayscale conversion of the base color, not the RGB image directly.

Depth Anything returns a mostly uniform depth map This happens with very flat textures like smooth metal or polished marble – the model sees no depth variation, which is technically correct. For these materials, use a flat mid-grey displacement map and rely on the normal map for surface detail instead.

ModuleNotFoundError: No module named 'scipy' The normal map generation uses scipy.signal.convolve2d. Install it:

1

pip install scipy

Generated texture has baked-in shadows Your prompt included terms like “dramatic lighting” or “sunlight”. For PBR textures, always specify “even lighting” or “flat lighting” in the prompt and add “shadows, directional light” to the negative prompt. The lighting should come from your 3D scene, not from the texture itself.