Game texture work is a grind. You need base diffuse maps, normal maps, and multiple material variants – all seamless, all tileable, all at consistent quality. SDXL generates textures that are genuinely useful for production pipelines if you set up the prompts and post-processing correctly. Here’s a full pipeline that goes from text prompt to game-ready texture sets.
1
| pip install diffusers transformers accelerate torch pillow numpy safetensors
|
You need a CUDA GPU with at least 12GB VRAM for SDXL. An RTX 4070 or better works well. If you’re on 8GB, drop to torch.float16 and enable attention slicing – shown below.
Set Up the SDXL Pipeline#
SDXL produces significantly better textures than SD 1.5 or 2.1 because of its higher native resolution and improved detail coherence. The key is configuring the scheduler and generation parameters for texture work specifically – you want sharp details and minimal noise artifacts.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
| import torch
from diffusers import StableDiffusionXLPipeline, EulerAncestralDiscreteScheduler
pipe = StableDiffusionXLPipeline.from_pretrained(
"stabilityai/stable-diffusion-xl-base-1.0",
torch_dtype=torch.float16,
variant="fp16",
use_safetensors=True,
)
pipe.to("cuda")
# EulerAncestral gives sharper texture detail than the default PNDM
pipe.scheduler = EulerAncestralDiscreteScheduler.from_config(pipe.scheduler.config)
# Memory optimizations for 12GB cards
pipe.enable_vae_slicing()
pipe.enable_vae_tiling()
def enable_seamless_tiling(pipeline):
"""Patch Conv2d layers to circular padding for seamless output."""
for module in pipeline.unet.modules():
if isinstance(module, torch.nn.Conv2d):
module.padding_mode = "circular"
for module in pipeline.vae.modules():
if isinstance(module, torch.nn.Conv2d):
module.padding_mode = "circular"
enable_seamless_tiling(pipe)
print("Pipeline ready with seamless tiling enabled")
|
The enable_seamless_tiling function patches every convolution layer to use circular padding. This means the left edge wraps to the right, top wraps to bottom. The model generates output that tiles without visible seams – no post-processing needed for the basic repeat pattern.
EulerAncestralDiscreteScheduler is the best choice for textures. It preserves fine grain detail better than DPM++ or DDIM, which tend to smooth out the micro-detail that makes textures look realistic at close range.
Generate Tileable Base Textures#
Good texture prompts follow a specific pattern: material type first, then surface characteristics, then lighting and camera angle. Always include “seamless texture” and “top-down orthographic view” to push the model toward flat, tileable output.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
| from PIL import Image
import os
TEXTURE_PROMPTS = {
"wood_planks": {
"prompt": "seamless texture, aged oak wood planks, grain detail, slight weathering, "
"top-down orthographic view, even studio lighting, 4k, PBR material",
"negative": "text, watermark, perspective, shadows, objects, people, uneven lighting",
},
"stone_wall": {
"prompt": "seamless texture, rough cut stone wall, mortar joints, grey limestone, "
"top-down orthographic view, even studio lighting, 4k, PBR material",
"negative": "text, watermark, perspective, vegetation, objects, cracks, uneven lighting",
},
"brushed_metal": {
"prompt": "seamless texture, brushed stainless steel, fine linear grain, metallic sheen, "
"top-down orthographic view, even studio lighting, 4k, PBR material",
"negative": "text, watermark, perspective, scratches, rust, dents, reflections of objects",
},
"fabric_linen": {
"prompt": "seamless texture, natural linen fabric weave, visible thread pattern, "
"top-down orthographic view, even studio lighting, 4k, PBR material",
"negative": "text, watermark, perspective, wrinkles, folds, shadows, objects",
},
}
output_dir = "generated_textures"
os.makedirs(output_dir, exist_ok=True)
generator = torch.Generator("cuda").manual_seed(12345)
for name, config in TEXTURE_PROMPTS.items():
image = pipe(
prompt=config["prompt"],
negative_prompt=config["negative"],
num_inference_steps=35,
guidance_scale=7.0,
height=1024,
width=1024,
generator=generator,
).images[0]
filepath = os.path.join(output_dir, f"{name}_diffuse.png")
image.save(filepath)
print(f"Saved {filepath}")
|
A few things matter here. The guidance scale of 7.0 keeps textures detailed but not over-saturated – go higher and you get cartoon-like contrast that looks wrong in a game engine. 35 inference steps is the sweet spot for SDXL texture quality; below 25 you lose fine detail, above 40 you waste compute for no visible gain. The 1024x1024 resolution is SDXL’s native size and produces the sharpest results.
Post-Process Textures for Perfect Tileability#
The circular padding trick handles most of the tiling, but some generated textures still show subtle brightness or color shifts at the edges. A cross-blend pass fixes this by smoothly interpolating the border regions.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
| import numpy as np
from PIL import Image
def make_seamless(image: Image.Image, blend_width: int = 64) -> Image.Image:
"""Cross-blend edges to remove any remaining seam artifacts."""
arr = np.array(image, dtype=np.float32)
h, w, c = arr.shape
# Create a horizontal blend: fade left edge into right edge
for i in range(blend_width):
alpha = i / blend_width
# Blend left columns with wrapped-around right columns
left_col = arr[:, i, :].copy()
right_col = arr[:, w - blend_width + i, :].copy()
blended = left_col * alpha + right_col * (1.0 - alpha)
arr[:, i, :] = blended
arr[:, w - blend_width + i, :] = left_col * (1.0 - alpha) + right_col * alpha
# Create a vertical blend: fade top edge into bottom edge
for i in range(blend_width):
alpha = i / blend_width
top_row = arr[i, :, :].copy()
bottom_row = arr[h - blend_width + i, :, :].copy()
blended = top_row * alpha + bottom_row * (1.0 - alpha)
arr[i, :, :] = blended
arr[h - blend_width + i, :, :] = top_row * (1.0 - alpha) + bottom_row * alpha
return Image.fromarray(np.clip(arr, 0, 255).astype(np.uint8))
# Apply to a generated texture
diffuse = Image.open("generated_textures/stone_wall_diffuse.png")
seamless = make_seamless(diffuse, blend_width=64)
seamless.save("generated_textures/stone_wall_diffuse_seamless.png")
print("Seamless texture saved")
|
The blend_width parameter controls how many pixels at each edge get blended. 64 pixels works well for 1024x1024 textures. For 512x512, use 32. Too wide and you get visible softening near edges; too narrow and seams still show.
Generate Normal Maps from Diffuse Textures#
Every game engine needs normal maps for surface detail without extra geometry. You can derive a reasonable normal map directly from the diffuse texture using the Sobel operator. This won’t match a proper baked normal map from a 3D sculpt, but for AI-generated materials it works well enough for most use cases.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
| import numpy as np
from PIL import Image, ImageFilter
def generate_normal_map(diffuse: Image.Image, strength: float = 2.0) -> Image.Image:
"""Generate a normal map from a diffuse texture using Sobel gradients."""
gray = diffuse.convert("L")
gray_arr = np.array(gray, dtype=np.float32) / 255.0
# Sobel kernels for x and y gradients
sobel_x = np.array([[-1, 0, 1],
[-2, 0, 2],
[-1, 0, 1]], dtype=np.float32)
sobel_y = np.array([[-1, -2, -1],
[ 0, 0, 0],
[ 1, 2, 1]], dtype=np.float32)
from scipy.signal import convolve2d
dx = convolve2d(gray_arr, sobel_x, mode="same", boundary="wrap") * strength
dy = convolve2d(gray_arr, sobel_y, mode="same", boundary="wrap") * strength
# Normal vector: (-dx, -dy, 1.0), then normalize
normal = np.zeros((*gray_arr.shape, 3), dtype=np.float32)
normal[:, :, 0] = -dx
normal[:, :, 1] = -dy
normal[:, :, 2] = 1.0
# Normalize each vector to unit length
length = np.sqrt(np.sum(normal ** 2, axis=2, keepdims=True))
normal = normal / (length + 1e-8)
# Map from [-1, 1] to [0, 255]
normal_img = ((normal + 1.0) * 0.5 * 255.0).astype(np.uint8)
return Image.fromarray(normal_img)
# Generate normal map for each diffuse texture
for name in TEXTURE_PROMPTS:
diffuse_path = f"generated_textures/{name}_diffuse.png"
diffuse = Image.open(diffuse_path)
normal = generate_normal_map(diffuse, strength=2.0)
normal.save(f"generated_textures/{name}_normal.png")
print(f"Saved normal map for {name}")
|
You’ll need scipy for convolve2d – add it with pip install scipy. The boundary="wrap" parameter is critical here. It makes the convolution treat edges as wrapping around, so the normal map stays seamless just like the source diffuse.
The strength parameter controls bump intensity. Use 1.0-2.0 for subtle surfaces like fabric or smooth metal. Crank it to 3.0-5.0 for rough stone or heavy wood grain. Preview in your engine and adjust – what looks right in a 2D viewer is often too subtle at game rendering angles.
Batch Generate Texture Sets#
A real game project needs dozens of texture variants. Here’s how to batch-generate complete texture sets with consistent naming that you can drop straight into Unity or Unreal.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
| import os
from pathlib import Path
def generate_texture_set(
pipe,
material_name: str,
prompt: str,
negative_prompt: str,
output_dir: str,
seed: int = 42,
num_variations: int = 3,
normal_strength: float = 2.0,
):
"""Generate a complete texture set: diffuse variants + normal maps."""
mat_dir = os.path.join(output_dir, material_name)
os.makedirs(mat_dir, exist_ok=True)
for i in range(num_variations):
gen = torch.Generator("cuda").manual_seed(seed + i)
image = pipe(
prompt=prompt,
negative_prompt=negative_prompt,
num_inference_steps=35,
guidance_scale=7.0,
height=1024,
width=1024,
generator=gen,
).images[0]
# Post-process for tileability
diffuse = make_seamless(image, blend_width=64)
diffuse_path = os.path.join(mat_dir, f"{material_name}_v{i:02d}_diffuse.png")
diffuse.save(diffuse_path)
# Generate and save normal map
normal = generate_normal_map(diffuse, strength=normal_strength)
normal_path = os.path.join(mat_dir, f"{material_name}_v{i:02d}_normal.png")
normal.save(normal_path)
print(f" [{i+1}/{num_variations}] {diffuse_path}")
# Batch run for all materials
materials = {
"oak_wood": {
"prompt": "seamless texture, polished oak wood planks, warm tone, fine grain, "
"top-down orthographic view, even studio lighting, 4k, PBR material",
"negative": "text, watermark, perspective, objects, shadows",
"normal_strength": 1.5,
},
"cobblestone": {
"prompt": "seamless texture, worn cobblestone pavement, grey and brown stones, "
"top-down orthographic view, even studio lighting, 4k, PBR material",
"negative": "text, watermark, perspective, grass, vegetation, cracks",
"normal_strength": 4.0,
},
"rusted_iron": {
"prompt": "seamless texture, rusted iron plate, orange rust patches on dark metal, "
"top-down orthographic view, even studio lighting, 4k, PBR material",
"negative": "text, watermark, perspective, objects, bolts, seams",
"normal_strength": 3.0,
},
"canvas_fabric": {
"prompt": "seamless texture, heavy canvas fabric weave, beige cotton, visible threads, "
"top-down orthographic view, even studio lighting, 4k, PBR material",
"negative": "text, watermark, perspective, wrinkles, stains, folds",
"normal_strength": 1.0,
},
}
output_root = "game_textures"
for mat_name, mat_config in materials.items():
print(f"Generating {mat_name}...")
generate_texture_set(
pipe=pipe,
material_name=mat_name,
prompt=mat_config["prompt"],
negative_prompt=mat_config["negative"],
output_dir=output_root,
seed=42,
num_variations=3,
normal_strength=mat_config["normal_strength"],
)
print(f"All textures saved to {output_root}/")
|
This produces a directory structure like game_textures/oak_wood/oak_wood_v00_diffuse.png alongside matching normal maps. Both Unity and Unreal can import these directly – just set the normal maps to “Normal Map” texture type in the import settings.
Each variation uses a different seed offset, so you get distinct but stylistically similar textures. Useful for avoiding obvious repetition when tiling across large surfaces.
Common Errors and Fixes#
RuntimeError: CUDA out of memory
SDXL at 1024x1024 needs around 10-12GB VRAM. If you’re running out, add these lines before generation:
1
2
| pipe.enable_model_cpu_offload() # moves modules to CPU when not in use
pipe.enable_attention_slicing(1) # trades speed for lower peak VRAM
|
This drops peak usage to around 6-8GB at the cost of roughly 30% slower generation.
ValueError: Pipeline expected no more than 77 tokens
SDXL supports 77 tokens per prompt encoder (it has two). If your texture prompts are getting clipped, shorten them. Remove redundant modifiers – “4k, high resolution, detailed, sharp” all do roughly the same thing. Pick one.
Normal maps look flat or washed out
Increase the strength parameter in generate_normal_map. For rough surfaces like stone, values of 3.0-5.0 produce visible depth. Also make sure you’re importing the normal map with the correct format in your engine – Unity expects OpenGL-style normals (green channel = up), Unreal expects DirectX-style (green channel = down). Flip the green channel if needed:
1
2
3
4
5
| # Flip green channel for DirectX-style normal maps (Unreal Engine)
normal_arr = np.array(normal_image)
normal_arr[:, :, 1] = 255 - normal_arr[:, :, 1]
normal_dx = Image.fromarray(normal_arr)
normal_dx.save("texture_normal_dx.png")
|
Textures have visible seams when tiling despite circular padding
This happens occasionally with certain prompts that produce strong directional lighting baked into the texture. Add “flat lighting, no shadows, no directional light” to your negative prompt. If seams persist, increase blend_width in make_seamless to 96 or 128.
ImportError: No module named 'scipy'
The normal map generation uses scipy.signal.convolve2d. Install it:
If you want to avoid the scipy dependency entirely, use PIL’s built-in ImageFilter.FIND_EDGES as a rougher alternative, but the Sobel approach gives much better normal maps.