Manufacturing defect detection is one of those problems where supervised learning falls apart fast. You rarely have enough labeled defect images, and the defect types keep changing. Anomaly detection flips the script: train only on “good” parts, then flag anything that looks different. PatchCore, available through the anomalib library, is one of the strongest approaches for this. It extracts patch-level features from a pretrained backbone, stores them in a memory bank, and uses nearest-neighbor comparison at inference time to produce pixel-level anomaly heatmaps.
Here’s how to build a full visual inspection pipeline from dataset setup through batch inference with OpenCV.
Set Up Anomalib and PatchCore#
Install the dependencies first:
1
| pip install anomalib opencv-python-headless
|
Anomalib ships with PatchCore and several other anomaly detection models. The Engine class handles training, testing, and prediction. Here’s the minimal setup to train PatchCore on the MVTec AD dataset (the standard benchmark for visual anomaly detection):
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
| from anomalib.data import MVTecAD
from anomalib.models import Patchcore
from anomalib.engine import Engine
# MVTecAD auto-downloads. Pick a category like "bottle", "cable", "hazelnut", etc.
datamodule = MVTecAD(
root="./datasets/MVTecAD",
category="bottle",
train_batch_size=32,
eval_batch_size=32,
)
# PatchCore config: wide_resnet50_2 backbone, extract from layer2 and layer3
model = Patchcore(
backbone="wide_resnet50_2",
layers=["layer2", "layer3"],
num_neighbors=9,
coreset_sampling_ratio=0.1,
)
# Engine wraps PyTorch Lightning. max_epochs=1 is fine for PatchCore
# because it builds a memory bank in a single pass, no gradient updates.
engine = Engine(max_epochs=1)
engine.fit(datamodule=datamodule, model=model)
|
PatchCore doesn’t do backpropagation. The “training” pass extracts features from every good image and stores them. The coreset_sampling_ratio controls how much of the memory bank to keep after coreset subsampling – 0.1 means 10%, which cuts memory usage without hurting accuracy much.
Prepare a Custom Dataset#
Real-world inspection data won’t follow MVTec’s download structure. You need to organize your images into a folder layout that anomalib’s Folder datamodule expects:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
| datasets/
my_product/
good/ # Normal/defect-free images (training + validation)
img_001.png
img_002.png
...
defect/ # Abnormal images (test only, optional)
img_100.png
img_101.png
...
mask/ # Binary masks for defect regions (optional, for evaluation)
defect/
img_100.png
img_101.png
|
The key point: you only need the good/ folder to train. Defect images and masks are only used for evaluation. This is the whole appeal of anomaly detection for manufacturing – you collect images of passing parts off the line and train on those.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
| from anomalib.data import Folder
# Classification mode (image-level: pass/fail)
datamodule = Folder(
name="my_product",
root="datasets/my_product",
normal_dir="good",
abnormal_dir="defect",
normal_split_ratio=0.2, # Hold out 20% of good images for validation
)
datamodule.setup()
# Check the splits
print(f"Train samples: {len(datamodule.train_data)}")
print(f"Val samples: {len(datamodule.val_data)}")
print(f"Test samples: {len(datamodule.test_data)}")
|
If you only have good images and no defect samples at all, use synthetic anomaly generation for the test split:
1
2
3
4
5
6
7
8
9
10
| from anomalib.data import Folder
from anomalib.data.utils import TestSplitMode
datamodule = Folder(
name="my_product",
root="datasets/my_product",
normal_dir="good",
test_split_mode=TestSplitMode.SYNTHETIC,
)
datamodule.setup()
|
Anomalib will generate artificial defects on held-out good images so you can still evaluate detection performance.
Train and Run Inference#
With your data ready, training is identical to the MVTec example:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
| from anomalib.data import Folder
from anomalib.models import Patchcore
from anomalib.engine import Engine
datamodule = Folder(
name="my_product",
root="datasets/my_product",
normal_dir="good",
abnormal_dir="defect",
normal_split_ratio=0.2,
)
model = Patchcore(
backbone="wide_resnet50_2",
layers=["layer2", "layer3"],
num_neighbors=9,
coreset_sampling_ratio=0.1,
)
engine = Engine(max_epochs=1)
engine.fit(datamodule=datamodule, model=model)
# Evaluate on test set
test_results = engine.test(datamodule=datamodule, model=model)
print(test_results)
|
For inference on new images, use PredictDataset and engine.predict:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
| from anomalib.data import PredictDataset
dataset = PredictDataset(
path="datasets/my_product/incoming_batch",
image_size=(256, 256),
)
predictions = engine.predict(
model=model,
dataset=dataset,
ckpt_path="results/Patchcore/my_product/latest/weights/lightning/model.ckpt",
)
for pred in predictions:
print(f"Image: {pred.image_path}")
print(f"Score: {pred.pred_score:.4f}")
print(f"Label: {'DEFECT' if pred.pred_label else 'PASS'}")
# pred.anomaly_map contains the pixel-level heatmap as a tensor
|
Each prediction gives you an image-level anomaly score, a binary label, and a pixel-level anomaly map you can overlay on the original image.
Build a Batch Inspection Pipeline with OpenCV#
In production, you want to process a folder of images, generate annotated results, and save a report. Here’s a complete pipeline that loads a trained PatchCore model, runs inference, overlays anomaly heatmaps on the original images, and writes results to disk:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
| import cv2
import numpy as np
import csv
from pathlib import Path
from anomalib.data import PredictDataset
from anomalib.models import Patchcore
from anomalib.engine import Engine
def run_inspection_pipeline(
image_dir: str,
checkpoint_path: str,
output_dir: str,
threshold: float = 0.5,
):
"""Run batch visual inspection and save annotated results."""
output_path = Path(output_dir)
output_path.mkdir(parents=True, exist_ok=True)
(output_path / "pass").mkdir(exist_ok=True)
(output_path / "fail").mkdir(exist_ok=True)
# Load model and run predictions
model = Patchcore()
engine = Engine()
dataset = PredictDataset(path=image_dir, image_size=(256, 256))
predictions = engine.predict(
model=model,
dataset=dataset,
ckpt_path=checkpoint_path,
)
results = []
for pred in predictions:
img_path = pred.image_path
score = float(pred.pred_score)
anomaly_map = pred.anomaly_map.squeeze().cpu().numpy() # H x W float array
label = "FAIL" if score > threshold else "PASS"
# Load original image with OpenCV
original = cv2.imread(str(img_path))
if original is None:
print(f"Warning: could not read {img_path}, skipping")
continue
h, w = original.shape[:2]
# Resize anomaly map to match original image dimensions
anomaly_resized = cv2.resize(anomaly_map, (w, h))
# Normalize to 0-255 and apply colormap
anomaly_norm = (anomaly_resized * 255).astype(np.uint8)
heatmap = cv2.applyColorMap(anomaly_norm, cv2.COLORMAP_JET)
# Blend heatmap with original image
overlay = cv2.addWeighted(original, 0.6, heatmap, 0.4, 0)
# Add label text
color = (0, 0, 255) if label == "FAIL" else (0, 255, 0)
cv2.putText(
overlay,
f"{label} ({score:.3f})",
(10, 30),
cv2.FONT_HERSHEY_SIMPLEX,
0.8,
color,
2,
)
# Save to pass/fail subfolder
dest_folder = output_path / label.lower()
filename = Path(img_path).name
cv2.imwrite(str(dest_folder / filename), overlay)
results.append({
"image": filename,
"score": score,
"label": label,
})
# Write CSV report
csv_path = output_path / "inspection_report.csv"
with open(csv_path, "w", newline="") as f:
writer = csv.DictWriter(f, fieldnames=["image", "score", "label"])
writer.writeheader()
writer.writerows(results)
total = len(results)
failed = sum(1 for r in results if r["label"] == "FAIL")
print(f"Inspection complete: {total} images, {failed} defects found")
print(f"Report saved to {csv_path}")
return results
# Usage
results = run_inspection_pipeline(
image_dir="datasets/my_product/incoming_batch",
checkpoint_path="results/Patchcore/my_product/latest/weights/lightning/model.ckpt",
output_dir="inspection_results",
threshold=0.5,
)
|
This gives you annotated images sorted into pass/fail folders and a CSV report you can feed into your QA system. Adjust the threshold based on your tolerance – lower catches more defects but increases false positives.
Common Errors and Fixes#
RuntimeError: No images found in the path
The PredictDataset expects image files directly in the folder you point it at. It won’t recurse into subdirectories by default. Make sure your images sit directly inside image_dir, not nested in subfolders.
CUDA out of memory during training
PatchCore’s memory bank grows with the number of training images. Two fixes: lower coreset_sampling_ratio (try 0.01 instead of 0.1), or reduce train_batch_size in your datamodule. You can also switch to a lighter backbone like resnet18 instead of wide_resnet50_2.
1
2
3
4
5
6
7
| # Lighter config for limited GPU memory
model = Patchcore(
backbone="resnet18",
layers=["layer2", "layer3"],
num_neighbors=9,
coreset_sampling_ratio=0.01,
)
|
ValueError: normal_dir does not exist
Double-check your folder structure. The root path in the Folder datamodule should point to the parent directory that contains good/ (or whatever you set normal_dir to). If your good images are at datasets/my_product/good/, then root must be datasets/my_product.
High false positive rate on test images
This usually means the training set doesn’t cover enough variation. Add more good images that capture different lighting conditions, camera angles, and part orientations. PatchCore’s accuracy scales directly with how representative your normal training set is. Aim for at least 50-100 diverse good images.
Anomaly heatmap is all zeros or all ones
The anomaly map values depend on normalization. If you’re getting flat maps, check that pred.anomaly_map is not None and that you’re squeezing the batch and channel dimensions correctly. The raw tensor shape is typically (1, 1, H, W) – you need .squeeze() to get (H, W).