Every team that ships an LLM-powered feature hits the same wall: the model returns whatever it feels like. Sometimes it’s valid JSON. Sometimes it’s markdown with a preamble. Sometimes it hallucinates a field you never asked for. Structured output schemas fix this by forcing the model to conform to a contract you define up front.

The fastest path is combining OpenAI’s response_format parameter with Pydantic models. Here’s the minimal version that works right now:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI()

class SentimentResult(BaseModel):
    sentiment: str
    confidence: float
    reasoning: str

completion = client.beta.chat.completions.parse(
    model="gpt-4o-2024-08-06",
    messages=[
        {"role": "system", "content": "Analyze the sentiment of the user's text. Return sentiment as 'positive', 'negative', or 'neutral'."},
        {"role": "user", "content": "The new API is incredibly fast but the docs are a mess."},
    ],
    response_format=SentimentResult,
)

result = completion.choices[0].message.parsed
print(result.sentiment)     # "neutral"
print(result.confidence)    # 0.78
print(result.reasoning)     # "Mixed sentiment: positive about speed, negative about documentation"

The .parse() method on client.beta.chat.completions handles the schema conversion automatically. OpenAI converts your Pydantic model into a JSON schema, constrains the model’s token generation to only produce valid output matching that schema, and then deserializes the response back into your Pydantic object. No regex parsing. No “please return JSON” in your prompt. The model literally cannot produce invalid structure.

Defining Pydantic Models for LLM Outputs

Your Pydantic model is the contract. Think of it as the type system for your LLM’s output. Be specific about field types, use enums for constrained values, and add descriptions that guide the model.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

from enum import Enum
from typing import Optional
from pydantic import BaseModel, Field

class RiskLevel(str, Enum):
    LOW = "low"
    MEDIUM = "medium"
    HIGH = "high"
    CRITICAL = "critical"

class ContentModeration(BaseModel):
    is_safe: bool = Field(description="Whether the content is safe for display")
    risk_level: RiskLevel = Field(description="Overall risk assessment")
    flagged_categories: list[str] = Field(
        description="List of policy categories that were triggered, e.g. 'hate_speech', 'self_harm'"
    )
    explanation: str = Field(
        description="Brief explanation of the moderation decision in one sentence"
    )
    suggested_action: str = Field(
        description="One of: 'allow', 'flag_for_review', 'block'"
    )
    confidence_score: float = Field(
        ge=0.0, le=1.0,
        description="Confidence in the moderation decision between 0 and 1"
    )
    pii_detected: Optional[list[str]] = Field(
        default=None,
        description="Types of PII found, if any: 'email', 'phone', 'ssn', 'address'"
    )

A few things matter here. The Field(description=...) strings aren’t just documentation – OpenAI passes them into the JSON schema, and the model uses them as guidance for what to put in each field. The ge=0.0, le=1.0 constraints on confidence_score are enforced by Pydantic during deserialization, not by the model itself. The RiskLevel enum restricts the model to exactly four valid values.

Use Optional fields with defaults when a field may not always apply. The pii_detected field only matters when PII is actually present, so None is a reasonable default.

Using OpenAI’s response_format Parameter

OpenAI supports structured outputs through the client.beta.chat.completions.parse() method. Pass your Pydantic class directly as response_format:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

from openai import OpenAI
from pydantic import BaseModel, Field
from enum import Enum

client = OpenAI()

class RiskLevel(str, Enum):
    LOW = "low"
    MEDIUM = "medium"
    HIGH = "high"
    CRITICAL = "critical"

class ContentModeration(BaseModel):
    is_safe: bool
    risk_level: RiskLevel
    flagged_categories: list[str]
    explanation: str
    suggested_action: str
    confidence_score: float = Field(ge=0.0, le=1.0)

def moderate_content(text: str) -> ContentModeration:
    completion = client.beta.chat.completions.parse(
        model="gpt-4o-2024-08-06",
        messages=[
            {
                "role": "system",
                "content": (
                    "You are a content moderation system. Evaluate the given text "
                    "against standard content policies. Be precise with risk levels: "
                    "low = benign content, medium = borderline, high = likely violation, "
                    "critical = clear violation requiring immediate action."
                ),
            },
            {"role": "user", "content": text},
        ],
        response_format=ContentModeration,
    )

    message = completion.choices[0].message

    # Handle refusal (model may refuse harmful requests)
    if message.refusal:
        raise ValueError(f"Model refused to process: {message.refusal}")

    return message.parsed

result = moderate_content("I love building apps with Python!")
print(result.is_safe)          # True
print(result.risk_level)       # RiskLevel.LOW
print(result.flagged_categories)  # []

The message.refusal check is important. When the model refuses a request (safety filters, for example), it sets .refusal to a string and .parsed to None. Always check for this before accessing the parsed result.

Adding Custom Validators

Pydantic’s field and model validators add a second layer of enforcement that catches things the schema alone cannot. The model might produce structurally valid JSON that’s semantically wrong – a confidence score of 0.99 on a “neutral” sentiment, or an empty explanation string.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

from pydantic import BaseModel, Field, field_validator, model_validator

class AnalysisResult(BaseModel):
    sentiment: str
    confidence: float = Field(ge=0.0, le=1.0)
    key_topics: list[str]
    word_count: int = Field(ge=0)
    explanation: str
    language: str

    @field_validator("sentiment")
    @classmethod
    def validate_sentiment(cls, v: str) -> str:
        allowed = {"positive", "negative", "neutral", "mixed"}
        if v.lower() not in allowed:
            raise ValueError(f"sentiment must be one of {allowed}, got '{v}'")
        return v.lower()

    @field_validator("key_topics")
    @classmethod
    def validate_topics_not_empty(cls, v: list[str]) -> list[str]:
        if len(v) == 0:
            raise ValueError("key_topics must contain at least one topic")
        if len(v) > 10:
            raise ValueError("key_topics should not exceed 10 items")
        return [topic.strip().lower() for topic in v]

    @field_validator("explanation")
    @classmethod
    def validate_explanation_length(cls, v: str) -> str:
        if len(v.strip()) < 10:
            raise ValueError("explanation must be at least 10 characters")
        return v.strip()

    @model_validator(mode="after")
    def validate_consistency(self) -> "AnalysisResult":
        # High confidence neutral sentiments are suspicious
        if self.sentiment == "neutral" and self.confidence > 0.95:
            self.confidence = 0.85  # Cap it -- neutrality is inherently uncertain

        # Mixed sentiment should never be high confidence
        if self.sentiment == "mixed" and self.confidence > 0.8:
            self.confidence = 0.7

        return self

The @field_validator decorators run on individual fields. The @model_validator(mode="after") runs after all fields are validated, so you can enforce cross-field consistency. This is where you catch things like contradictory confidence-sentiment pairs.

One caveat: if validation fails, Pydantic raises a ValidationError. The .parse() method will propagate this, so you need to handle it.

Handling Validation Failures and Fallbacks

Production systems need graceful degradation. When a structured output fails validation, you have three options: retry, fall back to a simpler schema, or return a safe default.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78

from openai import OpenAI
from pydantic import BaseModel, Field, ValidationError
import logging

client = OpenAI()
logger = logging.getLogger(__name__)

class StrictAnalysis(BaseModel):
    sentiment: str
    confidence: float = Field(ge=0.0, le=1.0)
    topics: list[str]
    explanation: str

class FallbackAnalysis(BaseModel):
    """Simpler schema with fewer constraints for retry attempts."""
    sentiment: str
    explanation: str

def analyze_with_guardrails(text: str, max_retries: int = 2) -> dict:
    # Try strict schema first
    for attempt in range(max_retries):
        try:
            completion = client.beta.chat.completions.parse(
                model="gpt-4o-2024-08-06",
                messages=[
                    {"role": "system", "content": "Analyze this text. Be precise with your confidence score."},
                    {"role": "user", "content": text},
                ],
                response_format=StrictAnalysis,
            )

            if completion.choices[0].message.refusal:
                logger.warning(f"Model refused on attempt {attempt + 1}")
                continue

            result = completion.choices[0].message.parsed
            return result.model_dump()

        except ValidationError as e:
            logger.warning(f"Validation failed on attempt {attempt + 1}: {e}")
            continue

    # Fall back to simpler schema
    logger.info("Falling back to simple schema")
    try:
        completion = client.beta.chat.completions.parse(
            model="gpt-4o-2024-08-06",
            messages=[
                {"role": "system", "content": "Analyze this text briefly."},
                {"role": "user", "content": text},
            ],
            response_format=FallbackAnalysis,
        )

        if completion.choices[0].message.parsed:
            result = completion.choices[0].message.parsed
            return {
                "sentiment": result.sentiment,
                "confidence": 0.5,  # Low confidence for fallback
                "topics": [],
                "explanation": result.explanation,
                "fallback": True,
            }
    except (ValidationError, Exception) as e:
        logger.error(f"Fallback also failed: {e}")

    # Last resort: safe default
    return {
        "sentiment": "unknown",
        "confidence": 0.0,
        "topics": [],
        "explanation": "Analysis could not be completed",
        "fallback": True,
        "error": True,
    }

output = analyze_with_guardrails("This product is amazing but overpriced.")
print(output)

The retry strategy works because model outputs are non-deterministic. A second call with the same prompt often succeeds where the first failed. The fallback schema drops the fields most likely to cause validation issues (numeric ranges, list constraints). The safe default ensures your application never crashes regardless of what happens upstream.

Common Errors and Fixes

BadRequestError: Invalid schema for response_format OpenAI’s structured outputs don’t support every Pydantic feature. Union types, recursive models, and dict with arbitrary keys are not allowed. Stick to BaseModel, list, Optional, Enum, and primitive types. If you need a dict, use a list of key-value pair models instead.

ValidationError on a response that looks correct Check your Field constraints. A ge=0.0, le=1.0 constraint on a float will reject 1.0000001, which the model sometimes produces. Give yourself small margins: use le=1.01 if you want to allow minor floating-point drift, or round in a validator.

AttributeError: 'NoneType' object has no attribute 'sentiment' You’re accessing .parsed without checking .refusal. If the model refuses the request, .parsed is None. Always check message.refusal first.

Model ignores field descriptions Field descriptions are passed as part of the JSON schema but they’re hints, not hard constraints. If you need strict values, use Enum types or Literal instead of relying on description text alone.

TypeError: Object of type Enum is not JSON serializable When serializing results, use result.model_dump(mode="json") instead of result.model_dump(). The mode="json" flag converts enums to their string values.

Response is slow with large schemas Structured output adds latency because the model must constrain its token generation. Keep schemas under 15-20 fields. If you need more, split into multiple calls with smaller schemas and combine the results.