Validate AI Output with Pydantic Before It Reaches Your CRM

An AI assistant can return valid JSON and still put the wrong information into your business system. A budget might arrive as a string, a response might belong to another request, or a plausible summary might claim that a customer agreed to something they never said.

A useful application needs a boundary between the model’s suggestion and the operation that changes business records. This tutorial builds that boundary with Pydantic: validate the shape, check the request and source identifiers, then produce a candidate for human review. You can reuse the pattern for enquiry triage, project briefs or internal document extraction.

Fictional example: Harbor Studio and its enquiry are invented. The sample response was written as a test fixture; it is not output from a benchmarked model. The code below was executed with Python 3.14.3 and Pydantic 2.13.4 on Windows on September 15, 2026. No customer system or paid model API was used.

Separate three questions that often get mixed together

Pydantic provides typed models and configurable field handling. Its validation checks the resulting structure against declared constraints; that is different from verifying the truth of a summary. Pydantic model documentation.

The application should supply the expected request identifier from its own job record. It should also supply the set of source identifiers from the documents actually given to the model. Do not derive those trusted values from the response you are trying to validate.

Make unknown information explicit

The fictional source is short:

Fictional source, identifier message-0042:
We are Harbor Studio. We want a shared place to review new website enquiries.
We have not agreed a budget yet. Please prepare a proposal for discussion.

The proposed response includes budget_usd: null. That means the budget is unknown. It is different from a zero-dollar budget and different from omitting the field entirely. This contract requires the field so a downstream screen can display “Not supplied” deliberately.

{
  "request_id": "req-0042",
  "service": "automation",
  "summary": "Harbor Studio wants a shared place to review new website enquiries.",
  "budget_usd": null,
  "evidence_ids": ["message-0042"]
}

The service vocabulary is intentionally small. unknown is a valid result when the text does not fit. A limited list of categories is easier to route consistently than a free-form service label that changes with every response.

Use strict types and reject extra fields

Pydantic normally permits useful conversions, such as a numeric string becoming an integer. Strict mode reduces that coercion; exact behavior depends on the type and whether validation starts from Python objects or JSON. This example parses JSON first and validates the resulting Python object with strict model settings. Pydantic strict mode.

The model also uses extra="forbid". An unexpected field such as approved: true becomes an error rather than quietly joining a future workflow. The returned needs_human_review status is created by application code, outside the model’s response.

Save this as validate_output.py, and save the response above as response.json beside it:

"""Validate fictional AI output; return a proposal for review, never a CRM write."""
import json
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field


class Proposal(BaseModel):
    model_config = ConfigDict(strict=True, extra="forbid")
    request_id: str = Field(pattern=r"^req-[0-9]{4}$")
    service: Literal["website", "automation", "training", "unknown"]
    summary: str = Field(min_length=1, max_length=400)
    budget_usd: int | None = Field(ge=0, le=1_000_000)
    evidence_ids: list[str] = Field(min_length=1, max_length=8)


def unique_object(pairs):
    obj = {}
    for key, value in pairs:
        if key in obj:
            raise ValueError("Duplicate JSON field")
        obj[key] = value
    return obj


def review_candidate(raw, expected_request_id, available_evidence_ids):
    if len(raw.encode("utf-8")) > 16_384:
        raise ValueError("Response exceeds 16 KiB")
    data = json.loads(raw, object_pairs_hook=unique_object)
    proposal = Proposal.model_validate(data)
    if not proposal.summary.strip():
        raise ValueError("Summary is blank")
    if proposal.request_id != expected_request_id:
        raise ValueError("Response belongs to another request")
    if not set(proposal.evidence_ids) <= set(available_evidence_ids):
        raise ValueError("Evidence identifier is not in the supplied source set")
    # The application owns workflow state; a model cannot supply approval.
    return {"status": "needs_human_review", "proposal": proposal.model_dump()}


if __name__ == "__main__":
    from pathlib import Path
    sample = Path(__file__).with_name("response.json").read_text(encoding="utf-8")
    print(json.dumps(review_candidate(sample, "req-0042", {"message-0042"}), indent=2))

The JSON parser rejects duplicate field names. Without that check, a repeated field could be interpreted differently by different components. The byte limit bounds the accepted response size; a real network adapter should also limit how much it reads before this function receives the text.

Run it in an isolated Python environment

Create a project folder and a virtual environment. These Windows commands use the same Pydantic version as the executed example:

python -m venv .venv
.venv\Scripts\python.exe -m pip install pydantic==2.13.4
.venv\Scripts\python.exe validate_output.py

On macOS or Linux, use .venv/bin/python in place of .venv\Scripts\python.exe. The validation and test commands were run against the available Windows installation; the other operating systems were not tested for this article.

The result starts with "status": "needs_human_review". It includes the typed proposal and retains the unknown budget. Nothing is written to a CRM, no message is sent, and the application does not interpret a summary as permission to act.

Try changing the budget to "5000", adding an approval field, or replacing the evidence identifier with invented. Those changes should fail. Changing the budget to the integer 5000 can pass the type check, which is exactly why a type check cannot establish that the budget was actually agreed.

Test both the protection and its limits

The complete suite passed 10 tests. It checks valid output, string and Boolean budgets, negative budgets, unexpected services, attempted approval fields, wrong request IDs, unknown evidence, missing fields, blank text, duplicate keys, malformed JSON and oversized responses. One test deliberately supplies a false summary that has the right structure. It passes structural validation and still ends in human review.

import json
from pathlib import Path
import unittest
from validate_output import review_candidate


class OutputTests(unittest.TestCase):
    def setUp(self):
        self.data = json.loads(Path(__file__).with_name("response.json").read_text())

    def check(self, data):
        return review_candidate(json.dumps(data), "req-0042", {"message-0042"})

    def test_valid_output_is_only_a_review_candidate(self):
        result = self.check(self.data)
        self.assertEqual(result["status"], "needs_human_review")
        self.assertIsNone(result["proposal"]["budget_usd"])

    def test_string_and_boolean_budgets_are_rejected(self):
        for value in ["5000", True]:
            with self.subTest(value=value), self.assertRaises(ValueError):
                self.check(dict(self.data, budget_usd=value))

    def test_negative_budget_is_rejected(self):
        with self.assertRaises(ValueError):self.check(dict(self.data, budget_usd=-1))

    def test_unapproved_service_is_rejected(self):
        with self.assertRaises(ValueError):self.check(dict(self.data, service="send_payment"))

    def test_model_cannot_supply_approval(self):
        with self.assertRaises(ValueError):self.check(dict(self.data, approved=True))

    def test_wrong_request_is_rejected(self):
        with self.assertRaises(ValueError):self.check(dict(self.data, request_id="req-0001"))

    def test_unknown_evidence_is_rejected(self):
        with self.assertRaises(ValueError):self.check(dict(self.data, evidence_ids=["invented"]))

    def test_missing_and_blank_fields_are_rejected(self):
        missing = dict(self.data);del missing["budget_usd"]
        for data in [missing, dict(self.data, summary="   "), dict(self.data, evidence_ids=[])]:
            with self.subTest(data=data), self.assertRaises(ValueError):self.check(data)

    def test_duplicate_keys_malformed_and_oversized_json_are_rejected(self):
        for raw in ['{"service":"website","service":"automation"}', "not json", " " * 16_385]:
            with self.subTest(raw=raw[:60]), self.assertRaises(ValueError):
                review_candidate(raw, "req-0042", {"message-0042"})

    def test_structural_validation_does_not_prove_truth(self):
        false_claim = dict(self.data, summary="The customer agreed a million-dollar budget.")
        result = self.check(false_claim)
        self.assertEqual(result["status"], "needs_human_review")
        self.assertIn("million-dollar", result["proposal"]["summary"])


if __name__ == "__main__":unittest.main(verbosity=2)

Save the tests beside the other files and run:

.venv\Scripts\python.exe -m unittest -v test_output.py

That false-summary test is a useful reminder of the boundary. An evidence identifier can exist while the claimed meaning is unsupported. For a stronger application, display the source passage beside the extracted field, build an evaluation set of representative enquiries, and measure extraction errors separately from schema failures. Those evaluations require real model outputs; this article does not report any.

Connect it to a business workflow deliberately

Put this function after response collection and before the code that creates a durable proposal. Keep the original response available under an appropriate retention policy so a reviewer can investigate a failure. Record a request ID and an error category in operational logs; avoid dumping full enquiries or validation errors containing customer text into general logs.

Decide which failures deserve a retry. An interrupted response might justify another bounded attempt. An unsupported service or unknown budget may need a person. Repeatedly asking a model to “make it valid” can produce a compliant-looking answer while losing the uncertainty the business needed to see.

Once a person approves a proposal, the write to a CRM or another application should be a separate operation with its own authorization and retry behavior. Schema validation, factual evaluation and permission to act are separate responsibilities. Keeping them separate makes failures easier to explain and changes easier to review.

Jason Ead publishes AlgoThoughts and founded My Biz Heroes, connecting websites, automation and AI tools to business workflows. Explore the topic library for more practical engineering guides.

Research and tests checked September 15, 2026. This article was prepared with AI assistance under the editorial policy. Featured image: an AI-generated conceptual illustration of a measuring frame accepting compatible shapes, with a mismatched shape set aside.