Battle-tested Pydantic schemas for LLM data extraction -- e-commerce, jobs, real estate, news, and finance.
LLM extraction fails silently. Wrong types, missing fields, hallucinated dates. These schemas are battle-tested on 100+ real websites and designed to catch the errors that LLMs produce when extracting structured data from unstructured text.
Large Language Models are surprisingly good at reading a webpage and pulling out structured data -- a product name, a price, a job title. But without strict validation, they hallucinate fields, invent prices with wrong decimal places, return dates in formats your code cannot parse, and silently drop required information. The result is data that looks correct at first glance but breaks downstream pipelines, corrupts databases, and produces wrong analytics.
This library provides a set of Pydantic v2 schemas purpose-built for LLM extraction. Every schema includes field validators that handle the messy reality of LLM output: price strings with currency symbols, salary ranges written as "80k-120k", dates in a dozen different formats, and optional fields that may or may not appear depending on the source. The schemas enforce types, ranges, and formats at the validation boundary so your application code never has to guess.
Extract a product in five lines:
from schemas.ecommerce.product import Product, ProductPrice
from decimal import Decimal
product = Product(
name="Wireless Headphones",
price=ProductPrice(amount="$79.99", currency="USD"),
rating=4.3,
review_count=1247,
categories=["Electronics", "Audio"],
brand="SoundMax",
)
print(product.model_dump_json(indent=2))The ProductPrice validator automatically strips the dollar sign from "$79.99" and converts it to a Decimal. If the LLM returns a rating of 6.0 (outside the 0-5 range), Pydantic raises a ValidationError immediately rather than letting bad data propagate.
| Domain | Schema | Key Fields | Validators |
|---|---|---|---|
| E-commerce | Product, ProductPrice |
name, price, rating, categories, brand | Currency symbol stripping, rating bounds (0-5), discount calculation |
| Jobs | JobPosting, Salary |
title, company, salary, requirements, remote | Salary "k" notation (80k -> 80000), currency cleaning |
| Real Estate | PropertyListing |
title, price, bedrooms, bathrooms, area | Price string cleaning, non-negative constraints |
| News | Article |
title, author, published_date, content, tags | Multi-format date parsing (ISO, American, European) |
| Finance | StockQuote |
symbol, price, change, volume, market_cap | Symbol length validation, non-negative volume |
Every schema in this library leverages Pydantic v2 features for maximum reliability:
Field Validators handle the gap between what LLMs produce and what your code expects. For example, the ProductPrice.clean_price validator runs before type coercion and strips currency symbols, commas, and whitespace from price strings. This means "$1,299.99", "EUR 1299.99", and "1299.99" all produce the same Decimal("1299.99").
Field Constraints use Pydantic's built-in Field parameters to enforce business rules. Product ratings are constrained to ge=0.0, le=5.0. Review counts must be non-negative. Stock symbols have a maximum length of 10 characters. These constraints catch obvious LLM errors at parse time.
Optional Fields with Defaults reflect the reality that LLMs do not always extract every field. Most fields are optional with sensible defaults (empty lists for categories, None for missing descriptions). Only truly required fields like product name and price are mandatory.
Computed Properties like ProductPrice.discount_pct derive useful values from the raw extracted data without requiring the LLM to calculate them (which it would likely get wrong).
The library includes an extractor framework for calling LLM APIs directly. The ClaudeExtractor sends your text to the Anthropic API along with the schema definition and returns a validated Pydantic model:
from extractors.claude_extractor import ClaudeExtractor
from schemas.ecommerce.product import Product
extractor = ClaudeExtractor(api_key="your-key")
product = await extractor.extract(raw_html_text, Product)The BaseExtractor abstract class defines the interface so you can implement extractors for other providers (OpenAI, Gemini, local models) with the same API.
Price Cleaning: Currency symbols ($, EUR, GBP), thousands separators (commas), and whitespace are all stripped before conversion to Decimal. This handles the wide variety of price formats LLMs produce.
Date Parsing: The Article schema tries four date formats in sequence: ISO (2026-04-07), ISO with time (2026-04-07T14:30:00), American English (April 7, 2026), and European (07/04/2026). If none match, it passes the value through for Pydantic's default datetime parsing.
Salary Notation: Job salary fields accept "k" notation ("80k" becomes 80000.0), dollar signs, commas, and euro signs. The validator handles all combinations so your code always gets a clean float.
Missing Data: All schemas are designed with the assumption that LLMs will miss fields. Required fields are kept to the absolute minimum (usually just a name/title and a price/symbol). Everything else is optional.
Creating a new schema follows this pattern:
- Create a new directory under
schemas/for your domain (e.g.,schemas/healthcare/). - Add an
__init__.pyfile. - Define your Pydantic model in a module file (e.g.,
patient_record.py). - Add
field_validatordecorators for any fields that need cleaning (prices, dates, encoded values). - Use
Field()constraints for business rules (ranges, string lengths, patterns). - Export from the top-level
schemas/__init__.py. - Add tests in
tests/covering valid data, edge cases, and validation errors.
Example skeleton:
from pydantic import BaseModel, Field, field_validator
class PatientRecord(BaseModel):
patient_id: str = Field(min_length=1)
diagnosis: str
medications: list[str] = Field(default_factory=list)
@field_validator("patient_id", mode="before")
@classmethod
def clean_id(cls, v):
if isinstance(v, str):
return v.strip().upper()
return vThe utils.schema_validator module provides helpers for testing and introspection:
validate_with_sample(schema, data)-- returns(True, instance)or(False, error_string)for safe validation without exceptions.get_required_fields(schema)-- lists all required field names.get_optional_fields(schema)-- lists all optional field names.
These are useful for building dynamic UIs, generating documentation, or writing parameterized tests.
The ClaudeExtractor currently makes a single API call and parses the response. If the LLM returns malformed JSON (missing closing braces, trailing commas, text outside the JSON block), json.loads will raise an exception. For production use, consider adding retry logic that feeds the validation error back to the LLM and asks it to fix the output. Libraries like Instructor automate this retry loop and also support constrained decoding for more reliable structured output.
- Instructor -- A popular library that patches LLM client libraries (OpenAI, Anthropic, etc.) to return validated Pydantic models directly. It handles retries, partial streaming, and function-calling mode automatically. If you want a batteries-included solution rather than building your own extractor, Instructor is the standard choice.
- Tool/function calling -- Most LLM APIs now support structured output via tool-use or function-calling modes, which constrain the output to valid JSON matching a schema. This is more reliable than asking the model to produce JSON in a text response.
- secure-mcp-boilerplate -- Production-ready MCP server template with security best practices.
- python-mcp-server-starter -- Minimal Python MCP server for tool integration.
pip install -e ".[dev]"pytest tests/ -vruff check schemas/ extractors/ utils/ tests/MIT License 2026.