Skip to content

plan.md

Latest commit

 

History

History
307 lines (219 loc) · 10.2 KB

plan.md

File metadata and controls

307 lines (219 loc) · 10.2 KB
Raw

pAIge — Synthetic Document Generation Pipeline

An AI-powered pipeline that generates realistic synthetic PDF documents from text prompts and/or reference images. Uses LLMs (Gemini / local) to design layouts, ReportLab to build PDFs, Faker to populate data, and Augraphy for visual post-processing.

Tech Stack

  • API: FastAPI + Uvicorn
  • LLM: Gemini (online) / local model via LangChain
  • PDF: ReportLab
  • Data: Faker
  • Augmentation: Augraphy
  • Validation: Pydantic

Pipeline Breakdown

graph TD
    A["User Input (Text/Image)"] --> B{"Manifest Provided?"}
    B -- No --> C["Step 1: Architect (LLM)"]
    C --> D["Step 2: Pydantic Validation"]
    D -- "Fail" --> E["Step 3: Auditor (LLM Correction)"]
    E --> D
    D -- "Pass" --> F["Step 4: Template Building (ReportLab)"]
    B -- "Yes" --> F
    F --> G["Step 5: Data Filling (Faker/LLM)"]
    G --> H["Step 6: Document Rendering (PDF)"]
    H --> I{"Apply Filters?"}
    I -- "Yes" --> J["Step 7: Visual Rendering (Augraphy)"]
    J --> K["Final Output (PDF/Image)"]
    I -- "No" --> K
    K --> L["Step 8: Manifest Export (JSON)"]
Loading
  1. Schema Generation: The Architect LLM takes the user prompt or reference image and generates a structured DocumentManifest (JSON).
  2. Validation & Correction: Pydantic validates the JSON. If it fails, the Auditor LLM is called with the error log to fix the manifest.
  3. Template Engineering: The manifest is translated into a ReportLab layout object with defined frames, tables, and styles.
  4. Synthetic Data Injection: Placeholders in the layout are filled using Faker (or the Data Synth LLM) based on field types defined in the manifest.
  5. PDF Generation: The filled layout is rendered into a standard PDF byte stream.
  6. Visual Augmentation (Optional): The PDF is rasterized, processed through Augraphy (scanned effect, stains, folds), and converted back to PDF.
  7. Export: The final document is saved, and the manifest is tagged with a UUID and exported for reproducibility.

System Prompts (Personas)

  1. Architect — generates a JSON manifest (DocumentManifest) based on user input and/or reference image
  2. Auditor — reviews a manifest + validation errors and returns a corrected version (retry loop)
  3. Data Synth (optional) — generates domain-aware synthetic data via LLM instead of Faker

Open Design Questions

Decisions needed before implementation:

  1. Should Data Synth persona use the LLM for domain-aware data, or is Faker sufficient?
  2. How many Auditor retry loops before failing? (suggested: 3)
  3. What DPI for the render step's PDF → image → PDF roundtrip? (suggested: 300)
  4. Should render() also support returning images directly (useful for ML training), or always PDF?

Project Structure

pAIge/
├── app/
│   ├── main.py              # FastAPI app + route definitions
│   ├── config.py             # Settings (API keys, defaults, DPI, retry limits)
│   ├── models/
│   │   ├── schemas.py        # Pydantic: manifest, API request/response
│   │   └── filters.py        # Pydantic: filter config
│   ├── services/
│   │   ├── pipeline.py       # generate() orchestrator
│   │   ├── schema_gen.py     # generate_schema()
│   │   ├── template.py       # build_template()
│   │   ├── data_fill.py      # layout_data()
│   │   ├── builder.py        # build_document()
│   │   ├── renderer.py       # render()
│   │   └── manifest.py       # export_manifest()
│   ├── llm/
│   │   ├── client.py         # query_model()
│   │   └── prompts.py        # System prompts
│   └── utils/
│       └── errors.py         # Custom exception types
├── tests/
│   ├── test_pipeline.py
│   ├── test_schema_gen.py
│   ├── test_template.py
│   ├── test_data_fill.py
│   ├── test_builder.py
│   ├── test_renderer.py
│   └── test_manifest.py
├── requirements.txt
└── README.md

API Routes

POST /generate

Main endpoint that builds documents.

Expected input (GenerateRequest):

Field Type Required Default
prompt str optional —
reference_image UploadFile optional —
mode str no "online"
file_path str yes —
apply_filter bool no false
filter_config dict[str, float] no —
count int no 1
manifest DocumentManifest optional —

Edge case: error if both prompt and reference_image are empty (and no manifest provided).

Flow:

for i in range(count):
    if manifest is empty:
        manifest = generate_schema(prompt, reference_image)

    layout   = build_template(manifest)
    filled   = layout_data(layout, manifest)
    pdf      = build_document(filled)

    if apply_filter:
        pdf = render(pdf, filter_config)

    save pdf to file_path

manifest_id = export_manifest(manifest)
return GenerateResponse(file_paths, manifest_id)

GET /health

Health check endpoint.

Functions

generate_schema(prompt?, reference_image?) → DocumentManifest

Generates a validated JSON manifest via LLM.

call query_model(ARCHITECT, prompt, image)
validate with Pydantic (DocumentManifest)

if valid:
    return manifest
else:
    for attempt in range(MAX_AUDITOR_RETRIES):
        append validation errors to context
        call query_model(AUDITOR, manifest + errors)
        validate again
        if valid: return manifest
    raise SchemaGenerationError

query_model(persona, mode, text_input?, image_input?) → dict

Dispatches to online (Gemini) or local model via LangChain.

Input:

  • persona — system prompt (ARCHITECT, AUDITOR, or DATA_SYNTH)
  • mode — "online" or "offline"
  • text_input — user text prompt (optional)
  • image_input — reference image (optional)

Edge case: raise ValueError if both text and image are empty.

build_template(manifest: DocumentManifest) → ReportLabLayout

Translates manifest elements into ReportLab Flowable / Frame / PageTemplate objects. Returns a structured layout (not yet a PDF).

Input: JSON manifest Output: ReportLab layout object

layout_data(layout: ReportLabLayout, manifest: DocumentManifest) → ReportLabLayout

Populates placeholder slots in the layout with synthetic data.

Input: ReportLab layout + manifest (for field type hints) Output: Filled ReportLab layout

Uses Faker, seeded by manifest field types (text, numbers, dates, addresses, etc.).

build_document(filled_layout: ReportLabLayout) → bytes

Renders the filled ReportLab layout to PDF bytes.

Input: Filled ReportLab layout Output: PDF byte stream

Note: only needs the filled layout, not the manifest.

render(pdf_bytes: bytes, filter_config: dict, dpi: int) → bytes

Applies Augraphy visual augmentations.

Input: PDF bytes + filter config {filter_name: intensity} + DPI Output: Modified PDF bytes

convert PDF to image (via pdf2image at configured DPI)
map filter_config keys to Augraphy augmentation classes
apply augmentations in order
convert image back to PDF

⚠️ This is a lossy operation — text selectability and vector graphics are lost (intentional for training-data use cases).

build_document(filled_layout: ReportLabLayout) → bytes

Renders filled ReportLab layout to PDF.

export_manifest(manifest: DocumentManifest, output_dir: str) → str

Assigns a UUID, writes manifest to disk as JSON, returns the UUID for reproducibility.

Error Handling

PaigeError (base)
├── SchemaGenerationError   — LLM/validation failures after retries
├── TemplateError           — ReportLab layout construction failures
├── BuildError              — PDF rendering failures
└── RenderError             — Augraphy/image conversion failures

Each error carries context (manifest, step, attempt number). FastAPI exception handlers map these to HTTP status codes.

Data Models

DocumentManifest

The core schema the LLM must produce. Strict Pydantic validation.

  • Page dimensions (width, height, units)
  • Elements list: text blocks, tables, images, lines, shapes
  • Each element: type, position (x, y, w, h), font, size, style, content placeholder

FilterConfig

  • Ordered dict: {augraphy_filter_name: intensity}
  • Validator ensures names match valid Augraphy augmentation classes
  • Intensity range: 0.0–1.0

Verification Plan

Automated Tests

pytest tests/ -v
Test File Covers
test_schema_gen.py Pydantic validation pass/fail, Auditor retry loop, exhaustion
test_template.py Manifest → ReportLab layout for various element types
test_data_fill.py Faker populates correct field types; empty manifest edge case
test_builder.py Produces valid PDF bytes (check %PDF magic bytes)
test_renderer.py Augraphy filter application; invalid filter name rejection
test_manifest.py UUID assignment; JSON file written to disk
test_pipeline.py Full end-to-end (mocked LLM); count > 1; filter on/off

LLM calls are mocked in tests to avoid API costs and ensure determinism.

Manual Smoke Test

  1. Start: uvicorn app.main:app --reload
  2. Open: http://localhost:8000/docs
  3. Send POST /generate with prompt: "Generate a single-page invoice with header, company logo, line items table, and total"
  4. Verify: output PDF opens correctly, layout matches prompt
  5. Test with apply_filter=true, confirm output looks visually augmented