Web Data for LLM Fine-Tuning

Collect real web data with Zenrows, validate it, and format it into fine-tuning pairs for extraction, preference, and domain-continuation training.

Features

• Fetches clean Markdown and structured JSON from web pages using the Zenrows API
• Validates extracted data against a JSON schema, dropping empty extractions, schema mismatches, and duplicates before they reach the training set
• Formats validated pairs into instruction-output (SFT) and domain-continuation fine-tuning formats
• Fetches and extracts text from PDF sources, including the js_render requirement PDF mode needs
• Chunks long PDF documents by page for use in fine-tuning pairs
• Tracks source URL, retrieval date, and license per record for auditability

Prerequisites

• Python 3.9 or later
• A Zenrows API key

Installation

1. Clone the repository

git clone https://github.com/ZenRows/web-data-for-llm-fine-tuning.git
cd web-data-for-llm-fine-tuning

2. Create a virtual environment

python -m venv .venv

3. Activate the environment

# macOS/Linux
source .venv/bin/activate

# Windows
.venv\Scripts\activate

4. Install dependencies

pip install -r requirements.txt

Configuration

Create a .env file in the project root:

ZENROWS_API_KEY=your_zenrows_api_key

Project structure

.
├── validate.py          # Collect + validate one HTML source into training_data.jsonl
├── domain.py             # Same as validate.py, plus a domain-continuation example
├── pdf.py                 # Basic PDF fetch
├── pdf_2.py               # PDF fetch with retry and timeout handling
├── pdf_chunking.py        # PDF fetch + page-based chunking
├── pdf_metadata.py        # PDF fetch + license lookup + provenance metadata
├── training_data.jsonl    # Output: formatted fine-tuning pairs
├── requirements.txt
└── README.md

How it works

Source URL
   ↓
Zenrows Fetch (Markdown for pages, PDF for documents)
   ↓
Zenrows Extract (CSS schema, HTML sources only)
   ↓
Validation (schema check, length check, duplicate check)
   ↓
Formatted fine-tuning pair (JSONL)

For HTML sources, validate.py fetches the page twice: once for Markdown (the training input), once with a CSS extraction schema for structured JSON (the training label). Both are checked with keep() before being written out. For PDF sources, pdf.py and its variants fetch the raw file and extract text with pypdf as a separate step, since Zenrows returns the file, not its contents.

Running the project

python3 validate.py         # collect + validate a single HTML source
python3 domain.py           # same, plus a domain-continuation example
python3 pdf.py               # fetch a PDF source
python3 pdf_2.py             # fetch a PDF source with retry/timeout handling
python3 pdf_chunking.py      # fetch a PDF and split it into page-based chunks
python3 pdf_metadata.py      # fetch a PDF plus its license from the source's landing page

Each script is standalone. Swap the urls list and css_schema in validate.py/domain.py for your own target site — selectors are site-specific and won't carry over as-is.

Output

• training_data.jsonl — one JSON object per line, each with messages (system instruction, page content, extracted JSON)
• who-clinical-trials-guidance.pdf / public-health-references.pdf — PDF files saved locally by the PDF scripts
• Console output reporting how many candidates were collected versus how many passed validation

Technologies

• Python
• Zenrows
• jsonschema
• pypdf
• python-dotenv
• requests

Data sources

• priceoye.pk — e-commerce product pages (HTML extraction example)
• WHO clinical trials guidance — CC BY-NC-SA 3.0 IGO (PDF example)
• CDC public health references — U.S. federal publication, public domain (PDF example)

Related article

This repository accompanies this blog: Web Data for LLM Fine-Tuning: A Practical 2026 Guide