Text-First Extraction Is an Epistemic Error

Production RAG pipelines degrade in a pattern consistent enough to be structural: a PDF parser runs upstream of everything, confidently destroying the document's semantic structure before a single token reaches the model. The parser returns text. The text looks plausible. Nobody checks what was lost. And then retrieval quality is blamed on the embedding model, the chunking strategy, or the prompt.

The parser is the problem. More precisely: the assumption that a PDF is a text container is the problem. A PDF is a visual layout specification. Treating it as text-first is an epistemic error that compounds through every downstream stage.

The PDF Is Not a Document. It Is a Rendering Instruction.

PDF (Portable Document Format) was designed in 1993 to guarantee visual fidelity across printers and screens. Its internal representation is a stream of drawing commands: place this glyph at coordinate (x, y), draw this line, render this image at this bounding box. There is no semantic layer. No concept of "this is a table header" or "this text is a caption for the figure above it". Those relationships exist only in the visual arrangement — in the two-dimensional geometry that a human eye resolves instantly and that a text extractor discards entirely.

When pdfminer or Tika extracts text from a PDF, it reconstructs a linear character sequence from those drawing commands using heuristics: sort glyphs by vertical position, group them into lines, concatenate. For a single-column prose document, this works. For the actual documents that matter in enterprise pipelines — financial reports, technical specifications, regulatory filings, research papers — it produces structured noise.

This is not a new problem disguised as new vocabulary. OCR, layout-aware parsing, document AI, and information extraction have always had to decide whether geometry is part of the data. The LLM-specific mistake is moving that decision upstream of the model: flatten the page first, then ask a language model to reason over the flattened residue.

Four Ways Text-First Extraction Destroys Semantic Structure

The failures are not edge cases. They are the predictable consequence of applying a linearization algorithm to a two-dimensional layout specification.

Layout collapse is the most common. A two-column regulatory document, when linearized, interleaves the left and right columns by vertical position. The resulting text is grammatically coherent but semantically incoherent — sentences from unrelated sections merged into the same paragraph, mid-sentence.

Table destruction is the most damaging for retrieval. A financial table with 12 columns and 40 rows becomes a flat string of numbers with no structural relationship to their headers. A RAG system asked "what was the Q3 operating margin?" retrieves the number — but without the column header that identifies it as operating margin rather than gross margin. The answer is confidently wrong.

Figure blindness is silent. Charts, diagrams, and graphs produce no text output at all. The extraction succeeds with no error. The data encoded in those figures simply does not exist in the vector store. A retrieval system built on this index has a systematic blind spot it cannot report.

Reading order inversion is the subtlest. Footnotes, sidebars, callout boxes, and captions are positioned by their visual relationship to the content they annotate. A text extractor sorts them by y-coordinate, injecting them into the middle of body paragraphs. The semantic relationship — "this footnote qualifies that claim" — is inverted into noise.

Text-first PDF extraction does not degrade gracefully. It produces four distinct failure modes — layout collapse, table destruction, figure blindness, and reading order inversion — each of which is invisible to the pipeline stages that follow.

Document Type	Text-First Quality	Dominant Failure Mode	Vision-First Advantage
Single-column prose	Acceptable	Minimal	Marginal
Multi-column report	Poor	Layout collapse	High
Financial statement	Unusable	Table destruction	Critical
Technical specification	Poor	Figure blindness	High
Regulatory filing	Poor	Reading order inversion	High

The Vision-First Pipeline: Code as Specification

The vision-first approach treats the PDF as what it is: a visual artifact. Extraction begins by rasterizing each page to an image — 72 dpi is sufficient for text-dense documents, 150 dpi for documents with fine-grained tables or diagrams. Those images are then routed to a vision-capable model (Claude Sonnet 4.6 or Gemini 3 Flash are the current practical choices for throughput at cost) with a structured extraction prompt. The model returns JSON conforming to a Pydantic schema. The schema is the contract.

This is the architectural shift that matters — Code as Specification applied to document pipelines. The output of extraction is no longer raw text. It is a validated data structure. A table is a Table object with headers, rows, and caption. A figure is a Figure object with a description and data_summary. A section is a Section object with title, body, and reading_level. Semantic loss is no longer a silent degradation — it is a schema validation failure. The pipeline fails loudly instead of succeeding quietly with garbage.

This pattern inverts the conventional architecture: Code is the Constitution, the model is the Subject. The extraction prompt tells the model what to produce; the Pydantic schema specifies what output is structurally admissible. The model cannot return a table without headers because a headerless table is unrepresentable in the schema. Hallucinated structure — a predictable consequence of non-deterministic generation — fails validation at extraction time — not silently at retrieval time three weeks later.

import base64
from pathlib import Path
from typing import Optional
import anthropic
from pdf2image import convert_from_path
from pydantic import BaseModel, Field


class TableBlock(BaseModel):
    headers: list[str]
    rows: list[list[str]]
    caption: Optional[str] = None


class FigureBlock(BaseModel):
    description: str
    data_summary: Optional[str] = None


class SectionBlock(BaseModel):
    title: Optional[str] = None
    body: str
    tables: list[TableBlock] = Field(default_factory=list)
    figures: list[FigureBlock] = Field(default_factory=list)


class PageExtraction(BaseModel):
    page_number: int
    sections: list[SectionBlock]
    reading_order_notes: Optional[str] = None


EXTRACTION_PROMPT = """
Extract the semantic structure of this document page as JSON.
Preserve table structure with headers and rows intact.
Describe figures and charts with a data_summary field.
Respect reading order: body text, then captions, then footnotes.
Return only valid JSON matching the PageExtraction schema.
"""


def extract_page_vision_first(
    pdf_path: Path,
    page_number: int,
    client: anthropic.Anthropic,
    dpi: int = 150,
) -> PageExtraction:
    images = convert_from_path(
        pdf_path,
        dpi=dpi,
        first_page=page_number,
        last_page=page_number,
    )
    image = images[0]

    import io
    buffer = io.BytesIO()
    image.save(buffer, format="PNG")
    image_b64 = base64.standard_b64encode(buffer.getvalue()).decode("utf-8")

    response = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=4096,
        messages=[
            {
                "role": "user",
                "content": [
                    {
                        "type": "image",
                        "source": {
                            "type": "base64",
                            "media_type": "image/png",
                            "data": image_b64,
                        },
                    },
                    {"type": "text", "text": EXTRACTION_PROMPT},
                ],
            }
        ],
    )

    raw = response.content[0].text
    # Strip markdown fences if the model wraps JSON in ```json ... ```
    if raw.startswith("```"):
        raw = raw.split("```")[1]
        if raw.startswith("json"):
            raw = raw[4:]

    return PageExtraction.model_validate_json(raw.strip())

Two implementation details matter. First, the dpi parameter is a tunable cost lever: lower-resolution images are often enough for prose-heavy pages, while fine-grained tables usually need higher-resolution rendering. The right value is corpus-dependent and should be measured against a small labeled extraction set. Second, the Pydantic validation call is not optional — it is the mechanism that converts silent failure into loud failure. A model that returns malformed JSON fails at the extraction stage, not silently at retrieval time three weeks later.

The architectural shift in vision-first ETL is not the vision model — it is the schema contract. Replacing raw text output with a validated Pydantic schema converts semantic loss from a silent degradation into a hard pipeline failure.

Where Vision-First Breaks

The approach has real constraints that determine where it belongs in a pipeline portfolio.

Latency and cost scale with page count. A vision model call per page is materially more expensive than a text extraction call. For a 400-page technical manual, this is material. The mitigation is a routing layer: run a fast structural classifier first (does this page contain tables or figures?) and route text-only pages through a conventional extractor. For mixed corpora, a routing classifier can remove many unnecessary vision calls, but the ratio depends heavily on corpus composition.

Scanned documents with poor scan quality degrade accuracy. A vision model reading a low-quality scan of an old financial report may hallucinate table structure. The schema validation catches gross failures, but subtle misreadings — a "1" read as "7" in a financial table — are not catchable without a ground-truth comparison layer. This is a known limitation of the approach, not a reason to avoid it, but it requires explicit acknowledgment in any system that handles archival or low-quality scans.

Handwritten annotations are partially handled. Current vision models (Claude Sonnet 4.6, Gemini 3 Flash) read printed text with high accuracy. Handwritten marginalia are transcribed with variable accuracy and should be treated as best-effort, not authoritative, in the schema output.

The schema is a design artifact, not a given. The quality of extraction is bounded by the quality of the schema. A PageExtraction schema that has no FigureBlock type will cause the model to either ignore figures or force them into SectionBlock.body as prose descriptions. Schema design is the highest-impact engineering decision in a vision-first pipeline, and it requires domain knowledge about the document corpus — not just software architecture.

The Upstream Fix That Unlocks Everything Downstream

Chunking strategy, embedding model selection, hybrid search, reranking — these are all downstream optimizations. They compound on whatever extraction quality they receive. A vision-first pipeline with a well-designed schema gives every downstream stage clean, structured, semantically coherent inputs. A text-first pipeline with a sophisticated reranker is still retrieving from an index built on structural noise.

The structural noise isn't passive. Every malformed token — a number severed from its column header, a caption injected mid-paragraph — occupies attention surface that would otherwise carry semantic signal. Rerankers and hybrid search operate downstream of this damage; they cannot recover structure that was destroyed before they ran. The garbage goes in, and the garbage compounds.

The principle generalizes beyond PDFs. Any ingestion pipeline that converts a rich, structured artifact into a flat text stream before applying intelligence is making the same epistemic error: destroying information at the source and then trying to recover it downstream. The fix is always the same — move the intelligence to the extraction boundary, not the retrieval boundary.

The deeper question this architecture raises is about the retrieval contract itself. If extraction now produces typed, structured objects — TableBlock, FigureBlock, SectionBlock — then embedding them all into the same undifferentiated vector space is still discarding structure. A table and a paragraph are not the same semantic unit and should not be retrieved by the same mechanism. The vision-first pipeline solves the extraction problem. What it reveals is that the retrieval architecture has been assuming a flat document model that was never true.

That's the next problem.

Related Reading

• The Golden Dataset Problem — ETL quality determines whether your golden dataset is actually golden
• Semantic Compression — Vision-first ETL is semantic compression applied to document pipelines
• The Agentic Contract — ETL pipelines need contracts between extraction and loading stages