pdf-inspector: The Fast Rust Library With 8,700+ GitHub Stars That Processes PDFs in Under 200ms

Why PDF Processing Is Still a Headache in 2026

PDFs remain the world's most common document format — invoices, research papers, legal contracts, financial reports, and technical specifications all live in PDF. Yet processing them programmatically is notoriously painful.

The core challenge? You never know what you're dealing with until you open the file. Some PDFs are clean, text-based documents that parse in milliseconds. Others are scanned images masquerading as documents, requiring expensive OCR services that take 2-10 seconds per page and still produce imperfect results.

Most pipelines send every PDF through OCR "just in case," burning money and latency on documents that didn't need it. That's where pdf-inspector changes the game.

What Is pdf-inspector?

pdf-inspector is a pure-Rust library built by the team at Firecrawl — the web scraping and data extraction platform. It solves the PDF routing problem with an elegant two-stage approach:

1. Classify first — In 10-50ms, determine whether a PDF is text-based, scanned, image-based, or mixed
2. Extract smartly — For text-based PDFs, extract and convert to Markdown in ~150ms. For scanned ones, route to OCR

The result: you skip OCR entirely for the majority of PDFs, saving both cost and time at scale.

Smart Classification: Know Your PDF Before You Process It

The classification engine is pdf-inspector's killer feature. It samples content streams across pages looking for text operators (Tj/TJ) and image operators (Do), then classifies the document into one of four types:

• TextBased — Standard PDFs with selectable text (reports, papers, invoices)
• Scanned — Image-only PDFs from scanners or fax machines
• ImageBased — PDFs composed primarily of embedded images
• Mixed — Some pages have text, others are scanned

Each classification comes with a confidence score (0.0-1.0) and a per-page OCR routing list — so you can send only the specific pages that need OCR, not the entire document.

import pdf_inspector

result = pdf_inspector.process_pdf("financial_report.pdf")
print(result.pdf_type)        # "text_based"
print(result.confidence)      # 0.97
print(result.pages_needing_ocr)  # [] — no OCR needed!

Text Extraction That Actually Understands Layout

Most PDF text extractors dump characters in file order and hope for the best. pdf-inspector goes further with position-aware extraction:

• Font info and coordinates — Every TextItem includes X/Y position and font metadata
• Multi-column detection — Automatically identifies newspaper-style columns and orders text correctly
• RTL text support — Handles right-to-left languages like Arabic and Hebrew
• CID font support — ToUnicode CMap decoding for Type0/Identity-H fonts, UTF-16BE, UTF-8, and Latin-1
• Encoding issue detection — Flags broken font encodings so you can fall back to OCR

Markdown Conversion: From PDF to LLM-Ready in One Step

The Markdown converter is where pdf-inspector truly shines for AI/LLM pipelines. It produces clean, structured Markdown with:

Element	Detection Method
Headings (H1-H4)	Font size tiers relative to body text
Bold/Italic	Font name pattern matching
Code blocks	Monospace font detection (Courier, Consolas, etc.)
Tables	Rectangle-based + heuristic alignment detection
Lists	Bullet, numbered, and letter list patterns
URLs	Auto-linked as Markdown links

For the AI developer, this means you can feed a research paper or financial report directly into your LLM pipeline as clean, structured Markdown — no messy text blobs.

Benchmark Results: pdf-inspector vs. The Competition

The team benchmarked pdf-inspector against four other local PDF parsers on the opendataloader-bench corpus (200 PDFs) using an Apple M4 Pro. OCR was disabled for all engines to ensure a fair comparison of native text parsing:

Engine	Overall	Reading Order	Tables	Headings	Speed (200 docs)
pdf-inspector	0.875	0.915	0.814	0.788	0.470s
liteparse	0.873	0.913	0.693	0.811	0.750s
opendataloader	0.831	0.902	0.489	0.739	2.569s
pymupdf4llm	0.735	0.886	0.401	0.424	17.117s
markitdown	0.589	0.844	0.273	0.000	16.165s

pdf-inspector leads in overall score, reading order accuracy, table detection, and speed — processing 200 documents in under half a second, 36x faster than pymupdf4llm.

Getting Started: Three Languages, One Library

Python

# Install
# pip install pdf-inspector

import pdf_inspector

# Full processing: classify + extract + convert
result = pdf_inspector.process_pdf("contract.pdf")

print(f"Type: {result.pdf_type}")      # "text_based"
print(f"Markdown:\n{result.markdown}") # Clean Markdown output

# Detection only (fastest — ~20ms)
detection = pdf_inspector.classify_pdf("contract.pdf")
print(f"Needs OCR: {detection.pages_needing_ocr}")

Node.js

// npm install @firecrawl/pdf-inspector

import { readFileSync } from 'fs';
import { processPdf, classifyPdf } from '@firecrawl/pdf-inspector';

const pdfBuffer = readFileSync('report.pdf');

// Full processing
const result = processPdf(pdfBuffer);
console.log(result.pdfType);   // "TextBased"
console.log(result.markdown);  // Clean Markdown string

// Classification only
const detection = classifyPdf(pdfBuffer);
console.log(detection.confidence); // 0.95

Rust (Native)

// cargo add pdf-inspector

use pdf_inspector::process_pdf;

let result = process_pdf("invoice.pdf")?;
println!("Type: {:?}", result.pdf_type);

if let Some(markdown) = &result.markdown {
    println!("{}", markdown);
}

Browser (WASM)

// npm install @firecrawl/pdf-inspector-wasm

import init, { processPdf } from '@firecrawl/pdf-inspector-wasm';

await init();
const response = await fetch('/document.pdf');
const pdf = new Uint8Array(await response.arrayBuffer());
const result = processPdf(pdf);

console.log(result.pdfType);
console.log(result.markdown);

Real-World Example: Building a Smart PDF Pipeline for AI

Let's say you're building an AI-powered document analysis tool that processes thousands of uploaded PDFs daily. Here's how pdf-inspector fits into a production pipeline:

import pdf_inspector

def smart_pdf_pipeline(pdf_path: str) -> str:
    """Process PDF with intelligent OCR routing."""
    
    # Step 1: Classify (~20ms)
    detection = pdf_inspector.classify_pdf(pdf_path)
    
    # Step 2: Route based on type
    if detection.pdf_type == "text_based" and detection.confidence > 0.9:
        # Fast local extraction (~150ms) — no OCR cost
        result = pdf_inspector.process_pdf(pdf_path)
        return result.markdown
    
    elif detection.pdf_type == "mixed":
        # Extract text pages locally, OCR only scanned pages
        result = pdf_inspector.process_pdf(pdf_path)
        ocr_pages = detection.pages_needing_ocr
        
        for page_num in ocr_pages:
            ocr_text = call_ocr_service(pdf_path, page_num)
            # Merge OCR text into markdown
            result.markdown = merge_page(result.markdown, page_num, ocr_text)
        
        return result.markdown
    
    else:
        # Fully scanned — send to OCR
        return call_full_ocr(pdf_path)

# Usage
markdown = smart_pdf_pipeline("quarterly_report.pdf")
send_to_llm(markdown)  # Feed clean Markdown to your AI model

This approach saves significant cost at scale. If you process 10,000 PDFs per day and 54% are text-based, you skip OCR for 5,400 documents — potentially saving hundreds of dollars daily depending on your OCR provider.

Key Benefits

• ⚡ Blazing speed — Classification in 20ms, full extraction in 200ms
• 💰 Cost savings — Skip expensive OCR for 54% of PDFs that don't need it
• 🎯 Smart routing — Per-page OCR decisions, not all-or-nothing
• 🌐 Multi-language bindings — Rust, Python, Node.js, and browser WASM
• 📊 Best-in-class benchmarks — Top scores in reading order, tables, and overall accuracy
• 🔒 Privacy-first — Runs entirely locally, no external services or data transmission
• 🪶 Lightweight — Pure Rust, single dependency, no ML models required
• 📄 Table detection — Dual-mode (rectangle-based + heuristic) for financial and data tables
• 🤖 LLM-ready output — Clean Markdown perfect for AI pipelines and RAG systems

Frequently Asked Questions