N Noer

Firecrawl pdf-inspector is strongest as a confidence-aware OCR router

pdf-inspector shines when you use its CLI and APIs to inspect outputs, route only the needed pages to OCR, and handle confidence, encoding, and layout failures before they spread.

The first mistake most document pipelines make is treating OCR as the default intake step. That is the wrong starting point for a PDF processor that cares about speed, structure, and failure handling. OCR should be the fallback for pages that truly need it, not the universal first pass for every file that reaches the system.

Firecrawl's pdf-inspector is useful because it turns that idea into a practical local workflow. The project is a Rust PDF classification and extraction engine with bindings for Python, Node.js, browser WebAssembly, and a command-line interface. It detects whether a PDF is TextBased, Scanned, ImageBased, or Mixed, returns confidence and per-page OCR routing hints, and extracts position-aware text into clean Markdown without calling OCR, machine learning models, or external services. That combination matters because many document systems do not fail on obviously bad scans; they fail on the middle ground where some pages are machine-generated, some pages are images, and the output has to stay readable enough for downstream search, review, and ingestion.

Why the CLI matters even when you have APIs

It is easy to think of a CLI as a convenience layer, but in document operations it often becomes the fastest way to prove behavior. pdf-inspector ships with two binaries that expose the operational split clearly: pdf2md for extraction and detect-pdf for classification and layout analysis. The README shows the core commands: pdf2md document.pdf, pdf2md document.pdf --json, pdf2md document.pdf --items-json, pdf2md document.pdf --raw, pdf2md document.pdf --compact, pdf2md document.pdf --pages, and pdf2md document.pdf --select-pages 1,3,5-10. It also shows detect-pdf document.pdf, detect-pdf document.pdf --json, and detect-pdf document.pdf --analyze --json.

That separation is not cosmetic. In production you often want a fast gate before any expensive downstream work. detect-pdf can tell you whether a file is worth local extraction, whether it is mixed, and which pages need OCR. pdf2md then turns the text-bearing pages into structured output. If the file is mostly readable text, the CLI gives you a cheap, deterministic path from byte stream to reviewable Markdown. If the file is partially scanned, the same toolchain still gives you a page-level routing plan instead of collapsing the whole document into a single yes/no decision.

That kind of operational decomposition is what keeps a pipeline honest. You can log the detection result, inspect the Markdown, and route only the pages that actually need image recognition. You do not have to guess where the failure is hiding.

What the API surface is really for

The Python, Node.js, Rust, and WebAssembly bindings are not separate products; they are the same core engine exposed in different operational contexts. The README points to the Python API in docs/python.md, the Node binding in napi/README.md, the Rust API in docs/rust-api.md, and browser support in wasm/README.md. That coverage matters because the best integration point depends on where the document enters your system.

If your workflow is an ingestion service, the Rust API is the natural core. If your document step lives in a web application or a worker process, Node or Python may be easier to wire in. If you need local browser-side handling for a privacy-sensitive upload flow, WebAssembly lets you keep the parser on the client and avoid a server round trip entirely. The architectural point is consistent across all of them: inspect first, route second, OCR only where needed.

pdf-inspector is especially strong as an API when you want to inspect outputs rather than trust a single final string. The library does not just say whether a file can be read. It returns type information, page-level OCR hints, text items with coordinates, and Markdown that reflects layout decisions. That means a caller can build guardrails around the result. For example, a pipeline can reject a supposedly text-based PDF if the confidence is low, or can compare extracted pages against expected section patterns before deciding the file is ready for indexing.

Confidence is not decoration; it is a routing input

Many classification tools return a label and stop there. pdf-inspector is more useful because the label comes with a confidence score from 0.0 to 1.0 and a list of pages needing OCR. That gives you the raw material for policy. A conservative pipeline can require a high-confidence TextBased result before allowing direct extraction. A more flexible one can accept Mixed documents, split the file into routes, and send only the problematic pages into an OCR system.

This matters in real operations because document quality is rarely binary. A birth certificate scan, a report with one scanned appendix, or a legal bundle with embedded screenshots does not need the same treatment as a fully digital PDF. A confidence-aware workflow can preserve the speed advantage of local parsing while still respecting the pages that do not have usable text. It also helps prevent silent degradation. If the classifier says a file looks text-based but the output is suspiciously thin, low confidence can trigger a second pass or manual review instead of allowing a bad ingestion to pass as success.

The key is to inspect the result object, not just the output text. When a system only reads the rendered Markdown, it is blind to the reason the output exists. When it also stores classification, confidence, and page routing, it can explain why a document was processed one way rather than another.

How encoding failures should change the route

PDF text extraction often breaks on font encoding rather than on OCR. That is one of the most annoying failure modes in document work because the page may look readable, but the internal character mapping is broken. The README calls out CID font support, ToUnicode CMap decoding for Type0 and Identity-H fonts, UTF-16BE, UTF-8, and Latin-1 encodings, plus automatic detection of broken font encodings so callers can fall back to OCR. In other words, pdf-inspector is not just extracting text; it is actively watching for cases where the text layer is technically present but semantically unreliable.

That failure-aware behavior should become part of the routing policy. A page with broken encoding should not be trusted just because it contains glyphs. If the extracted characters do not map cleanly, the pipeline should treat that page like a candidate for OCR or special handling. That is a stronger engineering posture than blindly accepting text because text exists. It recognizes that text quality is a property of both content and encoding, not only of character count.

In practice, this is where many systems get stuck between speed and accuracy. A local parser is fast, but only if you are willing to check its signals. pdf-inspector makes that check easier by surfacing the condition directly. The caller can route low-quality encodings to OCR, log them for analysis, or quarantine them for manual correction. That is exactly how a mixed pipeline should behave: trust the text layer when it is good, and stop trusting it the moment the evidence says otherwise.

Layout failures are usually reading-order failures

PDF layout bugs are often described as extraction bugs, but the real problem is usually reading order. The README highlights position-aware extraction, automatic multi-column reading order, and RTL text support. It also describes table detection through rectangle-based detection from drawing operations and heuristic detection from text alignment. Those are not separate niceties; they are the mechanisms that keep Markdown usable after extraction.

For a developer building against the CLI or API, the practical question is not whether text came out. It is whether the text came out in the right order, with headings where expected, tables preserved, lists intact, and page breaks or captions recognized. A columnar paper that becomes a scrambled paragraph is operationally broken even if every character is technically present. A table that loses row alignment may still be readable to a human, but it is no longer dependable for downstream parsing.

That is why the --analyze flag on detect-pdf matters. It gives you layout information before you commit to the extraction path. A system can run detection, inspect page composition, and decide whether a document is safe to accept as clean Markdown or whether it needs extra handling. This is also where the project's mixed document logic pays off. A report with a good text layer and a scanned appendix should not force the whole file into the same path. The structured result lets you keep the readable pages in the fast lane and isolate the difficult pages.

When layout fails, the correct response is usually not “run OCR on everything.” The correct response is “figure out whether the failure is local to a page, a font, a column, or a table region, then route accordingly.” pdf-inspector gives you enough information to make that decision instead of guessing.

Why local-only matters

The project's restraint is part of its value. It is pure Rust, with no ML models and no external services. That keeps the operational blast radius small. Sensitive documents stay local. Latency becomes predictable. Deployment does not require a separate OCR backend just to handle files that already contain usable text. The README even frames the project as a way to skip expensive OCR services for the large share of PDFs that do not need them.

That is especially important in workflows where PDFs are not final user-facing products but internal intermediates: uploaded invoices, research papers, exports from other systems, or legal bundles entering review. In those settings, the goal is not to be clever. The goal is to produce stable, inspectable outputs with minimal infrastructure and a clear failure path.

The benchmark section reinforces that point with a useful detail rather than a vague claim. On the opendataloader-bench corpus of 200 PDFs, with OCR disabled, refreshed on 2026-07-16 on an Apple M4 Pro, pdf-inspector is reported at 0.875 overall, 0.915 for reading order, 0.814 for tables, and 2.8 seconds median speed. Those numbers are not a promise for every workload, but they do show the design center: strong local extraction, especially where reading order and tables matter, without depending on OCR or model-based PDF parsing.

That benchmark framing is important because it sets expectations correctly. This is not a universal replacement for OCR. It is a disciplined way to avoid OCR when OCR is unnecessary, while still exposing the pages and conditions that really do need it.

Best-fit operational pattern

The healthiest workflow looks like this: ingest the file, run classification, inspect confidence and pages_needing_ocr, choose a route, and then verify the output itself. TextBased documents with high confidence can move directly into local extraction and Markdown conversion. Mixed documents can be split so only the trouble pages get OCR. Documents with broken encodings or suspiciously poor layout can be escalated before they contaminate search or review systems. This is a routing problem, not just a parsing problem.

If you are building a CLI-first tool, that pattern is easy to express in scripts and batch jobs. If you are building a service, it becomes an API contract. If you are building a browser workflow, the WebAssembly build makes local inspection possible without shipping the document elsewhere. The surrounding environment changes, but the logic stays the same: inspect, evaluate, route, and verify.

That is why pdf-inspector stands out. It does not try to do every part of document understanding. It does the part that should happen before OCR, and it does it with enough structure that the next step can be smarter. In document automation, that is often the difference between a reliable pipeline and a pile of brittle heuristics.