N Noer

Operating Baidu Unlimited-OCR: Transformers, vLLM, SGLang, and the Evidence Trail

Noer Unlimited-OCRInferenceSGLang

An operations-focused guide to running Baidu Unlimited-OCR with clear GPU assumptions, serving choices, batch controls, observability, and validation gates.

Unlimited-OCR should be evaluated as an inference operations problem before it is evaluated as a library choice. The model is compelling because it targets long-horizon OCR, but the production question is more concrete: can a team turn PDFs into images, run the parser on known hardware, preserve layout, capture evidence, and recover from bad requests without turning every batch into a manual rescue job?

The official Baidu repository presents the project as one-shot long-horizon parsing and publishes code and model weights under the MIT license. The linked arXiv paper explains the pressure point: end-to-end OCR models that use an LLM-style decoder can benefit from language priors, but long outputs create growing KV-cache cost and slower generation. Unlimited-OCR replaces decoder attention with Reference Sliding Window Attention, aiming to maintain a constant KV cache while parsing long documents.

The operational boundary starts with hardware

The first boundary is not subtle. The README's Transformers section says inference uses Hugging Face Transformers on NVIDIA GPUs and lists requirements tested on Python 3.12.3 with CUDA 12.9. The example loads the model with bfloat16 and ends with model.eval().cuda(). That is the official reproduction path. It should not be rephrased as official CPU support.

CPU-only experiments may be interesting, but they are separate experiments. If a batch pipeline or customer-facing product depends on Unlimited-OCR, the deployment plan should name the GPU class, CUDA stack, model revision, dependency pins, memory budget, and maximum document size. Without that inventory, a successful demo on one machine says little about the next environment.

Transformers is the reproducibility lane

The Transformers route is the best first lane because it is closest to the model interface. It uses AutoTokenizer.from_pretrained and AutoModel.from_pretrained with trust_remote_code=True, loads baidu/Unlimited-OCR, and exposes two modes for single images. The gundam configuration uses a 1024 base size, 640 image size, and crop mode. The base configuration uses 1024 image size without cropping.

For multi-page work, the example uses infer_multi with base mode, max_length=32768, and a larger no-repeat n-gram window. PDF processing is not magic inside the model call; the README converts each page to a PNG using PyMuPDF at a chosen DPI and passes the image list to infer_multi. That detail is operationally important because PDF rendering becomes part of the pipeline's quality surface.

A disciplined first run saves every input and output. Keep the original PDF, rendered page images, command or notebook, dependency lock, model revision, generated text, and manual review notes. That evidence turns an OCR test from an anecdote into a regression asset.

vLLM is the serving lane

vLLM matters when the team wants to host Unlimited-OCR behind a service boundary rather than run ad hoc scripts. The README points to an official vLLM recipe and provides Docker images for different CUDA targets, including a default CUDA 13.0 image and a Hopper-oriented CUDA 12.9 image. The decision is therefore less about whether vLLM can start and more about whether it fits the existing inference platform.

Before using vLLM for document parsing, define request size limits, image upload limits, timeout policy, retry policy, and output storage. OCR requests often carry large base64 images or many rendered pages. If those payloads move through a general-purpose chat gateway, logging and retention rules can become unsafe. A dedicated service path is usually easier to audit.

SGLang is the batch-control lane

The SGLang path in the README is concrete enough to reveal the serving shape. It launches a server with model baidu/Unlimited-OCR, served name Unlimited-OCR, context length 32768, custom logit processor support, and an OpenAI-compatible chat completions endpoint. The request example streams responses and passes image content as data URLs.

For production batch work, the more interesting part is infer.py. The README shows it processing an image directory or a PDF, writing to an output directory, setting concurrency, selecting image mode, passing a local path or Hugging Face model ID, choosing a GPU through CUDA_VISIBLE_DEVICES, and writing a server log. Those controls are the beginning of an operational batch job: they expose concurrency, logs, input source, output location, and retry behavior.

SGLang also requires the custom no-repeat n-gram processor in the request path. That is not cosmetic. Long generation can repeat, and OCR repetition is dangerous because repeated legal clauses, table rows, or numeric values may look plausible to a downstream system. If a serving layer drops the processor or changes sampling behavior, the validation pack should catch it.

Observability requirements for document OCR

Document OCR observability should be designed around documents, not only requests. Each run should have a manifest that records source file hash, page count, render DPI, image mode, model revision, server route, hardware, start time, end time, exit status, and output path. For failures, preserve the rendered pages and the server log. For partial success, record which pages or chunks were accepted and which require review.

The output should also carry review status. A practical set of labels is: raw, machine-usable for search, human-reviewed for citation, rejected for structure, and rejected for recognition. That prevents a search index from silently mixing verified and unverified parses. It also helps measure whether a new model or serving configuration actually reduces human work.

Security and dependency review

The official examples use trust_remote_code=True, custom model code, GPU containers, PDF rendering, and network model downloads. None of these are unusual for modern AI infrastructure, but they are not zero-risk defaults. Run first tests in a disposable environment. Review model code and container provenance. Separate input documents from general application storage. Avoid sending sensitive PDFs through shared logs or debugging traces.

PDF rendering deserves special caution. PDFs are complex containers, and the renderer is part of the attack surface. Keep PyMuPDF current according to your security policy, restrict file access, and treat untrusted PDFs as untrusted input. If the OCR system handles external uploads, isolate conversion and inference from administrative networks and credentials.

Failure modes to test before rollout

Test clean failures before testing scale. Include a password-protected PDF, a malformed PDF, a scanned page rotated ninety degrees, a document with many repeated headers, a table split over two pages, a document with small footnotes, and a page containing handwriting or stamps. Check whether the pipeline fails explicitly, produces partial output, or returns confident nonsense.

Then test load. Increase concurrency slowly and watch GPU memory, latency, server errors, and output repetition. A batch parser that works at concurrency one may become unstable at concurrency eight. If the system processes queues, define backpressure rather than letting users submit unlimited PDFs into a GPU bottleneck.

When Unlimited-OCR is the right choice

Unlimited-OCR is a strong candidate when the workload is long, structured, and expensive to repair after page-level OCR. Examples include internal research libraries, compliance archives, technical documentation ingestion, legal review preparation, and RAG pipelines that need page order and layout fidelity. It is a weaker fit when the task is occasional single-page OCR, when only CPU resources are available, or when the organization cannot review generated output before relying on it.

The model's long-horizon design is valuable, but long-horizon parsing increases the blast radius of a bad run. A wrong page-level parse damages one page. A wrong long-document parse can damage a whole index or case file. Adoption should therefore begin with gates: representative corpus, stable rendering, pinned runtime, observable batch jobs, and explicit human review status.

Decision checklist

  • Reproduce the official Transformers path on NVIDIA GPU/CUDA before judging quality.
  • Choose vLLM only if the platform is ready for large multimodal request payloads.
  • Choose SGLang when batch concurrency, server logs, and OpenAI-compatible requests are the main requirements.
  • Store PDF render settings and page images as part of the evidence trail.
  • Validate reading order, tables, footnotes, and layout, not only raw character recognition.
  • Do not publish CPU feasibility, speed, or memory claims until measured on the target hardware.

The best conclusion is practical rather than promotional. Baidu Unlimited-OCR gives infrastructure teams a serious long-document OCR option, but it belongs behind the same controls as any GPU inference service: pinned environments, explicit hardware assumptions, isolated document handling, logs, metrics, retry boundaries, and review gates. Treat it that way and the model can be tested fairly. Treat it as a drop-in OCR shortcut and the hard parts will simply move downstream.