Nine Lines of Cargo.toml Beat a Benchmark Score
A new document converter went past me this week with a number attached: 81 quality, 4.4 milliseconds, fourteen of fourteen formats. anydoc, from Firecrawl — Rust, MIT licensed, documents in and Markdown out. Four days old, ten thousand stars.
I run agent systems that eat documents for a living, so a fast local converter is not an abstract interest. What decided the question turned out to have nothing to do with the benchmark.
The number is about something else
The headline everywhere is "sub ~5ms markdown parsing for PDFs, Word docs, slide decks." The
benchmark behind it opens with this, in the first three lines of
bench/README.md:
"Office, text, and presentation documents; PDFs are out of scope." The comparison table has no
PDF row.
So the 81 and the 4.4 ms are evidence about .docx, .pptx, .xlsx and .epub, and evidence
about nothing else. The tool does read PDFs, and that path is unmeasured — by this benchmark or any
other I could find. It also does no OCR at all. None of this is hidden. It is stated plainly, by
the vendor, one scroll below where reading stops.
What actually decided it
I handle client architecture material and my own GmbH's financials. The question that gates adoption is not quality-versus-markitdown. It is: does this file leave the machine?
Usually that is unanswerable without taking someone's word for it. Here it took nine lines.
anydoc's entire
dependency list is
calamine, cfb, csv, flate2, encoding_rs, log, pdf-inspector, quick-xml, zip. No HTTP client. No async runtime. No telemetry. No model
weights to download on first run — the quiet network call most Python document tooling makes and
nobody notices.
That is a checkable claim about confidentiality rather than a promise, and I can re-verify it in thirty seconds on every upgrade.
The number a vendor optimises is rarely the constraint that binds in your context. Find the one that binds — for me, whether the file leaves the machine — and evaluate against that first. Everything else is a tiebreak.