PullMD Part 9: Wiring in Microsoft's MarkItDown — and ripping three parts back out
PullMD is my self-hosted tool that turns web pages and URLs into clean Markdown. The series so far was about building, hardening, and shipping it. This part doesn’t start with a plan. It starts with an itch.
I had a few HTML files sitting on disk that I wanted to hand to Claude. PullMD already extracts Markdown from any URL anyway. So why couldn’t I just drop those files in? The answer was a small feature: local file conversion through the same pipeline.
The trivial itch
The guardrails around it came mostly from Claude: no cache.put() for uploaded files, no Playwright for uploads, the filename in an X-Filename header instead of the query param, so it wouldn’t land in the reverse proxy’s access logs. All defensible. My own share of that list was more mundane: I simply saw no point in caching a file you convert once by drag-and-drop. Why would you ever fetch it again later? PullMD is first and foremost a local tool with authentication; the privacy drama Claude used to justify the points I never fully shared.
The whole thing is triggered by drag-and-drop. While testing, the obvious follow-up question came to me afterward:
“but on mobile you could turn it into ‘or open file’, right? a downloaded html file is something you might want to convert on mobile too.”
That was my prompt, translated from German. So the drop box became tappable and has opened a native file dialog ever since: the same convertHtmlFile() behind it, just a <label> with a hidden file input in front. Desktop and mobile.
MarkItDown in
The upload feature ran only on my test instance at first. Two days later I happened to stumble across Microsoft’s freshly released MarkItDown, MIT-licensed, in Python. And it clicked. If HTML files work, why not PDF, Word, PowerPoint, Excel, EPUB? And why not YouTube transcripts, for that matter? Anyone who has ever landed on one of those bloated transcript pages knows why you’d rather have it inside your own tool.
I gave Claude the repo: “Take a look at the following repo. Can we integrate it into PullMD in a sensible way?” Claude analyzed the library and came back with a plan that made immediate sense to me. MarkItDown handles only the non-HTML file types. For HTML it would be a step backward: at its core it does the same thing PullMD has long done on the raw body, and as a “fourth HTML extractor” it adds nothing. So another sidecar. PullMD is already a small cluster of containers with its own services for extraction and browser rendering; what’s new is a Python service that gets called when a document format comes in that the existing pipeline can’t handle.
So far, so clean. A third-party library for the boring grunt work, because parsing Office formats is nobody’s favorite pastime; MarkItDown covers a whole zoo of them: PDF, DOCX, PPTX, XLSX, EPUB, CSV, JSON, XML. Exactly the kind of work you sensibly hand off instead of maintaining it yourself. Then I looked at Claude’s plan in detail, and “just wire it in” turned into “wire it in, but not three parts of it.”
Three parts back out
Reading through Claude’s integration plan, two things caught my eye: the YouTube converter was missing without comment, of all things the part that had tempted me the most. And for audio it listed “privacy” as the rationale while the plan still called OpenAI. I pushed back:
“why did you leave out YT transcript? … and what does ‘privacy’ mean if you use openai anyway?”
Claude’s answer walked through the reasons along the code, and that became the actual work of this release: not the wiring in, but the deliberate taking back out.
Audio. MarkItDown’s default sends audio files silently to Google’s Web Speech API. For a self-hosted tool, that decision belongs to the operator, not to the default. So a dedicated endpoint with an operator-configured backend: faster-whisper or Ollama locally, Groq or OpenAI for those who want it. A conscious choice, including a real local option instead of a silent Google default.
YouTube. MarkItDown’s YouTubeConverter was ripped out entirely. Claude’s look into its code found: ytt_api.list() sat outside the try/except. A blocked or transcript-less video drags the whole conversion down with it: you lose even the title and metadata, even though they were right there in the HTML. No proxy support on top of that. In its place came a dedicated /youtube handler with graceful fallback, modeled on a keyless implementation I already had elsewhere.
Image captioning. Not a wrapper around MarkItDown’s llm_client, but the vision call directly in our own provider layer. Zero MarkItDown coupling for the LLM part: a MarkItDown update can’t break any of it.
The principle went into the project memory as one sentence: MarkItDown only for file types; fragile and third-party paths we build ourselves.
The next morning that sentence grew into a larger architecture brainstorm. I put the open question:
“are we building our own markitdown replacement? would that be bad design?”
Claude’s answer was no, and it was the right one. Don’t rebuild the whole Office, EPUB, and XLSX parser zoo; nobody wants to maintain that themselves. But the LLM layer belonged out. Vision and speech-to-text moved entirely out of the Python sidecar and into Node. My reasoning for it: “I don’t like making our LLM calls depend on MarkItDown like that. If someone doesn’t need documents at all, they still have to run MarkItDown alongside.” The sidecar thereby shrank to what it does well: documents and YouTube.
At the same spot a fourth piece nearly wandered in: Docling, IBM’s table engine, which gives the best results locally. Claude had mentioned it as an alternative, and I was briefly thrilled, until Claude supplied the numbers: a 4.4 GB image, several gigabytes of RAM per request, 60 to 140 seconds of cold start. What tipped it in the end wasn’t a technical trade-off but the picture of how that comes across on a first try:
“if there’s some endless warmup like that, I get frustrated and pin it on ‘what a bad project, this pullmd’.”
Docling is now documented only as a pluggable URL endpoint, not in the standard image. The guiding line behind it: a light hosted door for everyone, a heavy self-host door only for those who consciously open it. The same logic for the PDF tables: Mistral OCR as a reference adapter, but opt-in via query param, so it never silently costs money.
In the end exactly one real breaking change remained: the body now consists only of title and content; source and fetch date live solely in the YAML frontmatter instead of additionally as a line in the text. Nothing is duplicated anymore, agents read fewer tokens; anyone who needs the old line gets it back via an env flag. That turned the release into v3.0.0.
The new model as bouncer
That evening, when v3 was finished and the release tag planned, Fable 5 appeared — Anthropic’s new flagship model. A brand-new model whose class is built for finding vulnerabilities, the evening before a release that had just gained a public upload path: new attack surface that nobody had yet looked at with hostile eyes. The most obvious first task could only be one thing.
First use, straight on the real diff:
“please do a code review and find bugs and security issues.”
Seven real findings, one false positive. The two that gave me the most pause were security:
A YAML injection via \r. The function that quotes strings for the frontmatter escaped only \n, not \r. YAML treats a bare \r as a line break: an attacker-controlled page title could have used it to inject its own frontmatter line. The fix was a single extra character in the regex: [\r\n]. A one-character hole. Claude had written the code, and it had slipped through every review before this one regardless.
A zip bomb, or DoS, on the public upload path: arbitrary documents tore through the parser unguarded, and the sidecar was internally unauthenticated. The fix: conversion in a throwaway subprocess with a wall-clock timeout and a memory cap. Deliberately spawn instead of fork, because fork from a worker of a multithreaded server can deadlock with MarkItDown’s lazy imports. This point first landed on the review’s “needs real design work” list; I then had it fully implemented right away anyway, because a public upload path without a resource ceiling is not something you defer to later.
Plus the bugs:
- try/catch missing: a vision/STT error would have triggered a 502 instead of a graceful degradation; the reassuring comment “no behaviour change” held only for the null case, not for the throw case.
- Cache hit lost media fields when the frontmatter flag was set; the fix consolidated the triple-duplicated merge block along the way.
- MCP tool drifted from the HTTP API: the PDF OCR path wasn’t reachable via MCP at all.
- One false positive (a reported exception-string leak): harmless, because the affected body is discarded before it ever reaches the client.
Reading the review still stays mandatory; a finding is a hypothesis, not a verdict.
The next morning I spotted a single broken image on a share page myself. I gave it to Claude to track down and, while at it, asked it to go over the whole release once more. A full sweep came along with it. Relative <img src> URLs were never absolutified against the source domain. An OAuth secret hadn’t been passed through in any compose file since v2.3. And the opt-out flag for the breaking change wasn’t reachable through any path — the most important gap, because a breaking change without a reachable opt-out is simply a break.
After all that, 721 Node tests plus 4 sidecar-guard tests were green, up from 644 when the branch was opened.
Out the door
Squash-merge, tag, release. Minutes later the Reddit post. No triumphant tone: the thing is out, :latest now points to v3, done. Before the push, a privacy and secrets scan still ran across the whole branch; that mattered to me more than clean commit aesthetics, for obvious reasons that have already been a topic of their own in this series.
The split was the same across the whole arc: I made the decisions, Claude analyzed and built. What sets this part apart from the previous ones is that the most important decisions weren’t the wiring in but the deliberate taking back out. The boring half I handed off. The delicate half I kept.
A third-party library spares you the parser zoo. But every path that is either fragile or silently sends data out, you’d better not adopt unchecked: that one you build yourself, or you at least make it a conscious choice instead of a silent default.
Part 9 of 9 in the PullMD series.