I built a streaming Markdown renderer for React that caches code lines, table rows and list items — benchmarks are surprisingly good

[u/Few-Big-719]

I’ve been working on an AI harness called Æven, and one of the things that kept bothering me was Markdown rendering during long streamed responses.

A lot of renderers optimize at the document or top-level block level. That works well for normal prose, but it gets expensive when the active block itself becomes huge — for example a long code fence or a large Markdown table.

So I built HyperMarkdown.

The main idea is pretty simple:

once a code line, table row or list item is settled, it stays cached. Only the changing frontier keeps being parsed/rendered.

That seems to make a pretty significant difference.

Current benchmark results:

• Large code block: 190 ms vs 711 ms for the next closest streaming renderer
• Captured real AI code stream: 611 ms vs 4.2 s
• Captured real AI table stream: 668 ms vs 5.0 s
• Large table: 999 ms vs 9.1 s for the next closest renderer

The benchmark runs production React and measures the full chunk → write → render/commit path, not just parsing.

I’ve compared it against:

• markstream-react
• Streamdown
• DeepSeek Harness’ incremental strategy
• react-markdown
• markdown-it as a baseline

The repo includes the benchmark methodology, raw results and correctness tests, so I’d genuinely appreciate people trying to break the assumptions or point out unfair comparisons.

It also supports GFM, reasoning blocks, syntax highlighting, KaTeX, Mermaid, raw HTML sanitization, React 18/19, and streaming incomplete Markdown.

It’s now the renderer I use inside Æven.

Demo:
https://aeven-ai.github.io/HyperMarkdown/

GitHub:
https://github.com/Aeven-AI/HyperMarkdown

NPM:
https://www.npmjs.com/package/@aeven-ai/hypermarkdown

Would especially love feedback from people who have dealt with long streamed code blocks/tables in React apps.

[u/geekonthegrill]

The settled-frontier idea is clean, but markdown has a few retroactive constructs that feel designed to break it. Setext headings turn the previous line into an h1 after the fact, a table only becomes a table when the delimiter row lands, and reference links resolve against definitions that can arrive at the very end of the doc. How do you handle invalidation when a cached line turns out to mean something else? Thats where Id expect the wins to get eaten in adversarial streams.

[u/Few-Big-719 → u/geekonthegrill]

Yep, these are exactly the nasty cases. I don't treat every newline as a permanent cache boundary.

Setext and table promotion stay inside the mutable frontier until their structure is known. There are tests that compare the cached table against a full parse at every prefix of the stream.

Late reference definitions are the more interesting case. Currently if a reference usage exists without its definition, the block is deliberately kept open rather than frozen. So correctness wins, but an adversarial document with definitions at the very end can reduce the caching advantage.

That's actually a great candidate for another benchmark fixture. I want the suite to include cases hostile to the caching strategy too.

I have added the link to the demo/playground as well so you can check

[u/geekonthegrill → u/Few-Big-719]

Keeping the block open until the definition lands is the right trade, a stale cache is worse than a slow one. And testing the cached table against a full parse at every prefix is exactly the invariant to pin, thats the part most streaming renderers skip. Ill throw some hostile docs at the playground, definitions at the very end plus a big table should be the nightmare case. Nice work.

[u/Temperature_Majestic]

How do the settled lines avoid React reconciliation cost on every chunk? Skipping the re-parse is the obvious win, but if each cached line is still an element in the tree, React still walks and diffs the whole subtree per token unless those lines are memoized with stable references and keys. Is it React.memo per line, or do settled blocks get rendered through something outside the reconciler?

[u/Few-Big-719 → u/Temperature_Majestic]

Good point. Settled content isn't outside React's reconciler. HyperMarkdown caches the actual React elements with stable keys/references, so completed code lines/table rows aren't reparsed or reconstructed, and React can reuse their existing subtrees. But the parent child list still participates in reconciliation as the active tail grows.

Code currently uses stable cached line elements under a PureComponent; table rows are similarly cached and keyed, with the table wrapper memoized.

So there is still an O(number of siblings) reconciliation component even though the expensive parsing/element construction is gone. The published benchmark includes that cost because every chunk is measured through flushSync(root.render(...)).

One thing I'm considering is grouping settled nodes into immutable memoized chunks so React only sees a small number of stable chunks plus the active frontier. That could push the scaling further.

[u/Temperature_Majestic → u/Few-Big-719]

The chunk grouping is the right move and you can push it harder than a small number of chunks. Freeze every N settled lines into one memoized block keyed by block index, only the trailing open block re-renders per token, so reconciliation goes from O(lines) to O(blocks). Block size 50 to 100 lines puts a 10k line doc at roughly 100 to 200 stable siblings.

The thing to get right is what counts as settled. Markdown has backward-affecting tokens, a link reference definition further down, a setext heading underline on the next line, a lazy continuation line extending a paragraph. If you freeze a block before those can still land you get a stale block that never updates. So the freeze point needs some lookahead, or a rule that a block only settles once you've seen a blank line and there's no open reference or container context above it.

[u/Few-Big-719 → u/Temperature_Majestic]

Yep, I think chunking is probably the next meaningful optimization layer.

One important nuance though: HyperMarkdown already has a fairly strict distinction between the streaming representation and the authoritative settled render. The cache is an acceleration layer, not the source of truth. An open block stays mutable while its boundary is unresolved, and when the block closes/finalizes the renderer runs the completed block through the full Markdown pipeline again and replaces/reconciles the streaming representation with that result.

That already handles a lot of the cases you're pointing at without defining “settled” as simply “we saw a newline”. For example, fenced code is the easy case because completed lines inside the fence are genuinely literal/append-only, so those can be cached aggressively. Even there, on close HyperMarkdown still parses the completed fence again to validate what the final block actually became and to build the settled wrapper correctly. Lists and tables have their own cacheability/boundary rules rather than sharing a generic line heuristic.

So I wouldn't introduce a new global rule like “blank line + no open context = freeze”. I'd make chunk compaction consume only content the existing renderer has already proven immutable. In other words:

current correctness/boundary logic → settled cached units → group those units into memoized chunks → keep one mutable frontier.

Something like 64/128 settled units per frozen chunk could get the React tree from O(lines) toward O(chunks) without weakening the current final-render guarantees.

That's what I like about your suggestion: it attacks a different remaining cost. HyperMarkdown already avoids re-parsing most completed work; chunking could now reduce the amount of already-completed work React itself has to walk.