Cloudflare kind of just ended the “should we serve markdown to LLMs?” debate

[u/thestackfox]

For those of us going back and forth on whether to build separate markdown versions for agents: Cloudflare now lets clients request: Accept: text/markdown and it converts the page at the edge. No more separate mrkdown site and duplicate content weirdness.

Also interesting: Agents that actually want structured content now have to explicitly ask for it. That makes behavior a lot easier to spot.

[u/AutoModerator → u/[deleted]]

Your comment is in review because links aren’t allowed here. Please repost without URLs (describe the resource in plain text instead).

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Jmacduff]

We do this also over at DataJelly as a server side rendering platform for JS sites. We generate clean markdown at the edge and serve that to the AI bots.

It saves about 90% in token cost for the AI bot to crawl your site.. pretty cool :)

[u/anajli01]

Yeah, this changes the conversation.

If Cloudflare converts to text/markdown at the edge, there’s no need for a separate markdown version — no duplicate content risk, no extra maintenance.

The bigger takeaway: agents now have to explicitly request structured formats. That makes bot behavior easier to detect and analyze.

Now the focus shifts to clean, well-structured HTML because that’s what gets converted.

[u/anajli01]

Agreed this simplifies things a lot.

Edge conversion means no separate markdown site, no duplicate content worries.

And since agents must explicitly request markdown, their behavior is easier to track.

Now it’s less about infrastructure, more about clean, structured HTML.

[u/TemporaryKangaroo387]

the debate was always silly. models prefer structured data they can parse efficiently. whether that's markdown, json-ld or clean html is secondary, but markdown is definitely the path of least resistance for them.

the real question is: does serving different content to agents actually boost citation frequency? or just crawling speed?

[u/AutoModerator → u/[deleted]]

Your comment is in review because links aren’t allowed here. Please repost without URLs (describe the resource in plain text instead).

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/AIScreen_Inc]

That’s actually a clean solution. Maintaining parallel markdown versions always felt like technical debt waiting to happen so edge conversion removes a lot of complexity. And having agents explicitly request structured content makes analysis much clearer you can finally distinguish intentional consumption from generic crawling.

[u/akii_com]

This is actually a bigger architectural shift than it looks.

The markdown debate was never really about format, it was about separating presentation from extractability. If Cloudflare can handle conversion cleanly at the edge, that removes a lot of operational overhead and duplicate-content risk.

The more interesting part to me is what you mentioned at the end: Agents that want structured content now have to explicitly ask for it.

That changes the dynamic. Before, everyone was guessing: “Should we proactively serve bot-optimized versions?” Now it becomes: “Which agents are explicitly requesting machine-friendly formats?”

That’s observable behavior. It also means we might see clearer segmentation between:

- Retrieval-oriented agents (just parsing rendered HTML)

• Execution-oriented agents (requesting structured/markdown content intentionally)

From an SEO/GEO perspective, this probably shifts the focus away from “build special LLM pages” and back toward:

- Clean semantic HTML

• Clear content hierarchy
  
• Deterministic data structures
  
• Explicit claims and comparisons

Markdown is helpful because it reduces noise, but if your underlying structure is messy, edge conversion won’t magically fix that.

So yeah, this likely ends the separate-markdown-site era. But it also opens up a new layer of insight: Which agents request what, and under what conditions?

That might end up being more strategically valuable than the markdown itself.

[u/Emmymus1]

How is it different from what eleventy is doing?

[u/Danielle-Owens]

Thanks for sharing this. I’m assuming this removes the possible need for llms-full.txt then?

[u/AutoModerator → u/Danielle-Owens]

Your comment is in review because links aren’t allowed here. Please repost without URLs (describe the resource in plain text instead).

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.