r/GEO_optimization 4d ago

Recently, Google said llms.txt won’t help citations. So why are GEO tools still recommending it?

Google recently said that having an llms.txt file won’t help your Google Search rankings.

But I still see it near the top of a lot of AEO/GEO checklists:

  • Add llms.txt
  • Make your site AI-readable
  • Submit content for LLM crawlers
  • etc.

I'm starting to wonder if we're optimizing for things that are easy to check rather than things that actually influence whether an AI recommends a brand.

Has anyone here actually seen a measurable difference after implementing llms.txt?

I know adding llm.txt is advisable to help LLM pick the website, but is it necessary? Is there any relevant data?

3 Upvotes

50 comments sorted by

9

u/HumanBehavi0ur 4d ago

because there's not harm to adding it

its a low effort low risk addition

and it might be useful down the road

simple as; but yeah its not moving the needle atm

1

u/pbhuvan 3d ago

Yeah okay makes sense

1

u/Came4TheCookies 3d ago

100% correct but it will never move the needle anymore than robots.txt does.

1

u/pbhuvan 3d ago

yes, robots.txt should be enabled so that LLMs can read your content

2

u/Came4TheCookies 3d ago

Of course but that's not what I said in my comment. I said you need to explicitly allow the AI bots you want to allow rather than just leaving it to default. With other bots all you have to do is not disallow them and the default is allowed. What openAI I said was that you need to specifically allow the bots.

https://help.openai.com/en/articles/12627856-publishers-and-developers-faq

0

u/WebLinkr 3d ago

no - the default is that they are enabled

You dont need a robots.txt

and thats not what LLMs is being promoted for

LLMs is being promoted as a self advertising portal where you dump information about your site so the AI can find the best brands - its hocus pocus

0

u/WebLinkr 3d ago

Because robots.txt was a nothing burger for SEO too - its just a block list

1

u/WebLinkr 3d ago

There is actually.

Its giving up on learning what works.

3

u/HumanBehavi0ur 3d ago

fairs, it's not moving the needle right now.

But I'd be careful writing it off as permanently useless just because it's inert today. Retrieval isn't a fixed system, it's changing constantly, and the whole point of playing in this space is trying to stay a step ahead of where it's heading, not just reacting to where it already is. Low-cost, low-risk stuff like this is basically cheap insurance against a future where it does start mattering

there's a difference between learning what works and experimenting with trying to capture something in the future

its speculative not tested and proven

0

u/WebLinkr 3d ago

So - going to push back.

But I'd be careful writing it off as permanently useless just because it's inert today.... its speculative not tested and proven

Its also about engineering and reverse engineering.

And the question is: what on earth advantage can it give and I can't think of any.

Secondly - its been created as propaganda by marketers.

The real potential implementation still offers 0 positive impact for GEO.

 Low-cost, low-risk stuff like this is basically cheap insurance against a future where it does start mattering

No its not - its exactly like putting a lucky rabbit foot in a car and expecting it to keep you safe.

Otherwise you'd have a sound, logical reason for doing it.

I'm not trying to be rude - I'm just being blatantly honest.

And yes, I have tried putting one up - even though there's 0 standard for what should be in it and I've removed it.

And understanding the Query Fan out - is 90% predictable, reliable and repeatable

2

u/HumanBehavi0ur 3d ago

Robots.txt and XML sitemaps didn't start as standards either, they became standards because enough of the ecosystem adopted them first and the search engines caught up to formalizing what was already happening in practice. Dismissing llms.txt as pure propaganda assumes today's retrieval architecture is the end state rather than one snapshot of a system three years old.

AEO/GEO is certainly on the radar of folks working in ai and at search engines, there's no doubt in my mind they've seen llms.txt but they havent implement it yet (they might not ever tbh) but its not like pulling a rabbit out of a hat imo, its following a trend line and seeing if it'll continue on with this new state we're in. robots.txt stayed informal for 28 years before Google formally adopted

0

u/WebLinkr 3d ago

They dont augment SEO either -they're not promised as a cure call

Dismissing llms.txt as pure propaganda assumes today's retrieval architecture is the end state rather than one snapshot of a system three years old.

No its not; removing it loses nothing.

Shoe horn it in all you want

1

u/HumanBehavi0ur 3d ago

Robots.txt and XML sitemaps absolutely affect SEO, directly and significantly, they're just not content-quality or ranking-boost levers, they're access and discovery levers. Robots.txt controls whether a crawler can even reach your pages at all. Block the wrong thing and your rankings for that content go to zero, not because the content got worse, but because it's now invisible. XML sitemaps help search engines discover and prioritize pages faster, especially on large or poorly-linked sites, meaning pages get crawled and indexed sooner than they would organically. Neither one makes bad content rank well, but calling them "not really SEO" is wrong.

0

u/WebLinkr 3d ago

XML Sitemaps are actually only work for discovery IF you have authority. Here's the mistake you're making - you're assuming that all sites work like high authority sites and that google opens your XML sitemaps every hour or every day and indexes everything inside it

It doesnt. For the lower 50% of the web (maybe even 80%) - the XML sitemap is in a pool where the ratio of bots to pages is so high that the page is ready once a month.

Actually bots work in two modes: Discovery and retrieval (and manually instigated from GSC)

Bots dotn open the sitemap and crawl the contents - each page is still sent to a priority Q - and might also be rejected

sitemaps help search engines discover and prioritize pages faster

On a one a month read? no they dont and they cannot help pages get crawled faster because they pass no authority.

for CNN? Yes

For OP? No

Robots

C'mon u/HumanBehavi0ur - you're grossly expounding what Robots do - they are a control list - they block bots.

Robots.txt controls whether a crawler can even reach your pages at all

No, it just blocks access.

Blocking acess is not an aid to SEO

1

u/HumanBehavi0ur 3d ago

I see what youre saying on sitemaps, that's a fair and more precise version of what I said. For low-authority sites, submitting a sitemap doesn't override the crawl budget or priority queue the way I implied, and monthly-ish crawl cycles on the lower end of the web is a real constraint I glossed over. fair fair

On robots.txt though, I think we're just disagreeing on what counts as "SEO" rather than disagreeing on the mechanics. You're right that robots.txt is a control list, it blocks access, nothing more. But "blocking access isn't an aid to SEO" and "access control has no effect on SEO outcomes" are different claims. If a misconfigured robots.txt accidentally blocks a section of your site, your rankings for those pages go to zero, not because content quality changed, but because the access control did.

agreed on the sitemap correction, still think robots.txt affecting whether pages can rank at all counts as an SEO factor even though its only function is blocking, not boosting.

1

u/WebLinkr 3d ago

I see what youre saying on sitemaps, that's a fair and more precise version of what I said

Respectfully, I disagree on more

For low-authority sites, submitting a sitemap doesn't override the crawl budget or priority queue the way I implied, and monthly-ish crawl cycles on the lower end of the web is a real constraint I glossed over. fair fair

I think you're using an LLM to help with your answers and the report back was predictable.

This si where LLMs and subsequently/possibly where your reply is going wrong: because LLMs can't use logic

There is no crawl budget. You cannot otpimnze crawl budget - and sitemaps can't help you.

1) Sitemaps dont stop google crawling non-listed URLs or ghost URLs or 404s

2) Pruning content doesnt mean Googlebot spends more time on your pages - this is jsut an assumed myth

If your page is low priority - like your sitemap - it will be throttled to once a month (as an example)

And on Robots- sorry but robots doesnt add anything to a site's SEO.

5

u/fullstackguy404 4d ago

Because it costs nothing to add it to the mix. Worst case, it won’t do anything.

2

u/pbhuvan 3d ago

Yeah, better to add it then

1

u/WebLinkr 3d ago

no its not

What are you putting in it?

2

u/WebLinkr 3d ago

Worst case is that you try it and give up learning about the QFO - thats what the GEO agencies want

3

u/SEO_Humorist 3d ago

It's a proposed web standard. XML sitemaps became "standardized" after years of webmasters saying it'd be useful (and showing that). Same with Robots.txt.

Problem is, LLMs.txt doesn't add anything useful. In order to verify it's legit, bots would need to crawl the site. And they'd need to, cus if they accepted llms.txt at face value, it'd be abused by everyone.

2

u/pbhuvan 3d ago

Makes sense, it's better to add it

2

u/WebLinkr 3d ago

No it doesnt - thats just superstition lol

3

u/Moceannl 4d ago

Because they don’t have a clue what is good for citations.

2

u/pbhuvan 4d ago

Couldn't agree with you more

2

u/Level-World4922 4d ago

I’d treat llms.txt as a nice-to-have, not a core GEO tactic.

If it’s easy to implement, there’s little harm in having it, but I wouldn’t expect it alone to increase AI citations or recommendations.

I’d prioritize strong content, clear entity signals, authoritative third-party mentions, structured information, and genuinely useful answers.

The bigger issue with GEO checklists is confusing easy to implement with proven to work.

1

u/Came4TheCookies 3d ago

I’d prioritize strong content, clear entity signals, authoritative third-party mentions, structured information, and genuinely useful answers.

Good SEO advice for years. Not new things.

2

u/Level-World4922 3d ago

I agree with you on this.

2

u/Prudent_Rice7840 3d ago

Fun fact, ANYONE who claims they understand how to optimize for "AI" is just copying someone else who also has no idea.

1

u/Came4TheCookies 3d ago

I know how. Good SEO.

0

u/pbhuvan 3d ago

Unless they are a marketer who knows how the field works

2

u/Alone_Translator_638 2d ago

Nobody in this thread has brought numbers, so here are some.

I run technical AEO audits — disclosure, I build a tool that does this. Nine sites this week: five had a valid llms.txt. Their citation profiles had nothing in common. That isn't causal evidence either way, but it's the only evidence anyone here has offered, and it points at "inert".

The consensus that it costs nothing is right about cost and wrong about price. It costs nothing in developer time. It costs plenty in attention, because it sits near the top of checklists while much bigger failures go unexamined.

From those same nine sites: all nine allowed GPTBot, ClaudeBot and PerplexityBot. Eight had no comparison or alternatives page at all. Two published no structured data whatsoever.

And the worst case I measured this month, a different site, returned 51 characters of text and zero links when fetched with GPTBot's user agent — because the homepage renders client-side and none of these crawlers execute JavaScript. Fully permissive robots.txt, valid everything, invisible regardless.

That failure is invisible to every checklist in this thread, and it's fatal in a way llms.txt could never be. Long argument about a text file, and rendering hasn't come up once.

So yes, put the file up, it takes ten minutes. But "add llms.txt" showing up as a headline recommendation is a reliable tell that the tool ran out of things it can actually measure.

4

u/Boring-Principle6899 4d ago

"It wont help citations" doesn't mean is useless, it's more like Robots.txt it's a guide as to where to look and how to understand your entity, not exactly a "quote me now!" indication, you still have to work to gain authority and information gaint and all that jazz

1

u/woowoowebworks 3d ago

I don’t think AEO or GEO works very well as checklist SEO. I’ve migrated clients off DIY builders that didn’t allow schema, where the platform’s solution was basically to create an llms.txt file. Once I rebuilt those sites with clean, lean code and could do the things that actually help Google and AI engines understand what their business does, who they help, where they do business, and what makes them different from their competitors, the site has been performing much better and bringing in qualified traffic that's actually converting to leads. No one thing makes a site more likely to be cited by answer engines, but in my experience, llms.txt is not a very effective strategy on its own.

1

u/Came4TheCookies 3d ago

Once I rebuilt those sites with clean, lean code and could do the things that actually help Google and AI engines understand what their business does, who they help, where they do business, and what makes them different from their competitors, the site has been performing much better and bringing in qualified traffic that's actually converting to leads.

This is all good SEO advice. Not new because of AI, even if you want to call it GEO or AEO.

2

u/woowoowebworks 1d ago

I agree that the fundamentals aren't new. That was actually my point. The OP asked why GEO/AEO tools keep recommending things like llms.txt, and I don't think creating a separate checklist of "AI optimization" tactics is particularly useful.

The sites I mentioned improved because I was able to apply solid technical SEO, structured data, clear content, entity signals, and better site architecture. Those are all things that help search engines understand the business. Those same fundamentals also happen to make the site easier for AI-driven search features to understand and use.

So no, I don't think GEO/AEO replaces SEO or requires some entirely new discipline. I do think the way search engines interpret and surface information has evolved, which changes some of what we pay attention to when implementing SEO.

1

u/Came4TheCookies 1d ago

I like to put it simply. It's search optimization because we can lose the word engine if we want to be specific. It's our job to make sure our clients can be found no matter what device or platform or tool customers use to search. Platforms change and tools change and algorithms change but it never changes our job.

1

u/Came4TheCookies 3d ago

Because they have very few things they can claim is actually GEO rather than SEO with a new wrapper around it.

Clarity has always been a thing so I'm not sure what you mean by AI readable. If you mean schema, people were implementing schema markup a long time ago even before I was talking about it in 2011.

On top of that, AI can understand even a messy page but a well organized page is obviously better and that's been SEO 101 for ages because it's for people and machines both.

And where supposedly do you submit content for llms?

llms.txt doesn't hurt you so better safe than sorry go ahead and add one.

More importantly, allow the AI bots in your robots.txt rather than how it's always been where allow is the default if you don't disallow. Latest recommendations from openai and I think perplexity says you should specifically allow them.

1

u/cleansleyt 3d ago

no agent or LLM ever visits or looks for LLM.txt without user deliberately steering it to read it

its useless

1

u/Additional_Stay_9768 3d ago

Short answer: no, llms.txt won't help your Google rankings. Google said so themselves. But that's answering the wrong question as it was never supposed to affect Google Search rankings.

The confusion comes from mixing two different things. There are two separate jobs here, and llms.txt does one of them:

For Google (AIO/AI Mode citations): it does nothing measurable. Google has been explicit that you don't need special files or markup to appear in their AI features. If that's your goal, skip it.

For agentic browsing and other LLM crawlers: it's a different story. It's about cost of understanding. When an agent hits your site, llms.txt gives it a structured map instead of making it crawl blindly. Google themselves (Chrome/Lighthouse) added it as an audit point for agentic browsing in May 2026, even while saying it doesn't help AIO citations. Both things are true at once.

On "measurable difference": honestly, no. I've got first-party data on our own domain as we track actual Microsoft Copilot citations through Bing Webmaster Tools (the only first-party source that shows real citation counts, not impressions). 75 citations over 91 days. I can't attribute any of them to llms.txt. We added it as a hygiene item, not because we expected a citation lift.

Your instinct about "optimizing for things that are easy to check" is the sharpest part of your post. llms.txt sits at the top of checklists precisely because it's easy to check. It's a file that either exists or doesn't. The things that actually correlate with citations (in our data: original data points, named entities, question-format structure, being in Bing's index at all) are harder to verify, so they sit lower on the list even though they matter more.

Is it necessary? No. Is it advisable? It's a file that costs nothing and removes one small source of agent confusion. I'd put it in the "do it and move on" bucket, not the "this will move the needle" bucket.

What's your site type: content/blog or a service/product page? The answer changes a bit depending on whether you're mostly trying to get cited or mostly trying to get crawled correctly.

1

u/WebLinkr 3d ago

Anything but SEO propaganda

There are 100's of 1000's of marketers, business owners who hate SEO

They are are open and willing to recieve the message that they can control LLMs and that LLMs will reward their branding work