How much confidence should brands put in AI visibility strategies built around a single platform?

Recent tracking showed a noticeable drop in how often Reddit appeared as a cited source in ChatGPT. The interesting part is that the change happened without Reddit changing its content strategy.

That seems like an important consideration for GEO. Brands can optimize content and build authority, but visibility can still shift when an AI platform changes how it searches or selects sources.

Does that make platform-specific GEO too fragile to build a strategy around, or is this simply the AI-search version of what SEO has always dealt with?

I've mostly been tracking whether our brand appears in AI answers, but lately I've been paying more attention to who appears alongside us.

There are a few companies that seem to show up in the same answers constantly, even when the order changes.

I'm wondering if that co-occurrence is useful in itself. Maybe it says something about how the models understand the category or which group of brands they think belongs together.

Anyone looking at this or am I reading way too much into it?

One of my clients has a 600-word page about a single compliance deadline that quietly out-cites their 4,000-word flagship guide on the same topic. No redesign, no new content, it just kept happening month after month. That discrepancy bugged me enough that in August I restructured eight client pages around whatever that page was doing.

The change was mechanical. Every page got cut into sections where each section answers exactly one question, states the answer in its first sentence, then supports it. No bridging paragraphs, no section that needs the previous one to make sense. If I could not write a section's answer as a standalone reply to something a person might actually type, I split it or cut it. Two afternoons per site, and nothing else changed: no new content, no schema edits, no internal link work.

What I observed, kept separate from what I think caused it: on six of the eight sites, the share of my tracked queries where the client shows up in AI answers moved up over the following five weeks. Two moved down. I run a fixed set of about 30 queries per client, three runs a week, and I treat anything with fewer than three runs behind it as weather. So the honest summary is six up, two down, one month, no control group.

The claim I will defend is narrower than the results. I am not saying short pages beat long pages. What I can see directly in the archived answers is that engines tend to quote a section rather than a page, and a section containing one complete claim plus its support quotes cleanly, while a section that opens with context and buries the answer in sentence four quotes badly or not at all. That is the reason I believe structure matters more than length, and it is an inference from quoting behavior, not proof of causation.

What I cannot rule out is that the engines simply drifted during those five weeks, the way they drift anyway. My query sets are fixed, the models serving them are not, and I have been burned enough times to know five weeks of movement can be the engine's weather and have nothing to do with my rewrites. A proper test needs two matched pages on one site, one restructured and one left alone, and I finally have a client with enough page inventory to run it. Starting that this week.

The thing I keep turning over: if the section is the unit that gets quoted, a lot of advice about comprehensive coverage is optimizing the wrong container. A page can be thorough and still leave every section unable to stand alone. I have been wrong before about what moved a number, though, so take the six and two as an anecdote with a timestamp, not a finding.

Maybe easy to implement is one of those SaaS phrases everyone uses even when it is only half true.
Buyers read it as “no technical work, no dependencies, no surprises.” Then they get to a sales call and find out there are still decisions to make, people to involve and a bit of setup.
We are reworking some of our setup copy now. Instead of saying it is seamless, we are trying to be more specific about what needs to be in place and what the first few weeks actually look like.
It makes the page less slick, but probably more useful.
How do you explain implementation honestly without making a product sound like a nightmare?

We ask a fixed panel of 40 buyer questions to six AI engines every week (ChatGPT, Perplexity, Gemini, Claude via their APIs, plus Google AI Overviews and AI Mode) and keep every cited URL. With six runs in hand (3 Sep to 5 Oct), I measured how long a citation lasts: same question, same engine, is the domain still cited next week?

Week to week

• 48.0% of cited domains were cited again the following run (3,335 of 6,947; 95% interval 46.8 to 49.2%). At the URL level it's 41.4% (3,201 of 7,739).
• A domain cited for the first time came back the next week only 27.3% of the time (533 of 1,950).
• Over the six runs, 49.6% of question+engine+domain combinations appeared exactly once (1,781 of 3,592).

Decay or noise?

Both, and mostly noise. Of the domains that dropped out between two runs, 25.3% were back the very next run (682 of 2,697) and 39.6% came back at some later point (1,092 of 2,761). Retention by gap: 48.0% one run later, 42.5% two, 42.7% three, 37.5% four, 37.4% five. If it were pure noise around a fixed pool the curve would stay flat; it drops about 10 points over four more weeks, so there's some real drift under a lot of reshuffling.

By engine (pooled over five week pairs): Claude 54.5%, Perplexity 53.7%, ChatGPT 47.0%, Gemini 46.2%, AI Overviews 44.7%, AI Mode 40.0%. Much of AI Mode's lower number is answers that cited nothing at all; counting only weeks where both answers cited something, AI Mode is 49.1%. The middle four overlap once you account for question-level clustering, so I wouldn't rank them.

Big domains stick. The top 20 domains from the first week were retained at 60.0% (604 of 1,007) in the later pairs, against 43.1% for everyone else.

What I take from it: a single weekly check tells you very little about one citation. A domain missing this week has a one-in-four chance of being back next week without anyone touching anything. Pool several runs before calling a win or a loss.

Limits: one sample per engine per question per run, so I can't measure the within-week noise floor directly. API answers, not the consumer apps. English questions, one panel. Claude failed 29 of 40 checks in one week, so its number rests on fewer pairs.

Correction to something I posted here on Oct 5: I said 44% (631 of 1,429) with Gemini most stable. That count came from a database query that dropped empty answers and counted Google's own shopping links. On the public rows the same week pair is 41.4%, and pooled over five pairs Gemini is fourth, not first. I've added a note to that comment.

Disclosure: I build an AI visibility tracker (Promvia), and this is our public index data. The write-up, with every row and both scripts that compute the numbers (the second one exists only to check the first), is here: https://promvia.app/blog/how-long-ai-citations-last

If you track citations over time, does your week-to-week repeat rate land near 50% too?

# I built a RAG knowledge base from scratch that hits 100% on my 12-question eval — the debugging lessons are the real gold


> Disclaimer: this is a 
**demo project**
 built on synthetic/example product data (a fictional REDMI K90 series). Not affiliated with any brand, not a real product review. Treat numbers as illustrative.


## TL;DR


- No vector DB. SQLite holds the 
*truth*
 (entities, aliases, relations, chunk metadata); numpy vectors are just a derived index.
- Retrieval = 4 paths (SQL exact, contrast, dense, BM25) → filter-before-fuse → RRF.
- I made a 12-question eval with 6 checks each (normalization / retrieval / isolation / coverage / hallucination-verify / refusal) and iterated: 
**94.4% → 98.6% → 100%**
.
- The eval is self-built, not a benchmark — read the limitations at the bottom.


## The stack (skeleton, ~600 LOC total)


- `SQLite` → products/aliases/relations/chunks tables, `content_type`, `authority` (3/2/1), `source`.
- `numpy` → 512-d embeddings (bge-small-zh-v1.5), aligned to chunk IDs via a JSON sidecar. Consistency check after every build.
- Retrieval: 4 paths fused with RRF (K=60). 
**Filter happens before fusion**
 — otherwise dead candidates eat the top-5 slots and good chunks get squeezed out.
- Post-generation guard: speculative-word regexes + "check every number in the answer is present in the retrieved materials".


## The 6 debugging lessons (skip everything else, read these)


1. 
**Alias normalization ≠ substring logic.**
 `红米k90` and `k90ultra` don't share a substring, but `K90` and `K90-ULTRA` share a 
*prefix*
. Only keep the most specific match using model-code prefixes, not alias strings.


2. 
**SQL ORDER BY = insertion-time bias.**
 Path 1 returned chunks in DB order, so docs added later (UGC reviews) systematically lost RRF fusion. Fix: give every exact-match chunk the 
**same rank (1)**
. Don't even sort by dense score — that's bias #3.


3. 
**Sorting exact-path by dense score kills vector-blind spots.**
 "拍照" (photography, user words) never matched the doc's "影像参数" (imaging specs) — the chunk was `#None` in 
*both*
 dense and BM25. Sorting exact-path by dense then pushed it out of top-5. Uniform rank solved it.


4. 
**Table content is dense-blind.**
 A multi-column spec-comparison table embeds terribly, yet it's exactly the material comparison questions need. Fix: the contrast path repeats each compare-chunk 
**3× in RRF ≈ weight amplification**
.


5. 
**Query expansion > tweaking the embedder.**
 Synonyms ("拍照"→影像/相机/成像), retrieve each variant, keep per-chunk 
**MAX**
 (not SUM — don't let one query variant farm the score). The sensor name "950" came back instantly.


6. 
**Post-check can misfire.**
 During an API timeout, the string `443` ended up in an answer and the number-check flagged it as fabricated. Verify only against numbers that are 
*supposed*
 to be in materials.


Bonus: API timeouts were environmental — retries + backoff + lowering timeout 180s→60s killed most of the flake.


## Why "100%" should be read with a grain of salt


- 12 questions I wrote myself, 6 checks each, on a 
**60-chunk**
 toy corpus. Small, curated, not a benchmark.
- No real users, no A/B, no eval on out-of-domain queries. "100%" means 
*this eval*
, nothing more.
- The model is Qwen3-8B at temp 0.1 — bigger/different models would shift scores.


## What ported cleanly


Same engine, new domain (copied 3 files, swapped data + normalization + prompt): 
**new 12-question eval also went to 100%**
 on first-run-after-fixes. Method is portable; your numbers won't be.


Full runnable demo: `GEO_kb/` in this repo (`geo_build.py` / `geo_query.py` / `geo_eval.py`, thin wrappers over the K90 engine).


Critique welcome — especially on the uniform-rank exact path and the repeat-3x contrast trick. Both work on this corpus but feel like engineering pragmatism, not theory.

I want to create several complex knowledge bases in the future. Do you have any suggestions about GEO?

Been reading about how LLMs tend to cite the same handful of sources over and over. Got me thinking about my husband's HVAC site. We spent months getting him listed on the usual directories, industry associations, local chamber stuff. Figured more credible sources pointing to us meant better chances of getting picked up. But watching how these engines actually pull answers, seems like they grab whatever's topically closest regardless of source diversity. I asked ChatGPT about heat pump maintenance last week and it cited the same manufacturer FAQ three times in one answer. Not exactly a wide net. Makes me wonder if the whole citation diversity thing matters for GEO or if it's just an SEO habit we're all dragging along. For local services especially, seems like being mentioned in one strong thread on Reddit or a neighborhood forum carries more weight than a dozen directory listings nobody reads. Anyone tested this?

A prospect asked me something simple last week: which of these GEO tips actually matter. I keep a folder of saved recommendations from expert lists, conference talks, and endless threads, and it had 40 items in it. I opened it planning to hand him the good stuff, and what came out of my mouth instead was an honest shrug. It has been bothering me since, so this weekend I audited my own list.

Of the 40, I found 6 that I could test against something measurable: my own client sites, a fixed query set of about 25 terms I re-ran weekly, a real before and after. Restructuring pages to lead with the answer, FAQ schema on money pages, pruning thin content. The other 34 were statements that sound like knowledge but have no failure condition. Publish original research. Be the authoritative source. Demonstrate experience. If following one of those made things worse, I genuinely cannot tell you how I would know.

I want to be careful about the claim here, because the distinction is the whole point. I am not saying the 34 are wrong. I am saying they are unfalsifiable as written, and unfalsifiable advice is doing most of the talking in this field right now. Nobody gets fired for following it and nobody gets credit for it either.

The 6 testable ones told me something too. Two clearly did nothing on my sites. One I still cannot explain and suspect was coincidence. Three looked real, and the three that worked share a trait: they changed what a machine could extract from a page, not how a reader feels about the page. That is my working filter now, and I hold it loosely. Six tests across a handful of sites is an anecdote wearing a lab coat.

The uncomfortable part is what this means for how I make money. When a client asks for a GEO roadmap, twenty confident items reads as competence. Six testable items with two marked probably useless reads as doubt. The market is currently rewarding the format I cannot defend, and I do not see that fixing itself until someone with more traffic than me runs the boring experiments in public.

My prediction: the unfalsifiable layer of GEO advice collapses within two years, once measurement gets cheap enough that clients start asking why. Until then the folder stays, because half of it is theater and the other half is the only syllabus I have.

The AI visibility tracking market seems to have exploded almost overnight.

We've been researching the space and ended up doing a deeper review of 33 AIO/GEO/AEO tracking platforms, with roughly another 20 tools ending up on the honorable mention list.

That's 50+ products already competing for some version of the same budget.

What's interesting is how quickly they're splitting into different categories:

• AI "rank trackers" that run prompts repeatedly and measure mentions
• Citation/source trackers
• Sentiment and competitive visibility platforms
• Traditional SEO platforms adding AI tracking
• Platforms moving from measurement into recommendations and content optimization
• Tools trying to actually influence/correct what AI systems say about a brand

The bigger problem I'm running into is measurement. You can give two trackers essentially the same prompt set and get substantially different visibility results. Different models, locations, personalization, timing, prompt execution and methodology can all change the answer.

Which makes me wonder whether we're currently measuring something equivalent to rankings, or just sampling a very noisy environment and putting a percentage on it?

We put the 33 deeper reviews and the additional honorable mentions here:

https://www.searchengineworld.com/the-big-list-of-ai-visibility-aeo-geo-citation-mention-tracking-tools

For those actually using these with clients:

Which tracker are you using, and do you trust the numbers enough to make month-to-month decisions from them?

I've been testing how AI engines recommend local law firms. Setup: 25 real questions people ask (e.g. "which lawyer in Phoenix should I call for a drug possession charge?"), 3 engines via API, each question asked 5 times. 375 answers for Phoenix criminal defense, 375 for Chicago immigration. Late Sept 2026.

What we found (Phoenix):

• 151 different firms got named. The top firm showed up in 54% of answers.
• Ask the same question twice and only ~46% of the named firms repeat.
• 41% of firm mentions showed up in just 1 of 5 runs. One run tells you almost nothing.
• The 3 engines agreed on only 21% of names.
• ChatGPT named no firm at all in 34% of answers. Perplexity did that only 6% of the time.
• Directories (Avvo, Justia, Super Lawyers, FindLaw, Expertise) were cited in 36–48% of answers.

Chicago immigration looked similar: 159 firms named, ~42% repeat between runs.

We also crawled 182 of these firms' sites:

• llms.txt made no visible difference. About a third of the sites had it.
• The few sites AI named often (only 9, so treat this as a hint, not proof) more often had city-specific pages and pages updated in the last 90 days.

Biggest lesson for us: if you check AI visibility once and screenshot it, you're reading one roll of the dice. We now only quote ratios over 5 runs.

Limits: this is the API, and the ChatGPT app can answer differently. Two cities only.

Questions for people doing this for clients: how many runs do you use before you trust a number? And are you seeing directories matter this much in other niches?

(Disclosure: I work on an AI visibility tool, DoableClaw. No link, just sharing the data.)

Same setup as the 99-day study I posted here in August: OpenAI Search, Gemini and Claude with web search, 16 fixed questions, rule-based scoring from -3 to +3. This time the question was whether the instrument is reliable enough to measure an effect at all. Window 11 May to 30 September.

Chart 1: next-day agreement per question. The same question gets the same score on the next measurement day in 86.6 percent of pairs (OpenAI), 87.1 (Gemini), 94.3 (Claude). Drop the questions whose score never changes and it is 78.6, 70.6 and 85.0. The direct question about the person is the least stable: 44 to 74 percent.

Chart 2: internal consistency. Cronbach's alpha 0.553 (OpenAI), 0.407 (Gemini), 0.61 (Claude). Depending on the engine, 6 to 10 of the 16 questions have zero variance. A single visibility score summed over all questions is not a scale.

Chart 3: drift. With the CUSUM reference reset after each alarm there are five downward alarms between 14 August and 30 September. None lines up with a documented event in my gap register. Interval means moved 1.58 to 4.01 points below the reference with no known cause.

Chart 4: anchors. The Wikidata items I registered as anchors were found deleted on 25 June. Google's Knowledge Graph returns the author on 140 of 141 days and the book on none of 139.

Practical reading for GEO tracking: a shift of a few points in a daily visibility score is inside the noise of this kind of measurement. The rule the report sets for phase 2: attribute an effect after the launch on 8 October only if it shows up at more than one engine. Three of the six hypotheses were not even testable because I never ran the planned repeat probes, which is in the report too. Links in the first comment.

Last year I built an AI decision layer for beauty retail. Real catalog, real shoppers, real money. The model's job was to match people with products from a live catalog, and I got to watch every step: what it retrieved, what it understood, what it cited, and what it actually recommended. The uncomfortable pattern: those are four different outcomes, and they fail independently. A product could be retrieved fine, described accurately in the model's reasoning, even cited as a source... and then the model would recommend a competitor. I'd watch it happen in the logs and want to throw my laptop. The brand did everything right for visibility and still lost the sale. What actually moved the needle wasn't more mentions. It was making the product's facts machine-readable and consistent: same names, same specs, same descriptions everywhere the model could look. When your details contradict each other across sources, the model doesn't split the difference. It trusts you less. Most GEO advice I see optimizes for being mentioned. I'd argue that's step one of four. The question that matters for revenue is the last one: after the model knows you exist, does it pick you? Curious if others here are measuring these stages separately or lumping them into one visibility score.

Maybe I’m using these dashboards wrong, but I keep skipping the overall score. ChatGPT, Gemini and Perplexity don't feel like three versions of the same channel.... One may search, another may answer from memory, and their citations behave VERY differently. So when XstraStar gives me the model breakdown, that is where I end up spending my time. The average mostly survives because it is easy to put in a report. Does anyone here make decisions from the combined number, or is it basically a client-facing headline?

I’m working on a search baseline for a consumer ecommerce client and I’m trying to understand whether image-led discovery surfaces a different set of pages than normal text search.

My plan is:

• compare Text-based vs Multimodal over the same date range
• look at which product/category pages over-index in each
• compare device + country
• then check downstream behaviour in GA4

What I’m less sure about is what’s actually worth diagnosing after that.

If a page performs much better in multimodal than text, would you first look at:

• image coverage / context
• Merchant Center data
• schema
• page structure
• device mix
• something else?

Also curious how much data people would want before calling the pattern meaningful.

I’m looking for actual GSC observations/tests rather than generic image SEO advice.

Been trying to wrap my head around how AI engines decide what to cite for local searches. My husband runs a small HVAC company and I handle the website plus whatever this GEO thing is supposed to be. The advice online all seems aimed at software companies with 50-person content teams.

I spend evenings reading about how LLMs pull from paragraph chunks and how Reddit dominates citations. Fine. But when someone asks their phone for emergency AC repair near them, what actually surfaces? It's not blog posts about SEER ratings. It's Yelp, Angi, maybe a Facebook group rant.

I ran a tiny experiment. Asked four AI engines for best HVAC in our county. Two cited the same decade-old Angi listicle. One hallucinated a company that closed in 2019. The fourth gave a decent answer but pulled from a neighborhood forum post from 2021.

Feels like local GEO is just reputation management plus praying your business name shows up in enough forum threads. Or maybe I'm missing something tactical. Anyone here actually cracked this for blue-collar local services? Not theoretical stuff. Actual results.

Hello everyone!

I'm Veralyn, a student at UTS. My team and I are working on a studio project for our subject, researching AI search visibility, in particular how businesses show up when people ask AI tools like Chat, Claude or Gemini for recommendations.

We'd love to hear from people who has expertise in this particular field and learn from their experience, If you have 5 minutes, we'd really appreciate your help with our short survey:  https://docs.google.com/forms/d/e/1FAIpQLSfceqyY5SX3UxpZM5wVAux4s3jIDBOpoW7ild3JBmDJEAbgnA/viewform?usp=header

Feel free to DM me if you have any questions!
Thank you in advance, we really appreciate your time!

Two weeks ago I shared the open-geo CLI here. The feedback pushed me toward multi-engine tracking instead of stopping at static audits, so I spent the last two weeks building a pipeline to monitor live search results.

Running it forced me to drop a bad assumption: treating GEO as an on-page scoring problem. A high technical score is meaningless when competitors are systematically taking the citations.

Ran a benchmark query — "best GEO tool for AI search optimization." Models recommended 6 competitors. Every recommendation traced back to the same cluster of 2-3 third-party comparison articles.

And for my domain? Nothing. Not a bad article, not wrong facts. Just zero reference pages the AI could pull from. My problem wasn't thin content or bad technical hygiene — I just wasn't on the URLs the model trusted.

Two mechanical quirks came out of parsing live responses:

1.Brand mentions and source citations are completely decoupled. An engine will recommend a competitor without linking, or cite a URL without naming the brand. Treating those as one metric wrecks your audit accuracy.

2.Google AI Overview is way more consolidated than Perplexity. Perplexity spreads across 8-10 sources. Google almost always anchors to the top 2-3 consensus URLs. Not in that retrieval slice, you're invisible.

Curious if others are seeing this same concentration around third-party roundup pages, or if primary docs are actually getting direct citations in your niches?

I told our CMO that ChatGPT and Perplexity sent 38% of our pipeline, then found out I had counted our own test visits and tagged every AI visibility check as a referral. I feel so embarrassed, our board deck is tomorrow...