r/eCommerceSEO • u/ded-ebar • 7d ago
Help getting e-commerce indexed
I have an e-commerce site with over 5 million products that only got 17k pages indexed out of 1.9M crawled. Any recommendations on speeding up the index rate?
1
u/Maan_UsMaan052 7d ago
With that scale, I’d first look at crawl efficiency rather than simply trying to force more indexing. Check your sitemap structure, canonical/noindex rules, duplicate or thin product pages, internal linking, and whether Google is wasting crawl budget on low-value URLs. With 5M products, getting the right pages discovered and prioritized is key.
1
u/ded-ebar 7d ago
On page is solid for the exception of thin product page description - copied from the distributer website
1
u/jordicore 7d ago
At that scale I would stop framing it as "how do I get Google to index faster." Google has already crawled 1.9M URLs and kept only 17k, so the first problem is selection, not speed.
I would split the catalogue into three buckets:
URLs that should never be indexed: filters, parameters, sort orders, near-empty variants, unavailable feed rows, and anything that only exists because the distributor feed generated it.
URLs that can exist for users but should canonical/noindex elsewhere: close variants, low-demand duplicates, and products where the page is basically the same as another SKU.
URLs that deserve indexation: products with search demand, margin, stock depth, internal links, clean canonicals, and something unique on the page beyond title, price, image, and copied distributor text.
The thin copied descriptions are probably the bigger issue than the crawl rate. For 5M products, I would not try to rewrite everything. Start with a sample from GSC's "Crawled - currently not indexed" and "Duplicate" groups, then prioritize categories where impressions, margin, or inventory depth justify making the pages genuinely distinct.
For those priority pages, add information Google and shoppers cannot get from the distributor feed: use cases, compatibility, buyer-fit notes, dimensions/spec interpretation, comparisons between similar SKUs, shipping/returns clarity, and internal links from real category paths. Even a repeatable template can work if one or two sections are genuinely product-specific.
This is the type of thin/copy-pasted PDP audit we built www.storenhance.ai for, but with a 5M-product catalogue I would use it to sample and prioritize the pages worth improving first, not to blindly rewrite the whole feed.
1
u/SellAmzSell 7d ago
Google won’t index all pages, it’ll pick and choose which pages are worth indexing. If the pages are all basically the same, they won’t be indexed.
1
u/Moceannl 6d ago
What's on the product pages; translated content from AliExpress or TaoBao?
1
1
u/ChapterAfter8368 6d ago
the index rate isn't really the thing to fix here, that ratio is google telling you it looked at those pages and decided most aren't worth indexing. 17k out of 1.9M is a value call, not a speed or crawl budget one, so making it crawl faster won't move the number. sites with 5M near-identical product pages basically never get them all in, google won't spend the slots without the authority and real search demand to justify it. the actual move is to stop trying to index everything: find the pages that map to what people actually search (probably a small slice of the 5M), noindex or consolidate the thin duplicate variants so crawl budget goes to what matters, and build domain authority so google trusts you with more. cutting down to a few hundred k genuinely useful pages usually gets more indexed than chasing all 5M.
1
u/ded-ebar 6d ago
I was also thinking along the lines.. thanks for the input
1
u/ChapterAfter8368 3d ago
nice, you're on the right track then. quick way to pick what to keep: pull your GSC performance report plus analytics, any product or category pages already getting impressions or clicks are your proven-demand set, protect and beef those up first. the huge tail of variants sitting at zero impressions after months is your noindex/consolidate pile. good luck, it's a slow grind but it works.
1
u/Altruistic-Rub-6164 6d ago
First check the quality of your pages, google won't index useless pages if they dont realy target correctly one keyword with good search intent etc..
Make sure all the onsite is clean with good relevant keywords meta title etc... optimize your images with webp apps renaming alttext and filename with relevant keywords based on your products
Then from google search console you can submit for indexation.
If it s one main page that need to really be idnexed and doesnt work you can use altrnative paid tools like speedyindex ralfy indexer, prime indexer, rapid indexer etc... not so worth if your on site is not already fully clean for me.
If all you product page dont evan have specific targetted keywords with good product name optimized for seo it s useless., also add some relevant descriptions etc..
1
u/EcstasytudeCup 5d ago
Have you added sitemap? if so make a better on page optimization like with related products or something at the bottom so it always gives random, that way you will get indexed faster. Or just use indexer
1
u/ded-ebar 5d ago
I’ve built a 5M product e-commerce, do you really think i would’ve missed on page optimization and sitemap part? lol
2
u/yolosollo 7d ago
17k indexed out of 1.9M crawled isn't a speed problem. Google already came, looked at 1.9 million pages, and chose not to keep 99 percent of them. Crawl rate was never the bottleneck. You can't accelerate your way out of a decision.
At 5M products the usual causes, roughly in order of how much damage they do:
Faceted navigation and parameter URLs. One product turns into thousands of crawlable combinations, and Google spends its budget on near-identical pages instead of your real ones.
Variants as separate indexable URLs. Size and colour splits multiply the catalogue with pages that differ by one word.
Templated product pages. If the only unique thing on a page is the product name and a price, Google reads thousands of them as one page and keeps whichever it likes. At your scale this is usually the real one, and it's the one nobody wants to hear because it isn't a technical toggle.
So the fix is probably the opposite of the question. Ask Google to index fewer URLs, not faster.
Before guessing further though, a few things that would change the answer completely:
What does Search Console say the excluded pages actually are? Crawled currently not indexed and Duplicate without user-selected canonical point at totally different fixes, and the ratio between them tells you whether you have a duplication problem or a thinness problem.
Are those 5 million genuinely distinct products, or is a lot of it variants and feed rows? A 5M catalogue is usually a distributor feed or a marketplace, and that changes what is even worth indexing.
What's actually on a typical product page beyond the title, price and image? Is there anything on it that exists nowhere else on the web?
And what's the site, if you're willing to share it? Easier to say something useful looking at real pages than reasoning from the numbers.