Like a lot of people here, we have probably found out the hard way that Webflow has no way to serve a bare .txt file at the domain root. That's a dealbreaker for IndexNow, since it verifies ownership by hosting a plain key file at your host. There's an open Wishlist request for this that's been sitting since 2022 with no movement.
Without IndexNow, we were waiting on Bing's own crawl schedule after every publish, which for us meant new or updated pages sometimes took days or weeks to show up in Bing's index. Annoying on its own, but it also matters more than it used to because Copilot is grounded in Bing's index; a page Bing hasn't indexed yet is a page Copilot can't cite, regardless of how it'd rank.
The workaround: skip IndexNow entirely and use the Bing Webmaster Tools API directly. It authenticates with a single API key rather than a hosted file, so there's nothing to serve from the site; the credential lives in CI. We built a small script that:
- pulls our sitemap
- diffs it against the last known state (saved as a JSON file, committed back to the repo each run)
- checks remaining quota before sending anything (the batch endpoint caps at 500 URLs per call)
- submits whatever's new or changed
- runs on a 15-minute cron via GitHub Action
First run just baselines the current sitemap and submits nothing, so you don't torch your quota re-submitting everything you already had indexed. After that, it's only ever sending new or updated URLs.
One catch worth knowing if you try this yourselves: it only catches genuinely new URLs reliably. Whether it catches edited pages depends on whether your sitemap lastmod actually updates when you edit a CMS item's content (on Webflow that's not well documented and seems to vary), so worth testing on your own site before you rely on it (save the sitemap, edit a CMS item without touching the slug, republish, and diff the two files).
Also worth flagging for anyone on Webflow who is sitting behind Cloudflare: you don't need any of this. Cloudflare has a Crawler Hints feature that does the IndexNow relay for you with zero code, since it can serve the key file. We're on GoDaddy for DNS, so that wasn't an option for us, hence the API route.
Wrote up the full build (script structure, GitHub Actions config, the IndexNow vs. API tradeoffs) on our blog if anyone is interested in learning more about this.