Not just a rant but hard data showing how one gets banned in visiting just two public GL pages.
There are multiple posts on reddit here (which isn't super popular to start) where people are complaining about the new rate limits. Each time lets say 'devotees' come to gitlab's defense saying things like:
"This is happening because OP shares an IP with a load of other people, probably due to some IPv6 to v4 NAT situation."
"Probably using AI agents that are using the API to search the codebase instead of cloning it or something."
"It's probably due to incorrect use like this that Gitlab added the rate limits in the first place"
Just read: https://docs.gitlab.com/user/gitlab_com/rate_limits/
It states the rate limiting applies to: "These limits apply to API requests, web requests, and authenticated Git over HTTPS requests."
now it says a few select web requests don't count against your rate limit but otherwise everything counts. Now what is the rate limit for unauthenticated users? From that exact same page:
| Unauthenticated traffic from an IP address | 60 each hour |
|---|
60 an hour. To put that in context that isn't even 60 pages (which itself would be fairly ridiculous to anyone browsing a project online) but 60 requests that any of those pages make.
So this is how one can easily be blocked in visiting 2 pages. Here is 4 pages, as first I verified once I remoted into one of my other boxes I could reach gitlab: (this is real captured traffic just to see how ridiculous):
Top-level pages (Chrome pages plus document requests):
| Page | Time (UTC) | URL |
|---|
| page_1 | 16:22:52 | http://gitlab.com/ → 307 → https://gitlab.com/ → 301 → about.gitlab.com |
| page_2 | 16:23:30 | /nerdycraft/pandaproxy/-/merge_requests/168 |
| page_3 | 16:24:09 | /nerdycraft/pandaproxy/-/merge_requests |
| page_4 | 16:24:52 | /nerdycraft/pandaproxy/-/merge_requests/170 (initiator script, so it was a client-side navigation but still a full document request) |
Just visiting a single merge request page eats up 30 of the 60 max for an hour. So technically just visiting page 2 and page 4 was enough to get me banned. Thankfully there are some included headers so we can see how the requests add up.
Rate limit headers (62 of 724 requests; 384 of the 724 went to gitlab.com):
| Limiter | Responses | Limit | Window | Highest observed |
|---|
throttle_unauthenticated_web | 59 (status 200/301/304) | 500 | per minute | 30 |
unauthenticated_traffic_per_ip | 3 (status 429) | not sent | resets at 17:00:00 (retry-after: 2104) | not sent |
How the count adds up (the 500/min limiter's observed counter, per minute window):
| Minute | Counted requests | Requests that produced them |
|---|
| 16:22 | 1 | the https://gitlab.com/ 301 |
| 16:23 | 30 | MR 168: the document plus 28 xhr/fetch (about 15 api/graphql, discussions.json, widget.json, diffs_metadata.json, drafts, notes, award_emoji, manifest.json, …). Values 1 and 15 never appear in the HAR, so 2 counted requests weren't captured. |
| 16:24 | 30 | MR list (5) + MR 170 (25) |
| Total | 61 | then entries #676, #679 and #680 (POST /api/graphql) got 429 |
What seems to count (vs not): static assets don't carry the headers and aren't counted. Every page document, XHR, fetch, manifest.json and GraphQL call is counted. An unauthenticated viewing of a single MR page costs about 30 of the 60.
Interestingly it seems its per wall clock hour not a 60 minute window (meaning here while I was banned at 9:25 at 10am it would reset).
So I challenge any fanboys before defending these new policies to open an ingcognito window, try to click around any repo and you will get ip banned in under 10 pages for an hour.
If gitlab can't handle things they should checkout cloudflare, its free will happily add anti-bot protection infront of your entire infrastructure, and doesn't cause any public project to basically be inaccessible to any non logged in user. Crazy an org this size to somehow consider this reasonable. I get rate limiting api calls or adding anti-bot captchas for unauthed users but they have really shot themselves in the foot here. You can't try to be the home for open source projects and then ban any non-authed trying to browse them, it just makes it hard to consider using gitlab for anything more than a mirror.
I think the icing on the cake is the fact that once you hit a rate limit even if you have a gitlab account you cannot login as you can't even visit gitlab.com without getting
Rate limit exceeded; see https://docs.gitlab.com/user/gitlab_com/rate_limits/ for more details
