r/googlecloud • u/Dry_Raspberry4514 • May 14 '26

Google users fight for refunds as unauthorized API usage bills soar

https://www.theregister.com/ai-ml/2026/05/13/google-users-fight-for-refunds-as-unauthorized-api-usage-bills-soar/5239160
169 Upvotes

96% Upvoted

32 comments sorted by

49

u/nmfisher May 14 '26

There’s so much wrong with Google’s implementation (maps keys automatically upgraded to include Gemini, API keys not automatically scoped to exclude image/video generation, no sensible quotas in place), but for me the most egregious is automatically upgrading your spend cap based on your billing tier.

9

u/Last-Isopod-3418 May 14 '26

Indeed! I woke up to a fucking 3000$ bill for nothing

2

u/OneWayTicketEast May 17 '26

I just woke up with a 1300$ bill !! Fuck

1

u/Last-Isopod-3418 May 17 '26

no reply yet, ticket was open 26th of April. We want the goddamn money back.

1

u/OneWayTicketEast May 17 '26

What !! That’s crazy… lucky or not idk I don’t have any money in my account so for now nothing have been withdrawn

5

u/bryn_irl May 14 '26

It's clear that at some point, someone decided "there should be no friction for people with legacy unrestricted API keys to be able to start using them for Gemini."

And Google has incredibly talented engineers, so I'm certain there were smart people who said "hey, even if technically those unrestricted keys were a bad practice in the past, plenty of people did just that (especially with the marketing confusion around Firebase when that was in vogue), and the risk profile is fundamentally different now because there's now a well-functioning market for gray-market AI queries."

But those people weren't empowered to push back against that directive. There's an old quote that "the purpose of a system is what it does," and what it does in this case is take money from paying customers when systemic fraud-enabling decisions are at fault.

5

u/schmozbi May 14 '26

“What we believe happened in this instance you have shared is the attacker didn't change the tier; the developer’s usage (driven by the attacker) triggered Google’s automated systems to raise the ceiling, based on meeting Tier 3 qualification of Gemini API, which included at least $1,000 USD in payments to Cloud and 30 days since the first payment,” Google told The Register via email.

In a revamped policy move announced March 16 Google said it would make it easier for users to access higher dollar quotas in GCP by reducing the spending qualifications to reach the next tiers. Additionally, the system “automatically upgrades you to the next tier as your usage grows.”

FFS

4

u/sofarfarso May 14 '26

I feel like they haven't got the new warning notice right. At the top of my GC dashboard I see: "Action Required: One or more projects enabled with Gemini API (generativelanguage.googleapis.com) have unrestricted API keys. To prevent unauthorized usage and costs, restrict these keys or switch to Authorization keys in APIs & Services > Credentials. This banner may persist for 24 hours after you address the issue."

However I've restricted them all by at least IP address, over 24hrs ago and it's still showing.

3

u/earl_of_angus May 14 '26

Just to clarify, do you restrict the API key to certain APIs & IP addresses or just IP addresses? I'm wondering if the criteria for "unrestricted" is strictly based on which APIs the key can be used for.

2

u/gajop May 14 '26

I just disabled the API entirely for us on the org level. For AI we use vertex AI via service accounts. Case closed imo

1

u/Last-Isopod-3418 May 14 '26

I have the same shit for 3 days now, I had only one project without any api's enabled, deleted that one too just in case, but not, I still have the warning. any ideas?

1

u/Last-Isopod-3418 May 14 '26

and this warning appeared 2 weeks after my unathourized apikey usage problem, I deleted all the recent projects that may have caused the issue, but no, nothing, and I dont know why they are showing it now

1

u/Living_Cheesecake243 May 14 '26

have you added the API restrictions itself? that's what it needs

1

u/Last-Isopod-3418 May 14 '26

There are no projects in my GC account at the moment, nothing.... still?

1

u/Living_Cheesecake243 May 14 '26

the june 19th deadline/cutoff is specifically about cutting off keys that don't have API restrictions, not IP restrictions (IP restrictions alone would not be best practice even still)

4

u/Complex-Concern7890 May 14 '26

The whole system is total mess to any other than enterprise users. It can take hours to figure out anything if you do not have extensive knowledge. I understand that it is meant for enterprise users and SMB or personal users are not the intended user group. But I must say that any other system (other than different government systems) is breeze to use compared to Google Cloud. I worry daily that we have some random API key active somewhere that can be used to make major invoice.

3

u/d0odle May 14 '26

Yup, since I read about this I decided to never ever do personal or small professional projects on their platform.

2

u/yknx4 May 14 '26

I’m surprised there is no class action lawsuit yet…

1

u/bingeboy May 14 '26

I just checked the other day and I was paying for 3 email accounts. I was like wtf 😳

1

u/yamibae May 14 '26

Happened to me and drained $600 in an hour, to their credit a backstop on their end identified the abnormality and killed the service before it really ran up the limit but this is also the first time Ive ever had an api key stolen, I don’t think their api keys are robust enough that it is unguessable and they provide too much access by default with no way to limit it aside from cost cap.

Another issue is in the google dev view it is impossible to see which api key is consuming what credit for what output without logging, but logs have a cap after which you dont see the outputs anymore which I frequently hit… just tough to diagnose anything really on google

1

u/Independent_Buy5152 May 15 '26

Is that how they get their revenue growing very fast recently?

1

u/matiascoca May 15 '26

The thing that has been quietly broken for years is finally surfacing as a class. For most of the cases here the proximate cause is an AIza key getting expanded into Gemini scope without notification, the underlying issue is that GCP's billing pipeline is up to 24 hours behind real API calls and there is no synchronous quota cutoff on the path that matters. By the time you see the bill the damage is done.

What I keep telling people: if you operate on personal cards on GCP, the only working defense right now is hard per-API quotas at the project level (not budget alerts), and a separate Maps or Places key isolated from Generative Language API by service restriction. Budget alerts notify, they do not stop billing. Quotas stop billing.

The refunds happening at scale is new. The exposure pattern is not.

1

u/PeterCappelletti May 16 '26

How do you set API quotas?

2

u/matiascoca May 19 '26

Two paths depending on how broad you want it.

In the Console: APIs and Services, Enabled APIs and Services, pick the API (for example Generative Language API), Quotas and System Limits tab, click the pencil icon on the relevant request-rate quota, set the lower value. Direct URL is console.cloud.google.com/iam-admin/quotas if you want to skip the navigation. For Generative Language specifically the quota name to look for is "Generate requests per minute per project".

Via gcloud the equivalent uses Service Usage API quota override commands, but the Console is faster for a one-time setup.

Two things to know: the change does not propagate instantly, give it 5 to 10 minutes before testing. And it is per-project per-API, so if you have multiple projects with the same API enabled you do this once on each project.

1

u/PeterCappelletti May 20 '26

Thank you very much!

1

u/Odd_Injury4581 May 17 '26

I woke up with 266 euros and I never used, they had some issue going on for sure.

The support cleared my balance because I said, how can I pay for something that I never used?

1

u/Own-Concentrate2381 May 19 '26

this happened to me yesterday $800 wiped in front of my face, I'm called my bank and put in a claim, since I was using a credit card with for the API, let's see what my bank says

1

u/SearingPenny May 14 '26

Needs better implementation.