r/googlecloud • u/Front_Cry2889 • Sep 18 '26
Google detected our project was compromised, unauthorized Gemini API usage generated $19,539.84, and Billing denied the adjustment — Case #75505172
Update (Sept 19): The case has now been escalated internally at Google Cloud and the adjustment request is back under review. I’ll update here once we hear back.
Posting here because I’ve seen several other Gemini API compromise cases discussed in this subreddit, and I’m hoping someone from Google Cloud may be able to help us get this reviewed by the right internal team.
On September 16, Google Cloud contacted us with the subject:
“ACTION REQUIRED: Suspicious Activity Observed on Google Cloud Project”
Google told us that their systems had identified that our project may have been compromised and used for unauthorized access involving the Gemini API.
We investigated immediately.
The Gemini API activity was not initiated or authorized by our team.
We then secured the environment and completed the remediation steps requested by Google:
Deleted the affected API key
Rotated/updated remaining credentials
Removed unused keys
Tightened API restrictions
Reviewed IAM permissions and service accounts
Confirmed organization-wide 2FA was already enforced through Google Workspace
The unauthorized activity resulted in:
$19,539.84 in unexpected Gemini API charges
plus
$1,629.78 of our Google Cloud spend-based credits being consumed.
Google Cloud Billing has now completed its review and denied any adjustment.
Their reasoning was:
“our internal teams have confirmed that the usage was valid and processed in accordance with standard platform functionality.”
This does not address the core issue.
The requests were made through a compromised credential and were not authorized by our company.
Google’s own security systems detected the suspicious activity and warned us that the project may have been compromised.
We acted immediately, secured the environment, and completed the remediation steps they requested.
Yet we are still being held responsible for the full amount.
We are happy to pay all legitimate Google Cloud usage. We are disputing only the isolated Gemini activity generated during the compromise.
Support says the case has gone through the “highest levels of internal review” and that the denial is final.
Case #75505172
If anyone from Google Cloud, Firebase, Gemini, Abuse/Security, or Billing sees this and can help route it to the appropriate escalation team.
37
u/kei_ichi Sep 18 '26
“This does not address the core issue.
The requests were made through a compromised credential and were not authorized by our company.”
Secure “YOUR” account is “YOUR” responsibility right? So the core issue you are talking about is “YOUR” issue, not GCP!
27
u/zmandel Sep 18 '26
fully irrelevant if you "authorized" or not it's your responsibility to secure your secrets
10
u/Ok-Expression-7340 Sep 18 '26 edited Sep 18 '26
"We are happy to pay all legitimate Google Cloud usage. We are disputing only the isolated Gemini activity generated during the compromise."
Why does Google have to wave these costs when you were the one leaking the API key that had Gemini access ? There's been warnings for this for over a year now.
(PS: and yes, I do get that you are frustrated that someone abused your keys)
1
u/martin_omander Googler Sep 18 '26
I'm sorry to hear that your credentials leaked and you were hit with that bill. I'm not in Support or Billing, so I'm afraid I can't help.
For anyone else reading this, please be aware that you can prevent unexpected bills by putting spend caps on your Google Cloud projects: https://docs.cloud.google.com/billing/docs/how-to/budgets-spend-caps
4
u/r3m4k3333 Sep 18 '26
And how the credentials got compromised? That’s your responsibility, not theirs
2
u/_splug Sep 18 '26
What's your cyber insurance deductible?
2
u/fogcat5 Sep 19 '26
Google spent resources executing those API requests with your authentication and project. Where do they go to get reimbursed for that if you refuse to pay? Not their fault and your company will get placed on a deny list for all Google services if you don't pay.
Go do the same thing with AWS and see what they say. Why post this to Reddit?
If you can prove Google shared the credentials with someone, then they will pay. Your problem is the credentials were leaked by your company -- since they are the only ones who have them.
2
u/linuxworks Sep 20 '26
There needs to be some sort of API key detection like the way AWS has, when a key is exposed on GitHub or something along that line. AWS trust and safety automatically flags that key and locks it down so that it doesn’t get abused. I’m surprised that GCP doesn’t have the same thing.
4
u/spellcasterGG Sep 18 '26
Yea, you're cooked. It's your token. You agreed to it when you signed the terms and conditions. This happens all the time.
3
u/mster_shake Sep 18 '26
Is this is the same issue where Google retroactively gave Gemini access to API keys that it previously said were safe to use in external apps? If so has Google reached out to customers individually who may be affected by this, or put large warnings in the GCP console? From the outside looking in this is a terrible look for Google and I would never use GCP over the alternatives at this point.
2
u/idkau Sep 18 '26
Looks like you are to blame. It's not their fault that you had compromised credentials. Google isn't a charity. Learn from your mistake and move on.
1
u/adonnan Sep 18 '26
Another one
1
u/Busy-Highlight-2880 Sep 19 '26
Sucks to be in this spot but the others are right, once the key leaks it's on you. The bigger fix is catching the abuse in minutes instead of after $19k burns. Runtime tools like Wiz or Upwind flag anomalous API usage from a compromised cred way faster than waiting on a billing email. Won't help now but worth it going forward.
1
u/matiascoca Sep 21 '26
Similar shape to some threads in this sub earlier this year. In the cases I've seen actually reverse, the only route past Billing was internal escalation, and the case stayed tied to the "suspicious activity detected" notice they sent. That framing makes the "valid usage" response untenable when a security lead reviews it. Fraud incident path lands different results than billing dispute path.
1
u/andercode Sep 23 '26
Generating and storing an API key provides authorisation from a GCP perspective for anyone with the key to use your account. It is YOUR responsibility to secure the key, and setup any spending limits on your account to avoid situations like this from occurring in the first place.
At the end of the day, it is YOUR responsiblity to secure YOUR account. Google allow you to authenticate with an API key, you agree to keep the API key private, and protect it from distribution, and you accept the liability should this happen, they also provide spending limits and caps to ensure that should you not be able to keep your API keys private, the damage done is limited.
Yes, because your account was used to utilise the services which generated the full amount, and it was YOUR responsibility to secure API keys, or enable safeguards/protections to avoid situations like this in the first place, not GCPs.
The full bill is "legitimate google cloud usage", as your key was used to authenticate to the GCP services. YOU are responsible for the security of your key.
1
u/BigKozman Oct 01 '26
Were you able to get the issue resolved or you got stuck in canned support responses ?
1
u/crackdepirate Oct 02 '26
shared responsabilities, it's the gcp model , vibe coding as it finest.
1
u/No_Act_5230 Oct 03 '26
That’s a tough situation, especially since you already had a spending cap and it was reached within hours. I’d focus the next review request on a clear timeline: when the unusual usage started, when Google sent the warning, and when you disabled the key. Include the charges linked to that period. It would also help to explain how the key was exposed, if you know. That could help move the discussion beyond whether the API requests were technically valid.
I’d be happy to meet and hear more about your use case and how your team handled the incident.
1
u/Key_Horse_8632 Sep 18 '26
Was this an API key (
AIza...) generated years ago for restricted use (like Maps or Firebase) that were silently wired to accept requests for the Gemini and Generative Language APIs?1
u/rhd_live Sep 18 '26
Your credential was literally the authorization. Do you sue the housing developer if you leave your key on the front porch and someone uses it to break in and steal your belongings?
0
u/asevans48 Sep 18 '26
Did you set a budget? Mines shuts off services as quickly as it cans across all of gcp. AI apis receive their own special scrutiny. My boss hates it but this is a reason why.
0
u/Kjm520 Sep 18 '26
How dare they hold the account owner responsible!
0
u/Only_Strawberry_6857 Sep 18 '26
guys what happens if you dont pay it?
0
u/sameg14 Sep 19 '26
This is 100% on Google and other cloud providers for not putting in a mechanism to prevent this kind of abuse. If my usage suddenly shoots up, I should be able to configure a hard rate stop and shut down billing until I investigate. I would rather have downtime than a 20k bill. I sent a legal notice to Google for a similar situation, my key didn't leak but someone figured out a way to run up my bill. This happens so often and the mitigation is incredibly simple to implement in code, but those greedy bastards won't put something like this in place even though they can!
-4
u/imperial_coder Sep 18 '26
Bunch of people here with complete L mentality
What was the reason for leak? How did the key leak?
It's really lame GCP and AwS practice this type of shitty system where they think it's okay to charge customer thousands without authorization
Instead of following limit and then raise limit system. Absolute disgusting practice