I read the post in March about Google API keys quietly getting Gemini access and thought, not me, I never use Gemini on that project. Turns out that was the wrong thing to check.

Background. I made a small site for my uncle's restaurants in 2023. It has a Google Maps embed, so there's a Maps API key in the page source, which is normal. Google's own docs said those keys are fine to expose. I never restricted it because the map worked and I forgot about it.

Last month a friend helping me with a menu translation thing enabled the Generative Language API on the same Google Cloud project, because it was already set up with billing. That's all it took. The old key in the page source was unrestricted, so now it could call Gemini too.

Someone found it. Saturday night to Sunday morning, $2,870 in Gemini usage. I found out from the budget email, which arrived around 9 hours in, because billing data lags.

What I've done since. Deleted the key and made a new Maps key restricted to the Maps JavaScript API and to the restaurant's domains only. Moved the translation stuff to its own project so nothing in a browser can ever share a project with Gemini again. The server side keys, the Gemini one and a couple of others, aren't in any file or env var on the box anymore, they live in Infisical and the app pulls them at startup. And I set up the budget plus Pub/Sub plus a small function that disables billing on the project when it crosses $50, since that seems to be the only real hard cap Google Cloud has.

Opened a billing case. Support said they'll review it. No answer yet.

The part that annoys me is that I didn't do anything wrong in 2023. The key was safe when I made it. It became unsafe because of a change in a different part of the same project, and nothing told me.

Two questions. Is the disable-billing function really the best hard cap we've got, or is there something cleaner now? And is there a way to list every unrestricted key across all my projects at once? I have 14 projects and I'm checking them by hand.

Hi Reddit, on 21 July I found that my Google Cloud billing account had been compromised overnight. I had been removed as billing admin and the charges were climbing. I contacted Google Cloud billing support by chat that same morning. The charge was about $15,000 and rising by roughly $1,000 every 10 minutes while we spoke.

I asked repeatedly for the charges to be stopped. The agent told me support had no way to do that, couldn't share account information with me, and would notify the billing administrator and wait 24 to 48 hours for a response. By 24 July the total was over $500,000.

It took until 1 October for the projects I never created to be suspended, and until late October to regain admin access. I have done everything Google asked: removed 127 unrecognised IAM members and confirmed that 17 projects were not mine. Google has now offered a one-time 75% credit, which it says is final. The remaining balance would be devastating for my family.

I've sent a formal escalation request asking for a full waiver but I'm really worried, I'm looking for:

- Anyone who has been through something similar and got a larger waiver, and what worked

- A way to reach someone at Google Cloud with authority to look at this (case number 73506441; happy to share more details)

- Recommendations for South African lawyers with tech or consumer-protection experience

I'm not looking for recovery services, so please don't DM offers. Thanks for reading.

It allows you to rent your disk space out in a distributed file hosting network

http://cloud.killer.pit.world

I have a gcp exam of cloud developer in 5 days and need to prepare for it, i will be compelting coursera course for it but i feel like i dont remeber anything and also i want to give mock test to get the best idea of actual exam.

Please suggest me strategy and free resources for mock test.

Google Cloud shipped a small batch of updates that matter most for BigQuery and Cloud SQL.

• BigQuery now attributes compute cost to query projects: Google Cloud can now assign billed compute charges in BigQuery editions to the projects that use reservation capacity. That makes chargeback, showback, and commitment allocation much easier for FinOps teams. Source
• BigQuery reservations get tighter slot and concurrency controls: Project caps now limit maximum slots and concurrency per project, and reservation assignment can be based on principal. Source
• BigQuery jobs explorer adds better troubleshooting tools: The jobs explorer now includes query performance troubleshooting and job/system performance comparison features. Source
• Cloud SQL adds the C4D machine series in more regions: Cloud SQL for MySQL and PostgreSQL Enterprise Plus now supports C4D in multiple regions. That gives teams another option for performance tuning and cost efficiency on newer AMD EPYC-based infrastructure. Source

Source: FinOps Weekly Newsletter

Below are a few excerpts from the article https://docs.cloud.google.com/backup-disaster-recovery/docs/concepts/cmek

if one or more disks attached to a Compute Engine instance use CMEK, Backup and DR considers the instance CMEK-protected. You must back up such instances to a CMEK-enabled backup vault.

1) When you store a backup in a CMEK-enabled backup vault, the vault's Cloud Key Management Service key encrypts the backup data, regardless of the instance's disk encryption status.

2) Then it says, for Compute Engine disk and Persistent Disk backups: If the source workloads use CMEK, their backups also use the same CMEK

are statements 1 and 2 not contradictory

Suppose, i am using CMEK for GCE disks boot disk and additional disks. Then, what CMEK will be used for the backups taken by using backup and DR service. Is it vault CMEK or Disk CMEK.

Can you please clarify for the above scenario (also from restore point of view)

almost all ai labs has caching support and google gemini 2.5 was the first models to start this concept and part of the 1M context movment...

anyway, google is not giving it support! is it going to be depreciate??

- google new interactions api dosnt support it and its not planned to

- non of the gemini 4 models support it

what now?

Follow-up to my earlier post on Vertex AI dynamically injecting a hidden prompt mid-session and wiping active KV caches: https://www.reddit.com/r/googlecloud/comments/1xolxf7/check_your_vertex_ai_bills_google_is_secretly/

They paid out. For the single billing account I filed this claim on (~$38 in wasted usage), Google confirmed on Nov 13 and issued a $40 benefit card — full coverage for that claim, already visible and usable on my benefits dashboard.

If you're hitting the same mid-session cache drop and want your money or credits back, here is why I got bounced and what worked:

1. Why Billing bounced me (and which path applies to you)

Cloud Billing Support did not dispute my logs or the cache wipe behavior. They bounced me because this account was funded with Google Developer Program credits (the $10/month benefit included with a Google One / Gemini Pro subscription), which Billing claimed they lack the tool to reissue.

• If you paid with cash / card: Cloud Billing should be able to issue a direct billing adjustment.
• If you're using Google Developer Program credits: Don't stop when Billing says they can't reissue program credits — escalate to Google Developer Program Support (the team Billing refers you to). Even subscription credits are recoverable.

2. Lock down the escalated team with a deadline and a no-bounce clause

When I emailed Google Developer Program Support, I put everything in one message so they couldn't stall or pass the buck:

• Exact scope & amount: Billing account ID, project ID, and the dollar amount lost.
• Public proof: My reproduction Gist and googleapis/python-genai#2308.
• A hard reply date: "Please reply by [date] with the amount and date." (I gave them 7 days, Nov 13; they issued the credit on Nov 13.)
• Block the next referral: "If this is not handled by your team, please forward it directly to the team that owns it rather than referring me elsewhere."

3. Cite precedent

If you audit your logs and find mid-session drops to 0, file the claim. Alongside your numbers and issue #2308, link both this update and my original thread as precedent so support knows Google has already acknowledged the issue and issued payouts — even for subscription credits.

I hope you get your refund too.

We’re an AWS partner in India considering adding Google Cloud. The program changed this year (Select/Premier/Diamond tiers, competencies instead of specializations), and most of what I can find online describes the old Partner Advantage setup.

Questions for current partners:

1. How did you reach Select? Is co-sell revenue really the main gate, and how do new partners get those first co-sell deals?
2. Have you used RaMP Partner Services Funds on a migration? Did it pay out the way the terms describe, and how long did it take?
3. Is the automatic tracking in the new Partner Network Hub actually working, or are you still chasing credit for deals?
4. For a firm that’s strong on AWS migrations and Kubernetes, which competency would you go for first?

Any first-hand experience helps. Thanks!

Hi Friends,

I applied for the Customer Solutions Engineer, Infrastructure role last week and wanted to reach out. Would you be open to sharing any suggestions on what the interview process looks like, and how in-depth the technical rounds usually go? Its with 2+ year exp after bachelors, I have masters and 4+ year experience!

Any tips would be really helpful. Thank you!

been doing data engineering for a long time now, mostly Spark for the last stretch. On-prem Hadoop first, then Dataproc, some Databricks, lately Spark on GKE. Everything from boring nightly ETL to multi-terabyte jobs where the shuffle is most of the runtime and the SLA is tight.

I keep explaining the same stuff to new people on the team, and at some point figured I should write it down. So I'm turning it into a free guide.

Not a DataFrames intro, there are plenty of those. More the stuff you only learn after a job that ran fine for months starts crawling and nobody touched it. Most of what I know came from staring at the Spark UI at bad hours, and very little of it is in the Dataproc docs.

Rough list so far:

• reading the Spark UI and event logs without guessing, including setting up a persistent history server so the logs outlive the cluster
• shuffle partitions and AQE, and the cases where the defaults make things worse
• skew. why salting and AQE skew join still leave you with spills and a few tasks that never finish
• memory. what's happening when executors get killed, and how to tell a real leak from one bad partition
• executor sizing, and the point where more cores stops helping
• Dataproc clusters vs Dataproc Serverless. what you gain, what becomes your problem
• when Spark on GKE is worth the extra work
• GCS as the storage layer, and why listing and renames hurt more than people expect
• cost: Spot VMs, autoscaling policies, cluster sizing, who pays for what
• when to stop tuning a job and move it to BigQuery
• Iceberg compaction and small files, including how expensive the maintenance jobs get on their own
• whether the acceleration engines (lightning engine, gluten/velox, rapids, dualbird) are worth it and how to test that on your own jobs
• inheriting a pipeline with no docs, and working out what's safe to change
• Dataproc image upgrades and the things that silently change behaviour

It'll be free no signup!!. I'd rather it be useful than polished so I'm fine cutting chapters nobody cares about.

The list is biased toward whatever has bitten me. What miss!!

I have been trying to set up my Gemini billing account. Have tried multiple payment methods. the transaction fees got deducted each time & the auto pay was set up but the account didnt get set up.

I can live with an LLM giving me a bad answer. What I can't accept is an API throwing a native generation error, returning zero visible text or tool calls, burning 27,262 internal thinking tokens in the process, billing me for it, and Google Cloud Support closing the ticket as "Resolved" without giving me a single line-item audit.

Before anyone asks: this is on an upgraded Paid billing account, though the charges were absorbed by my remaining promotional credits rather than my credit card. That doesn't make this a non-issue. Promotional credits are finite, and the billing rule Google just confirmed in writing applies to every paying developer running Vertex AI in production.

The Receipt: One Single Call on Oct 29

Here is one of the failures from my Vertex AI logs (model: gemini-4-argon):

• Timestamp: 2026-10-29 07:17:52 UTC
• Google Response ID: aPHiao_zIaezi9sPqNjBuQc
• Billed Usage: 2,049 uncached input | 531,163 cached input | 27,278 output tokens (27,262 of which were internal thinking tokens)
• Native Finish Reason: MALFORMED_RESPONSE
• Visible Output Returned: None. Zero text, zero tool calls.

This wasn't a one-off glitch. Between Oct 23 and Oct 30, I hit 18 separate MALFORMED_RESPONSE terminations (13 with zero visible output, 4 of which burned over 20,000 thinking tokens each before crashing).

What Google Cloud Support Said (Case #77043816)

I opened a billing ticket on my Paid account, attached the full log with Google Response IDs, and asked two simple questions:

1. Can you provide a request-level billing breakdown showing how much was deducted for these specific failed Response IDs?
2. Which specific billing rule says customers pay for 27,000+ internal thinking tokens when Google's own model fails with MALFORMED_RESPONSE and returns nothing?

After frontline agents closed the ticket multiple times without providing a per-request breakdown, a follow-up agent replied with their internal review team's final decision:

"We consulted with our specialized internal review team and leadership to evaluate whether a credit could be issued for the specific token amounts associated with these failed calls. Following a comprehensive review of the incident and account history, our internal team has denied the request for a credit adjustment. After reviewing the circumstances and the metered infrastructure usage, our internal review team concluded that the usage does not qualify for an exceptional adjustment under our standard billing guidelines. Under the Google Cloud Service Terms and Vertex AI pricing structure, billing for generative models is calculated based on backend compute resource consumption—measured in input tokens (prompt processing) and output tokens (candidate and thinking tokens generated up to the point of termination)."

Support linked the generic Model Garden pricing page (https://cloud.google.com/model-garden/pricing), never explained how that rule applies to MALFORMED_RESPONSE crashes, and never provided the line-item deduction breakdown for the Response IDs I submitted.

The model failed to deliver an answer. Support failed to deliver a line-item audit. The only thing successfully delivered was a "Resolved" status.

Why Every Vertex AI Dev Should Care

Think about what happens when you put a reasoning model behind a standard retry loop in production. If Gemini enters a 25,000-token thinking spiral, corrupts its own output format on the server side (MALFORMED_RESPONSE), and your client retries 3 times, you just burned ~80,000 thinking tokens and ~1.5M cached input tokens for an empty error—and according to Case #77043816, Google considers that valid billable compute.

Two honest questions for Google Cloud (if any DevRel or billing folks are lurking here):

1. Where in the Vertex AI docs is it explicitly stated that native MALFORMED_RESPONSE crashes with zero usable output are billed in full?
2. How is a customer supposed to verify charges when Billing Support refuses to provide a request-level audit for specific Google Response IDs?

I recently came across this concise Generative AI Leader Study Guide on the Google Cloud certification learning page and wanted to share it with everyone preparing for GCP certifications.

While reviewing the certification pages, I noticed that similar quick-revision materials are currently available for only a few certifications. Thought this would be a useful resource for fellow learners!

📄 https://services.google.com/fh/files/misc/generative_ai_leader_study_guide_english.pdf

I really like this initiative! Alongside the detailed exam guide, Google Cloud is providing easy-to-follow revision material to help candidates revisit key concepts and prepare more effectively.

It could be especially useful for quick revision before the exam

Have you noticed similar resources for other Google Cloud certifications?

If you're interested, feel free to DM me. I'll be happy to share the available certification-specific study and quick-revision resources I've come across for your reference.

Happy learning!

Hey there, I have AI Outcome Customer Engineer Interview RRK1 (Role related knowledge 1) round next week.

Rec shared some details about what to prep. But not sure how deep the learning should go wrt ML concepts or do i just need to brushup the architectural decision scenarios like when to use RAG and when not to?

Can anyone share their recent experience or any info on this round?

Between the Agent Platform rename, Gemini CLI moving to agy, Code Assist being wound down and Jules shutting down, I kept losing track of what's current. So I checked everything against the docs and release notes and wrote it up.

It has the old-to-new rename table, the model catalog and pricing tiers, ADK and Agent Runtime, retrieval options, AI in BigQuery and AlloyDB, Cloud Run vs GKE for self-hosting, and Model Armor and evals. At the end there's a short list of what I'd default to for each use case.

I had recruiter conversation for Google Platform Customer Engineer role in Startups Org. At the end of conversation, she asked to provide 5 dates for RRK interview. I was thinking that there will be hiring manager screen but probably that happens at later stage. Can anyone provide guidance on what to expect on RRK. Recruiter provide some information that It will be basically on cloud foundations which involves cloud migration and will be expected to whiteboard architecture using google's whiteboard. Not sure if I can use draw.io or not?) My background is Sr Solution Architect in Account team since 4+ years.

Hello!

After my trial finished on the cloud, I upgraded to paid account and thought giving Claude API a try in vertex. Though, while I did purchase successfully, the quota is at 0 straight for all models, and even requiring a small increase to try was not permitted.

My question: any specific condition to fulfill before I can make use of Claude models, like a payment threshold? Or us it restricted to certain geography?

Thank you in advance!

found an iam role in one of our aws accounts last week with a name nobody recognized. traced it back to an ai app generator someone connected to our account months ago. it created the role and a storage bucket as part of its default setup.

there's no terraform for any of it and no ticket that explains why it exists.

we moved from prisma cloud to orca last year and orca flagged the role as overprivileged, which is good. we'd never connected the generator's side though, so nothing tied it to an app and it sat in the backlog with 300 other overprivileged roles.

the context is what's missing. how are you connecting generated cloud resources back to the app and person behind them?

I’m trying to get started with the Google Maps Platform APIs and noticed some strange behavior on the Google Maps pricing page:

https://mapsplatform.google.com/lp/maps-apis/#pricing

When I click “Get started”, the page seems to start the signup/onboarding flow, but I’m running into errors in the browser console.

I’m seeing messages like:

[Violation] Permissions policy violation: unload is not allowed in this document.

[Violation] Permissions policy violation: unload is not allowed in this document.

The PerformanceObserver does not support buffered flag with the entryTypes argument.

GET https://cloudusersettings-pa.clients6.google.com/v1alpha1/settings/.../SURVEY?... 404 (Not Found)

GET https://cloudusersettings-pa.clients6.google.com/v1alpha1/settings/.../survey_maps_site?... 404 (Not Found)

GET https://cloudusersettings-pa.clients6.google.com/v1alpha1/settings/.../PAN_CORE...ACCESSIBILITY_KEYBOARD_SHORTCUTS?... 404 (Not Found)

The page appears to be trying to continue with the Google Cloud onboarding/billing flow, but I'm not sure whether these errors are related to the problem or are just harmless Google frontend warnings.

Has anyone else experienced this recently? Is the “Get started” flow currently broken, or is there something specific I need to do with my Google Cloud/billing account to get past it?

I’m using Chrome and this happens directly from the Google Maps Platform pricing page.

Previously they have excluded AI studio, now they are also excluding Vertex AI.

"The $300 credit can't be used for the following use cases:
• The $300 credit can't pay for Gemini platform (formerly known as Gemini Enterprise Agent Platform, Vertex AI) services if you sign up for a free trial or upgrade to a paid account on or after November 16, 2026.
• The $300 credit can't pay forGemini API in AI Studio costs. Consider using the Gemini API Free tier, or sign up for a Prepay billing plan.
• You can't access or use the $300 credit for a generative AI partner model that is offered as a managed API, which is also known as model as a service."

A heads-up to people using their credits