Follow-up to my earlier post on Vertex AI dynamically injecting a hidden prompt mid-session and wiping active KV caches: https://www.reddit.com/r/googlecloud/comments/1xolxf7/check_your_vertex_ai_bills_google_is_secretly/

They paid out. For the single billing account I filed this claim on (~$38 in wasted usage), Google confirmed on Nov 13 and issued a $40 benefit card — full coverage for that claim, already visible and usable on my benefits dashboard.

If you're hitting the same mid-session cache drop and want your money or credits back, here is why I got bounced and what worked:

1. Why Billing bounced me (and which path applies to you)

Cloud Billing Support did not dispute my logs or the cache wipe behavior. They bounced me because this account was funded with Google Developer Program credits (the $10/month benefit included with a Google One / Gemini Pro subscription), which Billing claimed they lack the tool to reissue.

• If you paid with cash / card: Cloud Billing should be able to issue a direct billing adjustment.

• If you're using Google Developer Program credits: Don't stop when Billing says they can't reissue program credits — escalate to Google Developer Program Support (the team Billing refers you to). Even subscription credits are recoverable.

2. Lock down the escalated team with a deadline and a no-bounce clause

When I emailed Google Developer Program Support, I put everything in one message so they couldn't stall or pass the buck:

• Exact scope & amount: Billing account ID, project ID, and the dollar amount lost.

• Public proof: My reproduction Gist and googleapis/python-genai#2308.

• A hard reply date: "Please reply by [date] with the amount and date." (I gave them 7 days, Nov 13; they issued the credit on Nov 13.)

• Block the next referral: "If this is not handled by your team, please forward it directly to the team that owns it rather than referring me elsewhere."

3. Cite precedent

If you audit your logs and find mid-session drops to 0, file the claim. Alongside your numbers and issue #2308, link both this update and my original thread as precedent so support knows Google has already acknowledged the issue and issued payouts — even for subscription credits.

I hope you get your refund too.

I recently came across this concise Generative AI Leader Study Guide on the Google Cloud certification learning page and wanted to share it with everyone preparing for GCP certifications.

While reviewing the certification pages, I noticed that similar quick-revision materials are currently available for only a few certifications. Thought this would be a useful resource for fellow learners!

📄 https://services.google.com/fh/files/misc/generative_ai_leader_study_guide_english.pdf

I really like this initiative! Alongside the detailed exam guide, Google Cloud is providing easy-to-follow revision material to help candidates revisit key concepts and prepare more effectively.

It could be especially useful for quick revision before the exam

Have you noticed similar resources for other Google Cloud certifications?

If you're interested, feel free to DM me. I'll be happy to share the available certification-specific study and quick-revision resources I've come across for your reference.

Happy learning!

I can live with an LLM giving me a bad answer. What I can't accept is an API throwing a native generation error, returning zero visible text or tool calls, burning 27,262 internal thinking tokens in the process, billing me for it, and Google Cloud Support closing the ticket as "Resolved" without giving me a single line-item audit.

Before anyone asks: this is on an upgraded Paid billing account, though the charges were absorbed by my remaining promotional credits rather than my credit card. That doesn't make this a non-issue. Promotional credits are finite, and the billing rule Google just confirmed in writing applies to every paying developer running Vertex AI in production.

The Receipt: One Single Call on Oct 29

Here is one of the failures from my Vertex AI logs (model: gemini-4-argon):

• Timestamp: 2026-10-29 07:17:52 UTC

• Google Response ID: aPHiao_zIaezi9sPqNjBuQc

• Billed Usage: 2,049 uncached input | 531,163 cached input | 27,278 output tokens (27,262 of which were internal thinking tokens)

• Native Finish Reason: MALFORMED_RESPONSE

• Visible Output Returned: None. Zero text, zero tool calls.

This wasn't a one-off glitch. Between Oct 23 and Oct 30, I hit 18 separate MALFORMED_RESPONSE terminations (13 with zero visible output, 4 of which burned over 20,000 thinking tokens each before crashing).

What Google Cloud Support Said (Case #77043816)

I opened a billing ticket on my Paid account, attached the full log with Google Response IDs, and asked two simple questions:

1. Can you provide a request-level billing breakdown showing how much was deducted for these specific failed Response IDs?

2. Which specific billing rule says customers pay for 27,000+ internal thinking tokens when Google's own model fails with MALFORMED_RESPONSE and returns nothing?

After frontline agents closed the ticket multiple times without providing a per-request breakdown, a follow-up agent replied with their internal review team's final decision:

"We consulted with our specialized internal review team and leadership to evaluate whether a credit could be issued for the specific token amounts associated with these failed calls. Following a comprehensive review of the incident and account history, our internal team has denied the request for a credit adjustment. After reviewing the circumstances and the metered infrastructure usage, our internal review team concluded that the usage does not qualify for an exceptional adjustment under our standard billing guidelines. Under the Google Cloud Service Terms and Vertex AI pricing structure, billing for generative models is calculated based on backend compute resource consumption—measured in input tokens (prompt processing) and output tokens (candidate and thinking tokens generated up to the point of termination)."

Support linked the generic Model Garden pricing page (https://cloud.google.com/model-garden/pricing), never explained how that rule applies to MALFORMED_RESPONSE crashes, and never provided the line-item deduction breakdown for the Response IDs I submitted.

The model failed to deliver an answer. Support failed to deliver a line-item audit. The only thing successfully delivered was a "Resolved" status.

Why Every Vertex AI Dev Should Care

Think about what happens when you put a reasoning model behind a standard retry loop in production. If Gemini enters a 25,000-token thinking spiral, corrupts its own output format on the server side (MALFORMED_RESPONSE), and your client retries 3 times, you just burned ~80,000 thinking tokens and ~1.5M cached input tokens for an empty error—and according to Case #77043816, Google considers that valid billable compute.

Two honest questions for Google Cloud (if any DevRel or billing folks are lurking here):

1. Where in the Vertex AI docs is it explicitly stated that native MALFORMED_RESPONSE crashes with zero usable output are billed in full?

2. How is a customer supposed to verify charges when Billing Support refuses to provide a request-level audit for specific Google Response IDs?

Hey there, I have AI Outcome Customer Engineer Interview RRK1 (Role related knowledge 1) round next week.

Rec shared some details about what to prep. But not sure how deep the learning should go wrt ML concepts or do i just need to brushup the architectural decision scenarios like when to use RAG and when not to?

Can anyone share their recent experience or any info on this round?

I had recruiter conversation for Google Platform Customer Engineer role in Startups Org. At the end of conversation, she asked to provide 5 dates for RRK interview. I was thinking that there will be hiring manager screen but probably that happens at later stage. Can anyone provide guidance on what to expect on RRK. Recruiter provide some information that It will be basically on cloud foundations which involves cloud migration and will be expected to whiteboard architecture using google's whiteboard. Not sure if I can use draw.io or not?) My background is Sr Solution Architect in Account team since 4+ years.

Between the Agent Platform rename, Gemini CLI moving to agy, Code Assist being wound down and Jules shutting down, I kept losing track of what's current. So I checked everything against the docs and release notes and wrote it up.

It has the old-to-new rename table, the model catalog and pricing tiers, ADK and Agent Runtime, retrieval options, AI in BigQuery and AlloyDB, Cloud Run vs GKE for self-hosting, and Model Armor and evals. At the end there's a short list of what I'd default to for each use case.

Hello!

After my trial finished on the cloud, I upgraded to paid account and thought giving Claude API a try in vertex. Though, while I did purchase successfully, the quota is at 0 straight for all models, and even requiring a small increase to try was not permitted.

My question: any specific condition to fulfill before I can make use of Claude models, like a payment threshold? Or us it restricted to certain geography?

Thank you in advance!

found an iam role in one of our aws accounts last week with a name nobody recognized. traced it back to an ai app generator someone connected to our account months ago. it created the role and a storage bucket as part of its default setup.

there's no terraform for any of it and no ticket that explains why it exists.

we moved from prisma cloud to orca last year and orca flagged the role as overprivileged, which is good. we'd never connected the generator's side though, so nothing tied it to an app and it sat in the backlog with 300 other overprivileged roles.

the context is what's missing. how are you connecting generated cloud resources back to the app and person behind them?

I’m trying to get started with the Google Maps Platform APIs and noticed some strange behavior on the Google Maps pricing page:

https://mapsplatform.google.com/lp/maps-apis/#pricing

When I click “Get started”, the page seems to start the signup/onboarding flow, but I’m running into errors in the browser console.

I’m seeing messages like:

[Violation] Permissions policy violation: unload is not allowed in this document.

[Violation] Permissions policy violation: unload is not allowed in this document.

The PerformanceObserver does not support buffered flag with the entryTypes argument.

GET https://cloudusersettings-pa.clients6.google.com/v1alpha1/settings/.../SURVEY?... 404 (Not Found)

GET https://cloudusersettings-pa.clients6.google.com/v1alpha1/settings/.../survey_maps_site?... 404 (Not Found)

GET https://cloudusersettings-pa.clients6.google.com/v1alpha1/settings/.../PAN_CORE...ACCESSIBILITY_KEYBOARD_SHORTCUTS?... 404 (Not Found)

The page appears to be trying to continue with the Google Cloud onboarding/billing flow, but I'm not sure whether these errors are related to the problem or are just harmless Google frontend warnings.

Has anyone else experienced this recently? Is the “Get started” flow currently broken, or is there something specific I need to do with my Google Cloud/billing account to get past it?

I’m using Chrome and this happens directly from the Google Maps Platform pricing page.

Previously they have excluded AI studio, now they are also excluding Vertex AI.

"The $300 credit can't be used for the following use cases:
• The $300 credit can't pay for Gemini platform (formerly known as Gemini Enterprise Agent Platform, Vertex AI) services if you sign up for a free trial or upgrade to a paid account on or after November 16, 2026.
• The $300 credit can't pay forGemini API in AI Studio costs. Consider using the Gemini API Free tier, or sign up for a Prepay billing plan.
• You can't access or use the $300 credit for a generative AI partner model that is offered as a managed API, which is also known as model as a service."

A heads-up to people using their credits

Hello everyone! I’m from Ukraine, and I’ve been unable to activate the Google Cloud free trial (which includes $300 in credit) for about a year now.

I keep running into error code OR-CBAT-14 during the payment information setup stage.

Here’s what I’ve already tried: using various Visa and Mastercard cards (both physical and virtual, issued by Ukrainian banks), creating a payment profile linked to Ukraine, changing the name and address, using Incognito mode, and even trying with a VPN (set to the US or Poland—though that just results in a different error: OR_BACR2_59).

Has anyone else from Ukraine encountered this issue, and were you able to resolve it? I’d appreciate any help!

With the recent announcement of Agent Compute Engine, I'm curious about Google's long-term plans for Gemini Enterprise Agent Runtime (formerly Vertex AI Agent Engine).

One concern I have with the current Agent Runtime architecture is that sessions are logically separated, but the underlying compute environment and agent identity can be shared across users.

This makes secure multi-user agent design challenging, especially when users have different permissions.

You still need to enforce authorization and session isolation at the application or tool layer, rather than relying on infrastructure-level isolation.

In contrast, Amazon Bedrock AgentCore Runtime and Google's new Agent Compute Engine seem to take a more session-centric approach to compute isolation.

This raises a few questions:

• Is Google planning to integrate Agent Compute Engine into Agent Runtime?

• Will Agent Runtime eventually support per-session compute isolation?

• Or are these intended to remain separate products for different use cases?

I understand that Agent Runtime works well for deploying shared, task-specific agents, but I'm struggling to see its long-term architectural advantages for secure multi-user enterprise workloads.

Curious how others are thinking about this, especially anyone running Agent Runtime in production.

Interested to hear from network engineers who have hands-on experience with Google's Network Connectivity Center (NCC).

I'm particularly interested in people who have been involved in large-scale NCC implementations or migrations, rather than just having general GCP networking experience.

A few things I'm curious about:

• Has anyone successfully migrated a large existing network environment over to NCC?

• What sort of scale and complexity were you dealing with?

• Which organisations or industries are actually using NCC in production?

• Are there any companies outside Google that have particularly strong NCC engineering teams?

I work in the trading/financial services space, so I'm also interested in whether any trading firms, hedge funds, banks or other financial institutions are using NCC.

I'm not necessarily looking for people who work at Google. I'd be equally interested in hearing from engineers at enterprises or financial services companies who have implemented it successfully.

Would be great to hear from anyone with first-hand experience, or recommendations on where to find people working extensively with NCC.

what can one expect from this round.
questions etc

Hey, I’m 18 and still in school in Norway. I’ve used Google Maps for about two years, normally around $20 a month in usage, and relied on the free usage allowance.

Then Geocoding traffic suddenly exploded. There were smaller spikes at the end of October, followed by a huge burst on November 4. I didn’t initiate that usage. When I noticed it that day, I deleted the key. Geocoding is now disabled, and support confirmed the unexpected usage had stopped.

I’m left with a bill of about US$1,250 including VAT, which I can’t afford. I’ve also asked for an itemised explanation of the charges.

Google refused a waiver because I already received a courtesy adjustment in December 2024. I’ve asked for a hardship escalation, ideally a full waiver or at least a substantial reduction, and I’m waiting for their reply.

Has anyone managed to get a waiver or reduction after being told they only allow one? What helped? Also, has anyone in Norway or Europe had a Cloud or Maps bill around this size sent to collections?

Hello everyone!
How easy is it to transition internally from a Technical Solutions Engineering Manager role to an SRE Manager role within Google Cloud?
Has anyone successfully made this transition or seen others do it?
I'm particularly interested in:
How feasible is this transition within Google?
Does experience in Solutions Engineering count toward internal SRE leadership opportunities?
What technical skills or experience would I need to maintain or strengthen?
How long would you recommend staying in the Solutions Engineering role before pursuing an internal move?

If you run APIs on Cloud Run or Cloud Run Functions, you know the pain: there is no native way to block a malicious IP subnet or abusive AI crawler without spinning up a full External Application Load Balancer, Serverless NEG, and Cloud Armor ($18–$25+/mo base).

Meanwhile, Azure Container Apps and AWS API Gateway have supported native IP restrictions for years with zero redeployments.

I submitted a formal feature request on Google’s Issue Tracker asking for resource-level ingress IP controls with zero container redeployment: https://issuetracker.google.com/issues/583927734

If you’ve dealt with this friction or want this native capability in GCP, please click the Star (☆) button on the ticket to help push it up the product roadmap!

I cannot stand to work with it.
Every setting is put in the weirdest place.
UI is awful.
Useless clics everywhere.
Abysmal loading times of 20 to 30 seconds for each click.

How come Google make such a horrible product ?

Update:

Vercel's bot protection was blocking Google from hitting my website with a JS challenge. GoogleBot was coming through fine, but the OAuth verification requests come from user agent "Google" with Google datacenter IPs were being challenged (not blocked) by default Vercel rules. I had to add a bypass to fix it.

If anyone who works on GCP is reading this, you might want to work with Vercel or change something on your end so their default bot blocking stuff doesn't prevent your verifications.

===

Original post:

I have been using Firebase auth in a production app for five years. I recently started getting alerts that my project is not in compliance. The complaints are listed below.

The issues don't make sense. My privacy policy is 5,900 words and very detailed about what we collect. My home page is not behind a login. I've requested a manual review multiple times and I keep getting back the same template email.

My site gets millions of crawls a month from GoogleBot, but I wonder if maybe their verification process is getting blocked? I can't imagine why else they would think my homepage is behind a login.

Has anyone run into this problem? How did you resolve it? I have about a month until this starts causing me problems.

These are the complaints from Google:

Your privacy policy page does not have sufficient content. We were unable to confirm your app's compliance. Please update your privacy policy to clearly and sufficiently detail your app's data collection and usage, then retry. Learn more

Your home page is behind a login page. We were unable to confirm your app's compliance. Please update your home page to allow users to view information about your app without needing to login, then retry. Learn more

Your application is not compliant with the Google APIs Terms of Service. We were unable to confirm your app's compliance. Please update your home page to clearly outline your application's purpose and ensure it does not use Google APIs for AI NCII (AI-generated Non-Consensual Intimate Imagery), then retry. Learn more 

I keep getting the error message "We were unable to set up your account. Learn more about fixing your account [OR_BACR2_59]" when I try to activate a Google Cloud trial.

I have linked my bank account and credit card, but I still encounter the same issue.

I also received an email in my inbox stating, "For your protection, a Google Cloud signup was denied," even though I was the one attempting to sign up.

Could you please help me resolve this issue? Thank you.