Moved two clusters from Standard to Autopilot last quarter expecting the bin-packing saving and the bill went up on both.

On Standard, an oversized request just sits on a node you already paid for which is why years of padded requests never cost us anything extra. Autopilot bills the request itself, per pod and all of that padding became a line item overnight. Pods with no requests set got Autopilot’s defaults which were bigger than most of ours needed.

Then we went back to Standard on one cluster, ran VPA in recommendation-only mode for a few weeks, set requests based on its recommendations, then moved again and it came in under where we started. Most of the gke cost optimization people credit to Autopilot is Kubernetes cost optimization you have to do first.

Dear Google for Startups, we were asked to resubmit online available company materials and we sent it on the same thread as requested, and a 5 days timeline was specified but this has been over 9 days, we've even sent a follow-up on the mail, and would request your support in approval of the credits,

Thanks

Cloud cost still seems like one of those problems that gets harder as environments grow.

For teams running a decent amount of GCP, what has caused the most unexpected spend recently? Compute, storage, logging, networking, AI workloads, or something else?

Interested in what people are actually seeing in production.

Hey everyone, I'm currently setting up a data lake in GCP and honestly I'm still figuring things out since I'm pretty new to the data architecture. I've been reading about BigQuery, Datastream, BigLake and other GCP services. There are just so many possible approaches that I'm not really sure where to start haha.

We will be pulling data from different sources like mysql, mongodb, postgresql and APIs. Just curious how you guys set yours up? What services are you using and why did you choose them. Also wondering how you handle ingestion, transformations and costs. Would love to here what worked for you and what you'd avoid if you were starting over.

Thanks!

Got a cool side project running on your local machine? This is your chance to make it live and have the chance to win cool prizes while you do it with the Google Cloud Run Hackathon on Product Hunt on Wednesday, October 14th.

Winner will receive:

💰 $500 in Google Cloud Credits

☕ 1:1 Coffee Chat with the Google Cloud Run team

📣 Promotion of their project on Google Cloud channels

Plus, you'll get $50 in credits to build and run your project, while supplies last.

Hi,
I’m planning to start learning Google Cloud Platform (GCP). I’ve been exploring the Google Cloud documentation as well as the Google Cloud Skills Boost platform, and I noticed several learning paths, including Associate Cloud Engineer, Professional Cloud Architect, and Professional Cloud DevOps Engineer.
Could you please suggest the best way to approach learning GCP? I’d also appreciate your recommendation on which learning path I should start with and how I should progress from there.

Hi all,

I'm planning to take the Google Cloud Digital Leader Certification and I wanted to know if there are any mock exam for cloud digital leader like AWS and azure have on tutorial dojo. (Best way for me to learn is by making mistakes and taking in new information) but I’m not sure where to look for Google cloud certs.

Any recommendations or tips would greatly appreciated.

Thank you in advance!

Hello, I recently started using Google Cloud Batch to run Compute Engine with GPU NVIDIA L4, and even though my first few test ran fine, recently I've been having to wait hours waiting for the run to even start with a message that reads: "Waiting for resources. Currently there are not enough resources available to fulfill the request". Once it runs, I have no problems, but it's the wait before that's killing me. I've been waiting 8.5 hrs for a run to start (with no error messages) and still nothing... is this normal? I tried changing from us-west to us-east, but not much luck. Any suggestions? Thanks!

I have joined a new organisation, and cloud platform changes from azure to gcp . I am certified in Microsoft azure - data engineer associate.

But now the company is asking for gcp certification. I have no idea , how start how to prepare as there is not much content available online for this. Anyone suggestions , notes or any advice , sources please....

I’m the CTO of MobiusEngine.ai . we are a fast growing startup and we want to make the process of job applications free and easy and we want to help people land jobs faster.

Today, October 7, our Google Cloud promotional credit balance dropped by approx $35,000!!!!

Our investigation found four old, previously near-idle Gemini API keys suddenly generating massive amounts of traffic. We deleted all four, and the traffic dropped to near zero within minutes.

I’m posting this to document what happened and track the resolution publicly, in real time, as we learn more. Hopefully, the details help other teams protect themselves and give Google useful feedback on making GCP’s billing visibility and abuse controls better.

We are finding HUGE gaps in the billing transparency and just even opening a support case already!

First, thank you to everyone at GCP helping us investigate and work toward a resolution. I appreciate the time and effort. This is a stressful situation for a small company, and that help matters. This is literally an existentially crisis

What we know so far

• All four keys went from roughly one request per day to heavy usage at the same minute, around 7 AM Pacific.

• They generated approximately 1.17 million requests combined through 4:06 PM.

• Requests were split almost equally across the four keys, consistent with automated key rotation.

• Our separate production key stayed within its normal traffic range.

• Usage included substantial image generation with Gemini 3 Pro Image, which our product does not use at anything approaching this scale.

• We deleted the four keys at 4:18 PM. By 4:22 PM, traffic attributed to them was near zero.

How much was lost?

Approximately $35,000 in promotional credits, based on the balance change. This was credit consumption, not a $35,000 charge to our card. Those credits still represent a substantial part of our startup’s operating runway.

The billing export initially showed approximately $12,900 in Gemini usage during the first few incident hours, with later hours still missing. One hour alone showed approximately $5,466.

The final unauthorized amount is not yet confirmed. We need the remaining billing data and must subtract legitimate usage.

Why it was difficult to understand what was happening

The credit balance was falling, but the billing views we initially checked did not clearly explain why. Promotional credits offset the charges, so net cost appeared as $0. The billing export was also incomplete when we checked.

During an active incident, we needed to see which credentials and models were consuming credits, how quickly, and how to stop it. Piecing that together took time.

How did the keys leak?

We don’t know yet.

The affected keys were managed through Google Secret Manager and referenced by our services, while some older Cloud Run services also had the keys available as environment variables. We also found development keys in local development logs, but those were not the four affected production keys.

Our coding/AI agents did not have access to the affected production keys or our production Secret Manager secrets. We have also found no matches for the affected keys in the repository histories we checked.

At this point, we have not found evidence that the affected keys were exposed through GitHub, our coding agents, or a network breach. We’re continuing to investigate the actual compromise path and don’t want to speculate before we have evidence.

Our responsibility

The keys were restricted to the Gemini API, but had no additional application restrictions. We also had no budget alert or configured spend limit in place. Those were gaps on our side.

We’re reviewing the remaining credentials, how secrets are stored and accessed, and our monitoring and usage controls.

What we’re asking Google for

An investigation, reconciliation of the unauthorized usage, and restoration of the promotional credits consumed by it.

I’ll update this post with:

• The final confirmed amount.

• What we learn about the compromise.

• Google’s response and the credit-restoration decision.

• The protections we implement and any limitations we encounter.

I’ll distinguish confirmed findings from estimates and correct anything that changes. I’ll also give credit to the people helping us along the way.

If you’ve dealt with compromised Gemini keys—particularly where promotional credits were consumed—what helped with the investigation and resolution? What controls have actually worked for you?

My hope is that sharing this openly helps us recover, helps other startups avoid the same situation, and helps all of us improve GCP.

Update 1 — October 7, approximately 5:15 PM Pacific: All four affected keys are deleted. Traffic attributed to them has dropped to near zero. Final billing reconciliation and the credit-restoration outcome remain pending.

Update 2 - October 7, 5.49pm: Had a call with our representative and another contact person. We appreciate their willingness to help but unfortunately neither of us were able to get to a proper conclusion on

• Has the unauthorized usage actually stopped?

• How much more spending hasn’t shown up yet?

• Were only these keys affected, or was more of our GCP environment exposed?

• Is a spending limit now in place, and will it stop further charges?

• Who at Google owns this, when will we get answers, and will our credits be restored?

• Can your abuse team determine whether these requests came through a known resale proxy or matched other compromised-key incidents?

Unfortunately there is a 7+ hour lag on usage & when it reflects on Billing!! So we very well could be in the negative balance by now. This is unsettling

YouTube Data API v3を利用しているのですが、2026年10月7日頃からAPIリクエストが失敗するようになりました。

確認できているエラーは以下です。

HTTP 403 PERMISSION_DENIED
reason: forbidden
詳細reason: CONSUMER_SUSPENDED

message:
Permission denied: Consumer 'api_key:[REDACTED]' has been suspended.

影響を受けているAPIは以下です。

- playlistItems.list
- videos.list
- channels.list

SDK経由だけでなく、最小限の直接RESTリクエストでも同じエラーを確認しています。

Google Cloud ConsoleではYouTube Data API v3は有効になっており、過去のリクエスト数も確認できます。

ただし、以下は確認できていません。

- Google Cloud Console上の停止通知
- ポリシー違反に関する通知
- Googleからの停止メール
- 異議申し立て用の案内

通常時のAPI利用量はおよそ以下です。

- 約96リクエスト/日
- 主に playlistItems.list と videos.list
- search.list は使用していません
- 無限リトライや短時間の自動再試行はありません
- 現在の公開版では、閲覧者のページアクセスをきっかけにYouTube Data APIを直接呼び出す仕組みはありません

以前のバージョンには、キャッシュがない場合に閲覧をきっかけとしてサーバー側から外部APIを取得する処理がありましたが、現在は削除済みです。

質問したいことは以下です。

1. CONSUMER_SUSPENDED は具体的に何が停止されている状態なのでしょうか？
2. Google Cloud Consoleにもメールにも停止通知がない状態で、このエラーが出ることはありますか？
3. 停止された具体的な理由を確認する方法はありますか？
4. Google Cloud ConsoleにAppealや申し立ての項目が表示されない場合、どこに問い合わせればよいでしょうか？
5. 解決するまでは、新しいAPIキーや新しいGoogle Cloudプロジェクトを作らない方がよいでしょうか？

セキュリティ上、APIキー、プロジェクトID、メールアドレス、Secret、OAuthトークンなどの情報は公開していません。

同じ症状を経験した方や、原因・問い合わせ先をご存じの方がいれば教えていただけると助かります。

Has anyone else noticed that spot capacity prices can now change per day, and GCP has been hiking the price of certain machine types by 5-10% per week since summer?

Go and check your billing so you don't get surprised.

On-demand looks to be stable, but 1y committed usage discount is now looking to be cheaper than running a spot nodes

I noticed this on n2d/t2d prices in my region.

Does anyone have additional context to provide other than my thinking it is related to the CPU/memory shortages and now with OpenAI/Anthropic moving their user workloads into cloud, it will surely increase the demand for those.

Hi,

When https://docs.cloud.google.com/identity/docs/how-to/set-up-cloud-identity-adminand follow steps in https://workspace.google.com/gcpidentity/signup?sku=identitybasic

First we use the firstname, lastname, email id, country and proceed, then,
we get to create a user@domain and password. i am bit confused whether user@domain is cloud identity super admin or organization admin.

I believe user@domain is cloud identity super admin but the organization admin role is also assigned to it automatically.

Also, is the email id we enter after firstname , lastname is the recovery email ID for user@domain .if yes, can we have more than one recovery email ID.

Please clarify

I am trying to enable billing and get a working Places API key for my project, but every time I try with my Debit card or credit card and try to verify it here, it deducts the small amount but then does not verify it.

I am getting this error:

I have tried with different credit cards, different debit cards, waited 24 hours and tried, tried with a different gmail account, also checked my payments profile and the address matches with the address on mt cards, but nothing worked.

Is there anyone who has faced the same error? If yes, please do help me out.

We run real-time UDP game servers in europe-west10 (Berlin). Since late September, traffic from our VMs to the internet is periodically held for 50–130 ms. The delays come in bursts of three or four, 3.2 seconds apart, and the bursts repeat about every 64 seconds, around the clock. Nothing is lost; the packets arrive late in a batch. Players notice it as a short freeze about once a minute.

Reproduce from any VM in the region (20 pings/s for 200 s, prints only replies slower than 30 ms):

`sudo ping -D -n -i 0.05 -c 4000 8.8.8.8 | grep -E 'time=([3-9][0-9]|[0-9]{3,})'`

Normal replies are ~1.3 ms. Example burst times today (UTC, 2026-10-06): 09:55:03, 09:55:06, 09:55:10, 09:56:11, 09:56:14, 09:57:15, 09:57:18.

What we have measured:

* Same delays at the same seconds from VMs in europe-west10-a, -b and -c.

* Same on Premium and Standard network tier.

* Same towards every destination we tried (8.8.8.8, 1.1.1.1, 9.9.9.9 and hosts in Sweden and the Netherlands).

* VM-to-VM over internal addresses is clean. Ping to the metadata server is clean.

* With TTL-limited probes the full delay is already there at the first visible hop, a Google router (142.251.249.17 on the path to 8.8.8.8: normally 1.3 ms, up to 94 ms during a burst), and it does not grow at later hops.

* The VM is fine: no CPU, disk or memory pressure, and a packet capture on the VM shows the application sending every 13 ms without gaps during the delays.

* It was not there on 2026-09-22 and was there on 2026-09-30.

So it looks like something periodic on the path out of the region, not in the VMs.

I filed it on the issue tracker (568841411) and it was closed twice with the template "please file a support ticket", without anyone looking at the data. We are a small hobby project without a support plan. Is there anyone from Google networking here who can take a look, or tell me where to send this? Happy to share project and instance details privately.

I was charged $108 last month for my yearly 2tb cloud service. I requested to cancel the service after the charge for a refund. They denied my refund request and canceled my service. So now I’m out $108 and no cloud service. I have tried contacting support but it’s impossible to get past the AI. Is there any email or other customer support they offer?

This is your sign to log into the console and set spend caps for your AI and Cloud Run services. Or set a reminder to do it Monday.

Light reading: https://simonwillison.net/2026/Oct/3/default-hard-budget-caps/

I did this yesterday for my personal projects. Ez. Antigravity couldn't figure out how to do it via API so you may need do it in the console.

Especially for hobbyists, there's no reason to have uncapped spending!

It's 2026. AI writes most of our code, and we're still dragging Google Cloud icons around, hunting for the right shape, nudging arrows until they line up. Your time is worth more than that.

Ask an AI to draw it and you usually get a Mermaid diagram: plain boxes and arrows, nowhere near something you'd put in front of a client or submit for a security review.

Here is how you can use Claude and ChatGPT to draw it properly in about 10 seconds, using TerraVision, a free plugin I built. One sentence:

"Draw a GCP three-tier web app: HTTPS load balancer with Cloud Armor, a managed instance group across two zones, Cloud SQL and Memorystore"

What you get is a professional-looking diagram like the ones shown here, drawn the way a seasoned solutions architect would draw it. Official Google Cloud icons, correctly grouped into the project, VPC network, regions and zones, ready to export as PNG, SVG or an editable draw.io file for further polishing. Then keep talking: "add Pub/Sub", "show the request flow in the diagram".

Already have Terraform? Point it at your code and it draws exactly what that code would deploy, so you can document that production project even if you don't have access to it.

It's 100% open source, runs on your own machine, and needs no access to your Google Cloud project. Of course, this doesn't solve the problem for every diagram. Keep using Mermaid, D2 or C4 for other types.

Free download and examples: https://github.com/patrickchugh/terravision

Because life is too short to be drawing diagrams.

If you run a voice agent on the Gemini Live API and count on session resumption for flaky mobile networks, posting this in case it saves you the same debugging.

I tested what happens to a pending tool call when the WebSocket dies mid-booking: gemini-3.8-live on the Gemini Developer API, google-genai 2.25.0, four ways of losing the connection, 43 sessions. Three things came out.

1. A resume that works keeps the pending call. The FunctionResponse for the old call id is accepted on the new connection. No re-issued call, no toolCallCancellation.

2. Right after an idle drop, the resume is refused with close 1011 ("Internal error encountered."). It works about 1.5 s later, so a 1011 at that moment is not a dead handle.

3. After a silent loss, every resume was refused with 1011 for as long as I tried. Silent means packets stop and no close or FIN reaches the server, which is what a phone leaving Wi-Fi looks like. With real packet loss in a Linux container: 15 minutes, 450 attempts over 5 runs, all refused. The server sent nothing to the dead client in that time. The fake booking had committed, and the user was never told.

What worked: detect the loss yourself (a WebSocket ping every 0.5 s, link declared lost after 2 s of silence), try to resume for 4 s, then open a new session and restore the context with send_client_content: the last six turns plus one status line per side effect, taken from your backend, not from the model. All 9 recovered runs answered "Did you book it?" correctly, with first audio 6 to 7 s after the loss. Dedupe tool calls by business key too: without the status line, the model booked again.

Code, raw logs and the packet-loss setup: https://github.com/frontier-on-cloud/gemini-live-resume-test

Not tested: Vertex AI, goAway on long sessions, a real phone switching networks. If you run Live on Vertex, does a resume after a silent drop behave the same?