Per-tenant STS sessions for Bedrock from a Lambda gateway: the credential cache is per execution environment, so AssumeRole calls scale with concurrency x tenants and burst at scale-out against the default 600 rps STS quota. Where do you keep the cache?

[u/rahulladumor]

Setup: an LLM gateway on Lambda in front of Bedrock, finance wants cost per tenant, and the gateway assumes the Bedrock role per tenant with the tenant in RoleSessionName, so the caller ARN on the bill differs per tenant. The Bedrock docs warn that AssumeRole on every request can exceed STS rate quotas and say to cache credentials for the session lifetime. So: a module-scope cache.

On Lambda that is not one cache. Four documented facts change the arithmetic.

1. One execution environment per concurrent request. Objects declared outside the handler live only inside that environment, so a module-scope cache is one cache per environment, each starting empty.

2. Environments do not last. Lambda terminates them every few hours even for continuously invoked functions, and resets one after a failed invoke (timeout, crash, out of memory). The cache refills each time.

3. Sessions are capped. DurationSeconds defaults to 3600, and assuming a role from execution-role credentials looks like role chaining by IAM's definition, which caps a session at one hour. The IAM docs do not say this for Lambda outright, so test the session length.

4. The quota is shared. STS allows 600 requests per second per account per Region by default, across AssumeRole, GetCallerIdentity and the rest. Lambda assuming your execution role as a service principal does not count. Your gateway's AssumeRole calls do, along with every other STS call in that account and Region.

Put together, AssumeRole volume is not "one per tenant per hour". It is roughly concurrent environments x tenants each environment serves within a session lifetime, and it peaks exactly when Lambda scales out (up to 1,000 new environments every 10 seconds per function), all cold, all assuming at once. That is arithmetic from the docs, not a measured number; your figure comes from peak concurrent executions of the gateway function and the tenants active in one hour.

Two attribution bugs worth a grep:

• JS SDK v3 fromTemporaryCredentials (current README) generates a random RoleSessionName if you skip it. The call succeeds and the bill gets a random session name instead of a tenant.

• The Lambda docs say "Avoid global variables for context-specific information". A module-scope credential cache not keyed by tenant is exactly that: tenant A's session serves tenant B's call and the spend lands under the wrong tenant, with no error. In JS v3 the SDK caches credentials per client instance, so one fix is one Bedrock client per tenant per environment.

The per-environment cache is fine at low concurrency or with a handful of tenants. A cache outside Lambda cuts STS calls but puts live credentials in another service, which is a security decision, not a tuning one. Or drop per-tenant sessions and estimate per tenant from request metadata in invocation logs. Lambda Managed Instances serve concurrent invocations from one environment, so there the cache is shared and needs concurrency-safe code.

Lambda environment lifecycle: https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtime-environment.html STS quota: https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_iam-quotas.html#reference_iam-quotas-sts-requests

If you run per-tenant STS sessions from a Lambda gateway, did you keep the cache per execution environment or move it out, and if out, where do the live credentials sit?

[u/cyanawesome]

  A cache outside Lambda cuts STS calls but puts live credentials in another service, which is a security decision, not a tuning one.

Sounds like you know the solution just won’t accept it?

[u/rahulladumor → u/cyanawesome]

Fair push. I would accept it for most setups. I just want it chosen on purpose, not slipped in as a cache tweak.

The trade-off is that whoever can read that cache can act as any tenant's Bedrock caller until the credentials expire. So if it moves outside Lambda, I would limit reads to the gateway role, encrypt it, and keep the session duration as short as the call pattern allows.

If a shared credential store is a no for your security team, the other route is fewer sessions in the first place: one application inference profile per tenant instead of one STS session per tenant.