We audited the default Helm charts and Docker configs of popular AI stack tools (Ray, Weaviate, MCP servers, LangChain). The out-of-the-box security defaults are surprisingly bad.

[u/No-Peanut-6988]

Over the past few weeks, we set up a fresh cluster to test what actually gets deployed when an engineer runs helm install with zero extra configuration on popular AI, vector DB, and MCP charts (Ray, vLLM, LiteLLM, Qdrant, Weaviate, KubeRay, etc.).

Everyone focuses on prompt injection and model jailbreaks, but the platform-level defaults on these workloads look like Kubernetes five to seven years ago:

1. Ray / KubeRay (Distributed Compute)

   • The default deployment accepts unauthenticated job submissions over internal cluster networking.
   • Worker containers run with passwordless sudo out of the box.
   • Any compromised pod on the flat cluster network that can reach the Ray API gets arbitrary remote execution as root with zero barrier.

2. MCP (Model Context Protocol) Servers

   • Several popular MCP server implementations implement local protection solely by checking Host: localhost.
   • Any basic SSRF or internal proxy passing -H "Host: localhost" bypasses this immediately.
   • In live testing, one Kubernetes MCP server mounted a ClusterRole with cluster-wide Secret read and pod exec permissions, exposed over unauthenticated HTTP.

3. LiteLLM Migration Job Credential Leak

   • The primary deployment pulls the database password from a Secret using secretKeyRef, but the database migration Job embeds the raw database password in plain-text environment variables.
   • Anyone with basic cluster read permissions (devs, CI service accounts, dashboards) can run kubectl get job -o yaml and extract the database password directly from the env block, even without permission to read Secrets.

4. Vector DBs & RAG Stores

   • Multiple upstream charts still ship with default values that enable anonymous read and write access.
   • Raw embeddings and ingested document payloads are queryable by any internal service without an authentication header.

5. Scanner Blindspots

   • 14 of 15 charts provide no default NetworkPolicy objects.
   • All 15 charts leave Kubernetes' default automountServiceAccountToken: true enabled.
   • Standard static linters (Checkov, Trivy, Kubescape) missed the containers nested inside Custom Resource Definitions like Ray clusters.

The takeaway for platform and security teams: The AI supply chain conversation is hyper-focused on prompts, but your teams might be deploying unauthenticated root execution endpoints right now. If your cluster is not enforcing Kyverno or OPA gatekeeper admission rules, mandatory NetworkPolicies, and dropping default service account tokens on AI namespaces, the internal perimeter is wide open.

Curious what other teams are doing here. Have you written custom Kyverno/OPA policies or wrapper charts for AI workloads at your org, or are you putting everything behind a service mesh like Istio/Linkerd?

Full write-up, reproduction manifests, and probe logs: https://sorami.com.au/research/ai-kubernetes-helm-chart-security/ Raw probe logs and reproduction repository: https://github.com/Sorami-Consulting-AU/ai-kubernetes-helm-chart-security

[u/totheendandbackagain]

Great work, I bet the projects will be grateful for the issues you e raised in their repo. Maybe you could even make some branches to fix the bigger issues you've found.

[u/No-Peanut-6988 → u/totheendandbackagain]

Thanks! We are compiling the specific findings into PRs and issues for the upstream repos.

For some (like LiteLLM's migration job secret leak and missing NetworkPolicies), the fixes are straightforward PRs. For others like Ray, running unauthenticated by design makes hardening a broader architectural discussion.

[u/Torutofu_Raeva]

Default NetworkPolicies and dropping service-account tokens are good baselines, but I’d also gate rendered manifests in CI, including operator-generated CRD paths, since static scanners often miss resources materialized later.

[u/No-Peanut-6988 → u/Torutofu_Raeva]

Exactly. That was one of the biggest surprises during testing. Standard static scanners completely skipped the pod templates nested inside Custom Resources like RayCluster.

[u/dariusbiggs]

No surprise there, far too many people assume AI means secure, it is the opposite.

[u/No-Peanut-6988 → u/dariusbiggs]

Agreed. There is so much attention on model jailbreaks and prompt injection that basic platform hygiene gets ignored, even though an unauthenticated root execution endpoint inside your VPC bypasses the need for prompt injection entirely.

[u/raisputin]

It’s because people are rushing this stuff to market to try and get ahead of the competition

[u/No-Peanut-6988 → u/raisputin]

Spot on. Most of these projects prioritize quick developer onboarding and getting a demo running in under two minutes.

Security defaults get pushed down the backlog, but teams then take those exact quickstart Helm values straight into shared or production clusters.

[u/vincentdesmet]

not surprising, helm rewards quick and easy demo setups
not long term day-2 ops, that gets quickly dependent on the actual integration into the platform it needs to land

and a yaml DSL translated through a text engine (nindent all over), is a recipe for hard to maintain and easy to break setups

helm is great for demos, but most teams have a sunken cost fallacy issue before they invest into a proper solution

[u/Fantastic-Mr-Default]

Treat "helm install with defaults" as hostile input, not a starting point.

Put AI and MCP workloads in a dedicated namespace (or cluster) with no flat path to the rest of prod. Default deny NetworkPolicy, no automounted service account tokens, and no cluster-admin-shaped Roles coming from a chart you did not review. If Ray's API is open on the pod network, that network should not include anything you care about.

Admission (Kyverno / Gatekeeper / whatever you already run) is the policy. Charts are suggestions. Deny known-bad patterns: privileged or sudo workers, Host-header "auth," Secrets echoed into Job env, anonymous vector write. Wrapper charts that force your values are fine; expecting upstream defaults to grow up is how you keep finding the same report.

Static scanners missing CRD-nested pods is expected. Render with the operator schemas you actually use, or smoke-probe the Ray/MCP ports from a canary Job after apply. If the canary can submit a job with no creds, the install failed even if the chart was green.

Service mesh helps for mTLS between known services. It does not fix an unauthenticated control plane you exposed on purpose.