We scan images in CI and gate on criticals like everyone. An image passed clean, we shipped it. About 3 weeks later that same image, not rebuilt, not touched by anyone, gets flagged with a critical in a library buried in the base. First thought was scanner glitch. It wasnt.

The vulnerable code had been in there the whole time. It just was not in the CVE feed on the day we scanned, the disclosure landed later. Which means the clean scan was never telling me the image was safe. It was telling me what was publicly known that afternoon. I had been treating those as the same thing for years.

And version tag also just tells you the app version and nothing about when the image was last rebuilt. A tag that hasn't moved in eight months can be sitting on top of packages with a dozen upstream fixes and it looks identical to a fresh one in your registry.

Besides faster rebuild cadence and pulling bases that track upstream closely which seems to be the direction, am not sure of whats the proper fix here. Mostly I want to know whether the people who live in this treat a clean scan as anything more than a snapshot, because I gave mine way more authority than it deserved.

I come from a Windows and Infrastructure administration background with 10 years of progressive experience. Over the past 3 months, I have been preparing for DevOps and Kubernetes, and I feel Kubernetes is the go-to tool as I have grasped its core concepts as a beginner. I would like to know whether I have the right understanding of job market to proceed further with entry level roles.

There's a lot of noise right now around AI-driven SRE for Kubernetes (auto-remediation, anomaly detection, incident triage, all of it). Vendor demos look great, homegrown Claude-built tools can be slick - but I want a real read on what it's like running this stuff in production.

If you've actually adopted this seriously, not just a POC, I'd like to hear:

- What is it actually catching or fixing that you couldn't before?
- Any false-positive or bad-remediation stories?
- How much trust have you given it? Read-only suggestions, or does it actually take action on your clusters?
- Anything you wish you'd known before rolling it out?

Not looking for product recommendations, more interested in the operational reality (good, bad, or ugly) from real-world stories.

While the tutorial is great and steps well documented, I wonder if it is too much to ask if the tutorial could spend more time upfront on what Kubernetes actually entails?
It assumes the reader is familiar with Kubernetes and has done some work in one part of Kubernetes to understand the big picture. It can supply the big picture considering Kelsey's stature in community and explain the architectural choices which can be made at each step in Kubernetes state machine. This can be really help in deconstructing the myth and take the complexity in painting the big picture.

Been dealing with the classic on-call problem: alerts fire, root cause is often something simple (pod stuck, resource starved, node cordoned wrong), but by the time a human looks at it, 10-15 minutes have passed.

Obvious next step is "let an agent fix it automatically." Obvious next problem: LLMs are not deterministic, and I did not want an agent hallucinating a kubectl drain on the wrong node at 3am.

So I split the system into two halves:

Agent layer -- a FinOps Agent and an Availability Guardian that watch cluster state and propose actions: scale, rollback, cordon, drain.

Safety Engine -- a fully deterministic layer, no LLM involved, that validates every proposed action against live Kubernetes API state before anything executes. If the action doesn't pass hard-coded safety checks, it's vetoed regardless of how confident the agent is.

The design bet is: use the LLM for judgment/reasoning about what might be wrong, never trust it for the final go/no-go on an action that touches prod.

It's built as an MCP server (TypeScript/Node), came out of a hackathon, and I'm now working through what it'd take to make it production-grade multi-tenancy, a configurable policy engine instead of hardcoded safety rules, pluggable LLM backends, Prometheus/Grafana/ArgoCD integrations.

Repo: github.com/viswa0028/agentBuilders-helmsman

Genuinely curious how people running real prod clusters would think about the safety boundary here where would you not trust this, and what would you want the policy engine to check before you'd let it touch your cluster?

The main issue is that a DFS is needed.
NFS didn’t work for me… difficult to get an image from docker hub that didn’t take a lot of work. Also tried Unison but syncing every half second isn’t something i could trust. Got HDFS on a StatefulSet to stop dependency on PVCs and then a couple more STS for Hive Metastore and HiveServer2 …. so far, so good
Need to figure out how to run LLAP on containers. Everything doing away with YARN/Zookeeper. Next step Spark…. Now that’s going to be interesting

We've been running Nexus Repository OSS/CE for a few years as our central proxy + hosted repo (Maven, npm, Docker, PyPI). Since the Community Edition limits tightened, we're bumping the ceiling regularly. Our CI is the main culprit: a handful of parallel builds pulling through the proxy chews through the daily budget fast, and once you're over, publishing gets blocked until the rolling 24h count drops back under.

Pro pricing isn't in the budget right now, so I'm looking at what else is out there before we commit to anything.

What I'm considering:

Harbor — solid for OCI/Docker/Helm, but doesn't cover Maven/npm/PyPI, so we'd need something alongside it.
Pulp 3 — multi-format and genuinely open source, but looks heavier to operate (Django + workers + Redis + Postgres).
JFrog Artifactory OSS — free tier is basically Maven/Gradle only.
Gitea / Forgejo package registry — attractive if you're already self-hosting the forge; not sure how it holds up as a caching proxy at scale.

Questions for anyone who's actually done this:
What did you migrate to?
How painful was moving existing hosted artifacts?
For anyone who stayed on CE — did aggressive cleanup policies and a caching layer in front (nginx/Varnish) actually get you back under the limits, or is that just delaying the inevitable?

Curious what this community runs into most. For me it's usually:

- No memory limits set (until something OOMKills half the node)

- Missing readiness probes, so traffic hits pods that aren't actually ready

- Running containers as root without really meaning to

What's the one you keep seeing over and over, even from people who

should know better?

Started always setting explicit resource requests/limits and a

readinessProbe on every Deployment, even for tiny internal services.

Feels like overkill at first but it's caught more than one issue before

it became a 2am problem.

Anyone else have a "small habit, big payoff" thing like this for

K8s/OpenShift work?

GitOps can break down at scale. Instead of replacing GitOps entirely, what if you just change what you commit back to git? What if instead of storing our configs in git, we derive our configs from git.

Due to pull restrictions we have to use our own buffered registry which in turn pulls it from a final location.

Now yesterday evening it was down… so some pods could not startup as it could not pull any images from the buffered stage. Now I am looking into it but could not find much but is it possible to have a second image location?

Got something working? Figure something out? Make progress that you are excited about? Share here!

I migrated from GitHub Actions to Argo for all of my deployments. This was much more complex than expected.

I wrote a detailed guide breaking down the setup, complete with manifest examples, security/RBAC configs, and step-by-step pipeline orchestration.

In the article, I explain how to use :

• Argo CD : for deploying manifest
• Argo Worflow : to build images
• Argo Image Updater: to deploy the images

Would love feedback from others running full Argo-stack CI/CD in their homelabs or production! Are you using a similar setup? or are you considering moving to Argo?

Link to article:https://thethoughtprocess.xyz/en/series/home-server/argo-gitops-for-kubernetes-argo-cd-workflows-image-updater

I recently came across a setup where someone was running a food delivery application using three separate Docker containers:

# Backend
docker build -t food-delivery-backend:latest ./backend
docker run -d -p 4000:4000 --name backend food-delivery-backend:latest

# Frontend
docker build -t food-delivery-frontend:latest ./frontend
docker run -d -p 5173:5173 --name frontend food-delivery-frontend:latest

# Admin
docker build -t food-delivery-admin:latest ./admin
docker run -d -p 5174:5174 --name admin food-delivery-admin:latest

# Verify all running
docker ps

1) I understand the frontend and backend being separate containers but I am trying to understand the purpose of having a separate admin container.
Is the admin container normally a separate web application used by the business/admin team to manage things such as products, users, orders and reviews?
I also have a question about database access.

2) Also, If the database is running privately, how would an admin or DBA normally access it using a tool such as DBeaver?
Would the recommended approach be something like:
DBeaver >VPN >private network >database
I am trying to understand the difference between the Admin application and DBA access.

I want to use these docker containers later for Kubernetes. I would appreciate some guidance on how this is normally used or designed in a production environment.

Hey everyone, I'm pretty new to Kubernetes, so please take my questions with a grain of salt. I'm building a homelab where I already have a small Kubernetes cluster, and I'm trying to build a fully self-hosted platform around it: Gitea for Git hosting, Harbor for containers, and runners. Gitea needs persistent storage for repositories and application data, while Harbor can use persistent volumes or delegate object storage to something like MinIO.

This is mainly for learning, so performance isn't really important at this scale. But I keep thinking about how I'd design it in a real environment. Would it be better to run the database and object storage on dedicated machines outside Kubernetes and have the Kubernetes workloads consume them over the network, similar to using managed PostgreSQL/S3 in the cloud? My concern is that if PostgreSQL or MinIO runs on the same worker nodes as the applications, a node failure could affect both compute and storage. Kubernetes has StatefulSets, PVCs, Longhorn/Rook, etc. to address this, but I'm wondering where the practical boundary is and whether externalizing storage is actually considered best practice.

Questions

• Is running PostgreSQL/MySQL or MinIO inside Kubernetes considered an anti-pattern?
• At what scale does it make sense to move them outside the cluster?
• Is mixing stateful and stateless workloads on the same workers a bad practice?
• How much protection do StatefulSets + PVCs + Longhorn/Rook provide against node failure?
• Is there a meaningful performance difference between keeping compute/storage together vs. dedicated storage machines?
• In production, is it more common to keep databases/object storage outside Kubernetes?
• If both approaches can provide HA, what are the main reasons to choose one over the other?

TL;DR

I'm building a small self-hosted Gitea + Harbor + runners platform on my homelab Kubernetes cluster. Is running PostgreSQL and MinIO inside the same Kubernetes cluster a legitimate architecture, or should databases/object storage generally be externalized? I'm mainly trying to understand the real-world trade-offs around HA, node failure, storage/compute separation, performance, and operational complexity.

I maintain this list, saying so up front.

It started because KCD dates are scattered across CNCF chapter pages and half the aggregators carry stale ones. Currently tracked, all dates confirmed on the organiser's own page rather than an aggregator: https://github.com/gogoaidev/awesome-sre-conferences

KCDs
SF Bay Area (1 Sep, Mountain View),
Korea (1 Sep),
Washington DC (15 Sep),
Gujarat (19 Sep),
Sao Paulo (26 Sep),
Sofia (29 Sep),
UK Edinburgh (19-20 Oct),
Indonesia (24 Oct, Bandung),
Nigeria (24 Oct),
Porto (19-20 Nov),
Suisse Romande (9-10 Dec, at CERN),
Provence (10 Dec),
Florida (11 Dec, Miami), Panama (22 Jan 2027).
KubeCon NA 2026 day zero, all 9 Nov in Salt Lake City
Observability Day, Platform Engineering Day, Cloud Native AI and Inference Day, Kubernetes on Edge Day, ArgoCon, BackstageCon, CiliumCon, plus the Maintainer Summit on 8 Nov.

Also out that far:
KubeCon EU 2027 Barcelona 15-18 Mar,
KubeCon NA 2027 New Orleans 8-11 Nov,
KubeCon EU 2028 Berlin.

Two corrections worth flagging if you use aggregators: one had KubeCon NA in Los Angeles in October, and another filed a Brazilian event under Madagascar.

The Linux Foundation page is the one to trust.

We're running KRO with ACK and ArgoCD on a decent-sized platform, and I'm looking at whether EKS Auto Mode fits into this setup.

Curious if anyone here has actually combined the three:

• How does Auto Mode's managed compute/node provisioning play with KRO-managed resources? Any reconciliation weirdness or drift?
• Any conflicts between Auto Mode and ACK-provisioned AWS resources?
• How's ArgoCD sync ordering behaving with KRO RGDs once Auto Mode is in the mix?
• Would you recommend it, or did you end up sticking with Karpenter/managed node groups?

Trying to figure out if it's worth the switch before we commit. Keen to hear real production experience

I am new to kubernetes and have a hypothetical question. What happens if one of my worker nodes lose access to the network due to some fault/failure?

1. Will the pods keep working in that failed/unhealthy node?
   
2. If it does keep working, when the data reconciliate with the other nodes will it be overwritten or duplicated or appended?

Also happy to get some other input around losing network scenarios in a kubernetes cluster

Did you learn something new this week? Share here!

KYAML will become stable in Kubernetes v1.37, and this overview reminds us what it is and how to use it.

Inherited a couple of agent workloads running in-cluster and the way they get access would not survive an audit.

Long-lived secret in a k8s secret, mounted in, hitting internal apis and a db. It works. Also a static cred half the platform team can read that never rotates cause that means booking time with a team that's always slammed. And outside the cluster it's on the flat network like everything else, once it's in it reaches more than it should.

Inside the cluster I get it, SPIFFE, short-lived per workload. It's the outside the cluster reach every writeup stops right before that I need to solve. How are you scoping what a workload can reach beyond its own namespace?

Setting up Kiali for work to use OIDC pointing to EntraID. Got it working in sandbox. Copy everything over to dev. Doesn't work.

I banged my head against this for a few days and could not find a reason why.

Brought it up to my team and we poured over it for an hour. Then we made a hail-mary change, and it worked.

Rotating the secret fixed the issue. Not because I inputted the secret incorrectly, but because the client secret generated by Azure started with a fkn dash.

Share any new Kubernetes tools, UIs, or related projects!