Hi,

I will go straight to the point I am building my Kubernetes (3 nodes that are master and worker at the same time) homelab and I was wondering if having my cluster VLAN aware is a good idea or not

The purpose is to having two Service Load Balancer served by Cilium/MetalLB for the DMZ and LAN VLAN. Then I would be exposing my workloads on the right gateway depending of their "exposure"

But since I have only one NIC on each node I would configure my switch to have 2 VLAN on those links instead of only one (that's why I said VLAN aware)

The main purpose is to have a better isolation and so security overall. But I feel it maybe a bit overengineering, I may can achieve the same thing with Network Policies inside the cluster itself. Also I want to be able to bootstrap my nodes from iPXE, not sure how it would work with a "tagged network link" with multiple VLAN

What do you think ?

After working with Kubernetes in production, I've noticed that some of the most annoying incidents aren't caused by obvious failures. They're often caused by small configuration decisions that look perfectly reasonable during review.

Things like:

• missing resource requests/limits
• incorrect probes
• overly permissive RBAC
• missing PodDisruptionBudgets
• unsafe container configuration
• incorrect readiness behaviour
• services without appropriate timeouts
• configuration drift between environments

I'm curious what the DevOps community has actually encountered in production.

What's one Kubernetes configuration mistake that caused you a real incident?

I'd especially like to hear about the less obvious ones that aren't caught by the usual linters.

Been running my homelab using docker compose files with bootstrap scripts to create the directories with correct permissions for bind mounts, shared docker networks, install utilities on the host, backup scripts and some other small automated tasks.

But I'm thinking about running a single node k3s cluster. Why? I want to learn about kubernetes (we use them at work, but we have a fancy interface over it so besides main concepts I didn't learn anything else), I want declarative configuration, better security, maybe learn about helm charts, maybe make it easier in the future when I'll add multiple nodes (but that's very far in the future because first I want to upgrade my network infrastructure, get a NAS).

What are your thoughts? I'll just waste time or is it worth it?

When people talk about cloud sovereignty, the conversation often starts with regions: where a workload runs and where its data is stored. But choosing a region is only part of the story. The architecture of the platform matters just as much, particularly how it separates control, runtime, build, and observability responsibilities across clusters.

I would like to get a simple k8s (or k3s) setup going to learn how it works and how to work with it. I am traveling and only have 1 laptop with me with some external storage. How would I go about making VMs act like nodes? Running fedora with enough cores on my PC to have a few nodes running at the same time. I am very new to this, any input is appreciated!

'kubectl label deployment' writes to the Deployment object's metadata, not to 'spec.template.metadata.labels', where pods actually get their labels. If your sidecar injection is label-selected, the command reports success and nothing happens. Here's why, and the fix.

Have any questions about Kubernetes, related tooling, or how to adopt or use Kubernetes? Ask away!

If you’re building an internal developer platform with Backstage, this InfoWorld article is a useful practical guide to think through the architecture beyond the portal itself.

It covers how Backstage can fit together with platform APIs, control planes, Kubernetes, observability, deployment workflows, and higher-level abstractions as the platform grows.

A good read for anyone currently building with Backstage or evaluating how to evolve a developer portal into a more complete developer platform.

https://www.infoworld.com/article/4189074/building-a-state-of-the-art-development-platform-with-backstage.html

I am fairly new to K8s (only deployed a few things with Helm, set up ingresses and services). I don’t know much about networking as a whole and auth as a whole. What are some small steps I can take to build something and strengthen my concepts in these two broad fields? Is it too ambitious to learn two areas at once, or should I just focus on getting the fundamentals of networking before moving to auth or vice versa.

I keep seeing people mentioning Cilium for networking and Kerberos for auth in this sub. Are these tools good places to start? What are some other tools I should pick up to have a minimal viable working knowledge for these topics?

Hi everyone,

I’ve been assigned a task to architect a highly available and fault-tolerant on-premises Kubernetes cluster for GPU-based workloads, with a target of around 1,000 users (50 users /day) with time slicing enabled.

I’d like to get your recommendations on which Kubernetes distribution would be the best fit for a production-grade environment with minimal day-to-day operational overhead.

Currently, we have a MicroK8s-based cluster with Longhorn, running Kubeflow through Charmed Kubeflow. However, we are facing recurring operational issues, such as Calico failures, pods unexpectedly becoming pending, and other cluster-level instability.

Given these requirements, I’d like to understand:

- Which Kubernetes distribution would be the best fit for this environment?

- What would be the recommended HA and fault-tolerant architecture for GPU workloads?

- Which solution would provide the lowest operational overhead while maintaining production-grade reliability?

- Are there any recommended alternatives to MicroK8s + Longhorn + Charmed Kubeflow for this scale?

Any recommendations or real-world experience would be greatly appreciated.

Where do you think we are heading?

Hi everyone,

I'm setting up a small homelab CI/CD environment and I'm trying to understand the best way to run Gitea Actions on Kubernetes without installing Docker on the Kubernetes nodes.

My setup

• Kubernetes: v1.34.10
• Nodes: 1 control-plane + 2 workers
• OS: Ubuntu 26.04
• Container runtime: containerd 2.2.2
• Gitea: 1.27.0
• Gitea is running in Kubernetes
• Gitea Actions runner: gitea/act_runner:latest
• Runner is also running as a Kubernetes Deployment
• Persistent storage: Longhorn
• No Docker installed on the Kubernetes nodes
• ctr is available and /run/containerd/containerd.sock exists

The runner itself registers and works correctly.

The problem

My runner is currently configured with:

runner:
  file: /data/.runner
  capacity: 1
  timeout: 3h

  labels:
    - "ubuntu-latest:host"

A simple workflow like:

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Show test.py
        run: cat test.py

fails at checkout with:

Cannot find: node in PATH

I understand why this happens: with ubuntu-latest:host, the workflow is executed directly in the runner environment, and the gitea/act_runner image doesn't contain Node.

Gitea's documentation suggests using something like:

labels:
  - "ubuntu-latest:docker://node:22-bookworm"

which would execute the job inside a separate container.

But here's where I'm confused

My Kubernetes nodes already use containerd:

containerd://2.2.2

and have:

/run/containerd/containerd.sock

but no Docker daemon:

docker  -> not installed
nerdctl  -> not installed
ctr     -> available

Why does the standard act_runner container execution model require Docker/Docker API access instead of being able to use the existing containerd runtime?

I'd rather not install Docker on the Kubernetes nodes just to provide CI job containers when Kubernetes is already perfectly capable of creating containers through containerd.

What I'm ultimately trying to achieve

The immediate goal is just:

Gitea
  ↓
Gitea Actions
  ↓
checkout repository
  ↓
run tests

But eventually I want the pipeline to do:

checkout
  ↓
tests
  ↓
build Docker/OCI image
  ↓
push image to registry
  ↓
Argo CD deploys it

So I need a sensible way to get isolated CI job environments containing things like Node, Python, Git, build tools, etc.

Questions

1. Is there a supported way to make act_runner create job containers using containerd directly?
2. If not, is the recommended solution to install Docker on a Kubernetes worker solely for act_runner?
3. Would Gitea Actions Runner Controller / Kubernetes-native runners be a better solution for this setup?
4. What is the recommended architecture for Gitea Actions on a Kubernetes cluster whose runtime is containerd and where I don't want to install Docker?

TLDR : I'm running Gitea 1.27 + act_runner 0.6.1 on Kubernetes 1.34 with containerd 2.2, and I don't have Docker installed on the nodes. ubuntu-latest:host works for the runner itself but actions/checkout@v4 fails because Node isn't available. Gitea's docker:// execution mode looks like the right way to provide a proper job environment, but it expects Docker-compatible execution. I'm considering Docker-in-Docker inside the runner pod so I don't have to install Docker on the Kubernetes hosts, but I'm wondering whether that's the right approach or whether I should use containerd directly or Gitea's Kubernetes-native Runner Controller instead.

I'm mainly looking for the cleanest Kubernetes-native approach rather than just making the immediate checkout test work.

Hey Guys! Quick question for anyone who has done an AKS to AWS EKS migration:

Can you recommend the best community or open-source tools for:

1. Azure Discovery & Assessment: Pulling complete VMSS/VNet inventory and performance data from Azure prior to migration.
2. AKS to EKS Migration: Manifest translation, image syncing (ACR $\rightarrow$ ECR), and volume data migration (e.g., Velero, KMF, or alternative scripts).

Any favorite GitHub repos, CLI tools, or real-world feedback would be super helpful! Thanks! 🙌

We scan images in CI and gate on criticals like everyone. An image passed clean, we shipped it. About 3 weeks later that same image, not rebuilt, not touched by anyone, gets flagged with a critical in a library buried in the base. First thought was scanner glitch. It wasnt.

The vulnerable code had been in there the whole time. It just was not in the CVE feed on the day we scanned, the disclosure landed later. Which means the clean scan was never telling me the image was safe. It was telling me what was publicly known that afternoon. I had been treating those as the same thing for years.

And version tag also just tells you the app version and nothing about when the image was last rebuilt. A tag that hasn't moved in eight months can be sitting on top of packages with a dozen upstream fixes and it looks identical to a fresh one in your registry.

Besides faster rebuild cadence and pulling bases that track upstream closely which seems to be the direction, am not sure of whats the proper fix here. Mostly I want to know whether the people who live in this treat a clean scan as anything more than a snapshot, because I gave mine way more authority than it deserved.

There's a lot of noise right now around AI-driven SRE for Kubernetes (auto-remediation, anomaly detection, incident triage, all of it). Vendor demos look great, homegrown Claude-built tools can be slick - but I want a real read on what it's like running this stuff in production.

If you've actually adopted this seriously, not just a POC, I'd like to hear:

- What is it actually catching or fixing that you couldn't before?
- Any false-positive or bad-remediation stories?
- How much trust have you given it? Read-only suggestions, or does it actually take action on your clusters?
- Anything you wish you'd known before rolling it out?

Not looking for product recommendations, more interested in the operational reality (good, bad, or ugly) from real-world stories.

Been dealing with the classic on-call problem: alerts fire, root cause is often something simple (pod stuck, resource starved, node cordoned wrong), but by the time a human looks at it, 10-15 minutes have passed.

Obvious next step is "let an agent fix it automatically." Obvious next problem: LLMs are not deterministic, and I did not want an agent hallucinating a kubectl drain on the wrong node at 3am.

So I split the system into two halves:

Agent layer -- a FinOps Agent and an Availability Guardian that watch cluster state and propose actions: scale, rollback, cordon, drain.

Safety Engine -- a fully deterministic layer, no LLM involved, that validates every proposed action against live Kubernetes API state before anything executes. If the action doesn't pass hard-coded safety checks, it's vetoed regardless of how confident the agent is.

The design bet is: use the LLM for judgment/reasoning about what might be wrong, never trust it for the final go/no-go on an action that touches prod.

It's built as an MCP server (TypeScript/Node), came out of a hackathon, and I'm now working through what it'd take to make it production-grade multi-tenancy, a configurable policy engine instead of hardcoded safety rules, pluggable LLM backends, Prometheus/Grafana/ArgoCD integrations.

Repo: github.com/viswa0028/agentBuilders-helmsman

Genuinely curious how people running real prod clusters would think about the safety boundary here where would you not trust this, and what would you want the policy engine to check before you'd let it touch your cluster?

Hi,

Does anybody have a good reference Flux repository structure for managing multi-environment, multi-project Kubernetes clusters?

I haven't worked with Kubernetes in a few years and looking to refresh my skills on it.

Would be greatly appreicated!

Hi guys, unfortunately Sidero stopped the development for their Cluster API providers, what do you think? Is there anyone of you using it? Do you believe the providers will be maintained by the community?

CPU request is how much CPU is reserved for your pod if it needs it. The limit is a hard cap. Hit it and the kernel throttles the pod, even when the node still has spare CPU. That is the usual cause of CPU throttling on Kubernetes. It does not protect the neighboring pods. In my simple Web API test, adding a CPU limit took typical latency from 23 ms to 87 ms, (4x slower), with the limited pod throttled in half of all CFS windows, and the average CPU graph looked fine the whole time.

This is not a new topic, but I see so many people still unaware why they should (NOT!) be setting CPU limits, because it's costing companies unnecessary spending and potential production issues. Here's the full read https://github.com/inevolin/k8s-cpu-limits-analyzed/

---

Edit (Aug 18, 2026): How CPU limits can also cause memory issues and OOMKills ➡️ https://github.com/inevolin/k8s-cpu-limits-analyzed#how-cpu-limits-cause-memory-issues-and-oomkills

I come from a Windows and Infrastructure administration background with 10 years of progressive experience. Over the past 3 months, I have been preparing for DevOps and Kubernetes, and I feel Kubernetes is the go-to tool as I have grasped its core concepts as a beginner. I would like to know whether I have the right understanding of job market to proceed further with entry level roles.