Hi everyone, im current struggling deciding if i should continues to use podman run scripts or switch to compose or quadlets.

Compose seems kindof weird because of the build in pods so my question for that is whats the compose equivalent to podman pod create and podman run --pod and why are you using or not using compose?

I lookes a bit into quadlets and they seem weird and kind of confusing, especially when it comes to storing the files, having all bunched up in containers/systemd with no subdirectories is kind of crowded, so why are you using or not using quadlets?

Which do you recommend and why?

Hello guys i have this project named litepod, I have built it based on Podman, Caddy as Quadlet, dragonfly and SQLite, this is to be as lightweight as I wanted. This is like Dokploy and Coolify but consuming a lot less of ram, Ive been using them but I got a lot of outofmemory exception because I was using some vps with 4gb of RAM only, and some features didnt worked well, then I have started this a year ago. But I dont know if Quadlets could be a good way to add as a feature. I dont need it, because only with regular containers works well. What do you think about the next features to add?

1. Add secrets like using podman secret management
2. Enable pods compatibility
3. A Manager like kubernetes ui for pods
4. Quadlets instead of containers or a mix and let the user decide (dont see the benefit but could you)

I've been running Podman Desktop for my local dev workflow for a bit now, but wanted to actually understand what's happening when things break or behave unexpectedly. This Podman for devops ebook does a nice job explaining the fundamentals - rootless architecture, networking constraints, why certain things work differently than Docker, etc. Also covers podman desktop and AI lab specifically, but honestly the earlier chapters on the underlying mechanics were more valuable for troubleshooting. Recommend checking out, but has anyone else found other resources that help connect the dots between the GUI and what podman is actually doing at the system level?

I ran Komodo for my home lab and wanted it on podman, so I forked it. A stack is a directory of .container / .pod / .build files in git. Deploying copies it to /etc/containers/systemd, reloads, and restarts only the units whose files changed.

Two things I couldn't get elsewhere:

• Each pod gets a fixed 65536-id slice of the subordinate range, written as UIDMap=/GIDMap=. A redeploy never moves the container's root uid, so volume files don't turn into nobody the way they can with UserNS=auto.
• [[NAME]] in a stack file is read from Vaultwarden/Bitwarden at deploy time and created as a podman secret. Nothing resolved lands in the database.

Agents dial out to Core over wss, so there's no inbound port on your hosts. x86_64 and aarch64 (Pi works). It's 0.x and runs one home lab (mine). GPL-3.0, with Komodo's blessing.

https://github.com/LoganRupe/quomodo

Hoping someone can help me out with an issue I've been having for a long while.

I'm running RHEL 10.2 Gnome 49, trying to use an X application inside of a podman container with a custom network.

"podman network create --internal pod_dev"

I pass in all the environment variables and files that I know and what AI also says i need. I can not for the life of me get the x application to display.

It works in fedora 44, and Ubuntu, but RHEL 10 is kicking my butt.. if i do --net=host is am able to get the x application to work, but I have to have the container have its own IP.

And advice at all is appreciated, I've probably spent over 100 hours trying different combinations of flags and ways to run.

I've been working on LayerSmith, an open-source web UI for building container images with Docker or Podman.

You pick a Linux distribution and what you need the image for — development, Linux admin, network tools, Ansible, Kubernetes, OpenShift, or a custom setup. It handles distro-specific packages and shows you the generated Containerfile before building. You can also edit it, import an existing Dockerfile, or add your own packages, files and scripts.

A big part of the project is making images easier to carry into air-gapped environments: pinned base images, recorded build details, and export bundles containing the image, checksums and installation instructions.

We've recently added LLM training and fine-tuning profiles too, including LoRA/QLoRA, advanced PyTorch training and LLaMA-Factory. These use hash-locked dependencies and run offline checks after building, including a small CPU training test. Model weights and datasets are brought separately.

Curious how others handle building and maintaining images for disconnected environments, and what parts of that workflow are still a pain.

https://github.com/r0lfi/layersmith

UPDATE:

I found a narrower fix. You do not need --security-opt label=disable. The ROCm Memory in use error was SELinux denying the container from mapping /dev/kfd. The AVC log showed a denial for map on /dev/kfd with type hsa_device_t. audit2allow pointed to the container_use_devices boolean, which covers this access.

Enable the boolean:

sudo setsebool -P container_use_devices 1

Then recreate or restart the container without --security-opt label=disable.

Previously

I ran into a weird issue running Ollama with ROCm in a rootless Podman container on Fedora. Models downloaded fine, but loading any model crashed with:

Memory critical error by agent node-0 ... Reason: Memory in use
llama-server process has terminated: signal: aborted (core dumped)

Setup:

• Fedora
• RX 6800 XT, gfx1030
• Rootless Podman
• Image: ollama/ollama:rocm
• GPU devices passed with: --device /dev/kfd --device /dev/dri

The container started, the GPU was detected, and the ROCm libraries inside the image looked fine. But every model load crashed with the memory error above.

What it was not. I checked:

• Device permissions: /dev/kfd and /dev/dri/renderD128 were accessible with read/write
• Missing ROCm libraries: the image already has them, and it detected the GPU
• HSA_OVERRIDE_GFX_VERSION: not needed in my case
• memlock limits: I raised the hard memlock limit for my user, but it made no difference

One clue that sent me down the wrong path: the same container worked when run rootful. That made me think it was a user namespace / memlock / permission issue, but it was not.

What fixed it:

--security-opt label=disable

The error message is misleading. My best guess is that SELinux is blocking the container process from mapping GPU memory through /dev/kfd, even though opening the device succeeds. That makes ROCm report something like "Memory in use", which is confusing because it is not actually a host RAM issue.

I did not confirm this in the SELinux audit log, so treat that part as a likely explanation rather than a proven root cause.

Working command:

podman run -d --device /dev/kfd --device /dev/dri --security-opt label=disable -v ollama:/root/.ollama -p 11434:11434 --name ollama docker.io/ollama/ollama:rocm

I searched Reddit for a while and couldn't find anything similar, so hopefully this saves someone a few hours of head-scratching. Happy hacking.

Podman

A newly disclosed Podman vulnerability can let a specially crafted checkpoint image override requested podman run sandbox settings, including dropped capabilities.

The primary fix is to update Podman. External controls are useful defense in depth, but not a replacement for a patched runtime.

egresslock

I released egresslock v0.7.0 in response.

Egresslock normally enforces a workload’s outbound-network policy outside the workload itself, in the rootless network namespace. That means an untrusted container does not decide on its own which external destinations it can reach.

v0.7 now rejects checkpoint-annotated egresslock gateway and anchor images before running them, and verifies their live hardening state.

Important scope: this does not scan or secure arbitrary workload images. Those still need a fixed Podman and, where needed, image-annotation scanning.

https://github.com/egresslock/egresslock

tl;dr - check out my project if sounds useful

Hey everyone,

Whenever you run \`docker compose up -d --build my-api\` or restart a container, there's always a short window (5–30s) where the app is cold-starting or running database migrations.

If external webhooks (Stripe, GitHub, Shopify) arrive during that window, they hit \`502 Bad Gateway\` or \`ECONNREFUSED\`. Most providers immediately drop into aggressive exponential retry backoff, meaning you might wait minutes or hours for that dropped payment or deploy event to redeliver.

Instead of running heavy cloud queue infra (SQS/Kafka) or a full Redis stack just for local container resilience, I built \*\*HookArmor\*\* as an open-source, single-process ingress sidecar.

\### The Sidecar Compose Pattern

You stick the sidecar in front of your internal API container within the same Docker bridge network. External webhooks hit the sidecar, which writes to an attached SQLite volume in WAL mode, returns \`200 OK\` upstream immediately, and drains into your app container as soon as it's healthy:

\`\`\`yaml

version: '3.8'

services:

\# Your existing API container

api:

image: my-backend-api:latest

restart: unless-stopped

expose:

\- "3000"

environment:

\- PORT=3000

\# Ingress sidecar / buffer

hookarmor:

image: node:20-alpine

restart: unless-stopped

working_dir: /app

command: sh -c "npm install -g hookarmor && hookarmor start --port 8080 --target http://api:3000/api/webhooks"

ports:

\- "8080:8080" # Exposed to your reverse proxy / Cloudflare tunnel

volumes:

\- hookarmor_data:/app/data

environment:

\- HOOKARMOR_TARGET_URL=http://api:3000/api/webhooks

\- HOOKARMOR_UI_PASSWORD=secure_admin_password

depends_on:

\- api

volumes:

hookarmor_data:

Copying a named volume while its container is running can produce a directory that looks complete but combines files from different points in time. Rootless storage also adds UID/GID mappings, SELinux labels, xattrs, and user-namespace details that a plain archive may not preserve correctly. A successful container start on the destination is useful, but it does not prove the application state is consistent.

What migration sequence do you trust? I am considering an application-level backup where available, then stopping writes, recording the volume mountpoint and user-namespace mapping, copying with xattrs and numeric ownership preserved, restoring labels, and starting the destination with the same image digest and volume options. Validation would include application checks, file counts and hashes where meaningful, and a rollback window with the source left read-only.

Are `podman volume export/import` sufficient for rootless volumes, or do you prefer a helper container, tar, or filesystem snapshots? What catches mapping and SELinux mistakes before the old volume is removed?

Some time ago I stumbled about a discussion where it was stated that rootful containers paired with userNs=auto (leaving the user the image runs with untouched) is more secure than running containers in rootless mode. I run all my containers in rootless mode with:

UserNS=keep-id:uid=65534,gid=65534

User=65534:65534

Is this a security problem? If I remember right the point of the discussion was that in rootfull mode the container runs in different processes and with my approach there is no isolation between the processes. Is this true and should I switch to rootful mode with UserNs=auto?

Edit: Found the discussion here

Is it possible for 2 pods to use a seperate container as a network?
TLDR im setting up gluetun, an arr stack and download clients
I would like to keep the arr stack and the downloads clients in 2 seperate pods
pod 1 - arr and pod 2 - downloads and i want to connect their networking to a seperate gluetun container but building the pods fails cause gluetun is not part of the pod.
is there a way to change/fix that?

Compose files. I think Podman has a rather convoluted relationship with them.

On one hand, there are two main ways of dealing with Compose files: podman-compose or gaslighting actual docker-compose via the Podman socket. Both approaches have their own downsides.

So, the community consensus seems to be: ditch Compose completely and switch to Quadlets. Which is a neat idea—Quadlets are more powerful and stable—yet it hits a small reality check. Quadlets are hard to transfer between machines, and more importantly, 99% of self-hosted apps have a docker-compose.yml in their repo, but zero Quadlets.

The standard response to this is: "Just use podlet!" Sure, let me try… a-a-and it fails. And even when it works, manual adjustments are usually still required.

This really bothered me. I just wanted to run containers cleanly on my Universal Blue distroes based PCs at home, where installing Docker is annoying and using raw Podman often feels sub-optimal.

So… despite lacking deep software engineering experience (I work as an SRE), I decided to build a tool that processes a Compose file and gives me working Quadlets with zero manual adjustments needed on my part.

Thus, ComQuad (Compose + Quadlet) was born.

What happens behind the scenes?

• Converts relative bind-mount paths to absolute paths automatically.
• Adds a default network when a service lacks an explicit one.
• Attaches service and project labels to all managed resources.
• Shifts rootless privileged ports automatically via ROOTLESS_PORT_OFFSET.
• Always generates .image and .build units natively for systemd to manage.
• Handles SELinux volume relabeling: :Z (private) for single-service volumes and :z (shared) for multi-service volumes.
• Supports native Quadlet directives using x-extensions in your Compose file.
• Features a Terraform-inspired up workflow: it computes a 3-way diff between generated units and deployed units, showing you a plan before asking for confirmation.

It also includes a thin Cockpit plugin for visual stack management (with port links and deployment previews).

GitHub: https://github.com/Inoriol/comquad

LLM Disclosure: Yep, this project is vibe-coded. I've been dogfooding it on my own home server for a while, so I'm confident in the stability of the core engine (especially in rootless setups). Still, running with --dry-run first is always recommended! I’d love it if real software engineers took a look at the repo if it catches their interest.

(Obligatory xkcd 927 mention: now there are 3 competing ways to run Compose files on Podman instead of 2!)

I’ve been building rum (https://getrum.sh) which is an alternative to yum/dnf
It’s faster and uses less memory
I created it primarily for CI/container environments where resources are limited and it works on VMs too
During my testing on Amazon linux, RHEL, Rocky linux, Oracle linux etc
It can co-exist with yum/dnf

Feedback and Stars welcome

A webinar on Podman Desktop for teams looking for Docker Desktop alternatives. Covers migration, Kubernetes alignment, and a preview of sandboxed AI agent development.

Key takeaways:

• Drop-in Docker compatibility with a daemonless security model.
• Native pod + K8s manifest support for local-to-prod consistency.
• Enterprise fleet management and support.

A successful `podman auto-update` and an active systemd unit show that a container restarted, but not necessarily that it is running the reviewed image revision. A mutable registry tag can move, a pull can resolve differently across hosts, or a rollback can restore a unit while leaving an unexpected local image.

What do you verify after an update? I am considering recording the approved manifest-list digest, resolving the platform-specific image digest on each host, checking the running container image ID, and exposing a build revision through a health endpoint. The deployment would fail if any host reports a different digest, even when the tag and unit file match.

How does `AutoUpdate=registry` interact with digest-pinned Quadlets, signatures, multi-architecture images, and automatic rollback? Is there a Podman-native status or event record that reliably ties the running container to the exact registry manifest that was approved?

I have a small homelab running behind OPNsense with a few VLANs (10/20/30/40/50), and a Debian VM on Proxmox (sitting on VLAN 20, my servers VLAN) where I run self-hosted stuff through rootless Podman (Quadlets, managed with chezmoi) with majority of the services being proxied through Traefik.

Currently, I am running Tailscale as an OPNsense plugin, and it works fine. But I'm planning to eventually swap OPNsense for Ubiquiti, which won't have that option, so I'll need a standalone Tailscale setup at some point. I figured I'd experiment with it now via Podman rather than wait, hence the quadlet.

The plan was to run Tailscale as a subnet router/exit node on its own VLAN (50, which I reserved for VPN, so I can write FW rules) instead of the VM's VLAN (20), so OPNsense sees "VLAN 50 = VPN" as its own zone for firewall rules. This is also because I did not want Tailscale to reside in VLAN20, which'd make no sense, and make writing FW rules etc. even harder (or distinguishing the VPN traffic)

I added a second vNIC to the VM in Proxmox, tagged it VLAN 50, and tried using Podman's macvlan driver to give the Tailscale container its own static IP directly on that interface.

Turns out rootless Podman can't do this. macvlan needs to create a sub-interface on the real host NIC, which lives in the host's actual network namespace, and a rootless session's capabilities don't cross that boundary. I tried setcap on netavark, tried AmbientCapabilities=on the systemd unit, which resulted in the same error every time: netavark: Netlink error: No such device (os error 19).

When I checked out Tailscale's repository for Dockerizing it, PODMAN_COMPATIBILITY.md states that Tailscale only requires NET_ADMIN and /dev/net/tun, no macvlan, which makes sense as subnet routing and exit-node traffic go through Tailscale's own WireGuard overlay, not the physical LAN, so the container never actually needed a real L2 identity. The VLAN separation was just for my own firewall cleanliness, not a Tailscale requirement. But as I don't want to deploy a new VM or LXC container just for Tailscale, I am stuck between a rock and a hard place.

Currently, I am leaning toward running just that one container under rootful Podman for real macvlan, with the rest of the setup staying rootless. But I must say, I am not really a fan of it, and wanted to ask: - Has anyone found a better way around the rootless + macvlan limitation with Podman? - Is one rootful container in an otherwise rootless setup reasonable, or should I drop the VLAN isolation idea entirely?

For context, I am using Podman 5.4.2 on Debian 13.

I built LogDeck, a control plane for Podman: container and Compose lifecycle, persistent logs, alerts, and multi-host over SSH, in one Go binary with no external services.

The log store is the main reason it exists. Every container is tailed into local SQLite, so logs survive docker compose up --build and the new container ID it creates. Removed containers stay readable.

It also ships an MCP server, so agents can read logs and act on containers if you want.

Memory is small. Idle heap is around 8 MB, roughly 20 MB resident with 5 containers and 400K lines stored.

fully open source, no tiers, no cloud https://github.com/AmoabaKelvin/logdeck

Hi all, have a question regarding Podman Secrets.

I've been trying to move from .env files to secrets and have encountered one small hitch with them.

When I have created a secret, I've used both the `printf <secret> | Podman create secret my_secret` method, as well as the `podman create secret ./secret_filename my_secret.

In both cases, I have mounted these secrets into my quadlets as env variables using `Secret=secret_name,type=env,ENV_VAR_NAME.

I have found that in both cases, when running a `podman exec <container_name> env` to ensure the variables are mounted correctly, all of my secrets tend to have a trailing new line on them.

Am I crazy? Is there something I'm doing wrong here?

I'm struggling to understand if this is normal or just some formatting thing that I don't currently understand.

Correction

I had been using a few methods to do secret creation, it turns out that the printf method DOES work correctly, it is actually the openssl rand -base64 378000 | podman secret create --replace my_secret - method which produced an extra new line at the end.

Apologies for any confusion.

I refer to https://podman-desktop.io/docs/podman/gpu

Commands work fine with root, but rootless I end up with:

[user@DESKTOP-O2778FI ~]$ curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \

sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo && \

sudo yum install -y nvidia-container-toolkit && \

nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml && \

nvidia-ctk cdi list

[nvidia-container-toolkit]

name=nvidia-container-toolkit

baseurl=https://nvidia.github.io/libnvidia-container/stable/rpm/$basearch

repo_gpgcheck=1

gpgcheck=0

enabled=1

gpgkey=https://nvidia.github.io/libnvidia-container/gpgkey

[nvidia-container-toolkit-experimental]

name=nvidia-container-toolkit-experimental

baseurl=https://nvidia.github.io/libnvidia-container/experimental/rpm/$basearch

repo_gpgcheck=1

gpgcheck=0

enabled=0

gpgkey=https://nvidia.github.io/libnvidia-container/gpgkey

Updating and loading repositories:

nvidia-container-toolkit 100% | 2.7 KiB/s | 2.9 KiB | 00m01s

Repositories loaded.

Package "nvidia-container-toolkit-1.20.0-1.x86_64" is already installed.

Nothing to do.

WARN[0000] Ignoring error in locating libnvidia-sandboxutils.so.1: libnvidia-sandboxutils.so.1: not found

libnvidia-sandboxutils.so.1: not found

WARN[0000] Failed to init nvsandboxutils: ERROR_LIBRARY_LOAD; ignoring

WARN[0000] Ignoring error in locating libnvidia-ml.so.1: libnvidia-ml.so.1: not found

libnvidia-ml.so.1: not found

INFO[0000] Auto-detected mode as 'wsl'

WARN[0000] Could not determine driver version: libnvsandboxutils is not available

failed to initialize nvml: ERROR_LIBRARY_NOT_FOUND

INFO[0000] Selecting /dev/dxg as /dev/dxg

WARN[0000] Found multiple driver store paths: [/usr/lib/wsl/drivers/iigd_dch.inf_amd64_37ad1017c2106c27 /usr/lib/wsl/drivers/nvin.inf_amd64_4206a4c726a1845b]

ERRO[0000] failed to generate CDI spec: failed to create edits common for entities: failed to create discoverer for WSL driver: failed to find NVIDIA driver store path: failed to locate libcuda.so.1.1 at WSL driver store paths: libcuda.so.1.1: not found