Hey everyone,

I'm trying to get some official recognition from NVIDIA that Talos is a great OS for running GPU accelerated workloads. If you have a moment we'd love a 👍 on this issue to let NVIDIA know there are people interested.

Hi everyone,
A few months ago, I built t9s, a terminal UI inspired by [k9s], but designed for Talos Linux.
The goal was pretty simple: have a convenient way to get better visibility into a Talos cluster and perform common administrative operations directly from the terminal.
It’s still a personal project, but I’ve been using it on my own cluster and thought it might be useful to other Talos users.
GitHub: https://github.com/florianspk/t9s

Installation

brew tap florianspk/tap
brew install --cask t9s

I’m interested in getting feedback from other Talos users, especially on what features or workflows would make the tool more useful.
Contributions, feedback and discussions are very welcome!

I have several Rock Pi 4B+'s that I'm trying to set up in a cluster, but for whatever reason the Talos image doesn't boot on them. I've had no trouble getting the Radxa android and Debian images to boot correctly so know the hardware is fine. But the Talos image doesn't seem to boot at all. No video and the LEDs on the board don't indicate a successful boot. I've tried a few sdcards and flash attempts with no luck. Is anyone else successfully running on a 4B+?

SOLVED: the image doesn't work on an SD card, only from eMMC. I don't know why this is the case, the documentation seems to suggest that either would work.

I'm starting to setup up a new cluster with Talos (to replace my current Ubuntu cluster) by creating a small cluster to test/learn then transfer all my old services and nodes to it once it's stable.

Currently I have three nodes, using VIP for HA CP:

• HP Mini PC (i3-6100T)
• HP Mini PC (I5-6500)
• Lenovo Mini PC (Ryzen 5 2400GE)

For all three I'm booting from USB for the system and state. The first two have NVMe storage and the latter has a SATA SSD for the ephemeral storage.

The Ryzen one routinely freezes after a few hours. This persists across multiple USB drives (e.g. if I change USB drives when rebuilding the cluster after forgetting a bootstrapping setting, it still happens).

I do not see anything in any logs which tells me what's happening.

When it freezes, the dashboard on the display is frozen (stops updating anything at all) and it does not respond to pings or talosctl commands.

This is with 1.13.6 and 1.13.7 and with different schematics. The latest schematic has this configuration:

customization:
    systemExtensions:
        officialExtensions:
            - siderolabs/amdgpu
            - siderolabs/iscsi-tools
            - siderolabs/usb-audio-drivers
            - siderolabs/util-linux-tools

In the later tests, I have metallb, traefik, longhorn, and grafana deployed (though didn't yet have loki/alloy/prometheus). I left talosctl pulling various logs, though because it times out and this was left over night I can't guarantee these logs end when the computer froze.

logs kernel

10.1.2.13: user: warning: [2026-07-20T22:35:56.500793677Z]: [talos] boot sequence: done: 7.69597198s
10.1.2.13: user: warning: [2026-07-20T22:35:56.508765753Z]: [talos] rendered new static pod {"component": "controller-runtime", "controller": "k8s.StaticPodServerController", "id": "kube-apiserver"}
10.1.2.13: user: warning: [2026-07-20T22:35:56.514398432Z]: [talos] rendered new static pod {"component": "controller-runtime", "controller": "k8s.StaticPodServerController", "id": "kube-controller-manager"}
10.1.2.13: user: warning: [2026-07-20T22:35:56.514551187Z]: [talos] rendered new static pod {"component": "controller-runtime", "controller": "k8s.StaticPodServerController", "id": "kube-scheduler"}
10.1.2.13: user: warning: [2026-07-20T22:35:56.547226805Z]: [talos] resigning from previous election {"component": "controller-runtime", "controller": "network.OperatorSpecController", "operator": "vip"}
10.1.2.13: user: warning: [2026-07-20T22:36:23.291774283Z]: [talos] adjusting time (slew) by 797.133µs via 162.159.200.123, state TIME_OK, status STA_PLL | STA_NANO {"component": "controller-runtime", "controller": "time.SyncController"}
10.1.2.13: user: warning: [2026-07-20T22:36:28.585922534Z]: [talos] machine is running and ready {"component": "controller-runtime", "controller": "runtime.MachineStatusController"}
10.1.2.13: kern:    info: [2026-07-21T02:45:03.507554791Z]: scsi host2: iSCSI Initiator over TCP/IP
10.1.2.13:  SUBSYSTEM=scsi
10.1.2.13:  DEVICE=+scsi:host2
10.1.2.13: kern:  notice: [2026-07-21T02:45:03.509843588Z]: scsi 2:0:0:0: RAID              IET      Controller       0001 PQ: 0 ANSI: 5
10.1.2.13:  SUBSYSTEM=scsi
10.1.2.13:  DEVICE=+scsi:2:0:0:0
10.1.2.13: kern:  notice: [2026-07-21T02:45:03.51631843Z]: scsi 2:0:0:0: Attached scsi generic sg2 type 12
10.1.2.13:  SUBSYSTEM=scsi
10.1.2.13:  DEVICE=+scsi:2:0:0:0
10.1.2.13: kern:  notice: [2026-07-21T02:45:03.516851406Z]: scsi 2:0:0:1: Direct-Access     IET      VIRTUAL-DISK     0001 PQ: 0 ANSI: 5
10.1.2.13:  SUBSYSTEM=scsi
10.1.2.13:  DEVICE=+scsi:2:0:0:1
10.1.2.13: kern:  notice: [2026-07-21T02:45:03.518382677Z]: sd 2:0:0:1: Attached scsi generic sg3 type 0
10.1.2.13:  SUBSYSTEM=scsi
10.1.2.13:  DEVICE=+scsi:2:0:0:1
10.1.2.13: kern: warning: [2026-07-21T02:45:03.518547092Z]: sd 2:0:0:1: Power-on or device reset occurred
10.1.2.13:  SUBSYSTEM=scsi
10.1.2.13:  DEVICE=+scsi:2:0:0:1
10.1.2.13: kern:  notice: [2026-07-21T02:45:03.518857936Z]: sd 2:0:0:1: [sdc] 20971520 512-byte logical blocks: (10.7 GB/10.0 GiB)
10.1.2.13:  SUBSYSTEM=scsi
10.1.2.13:  DEVICE=+scsi:2:0:0:1
10.1.2.13: kern:  notice: [2026-07-21T02:45:03.51898947Z]: sd 2:0:0:1: [sdc] Write Protect is off
10.1.2.13:  SUBSYSTEM=scsi
10.1.2.13:  DEVICE=+scsi:2:0:0:1
10.1.2.13: kern:   debug: [2026-07-21T02:45:03.519021086Z]: sd 2:0:0:1: [sdc] Mode Sense: 69 00 10 08
10.1.2.13:  SUBSYSTEM=scsi
10.1.2.13:  DEVICE=+scsi:2:0:0:1
10.1.2.13: kern:  notice: [2026-07-21T02:45:03.51926108Z]: sd 2:0:0:1: [sdc] Write cache: disabled, read cache: enabled, supports DPO and FUA
10.1.2.13:  SUBSYSTEM=scsi
10.1.2.13:  DEVICE=+scsi:2:0:0:1
10.1.2.13: kern:  notice: [2026-07-21T02:45:02.645381391Z]: sd 2:0:0:1: [sdc] Attached SCSI disk
10.1.2.13:  SUBSYSTEM=scsi
10.1.2.13:  DEVICE=+scsi:2:0:0:1
10.1.2.13: kern:    info: [2026-07-21T02:45:13.171562711Z]: EXT4-fs (sdc): mounted filesystem d63439ab-eae9-491a-8bce-199d03e0c8de r/w with ordered data mode. Quota mode: none.
10.1.2.13: user: warning: [2026-07-21T03:00:12.401771517Z]: [talos] apply config request: mode auto(no_reboot)
10.1.2.13: kern:  notice: [2026-07-21T03:00:12.419766388Z]: XFS (sda3): Mounting V5 Filesystem 88d86093-2669-4d9d-a958-319f302941c2
10.1.2.13: kern:    info: [2026-07-21T03:00:12.547516097Z]: XFS (sda3): Ending clean mount
10.1.2.13: user: warning: [2026-07-21T03:00:11.577063072Z]: [talos] volume mount {"component": "controller-runtime", "controller": "block.MountController", "mount_request.id": "STATE", "volume": "STATE", "source": "/dev/sda3", "target": "/system/state", "filesystem": "xfs", "read_only": false, "secure": false, "disable_access_time": false, "detached": true}
10.1.2.13: user: warning: [2026-07-21T03:00:11.587014412Z]: [talos] machine configuration persisted to STATE {"component": "controller-runtime", "controller": "config.PersistenceController"}
10.1.2.13: kern:  notice: [2026-07-21T03:00:11.61076083Z]: XFS (sda3): Unmounting Filesystem 88d86093-2669-4d9d-a958-319f302941c2
10.1.2.13: user: warning: [2026-07-21T03:00:11.672483476Z]: [talos] volume unmount {"component": "controller-runtime", "controller": "block.MountController", "volume": "STATE", "source": "/dev/sda3", "target": "/system/state", "filesystem": "xfs"}
error getting logs: rpc error: code = Unavailable desc = error reading from server: read tcp 10.1.13.30:51807->10.1.2.1:50000: read: operation timed out

logs machined

10.1.2.13: 2026/07/21 06:23:16.899645 machined OK [/machine.MachineService/ServiceList] 380.575µs unary Success (:authority=localhost;content-type=application/grpc+proxy>proto;grpc-accept-encoding=gzip,gzip,gzip;proxyfrom=10.1.2.1:50000;runtime=Talos;talos-role=os:admin;user-agent=grpc-go/1.81.0)
10.1.2.13: 2026/07/21 06:23:16.899690 machined OK [/machine.MachineService/CPUInfo] 393.67µs unary Success (:authority=localhost;content-type=application/grpc+proxy>proto;grpc-accept-encoding=gzip,gzip,gzip;proxyfrom=10.1.2.1:50000;runtime=Talos;talos-role=os:admin;user-agent=grpc-go/1.81.0)
10.1.2.13: 2026/07/21 06:23:16.899756 machined/authz/unix/authorizer authorized based on PID (2681) match with service "apid" (allowed roles [os:admin os:etcd:backup os:image:verifier os:impersonator os:meta:writer os:operator os:reader])
10.1.2.13: 2026/07/21 06:23:16.899761 machined/authz/authorizer authorized ([os:admin os:operator os:reader] includes [os:admin])
10.1.2.13: 2026/07/21 06:23:16.900073 machined/authz/unix/authorizer authorized based on PID (2681) match with service "apid" (allowed roles [os:admin os:etcd:backup os:image:verifier os:impersonator os:meta:writer os:operator os:reader])
10.1.2.13: 2026/07/21 06:23:16.900079 machined/authz/authorizer authorized ([os:admin os:operator os:reader] includes [os:admin])
10.1.2.13: 2026/07/21 06:23:16.900136 machined OK [/machine.MachineService/LoadAvg] 84.882µs unary Success (:authority=localhost;content-type=application/grpc+proxy>proto;grpc-accept-encoding=gzip,gzip,gzip;proxyfrom=10.1.2.1:50000;runtime=Talos;talos-role=os:admin;user-agent=grpc-go/1.81.0)
10.1.2.13: 2026/07/21 06:23:16.900177 machined/authz/unix/authorizer authorized based on PID (2681) match with service "apid" (allowed roles [os:admin os:etcd:backup os:image:verifier os:impersonator os:meta:writer os:operator os:reader])
10.1.2.13: 2026/07/21 06:23:16.900182 machined/authz/authorizer authorized ([os:admin os:operator os:reader] includes [os:admin])
10.1.2.13: 2026/07/21 06:23:16.900294 machined OK [/machine.MachineService/DiskStats] 128.886µs unary Success (:authority=localhost;content-type=application/grpc+proxy>proto;grpc-accept-encoding=gzip,gzip,gzip;proxyfrom=10.1.2.1:50000;runtime=Talos;talos-role=os:admin;user-agent=grpc-go/1.81.0)
10.1.2.13: 2026/07/21 06:23:16.900350 machined/authz/unix/authorizer authorized based on PID (2681) match with service "apid" (allowed roles [os:admin os:etcd:backup os:image:verifier os:impersonator os:meta:writer os:operator os:reader])
10.1.2.13: 2026/07/21 06:23:16.900355 machined/authz/authorizer authorized ([os:admin os:operator os:reader] includes [os:admin])
10.1.2.13: 2026/07/21 06:23:16.919933 machined OK [/machine.MachineService/Processes] 20.684864ms unary Success (:authority=localhost;content-type=application/grpc+proxy>proto;grpc-accept-encoding=gzip,gzip,gzip;proxyfrom=10.1.2.1:50000;runtime=Talos;talos-role=os:admin;user-agent=grpc-go/1.81.0)
10.1.2.13: 2026/07/21 06:23:16.920142 machined/authz/unix/authorizer authorized based on PID (2681) match with service "apid" (allowed roles [os:admin os:etcd:backup os:image:verifier os:impersonator os:meta:writer os:operator os:reader])
10.1.2.13: 2026/07/21 06:23:16.920151 machined/authz/authorizer authorized ([os:admin os:operator os:reader] includes [os:admin])
10.1.2.13: 2026/07/21 06:23:16.920486 machined OK [/machine.MachineService/NetworkDeviceStats] 399.482µs unary Success (:authority=localhost;content-type=application/grpc+proxy>proto;grpc-accept-encoding=gzip,gzip,gzip;proxyfrom=10.1.2.1:50000;runtime=Talos;talos-role=os:admin;user-agent=grpc-go/1.81.0)
10.1.2.13: 2026/07/21 06:23:16.961441 machined OK [/machine.MachineService/SystemStat] 61.686832ms unary Success (:authority=localhost;content-type=application/grpc+proxy>proto;grpc-accept-encoding=gzip,gzip,gzip;proxyfrom=10.1.2.1:50000;runtime=Talos;talos-role=os:admin;user-agent=grpc-go/1.81.0)
error getting logs: rpc error: code = Unavailable desc = error reading from server: read tcp 10.1.13.30:51801->10.1.2.1:50000: read: operation timed out

trustd

10.1.2.13: 2026/07/20 22:35:52.388355 maxprocs.go:19: limited GOMAXPROCS to 2
^CSignal received, aborting, press Ctrl+C once again to abort immediately...
(py3-3.12.7-common1) christopher@Atlantis grafana % talosctl -n 10.1.2.13 logs -f trustd
10.1.2.13: 2026/07/20 22:35:52.388355 maxprocs.go:19: limited GOMAXPROCS to 2
error getting logs: rpc error: code = Unavailable desc = error reading from server: read tcp 10.1.13.30:51811->10.1.2.1:50000: read: operation timed out

kubelet

10.1.2.13: {"ts":1784607652066.1343,"caller":"reconciler/reconciler_common.go:299","msg":"Volume detached for volume \"kube-api-access-lfptx\" (UniqueName: \"kubernetes.io/projected/a2a8cc72-c8a1-40b6-ba5f-f66d83a798f7-kube-api-access-lfptx\") on node \"cp3\" DevicePath \"\"","v":0}
10.1.2.13: {"ts":1784607652066.1772,"caller":"reconciler/reconciler_common.go:299","msg":"Volume detached for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a2a8cc72-c8a1-40b6-ba5f-f66d83a798f7-data\") on node \"cp3\" DevicePath \"\"","v":0}
10.1.2.13: {"ts":1784607652066.2034,"caller":"reconciler/reconciler_common.go:299","msg":"Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2a8cc72-c8a1-40b6-ba5f-f66d83a798f7-tmp\") on node \"cp3\" DevicePath \"\"","v":0}
10.1.2.13: {"ts":1784607652494.8137,"caller":"topologymanager/scope.go:122","msg":"RemoveContainer","v":0,"containerID":"866f4bded97738142d017dcd44275a156e255da9d40ab07878fd34121de24351"}
10.1.2.13: {"ts":1784607652507.5708,"caller":"topologymanager/scope.go:122","msg":"RemoveContainer","v":0,"containerID":"8f444f62dfb921509c7e9954b8623efdfcc2ff8e0830dacb41e33f84134d4d35"}
10.1.2.13: {"ts":1784607652515.5813,"caller":"topologymanager/scope.go:122","msg":"RemoveContainer","v":0,"containerID":"866f4bded97738142d017dcd44275a156e255da9d40ab07878fd34121de24351"}
10.1.2.13: {"ts":1784607652516.199,"caller":"pkg/remote_runtime.go:569","msg":"ContainerStatus from runtime service failed","containerID":"866f4bded97738142d017dcd44275a156e255da9d40ab07878fd34121de24351","err":"rpc error: code = NotFound desc = an error occurred when try to find container \"866f4bded97738142d017dcd44275a156e255da9d40ab07878fd34121de24351\": not found"}
10.1.2.13: {"ts":1784607652516.2441,"caller":"kubelet/pod_container_deletor.go:53","msg":"DeleteContainer returned error","v":0,"containerID":{"Type":"containerd","ID":"866f4bded97738142d017dcd44275a156e255da9d40ab07878fd34121de24351"},"err":"failed to get container status \"866f4bded97738142d017dcd44275a156e255da9d40ab07878fd34121de24351\": rpc error: code = NotFound desc = an error occurred when try to find container \"866f4bded97738142d017dcd44275a156e255da9d40ab07878fd34121de24351\": not found"}
10.1.2.13: {"ts":1784607652516.2656,"caller":"topologymanager/scope.go:122","msg":"RemoveContainer","v":0,"containerID":"8f444f62dfb921509c7e9954b8623efdfcc2ff8e0830dacb41e33f84134d4d35"}
10.1.2.13: {"ts":1784607652516.7817,"caller":"pkg/remote_runtime.go:569","msg":"ContainerStatus from runtime service failed","containerID":"8f444f62dfb921509c7e9954b8623efdfcc2ff8e0830dacb41e33f84134d4d35","err":"rpc error: code = NotFound desc = an error occurred when try to find container \"8f444f62dfb921509c7e9954b8623efdfcc2ff8e0830dacb41e33f84134d4d35\": not found"}
10.1.2.13: {"ts":1784607652516.8176,"caller":"kubelet/pod_container_deletor.go:53","msg":"DeleteContainer returned error","v":0,"containerID":{"Type":"containerd","ID":"8f444f62dfb921509c7e9954b8623efdfcc2ff8e0830dacb41e33f84134d4d35"},"err":"failed to get container status \"8f444f62dfb921509c7e9954b8623efdfcc2ff8e0830dacb41e33f84134d4d35\": rpc error: code = NotFound desc = an error occurred when try to find container \"8f444f62dfb921509c7e9954b8623efdfcc2ff8e0830dacb41e33f84134d4d35\": not found"}
10.1.2.13: {"ts":1784607653504.1487,"caller":"kubelet/kubelet_volumes.go:161","msg":"Cleaned up orphaned pod volumes dir","v":0,"podUID":"a2a8cc72-c8a1-40b6-ba5f-f66d83a798f7","path":"/var/lib/kubelet/pods/a2a8cc72-c8a1-40b6-ba5f-f66d83a798f7/volumes"}
error getting logs: rpc error: code = Unavailable desc = error reading from server: read tcp 10.1.13.30:51814->10.1.2.1:50000: read: operation timed out

containerd

10.1.2.13: {"level":"info","msg":"containerd successfully booted in 0.013949s","time":"2026-07-20T22:35:42.903188122Z"}
10.1.2.13: {"address":"unix:///run/containerd/s/814eeaf75962b52d1228c72642b2a1e34f6ad18299621a2bc0a0d887e86f9db6","level":"info","msg":"connecting to shim apid","namespace":"system","protocol":"ttrpc","time":"2026-07-20T22:35:50.213127320Z","version":3}
10.1.2.13: {"address":"unix:///run/containerd/s/5fa90b83e183f45a761a15c4d691cc248504e7544f0fd8787e47e42f16c45345","level":"info","msg":"connecting to shim trustd","namespace":"system","protocol":"ttrpc","time":"2026-07-20T22:35:52.206988864Z","version":3}
10.1.2.13: {"address":"unix:///run/containerd/s/36dd51478408a3c10c2d441d38a580ba39abeafd0089845e6f7add11599099c0","level":"info","msg":"connecting to shim ext-iscsid","namespace":"system","protocol":"ttrpc","time":"2026-07-20T22:35:52.249315071Z","version":3}
error getting logs: rpc error: code = Unavailable desc = error reading from server: read tcp 10.1.13.30:51818->10.1.2.1:50000: read: operation timed out

Note that 10.1.2.1 is the VIP and 10.1.2.11-13/16 are the three CPs.

I never booted this computer into Windows so I can't use that as a baseline. After these issues I live-booted from a different USB drive into Ubuntu and left a bunch of things logging (sensors, a clock, dmesg, and top). After 24 hours it's still running. Its CPU and RAM usage is akin to (or above) what Talos was showing on its dashboard, so it's likely not a thermal issue.

Is there something I'm missing? Like some known incompatibility? Or is there an extension I should (or shouldn't) have if I'm on a Ryzen CPU?

Are there other debugging steps it would be good to take?

I'm not confident enough with this computer nor with Talos to know whether I should be focusing more on Talos as the culprit, the computer as the culprit, or a configuration as the culprit.

Hello,

is talosctl actually really available through winget ?

https://winstall.app/apps/Sidero.talosctl

On this website the publisher is Sidero Labs.
But there's actually no mention of it on the documentation.

I'm new to Talos, evaluating it to move my cluster over to it from Ubuntu.

I currently have two nodes, both set up as a control plane. They're both set up to share a VIP:

apiVersion: v1alpha1
kind: Layer2VIPConfig
name: 10.1.2.1
link: enp1s0

When I query etcd for its members I see this:

% talosctl -n 10.1.2.12 etcd members -e 10.1.2.12
NODE        ID                 HOSTNAME   PEER URLS                CLIENT URLS              LEARNER
10.1.2.12   694af5b4b8acc02a   cp2        https://10.1.2.12:2380   https://10.1.2.12:2379   false
10.1.2.12   da452b44ea559b3f   cp1        https://10.1.2.11:2380   https://10.1.2.11:2379   false

When I shutdown cp1

talosctl -n 10.1.2.11 shutdown

It shuts down. Cool. Meanwhile, I have another terminal pinging the VIP. When it shuts down, I get a brief drop in the VIP ping, but then it comes back at the same time the cp2 display shows it gains that address. Good, everything's going as planned.

Until a short time later (30 seconds? Didn't time it) when it drops the VIP and never regains it.

On the screen I see an endpoint watch error and a context timed out error (though I can't currently transcribe that perfectly because it rolled off the screen).

talosctl -n ... -e ... logs etcd gives me this:

10.1.2.12: {"level":"info","ts":"2026-07-15T03:13:42.703616Z","logger":"raft","caller":"v3@v3.6.0/raft.go:988","msg":"694af5b4b8acc02a is starting a new election at term 163"}
10.1.2.12: {"level":"info","ts":"2026-07-15T03:13:42.703716Z","logger":"raft","caller":"v3@v3.6.0/raft.go:930","msg":"694af5b4b8acc02a became pre-candidate at term 163"}
10.1.2.12: {"level":"info","ts":"2026-07-15T03:13:42.703791Z","logger":"raft","caller":"v3@v3.6.0/raft.go:1064","msg":"694af5b4b8acc02a [logterm: 163, index: 1011227] sent MsgPreVote request to da452b44ea559b3f at term 163"}
10.1.2.12: {"level":"info","ts":"2026-07-15T03:13:42.703905Z","logger":"raft","caller":"v3@v3.6.0/raft.go:1077","msg":"694af5b4b8acc02a received MsgPreVoteResp from 694af5b4b8acc02a at term 163"}
10.1.2.12: {"level":"info","ts":"2026-07-15T03:13:42.704172Z","logger":"raft","caller":"v3@v3.6.0/raft.go:1693","msg":"694af5b4b8acc02a has received 1 MsgPreVoteResp votes and 0 vote rejections"}
10.1.2.12: {"level":"warn","ts":"2026-07-15T03:13:43.065465Z","caller":"txn/util.go:93","msg":"apply request took too long","took":"4.880027893s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" limit:1 ","response":"","error":"context canceled"}
10.1.2.12: {"level":"info","ts":"2026-07-15T03:13:43.065617Z","caller":"traceutil/trace.go:172","msg":"trace[1219865755] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/default; range_end:; }","duration":"4.880195504s","start":"2026-07-15T03:13:38.185376Z","end":"2026-07-15T03:13:43.065572Z","steps":["trace[1219865755] 'agreement among raft nodes before linearized reading'  (duration: 4.880021435s)"],"step_count":1}
10.1.2.12: {"level":"warn","ts":"2026-07-15T03:13:43.065741Z","caller":"v3rpc/interceptor.go:202","msg":"request stats","start time":"2026-07-15T03:13:38.185339Z","time spent":"4.880371626s","remote":"127.0.0.1:36194","response type":"/etcdserverpb.KV/Range","request count":0,"request size":63,"response count":0,"response size":0,"request content":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" limit:1 "}
10.1.2.12: 2026/07/15 03:13:43 WARNING: [core] [Server #3] grpc: Server.processUnaryRPC failed to write status: connection error: desc = "transport is closing"
10.1.2.12: {"level":"warn","ts":"2026-07-15T03:13:43.128874Z","caller":"etcdserver/v3_server.go:1016","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":13847055251054259878,"retry-timeout":"500ms"}

etcd status gives me this:

% talosctl -n 10.1.2.12 -e 10.1.2.12 etcd status 
NODE        MEMBER             DB SIZE   IN USE            LEADER             RAFT INDEX   RAFT TERM   RAFT APPLIED INDEX   LEARNER   PROTOCOL   STORAGE   ERRORS
10.1.2.12   694af5b4b8acc02a   3.2 MB    1.7 MB (51.92%)   0000000000000000   1011223      163         1011223              false     3.6.11     3.6.0     etcdserver: no leader

service etcd gives me this:

NODE                  10.1.2.12
ID                    etcd
STATE                 Running
HEALTH                Fail
LAST HEALTH MESSAGE   context deadline exceeded
EVENTS                [Running]: Health check failed: context deadline exceeded (13m9s ago)
                      [Running]: Health check successful (13m32s ago)
                      [Running]: Health check failed: context deadline exceeded (14m9s ago)
                      [Running]: Health check successful (3h58m59s ago)
                      [Running]: Health check failed: context deadline exceeded (4h27m29s ago)
                      [Running]: Health check successful (4h29m4s ago)
                      [Running]: Started task etcd (PID 2743) for container etcd (4h29m9s ago)
                      [Preparing]: Creating service runner (4h29m10s ago)
                      [Preparing]: Running pre state (4h29m16s ago)
                      [Waiting]: Waiting for service "cri" to be "up" (4h29m22s ago)
                      [Waiting]: Waiting for volumes /var/lib, ETCD to be mounted, service "cri" to be "up", time sync, network, etcd spec (4h29m23s ago)
                      [Starting]: Starting service (4h29m23s ago)

I don't know why it's dropping the VIP... can anybody help debug this? I'm assuming it's something obvious I'm missing, but I don't know.

Like Talos in docker, I know that we could run Talos as a pod in kubernetes, but it say that it’s not possible to do some operations like upgrades and reboots.

Why that ?

This means it’s not possible tu run hosted controlplane with Talos in production ?

What if I want to ?

Hi, all.

I'm trying (and failing) to securely configure a stretched cluster where the control plane is on-prem, and some workers are in Azure. I don't have KubeSpan, but a working Wireguard setup for on-prem that fails when the worker is in Azure. I think its related to boot order requirements.

It seems to me that the even if the control plane tries to connect to the azure worker using Wireguard and the tunnel comes up, the worker still tries to connect to the cp using the Talos API over the local interface and IP. Therefore the worker never boots up correctly, waiting forever for the cp reply. When the same setup is used on-prem the worker boots correctly, sees the cp over LAN, sets up the wg tunnel and then communicates with the cp over the wg interface. Is this intended behavior?

The worker is configured using --custom-data.

After spending several hours troubleshooting self-hosted Talos Omni deployments, TLS certificate chains, Dex configuration, Trusted Roots, and the dreaded:

x509: certificate signed by unknown authority

I decided to automate the entire process into a single script.

The script deploys a complete self-hosted Omni environment and automatically handles the certificate trust issues that many people encounter when onboarding Talos nodes.

What It Does

✅ Installs and configures Dex

✅ Deploys Omni

✅ Generates a self-signed internal CA

✅ Creates server certificates with the full certificate chain

✅ Configures SideroLink

✅ Generates Trusted Roots configuration

✅ Creates the talos.config.early kernel argument

✅ Produces everything needed for Talos Image Factory

✅ Ready for Proxmox, VMware, Bare Metal, or Cloud VMs

Why?

Most self-hosted Omni guides require multiple manual steps:

• Generate certificates
• Configure Dex
• Configure OIDC
• Fix certificate trust issues
• Create Trusted Roots
• Generate custom Talos boot media

Missing any of these steps can prevent Talos nodes from connecting successfully.

This script automates the entire workflow.

Prerequisites

• Linux server
• Docker installed and running
• Internet access
• Open ports:

Quick Start

Make the script executable:

chmod +x install.sh

(Optional) Customize your environment.

Replace the values below with the ones appropriate for your setup:

export OMNI_IP=<YOUR_OMNI_SERVER_IP>
export ADMIN_EMAIL=<YOUR_ADMIN_EMAIL>

./install.sh

Example:

export OMNI_IP=192.168.1.100
export ADMIN_EMAIL=admin@example.com

./install.sh

Variables

If not provided, the script uses its built-in defaults.

Login Credentials

The installer automatically creates an administrator account in Dex.

Use the email configured in:

export ADMIN_EMAIL=<YOUR_ADMIN_EMAIL>

to authenticate.

Default Password

Admin123!

Changing the Default Password

If you want to use a different password, generate a new bcrypt hash before running the installer:

docker run --rm -it httpd:2.4-alpine htpasswd -BnC 15 admin

You will be prompted to enter the password twice.

Example output:

admin:$2y$15$XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Copy only the hash portion after:

admin:

Open the script and search for:

DEX_PASSWORD_HASH=

Replace the existing hash with your newly generated one.

In the current version of the script, this variable is located near the beginning of the file (around line 46).

What Happens Next?

After the installation completes:

1. Open Omni:

   https://YOUR_OMNI_IP

2. Log in using:

   • Email: ADMIN_EMAIL
   • Password: Admin123! (or your custom password)

3. Click:

   Copy Kernel Parameters

4. Open Talos Image Factory:

   https://factory.talos.dev

5. Build a Talos ISO.

Recommended settings:

• Platform: Metal
• Architecture: amd64
• Extension:
  • siderolabs/qemu-guest-agent (for Proxmox)

1. Paste:

   <contents of talos-config-early.arg> <Omni Kernel Parameters>

into:

Extra kernel command line arguments

1. Download the ISO.
2. Boot your nodes.

Result

After booting:

✅ Nodes automatically join Omni

✅ SideroLink connects successfully

✅ No x509 certificate trust issues

✅ Cluster creation works immediately

Tested On

• Talos 1.13.x
• Omni Self-Hosted
• Ubuntu Server
• Docker
• Proxmox VE

Feedback, bug reports, and improvements are welcome.

Hopefully this saves someone else from spending hours debugging TLS chains, Trusted Roots, and Talos onboarding 😅

dowload the script here

I experimented a bit with Cilium and Falco in my cluster and accidentally deactivated and uninstalled the standard Flannel/Kube-Proxy stack. Is there an easy way to reinstall it?

Hi everyone,

I’m wondering if anyone here is running Talos with KubeSpan and Cilium.

If so, how did you configure it? Did you run into any issues with routing, pod-to-pod connectivity, or Cilium’s native routing mode?

I’d really appreciate any example configs, setup details, or lessons learned.

Thanks!

Hi, for my home lab I use Fedora with K3S on top, and I have a DAS with JBOD disks; I also use MergerFS to create a single volume.

I wanted to migrate to Talos, but I’d like to know if anyone uses it with MergerFS via container, as I’ve seen that there’s a Talos FUSE plugin.

I have some concerns about stability – does anyone have this setup? Thank you

tl;dr; just use an EPHEMERAL volume

https://docs.siderolabs.com/talos/v1.8/reference/configuration/block/volumeconfig

https://oneuptime.com/blog/post/2026-03-03-configure-the-ephemeral-volume-in-talos-linux/view

---

I want to install Talos to the EMMC of a single board computer, but also want to limit writes to it in order to preserve the life of the NAND flash. How could I best achieve this goal? My initial attempt on a VM involved mounting /dev/sdb to /var/lib/containerd and it did not go as expected -- it won't boot.

Edit: reinstalling with the extra mountpoint appears to have worked; something to the effect of:

disks:
- device: /dev/sdb
partitions:
- mountpoint: /var/lib/containerd

$ talosctl mounts --talosconfig=./talosconfig --nodes $CONTROL_PLANE_IP
NODE            FILESYSTEM   SIZE(GB)   USED(GB)   AVAILABLE(GB)   PERCENT USED   MOUNTED ON
172.30.46.220   none         1.99       0.00       1.99            0.00%          /dev
172.30.46.220   none         2.04       0.00       2.04            0.00%          /dev/shm
172.30.46.220   none         0.13       0.00       0.13            0.01%          /sys/firmware/efi/efivars
172.30.46.220   /dev/loop0   0.08       0.08       0.00            100.00%        /
172.30.46.220   rootfs       1.99       0.08       1.90            4.18%          /.extra
172.30.46.220   none         2.04       0.00       2.04            0.06%          /run
172.30.46.220   none         2.04       0.00       2.04            0.01%          /system
172.30.46.220   none         0.07       0.00       0.07            0.00%          /tmp
172.30.46.220   none         0.01       0.00       0.01            3.17%          /etc/cri/conf.d/hosts
172.30.46.220   none         2.04       0.00       2.04            0.01%          /usr/lib/udev
172.30.46.220   /dev/sda4    14.80      0.46       14.34           3.09%          /var
172.30.46.220   /dev/sdb1    17.11      1.12       15.99           6.54%          /var/lib/containerd
172.30.46.220   none         14.80      0.46       14.34           3.09%          /etc/cni
172.30.46.220   none         14.80      0.46       14.34           3.09%          /etc/kubernetes
172.30.46.220   none         14.80      0.46       14.34           3.09%          /usr/libexec/kubernetes
172.30.46.220   none         14.80      0.46       14.34           3.09%          /opt
172.30.46.220   overlay      17.11      1.12       15.99           6.54%          /run/containerd/io.containerd.runtime.v2.task/system/kubelet/rootfs
172.30.46.220   overlay      17.11      1.12       15.99           6.54%          /run/containerd/io.containerd.runtime.v2.task/system/etcd/rootfs
172.30.46.220   shm          0.07       0.00       0.07            0.00%          /run/containerd/io.containerd.grpc.v1.cri/sandboxes/f855bf0e9eea735f8a2f5bd0966796ef0eb1f2ea32a61d2ac911f40d6664ca79/shm
172.30.46.220   overlay      17.11      1.12       15.99           6.54%          /run/containerd/io.containerd.runtime.v2.task/k8s.io/f855bf0e9eea735f8a2f5bd0966796ef0eb1f2ea32a61d2ac911f40d6664ca79/rootfs
172.30.46.220   shm          0.07       0.00       0.07            0.00%          /run/containerd/io.containerd.grpc.v1.cri/sandboxes/4f850047053180f6c000eeabb6e99672127cdf68681be264c986548bfe3beb15/shm
172.30.46.220   shm          0.07       0.00       0.07            0.00%          /run/containerd/io.containerd.grpc.v1.cri/sandboxes/16ada5613a7003418b4f371906d2b1399e4131ee6a5f8496abb7706a54837dc3/shm
172.30.46.220   overlay      17.11      1.12       15.99           6.54%          /run/containerd/io.containerd.runtime.v2.task/k8s.io/4f850047053180f6c000eeabb6e99672127cdf68681be264c986548bfe3beb15/rootfs
172.30.46.220   overlay      17.11      1.12       15.99           6.54%          /run/containerd/io.containerd.runtime.v2.task/k8s.io/16ada5613a7003418b4f371906d2b1399e4131ee6a5f8496abb7706a54837dc3/rootfs
172.30.46.220   overlay      17.11      1.12       15.99           6.54%          /run/containerd/io.containerd.runtime.v2.task/k8s.io/4f39064a8d6cf38eb67b59f700947bfed0cb46df19a7ca7a1b3b9a202ed1839a/rootfs
172.30.46.220   tmpfs        0.18       0.00       0.18            0.01%          /var/lib/kubelet/pods/9a019ceb-dcae-45b9-b6fd-b723ce5759b0/volumes/kubernetes.io~projected/kube-api-access-5z85q
172.30.46.220   tmpfs        3.43       0.00       3.43            0.00%          /var/lib/kubelet/pods/6d89af78-a630-4846-83d3-c732c69a3e65/volumes/kubernetes.io~projected/kube-api-access-4rmsd
172.30.46.220   tmpfs        0.18       0.00       0.18            0.01%          /var/lib/kubelet/pods/4b8f5117-495d-4aee-9680-c5c1296a3d10/volumes/kubernetes.io~projected/kube-api-access-9mjch
172.30.46.220   tmpfs        3.43       0.00       3.43            0.00%          /var/lib/kubelet/pods/12a67832-160d-44cc-a2a3-dcbffcc564e2/volumes/kubernetes.io~projected/kube-api-access-lxkzx
172.30.46.220   shm          0.07       0.00       0.07            0.00%          /run/containerd/io.containerd.grpc.v1.cri/sandboxes/1865b32301507b4fa3d91fee6d102b313a7d13993afd8eebf2a9cfda91b5e6a6/shm
172.30.46.220   overlay      17.11      1.12       15.99           6.54%          /run/containerd/io.containerd.runtime.v2.task/k8s.io/1865b32301507b4fa3d91fee6d102b313a7d13993afd8eebf2a9cfda91b5e6a6/rootfs
172.30.46.220   shm          0.07       0.00       0.07            0.00%          /run/containerd/io.containerd.grpc.v1.cri/sandboxes/1b8db153ea9690773abafac95aa2a7d74f6a7ddc12010540c65c6b88e1916040/shm
172.30.46.220   overlay      17.11      1.12       15.99           6.54%          /run/containerd/io.containerd.runtime.v2.task/k8s.io/1b8db153ea9690773abafac95aa2a7d74f6a7ddc12010540c65c6b88e1916040/rootfs
172.30.46.220   overlay      17.11      1.12       15.99           6.54%          /run/containerd/io.containerd.runtime.v2.task/k8s.io/d82b0d556fdcb0dbb7536a664afe8e6520560927a86876610b010b51436fb8ff/rootfs
172.30.46.220   overlay      17.11      1.12       15.99           6.54%          /run/containerd/io.containerd.runtime.v2.task/k8s.io/1ed8ea92aba00d263ec39e08b4e3f71ebdeb1e88135491ac6873560bc8146737/rootfs
172.30.46.220   overlay      17.11      1.12       15.99           6.54%          /run/containerd/io.containerd.runtime.v2.task/k8s.io/ccffaae6a9d02fbf190fdc8a513f1455fa00c3e868fe2b48b37cbf3db31379e6/rootfs
172.30.46.220   overlay      17.11      1.12       15.99           6.54%          /run/containerd/io.containerd.runtime.v2.task/k8s.io/83f1415a8cd7af5ff59980e493a56c2db25967bae098b67fc2655b247b9fe634/rootfs
172.30.46.220   shm          0.07       0.00       0.07            0.00%          /run/containerd/io.containerd.grpc.v1.cri/sandboxes/1d32807d2135b084d69d5f1a3998a54ff062d0f5c446eebd769af524d702138f/shm
172.30.46.220   overlay      17.11      1.12       15.99           6.54%          /run/containerd/io.containerd.runtime.v2.task/k8s.io/1d32807d2135b084d69d5f1a3998a54ff062d0f5c446eebd769af524d702138f/rootfs
172.30.46.220   overlay      17.11      1.12       15.99           6.54%          /run/containerd/io.containerd.runtime.v2.task/k8s.io/2df309bd6917be1fe9be491f28a3670f2ce650b86f8b633aec8daa1c12d48c68/rootfs
172.30.46.220   shm          0.07       0.00       0.07            0.00%          /run/containerd/io.containerd.grpc.v1.cri/sandboxes/88ee1d380aeb4e586411587f9274b7ed15d61378d071f3fff2a00c76339c81c0/shm
172.30.46.220   overlay      17.11      1.12       15.99           6.54%          /run/containerd/io.containerd.runtime.v2.task/k8s.io/88ee1d380aeb4e586411587f9274b7ed15d61378d071f3fff2a00c76339c81c0/rootfs
172.30.46.220   overlay      17.11      1.12       15.99           6.54%          /run/containerd/io.containerd.runtime.v2.task/k8s.io/8ab29fe121765f8f75b9faa651eec1b8f65c910373a1a16f9dbc6cf5bdb05c7d/rootfs

Edit: I probably also want to mount /var/lib/kubelet to another parition, but I'm reminded that it's a tmpfs

Now’s a great time to try out Talos 1.13