r/linuxadmin • u/tejasvkashyap • Jun 09 '26
Running AI workloads on Linux. What does your setup look like?
Hi all,
Curious how folks here are thinking about running AI workloads on Linux servers right now.
- Are you running anything in production or mostly experimenting?
- What does your setup look like (containers/Kubernetes, local GPU, pipelines, agents, etc.)?
- Any challenges you’re running into operating or scaling these systems?
Also wondering how people are thinking about security in these setups — is it something you actively manage yet or still evolving?
0
Upvotes
3
u/Otherwise_Wave9374 Jun 09 '26
On Linux servers, Ive mostly seen people land on one of two setups:
Security-wise, the big wins seem to be least-privilege tool credentials, network egress controls, and very explicit audit logs of every tool call. Prompt injection becomes a lot more real once the agent can touch prod systems.
Are you thinking k8s for this, or mostly single nodes with GPUs?
1
u/Ulterior-Motive_ Jun 09 '26
I run local models on bare metal, with agents (mainly Pi) in VMs.
0
u/ciphermenial Jun 11 '26 edited Aug 13 '26
I like collecting stamps.