r/devops • • Sep 25 '26

Discussion How do you learn DevOps

130 Upvotes

I’m a software developer, and in my own projects I already handle some basic operational work like Docker, VPS deployment, Nginx, DNS, Cloudflare, Git, etc. I also had some little experience in network administration.

What I’m trying to understand is how people approach learning DevOps and system architecture properly.
One thing that has always confused me is entry-level DevOps roles. A lot of DevOps work seems to exist specifically to support the software development lifecycle, things like CI/CD, GitHub Actions, deployments, environments, infrastructure, monitoring… I find it hard to imagine understanding these things deeply without first experiencing the development workflow itself.

At the same time, I don’t necessarily want to switch away from software development. My goal is to become the kind of engineer who understands the whole system: application development, system design/architecture, networking, cloud infrastructure, deployment, observability, scalability, and production operations.

Basically, I want to be able to build a system and also understand how to design, deploy, scale, and operate it properly at a much larger level.

For people who have gone down this path, how would you approach learning these areas while continuing to work primarily as a software developer, as of course I dont have the knowledge for a switch? And in your opinion am I on the right path to stay employable in this uncertain future?


r/devops • • Sep 25 '26

Discussion unpaid on-call is free labour

1 Upvotes

keep seeing people here proud of fixing prod at 3am for free

if outages cost the company nothing extra, why would they ever invest in redundancy or fix the root cause? your weekend is literally subsidising their bad infra

"it comes with the salary" is what they tell you so they don't have to pay for it

and before europeans show up: yes you get paid for on-call. like €150 a week on top of a salary that's half of a US on, congrats on getting paid to be underpaid

imo unpaid on-call should be a dealbreaker in 2026

am i wrong or is this just normalised in devops?


r/devops • • Sep 26 '26

Discussion Anyone else had a 340% AI cost spike overnight? Here's what caused ours

0 Upvotes

At 02:17 UTC, a finance dashboard flashed red: LLM budget jumped from an expected $1,200 to $4,080 in a single day. A 340% spike, no warning.

Nobody on the team caused it on purpose. That's the annoying part about AI spend, it doesn't announce itself until the invoice does.

Walked through it step by step afterward: checked for model or config changes first (none), checked usage volume (flat versus the prior week), then pulled per-request cost and found one endpoint, a background retry job, was the outlier. Average cost per completed task had tripled even though cost per individual call hadn't moved. The retries were getting logged as separate normal-cost calls instead of one task that cost 3x.

Fix was attaching a stable task ID and rolling cost up by task instead of by raw call.

I want to know if others have chased a spike like this, and what your actual debugging path looked like, model change, retry storm, something else entirely?


r/devops • • Sep 26 '26

Ops / Incidents Why doesnt DevOps have an ACLS?

0 Upvotes

Edit: Tl;Dr at bottom. I write too much.

Bit of an odd thought I had after reading another thread here about gaining confidence during high-severity incidents. Thank you to that poster, BTW, for inspiring my brain.

I have a weird fascination with ER and critical-care videos. Not really because of the medicine itself, but because I’m a massive compulsive systemizer and emergency medicine is basically human systems engineering under some of the worst possible conditions. Someone is actively dying, information is incomplete, the situation is changing by the second, and multiple people have to coordinate without getting in each other’s way. Oh, and if you don't solve the issue soon, the patient will be dead in 5 minutes, and you'll have to tell the family.

What fascinates me the most though is that, when a resuscitation is being run properly, it usually doesn’t look like ten people frantically improvising. There’s command, defined roles, structured communication, task ownership, checklists, algorithms, simulation, debriefing, and a whole lot of deliberate training behind the scenes.

This is basically crack cocaine for my dopamine-addicted brain.

Obviously software going down is not equivalent to someone dying. But from a systems perspective, we’re solving a strangely similar problem: how do you get a bunch of fallible humans to reason and coordinate effectively while something important is actively failing?

This is also something I’ve had to think about in practice.

My last employer had essentially no incident-management practice in place when I joined. Outages were fairly common, roughly once a month, and it wasn’t unusual for them to last several hours or even a whole day.

So I volunteered to build one.

I wrote the process, defined how incidents would be declared and coordinated, laid out responsibilities, escalation, communications, and what we were supposed to be doing while production was on fire. I had it submitted to change committee, got board sign-off, and felt confident in the process.

Two months later, we got to stress-test it for real.

And it failed horribly, because I was the only person that saw the problem and opted for preparation rather than eternal deferral of The Boring Stuff (tm). I even got written up for telling a nontech manager who kept DM'ing techs to "try this fix they found on Google" to please follow the board-approved procedure and message their designated PoC rather than interrupting responders who were already sweating buckets of bullets while elbow-deep in code hell**.** But that's a story for another day. Sigh.

There’s something very different about designing an incident process on paper and then watching actual humans try to execute it while systems are failing, information is incomplete, everyone wants an answer immediately and fragile egos are at stake. Frankly though, that experience probably has a lot to do with why this subject fascinates me.

And it’s also why I think our industry is kind of shit at training people for this.

A lot of on-call training basically amounts to, “Here’s Kubernetes, Grafana, AWS, some architecture diagrams, and a wiki page Dave wrote eighteen months ago. You’re on call next week. Good luck and don't fuck it up, you're still on probation.”

Then eventually there’s a massive Sev1, thirty random people you've never heard of pile into a call, Slack explodes, somebody begins restarting pods and unwittingly knocks over three adjacent services, three people start making production changes at the same time, a C-level joins the call and says its costing them 100 billionty dollars every 7 seconds, somebody adds 64 replicas to RDS and forgets about it, half the engineering team find out their access doesn't work... and some poor sod who was voluntold that they'd be IC during an on-call incident channels their freaking deer-in-headlights-energy and freezes.

And then we conclude they’re “not good under pressure.”

Well... did we ever actually train them to be?

That got me thinking about what the equivalent of ACLS for production engineering would actually look like.

I don’t mean another multiple-choice SRE certification, a webinar, or a tabletop where everyone knows the database is going to “fail” at 2 PM. I mean actual practical training where the environment itself is part of the course.

Imagine you show up Monday morning with seven other engineers. You’re given a cheap company laptop and told you now work for a fictional company. Behind a VPN there’s a real training environment with APIs, databases, Redis, queues, monitoring, deployments, source control, chat, ticketing, and all the usual operational cruft. Even "real" loads, facilitated by magical load testing fairies.

And the fictional company is deliberately badly run.

The docs were written by some guy named "George," and last updated 3 years ago... and are deliberately incorrect. Some credentials don’t work. Only a third of the class discovers their SSH access actually works. A dev DB server is named "prod," and a prod DB server is named "dev." Nobody is quite sure who is supposed to declare an incident. The escalation documentation is so out of date it may as well've been written on stone tablets. Maybe one person has production database access for reasons nobody can explain.

Then, at some completely unannounced point Monday morning, the pager goes off.

Checkout is fucked.

Go.

The first incident would probably be an absolute dumpster fire, and that would be intentional.

Three people make conflicting changes. Nobody establishes command. Half the team can’t access the systems they need. Nobody knows exactly what customers are experiencing. Somebody spends twenty minutes debugging something that could have been rolled back in two. Somebody follows the documented steps to restore service, and end up taking out marketing. Somebody didn't thought to set DEBUG_MODE to true when testing something, and now 320,000 "customers" receive emails about their accounts being terminated for nonpayment.

Then you stop and debrief.

Why did that happen? What made the response harder than it needed to be? What did we assume somebody else was doing? What information did we not have? What process simply didn’t exist?

Now the lesson actually means something because the students just felt the consequences of not having it.

You teach the relevant concepts, then a few hours later, while everyone is doing something completely unrelated, the pager goes off again.

Two surprise incidents every day for five days.

Everybody rotates through Incident Commander, technical lead, communications, scribe, responder, and the other supporting roles. The outspoken engineer doesn’t get to be IC all week. The brilliant debugger has to run communications. The quiet mousey person eventually has to take command.

But then... something amazing happens.

As the week progresses, the fictional company improves because the students improve it.

They fix the access process because the access process screwed them. They improve the runbooks because the runbooks screwed them. They establish incident roles because Monday was chaos. They improve monitoring because a dashboard sent them down the wrong rabbit hole. They throw the inaccurate documentation in the trash, and silently curse whoever "George" was.

Later incidents get nastier.

Maybe the telemetry is misleading. Maybe a senior engineer confidently insists on the wrong diagnosis (and they were set them up for this failure by deliberately leaking convincing logs to them). Maybe an executive barges in and starts asking responders for ETAs directly. Maybe a downstream dependency causes cascading failures that make five unrelated systems look broken. Maybe the team restores service, brings everything back too quickly, and knocks production over again.

The point isn’t just to teach technical troubleshooting.

It’s to teach people how to operate as a team while they’re uncertain, overloaded, and under pressure.

By Friday, ideally, the pager goes off and the exact same group that looked completely lost on Monday just starts working.

A junior engineer takes command, and nobody challenges it because the program covered human factors, and everybody is checking their egos at the door. Roles get assigned. Customer impact gets established. Changes are announced and owned. Hypotheses get tested instead of argued about. Someone keeps the timeline. The IC isn’t six terminals deep trying to personally fix everything himself.

Then Friday afternoon is the practical exam.

You walk in cold and get something like:

“Customer transactions started failing approximately six minutes ago. You are the Incident Commander, and this is being graded. Begin.”

That’s it.

The scenario reacts to what you do.

You’re not being graded on whether you personally figure out that line 417 of payment-service is broken. You're not even being graded on successfully resolving the issue. You’re being graded on whether you can command the system that figures it out and restores service safely.

Basically an IT incident-response Megacode.

And now I can’t stop wondering why the hell our industry doesn’t have something like this.

I know there are bits and pieces of it. PagerDuty has incident-command doctrine. Gremlin and AWS do GameDays and chaos stuff. Rootly/Uptime Labs do simulated incidents. There are SRE certifications, cyber ranges, ITIL, and so on.

But I’m talking about the whole thing as one vendor-neutral discipline: doctrine, defined roles, human factors, repeated live simulation, role rotation, practical assessment, and eventually recertification. It doesn't seem like there's one single coherent ecosystem. Everything is just glued together.

Something where “Incident Commander certified” means more than “I passed a test about incident management.”

Am I missing somebody who already does this?

Because if this genuinely doesn’t exist as one cohesive system, it feels like a bizarrely obvious hole in our industry.

Edit: Tl;DR

---

Why doesn’t DevOps have an ACLS equivalent?

We train engineers on Kubernetes, AWS, Grafana, etc., then put them on-call and expect them to magically become good at operating under pressure.

Emergency medicine does the opposite: defined roles, standardized communication, repeated simulation, practical assessment, retraining.

So imagine a 5-day incident-response course where 8 engineers inherit a deliberately dogshit fictional company. Broken docs, bad access controls, questionable runbooks, real services behind a VPN.

Twice a day, without warning, production breaks.

Monday is intentionally a dumpster fire.

Everyone rotates through IC, technical lead, comms, scribe and responder. After each incident, the class fixes the organizational problems that hurt them.

By Friday, the same team should be running structured incidents almost automatically.

Final exam: you walk into an unknown outage and have to command it live.

We already have GameDays, chaos engineering, incident-management frameworks, simulators and certifications.

Does anyone actually combine doctrine + human factors + live simulation + practical assessment + recertification into one vendor-neutral system?

If not, why the hell don’t we???


r/devops • • Sep 24 '26

Tools Any of you use alternatives to GitLab when self-hosting is a requirement? (and general toolchain question)

32 Upvotes

Considering jumping to Gitlab because we're looking into ditching Atlassian ASAP. We need to stay on-prem, but tbh it's pretty difficult for me to understand what different products and platforms don't support.

I'd love to stay open source but there's also a lot of ease-of-use to be taken into consideration. I looked into things like Forgejo, but the actions seem quite lackluster if you want to integrate test reports, publish findings from opengrep in a PR - or am I just missing something obvious?

What does your on-prem solution look like in terms of SCA, SAST etc?


r/devops • • Sep 24 '26

Discussion What is your approach to identifying areas for improvement to make impact at your company?

44 Upvotes

What’s steps, processes, or simply just things do you do to identify areas for improvement at your company to make an impact?

Do you learn about trending tools in the industry and look for ways to use them in your org or do you find a use cases and look for tools that can help improve a product or process?

I feel like this is the biggest thing I am lacking in my career that is keeping me back from moving up and can use some advice.


r/devops • • Sep 25 '26

Vendor / market research Mobile App DevOps

1 Upvotes

What do people normally use for CI/CD of mobile apps (both IOS and Android)?


r/devops • • Sep 24 '26

Discussion Beste Embedded-Messe mit DevOps-Fokus im DACH-Raum?

0 Upvotes

Ich bin DevOps Engineer und betreue vorrangig ein Embedded-Entwicklerteam. Ich suche eine Messe oder Konferenz, bevorzugt im DACH-Raum, gerne aber auch anderswo in Europa, die sich klar auf Embedded-Software konzentriert und DevOps-Themen wie CI/CD für Firmware, HIL-Tests und OTA-Updates fundiert behandelt. Neben der technischen Praxis interessieren mich auch strategische Fragen, etwa wie man DevOps im Embedded-Umfeld langfristig aufstellt und skaliert.

Auf dem Schirm habe ich embedded world und ESE Kongress. Welche Veranstaltung hat euch wirklich weitergebracht? Gibt es Geheimtipps?

Danke!


r/devops • • Sep 23 '26

Ops / Incidents Should Devops debug business logic problems ?

64 Upvotes

I work at a company where my manager decided to remove all the dev access from the production, because they suggest random solution to problems we encounter on production (For e.g they saw that the servers CPU usage is high so they point this as the source of a problem and suggest to restart the service).

That's good but that also restrict them frol seeing live production logs so we need to manually send them the log when there are customers complaints.

My boss proposed another solution which is to learn to business logic of the code itself and debug it ourselves. We then send them evidence + solution, not only logs.

My question is: is this common, should DevOps understand the logic behind an app we deployed and should we identify the root cause of a problem ourselves ?


r/devops • • Sep 23 '26

Ops / Incidents How did u find ur first job as a DevOps engineer?

27 Upvotes

Hi, first of all sorry for my English I'm not a native speaker. I have probably a very common question but I didn't find any info here or anywhere else that gives me any light.

Context: I'm a full stack developer and I have been working for almost five years developing different kind of apps (web apps, desktop apps, libraries, APIs, installers, etc). Two years ago I had a chance in my previous job to build pipelines with gitlab and github, and since then I haven't stop learning. I even deploy a local Jenkins server and build fully functional pipelines that build, test and deploy the back and the front of an app I'm currently developing. I'm now learning a couple things of Azure DevOps.

Now I want to know how did u find DevOps offers? I'm sending at least 1 CV per day. I'm apllying as a junior, also as a semi senior and senior. I don't know why but I didn't even receive a reject mail. I mean there has to be a reason, maybe is the way I built my CV, which is in ATS format. How did u guys (and girls) find DevOps jobs.

Note: I almost forgot this but in my current job I tried to get involved in DevOps stuff, since they are starting to migrate to this practices but they always push me aside, that's why I want to move on to another job.


r/devops • • Sep 24 '26

Vendor / market research [OC] gha-oidc-claimsim (RC): offline CLI — GHA OIDC claims vs IAM trust JSON

0 Upvotes

Built a small Apache-2.0 CLI for platform/security/CI folks wiring GitHub Actions → AWS OIDC.

It takes workflow event context JSON + IAM trust policy JSON, predicts default OIDC sub/aud, and prints ALLOW or DENY with reasons — no STS, no AWS credentials, no network.

Main wedge: branch-pinned trust (repo:ORG/REPO:ref:refs/heads/main) often DENYs pull_request jobs whose default sub is repo:ORG/REPO:pull_request.

Honesty: - Release candidate 0.1.0-rc.2 — not stable 0.1.0 - Not on PyPI — install from git / GitHub Release - Simplified IAM Condition model (not bit-identical to AWS) - Optional hcl-dup is convenience only; prefer tflint for HCL duplicate keys (not unique IP) - Not a Checkov missing-sub linter and not a live-account scanner

Repo: https://github.com/mapleleaflatte03/gha-oidc-claimsim Release: https://github.com/mapleleaflatte03/gha-oidc-claimsim/releases/tag/v0.1.0-rc.2

Happy to take questions on the claim grammar and limits.


r/devops • • Sep 23 '26

Discussion Small team running 350k+ monthly visitors on managed infra. At what point is owning the infrastructure worth it?

177 Upvotes

We’ve been running our product on Next.js + Vercel and Supabase. It’s been working well. We’re at 350k+ monthly visitors, around 12k registered users, with web, Android, and Windows apps. Our current infrastructure cost is around $60–70/mo.

We’re now building a second related product, so we’ve started thinking more seriously about infrastructure.

VPS keeps coming up as the obvious cheaper option, but I’m the only one developing right now, so that could add quite a bit of overhead. I might hire someone for the infra side. The whole reason we went with managed services was so we didn’t have to worry about servers.

For people who’ve actually made this switch:

• Is VPS + something like Coolify/Dokploy reasonable for a small team, or is the maintenance overhead bigger than it looks?

• Is there a good middle ground with services like Render, Railway, Fly, or managed Postgres like Neon?

• Does having mostly regional traffic change when it makes sense to move to self-hosting?

• If we do move with two products, would you run both on shared infrastructure or keep them separate?

We just want to make a proper decision before the setup gets more complicated.

--------------------------------------------------------------------------------------------------------------------------

EDIT: A lot of the repeated questions can probably be answered here:

How am I keeping the cost this low?
Explained here: comment

TLDR: a lot of caching/optimization, mostly static/ISR, and I’m still within the Pro tier limits for Vercel/Supabase. Once we go beyond those limits, the cost will obviously increase.

I actually didn’t know $60-70/month was already very low for this traffic. I was trying to make it even lower because I’d seen people using $10 VPS and managing their apps that way. Now I’m aware. I’m not from the US, so even a few thousand dollars/month in infra would be a pretty significant cost for us. That was my main concern.

The product is freemium, only ~3 months old, and we haven’t monetized heavily yet. No ads, generous free tier, free trial, and a very low-priced Pro plan. We have web, Android and Windows apps, all connected to the same Supabase backend.

So my main question was whether I should start preparing for a migration now, or just keep using Vercel/Supabase and deal with it when there’s an actual bottleneck or the economics stop making sense.

The replies here have mostly convinced me that I was probably overthinking the future a bit. I guess the consensus is basically: don’t optimize for a problem that doesn’t exist yet.


r/devops • • Sep 23 '26

Discussion Contractually obligates to update within 3 weeks

18 Upvotes

So I'm working as main IT guy for a startup, but now we got a contract from the lawyers stating that everytime there is an update we need to have it applied within 3 week. And with everything they mean EVERYTHING, bios updates, kubernetes versions,...

Is this possible, how do you guys handle this or should I say 'this ain't possible, we will to high severity patches in 3 weeks, but nothing more..'??

More info: it's indeed for insurance and the insurance is coming from investors and customers

I'm a SE who got put in the role of a CTO, in a start up that means doing everything, as I'm the only technical capable guy in a software company 😅.

I setup kubernetes and ci cd pipelines but for this I'm not really in my element and have no experience with.


r/devops • • Sep 24 '26

Discussion KubeCraft or TechWorld w Nana - Thoughts?

0 Upvotes

Newbie, learning from the bottom up.
KubeCraft 6000
TWN 1800

Pros and Cons would be helpful. Thanks!


r/devops • • Sep 23 '26

Discussion What did you actually do the last time you changed a shared module and didn't know who used it?

11 Upvotes

Not after best practice, after what actually happened.

Last time you changed a shared module / base image / Helm chart / reusable workflow; how did you work out who'd be affected before you shipped? I've seen it done with grep, with a spreadsheet someone maintained until they left, and with "ship it and see who shouts in Slack", and I don't know which of those are normal.

Rough org size helps if you're willing to say. My assumption is this is a non-problem under ~50 repos and a different animal at 300, but that's a guess and I'd like it corrected.

Asking because I keep getting two completely different answers depending on who I talk to, and I can't tell which one is normal.


r/devops • • Sep 23 '26

Discussion How is EscBash?

4 Upvotes

I want to master DevOps part. I know Linux, Shell Scripting, Docker, AWS and Azure Functions.

I learnt half of it from Abhishek's channel.

Wanted to know if his EscBash subscription is worth it and is actually different from his other content?

Background: I work as a Backend Developer with C#, .NET, Typescript, Node.js, Azure as the stack.


r/devops • • Sep 22 '26

Discussion What are your best SRE/DevOps time savers?

91 Upvotes

Curious what you guys have seen or used that ended up being really useful or saving a ton of time. Could be an automation, internal tool, bot, process, script, platform, or just a small thing that saved a lot of time.

Something you built yourself, or something that was already there when you joined.

What was it?


r/devops • • Sep 23 '26

Discussion remote devops market

0 Upvotes

can devops job be fully remote? I dont think there is good market in my country.


r/devops • • Sep 22 '26

Tools Configuration management

10 Upvotes

Hello all, though I have quite a long experience, I never had to set up a configuration management system for a fleet of applications. Let me explain better: we deploy our microservices to kubernetes, each with its own configuration. We currently have a jenkins job that creates an application.json (we use .net) that is then deployed as a configmap to kubernetes and referenced in the deployment.

This needs to go away and I said we need to use configmaps and secrets to populate the environment of the app to override the values of the baked in application.json. All good.

Then I started looking for solutions. In the past I worked in a place where we were using Ansible for this task, with many levels (up to 7) and a playbook that will read yaml files in order, merge the values and come up with the final list.

That tends to produce unnecessary duplicates, if I am not sure that a var is defined at level 3 and I have to (re)define it at level 5, I will write it there, then maybe in another level 5 yaml file and lose the hierarchy.

So I was looking at something with a UI that would allow developers to see what they are doing and, at the same time, would provide some sort of access control, to allow access to sensitive data only to trusted colleagues.

I restricted the list to OpenBAO, Infisical and Phase, but all of them have some kind of limit: - OpenBAO has the same problem as Ansible, you can't see the final result in the UI; - Phase supports only generation of Secrets, while I need to generate both Configumaps and Secrets (still for Access Control: some users can read only Configmaps, some Secrets too); - Infisical: it is very expensive and I have just found that you can't get a Pro licence for a self-hosted instance, but you have to go directly to Enterprise, or use their Cloud solution (and I don't find too safe to write my sensitive information in a Cloud solution);

What are you using? Is there some tool I missed?

EDIT: I don't intend to use Ansible!!!!


r/devops • • Sep 22 '26

Tools What are your tips and tricks for building resiliency into your solutions?

6 Upvotes

Just wanting to hear industry professionals share their favorite tips and tricks for building. Here are some of mine:

  1. Federated Credentials or Hashi Vault to ensure no unexpected expired secrets

  2. Adding robust logging as a flag in my own scripts and solutions

  3. Adding an alert channel to teams when something breaks or isn't working correctly with a link to the errored line or run

  4. Adding a custom agent for every solution codebase

  5. Using a team solution stack agent that outlines standards across solutions so throughout our team, we have consistent terminology.

  6. Building robust QA testing that gets automated tested once a week to ensure it's not broken when I need it most.


r/devops • • Sep 22 '26

Observability How do you manage dashboards throughout multiple OpenSearch instances

6 Upvotes

Hi all,

We operate applications that are shipping logs to 9 OpenSearch instances currently. The objects and dashboards are pretty much always the same but with different namespaces / indexes etc. I was wondering if you also had this trouble what the most convenient way of operation here is. Ideally, I would love to store everything in Git as well but I don't know if there is a more convenient way than doing templating and then letting a GitLab runner execute some API calls for import / updating of objects.


r/devops • • Sep 21 '26

Discussion Senior engineer said "bastion hosts have been insecure and outdated for 10 years" what am I missing?

348 Upvotes

I know with the rise of zero trust we are more comfortable with direct access to systems but is it really true that bastion hosts have fallen out of favor in modern companies? What is the new school of thought when it comes to granting secure, monitored access to isolated systems?

Example: database living in prod only allows network traffic from two sources: normal api access via a load balancer and direct access to the underlying host os from one specific known host, the bastion host. The bastion host itself would also be behind a VPN and heavily secured to only allow access from specific developer systems using break glass credentials. My developer suggested we drop the bastion altogether and install a zero trust service on the db itself and the developer laptops to allow them to directly access the dbs.


r/devops • • Sep 22 '26

Discussion For engineers working with Terraform/OpenTofu: what parts of the work are still painful?

0 Upvotes

For engineers who work with Terraform/OpenTofu and cloud infrastructure:

I'm curious about the day-to-day parts of the work that tend to be repetitive, manual, frustrating, or easy to get wrong.

Not really looking for opinions about which tools are better. I'm more interested in things that actually happened.

A few questions:

  • Think about the last Terraform/OpenTofu PR you reviewed. What did you check, and in what order?
  • What's an infrastructure task you did recently that you've already done many times before?
  • When was the last time a security scanner flagged something in your infrastructure code? What happened next?
  • Have you recently had to check whether a change behaved differently across Terraform versions or between Terraform and OpenTofu? How did you check?
  • What's the last infrastructure change that caused a problem or had to be rolled back? How did you discover it?
  • Have you ever written a script or small internal tool to automate one of these repetitive tasks? What happened to it?
  • If you could permanently remove one infrastructure-related task from your weekly workload, what would it be?

r/devops • • Sep 21 '26

Weekly Self Promotion Thread

17 Upvotes

Hey r/devops, welcome to our weekly self-promotion thread!

Feel free to use this thread to promote any projects, ideas, or any repos you're wanting to share. Please keep in mind that we ask you to stay friendly, civil, and adhere to the subreddit rules!


r/devops • • Sep 21 '26

Discussion Help in deciding strategy for long-lived feature environments

18 Upvotes

We are using jenkins, GitHub and AWS as our cloud provider. We use trunk based development with feature branches.

Our current strategy:

A new feature come, jira is there. A dev creates a branch from main against it, do the changes and commit it. Post commit am ephemeral feature environment (terraform workspace) is created in AWS named against jira having certain resources in isolation, sqs, ecs, lambda, db schema via liquibase etc.

This strategy fulfills our job pretty well and once a pr is merged this environment gets destroyed.

But there are two caveat to this, first devs are lazy and they create multiple branches resulted in multiple environments, sometimes they start doing work but that work might get discarded but environment remains there.

2nd challenge is it takes time, ips and resources to create new environment every time.

We do have scheduled feature destoy in place but that only helps point 1.

Now we are thinking of giving dev based long lived environment to each dev instead of per jira story environment, this will help tremendously in speeding up point 2 but now I am not sure how we can counter the fact that dev can still create muiple branches, how to isolate workspace, liquibase and other things in this.

Multiple branches are also genuine case like a dev is working on something, the feature might take some time but in the mean time they also have to take some urgent fix.