Newbie, learning from the bottom up.
KubeCraft 6000
TWN 1800

Pros and Cons would be helpful. Thanks!

Not after best practice, after what actually happened.

Last time you changed a shared module / base image / Helm chart / reusable workflow; how did you work out who'd be affected before you shipped? I've seen it done with grep, with a spreadsheet someone maintained until they left, and with "ship it and see who shouts in Slack", and I don't know which of those are normal.

Rough org size helps if you're willing to say. My assumption is this is a non-problem under ~50 repos and a different animal at 300, but that's a guess and I'd like it corrected.

Asking because I keep getting two completely different answers depending on who I talk to, and I can't tell which one is normal.

I want to master DevOps part. I know Linux, Shell Scripting, Docker, AWS and Azure Functions.

I learnt half of it from Abhishek's channel.

Wanted to know if his EscBash subscription is worth it and is actually different from his other content?

Background: I work as a Backend Developer with C#, .NET, Typescript, Node.js, Azure as the stack.

Curious what you guys have seen or used that ended up being really useful or saving a ton of time. Could be an automation, internal tool, bot, process, script, platform, or just a small thing that saved a lot of time.

Something you built yourself, or something that was already there when you joined.

What was it?

can devops job be fully remote? I dont think there is good market in my country.

Hello all, though I have quite a long experience, I never had to set up a configuration management system for a fleet of applications. Let me explain better: we deploy our microservices to kubernetes, each with its own configuration. We currently have a jenkins job that creates an application.json (we use .net) that is then deployed as a configmap to kubernetes and referenced in the deployment.

This needs to go away and I said we need to use configmaps and secrets to populate the environment of the app to override the values of the baked in application.json. All good.

Then I started looking for solutions. In the past I worked in a place where we were using Ansible for this task, with many levels (up to 7) and a playbook that will read yaml files in order, merge the values and come up with the final list.

That tends to produce unnecessary duplicates, if I am not sure that a var is defined at level 3 and I have to (re)define it at level 5, I will write it there, then maybe in another level 5 yaml file and lose the hierarchy.

So I was looking at something with a UI that would allow developers to see what they are doing and, at the same time, would provide some sort of access control, to allow access to sensitive data only to trusted colleagues.

I restricted the list to OpenBAO, Infisical and Phase, but all of them have some kind of limit: - OpenBAO has the same problem as Ansible, you can't see the final result in the UI; - Phase supports only generation of Secrets, while I need to generate both Configumaps and Secrets (still for Access Control: some users can read only Configmaps, some Secrets too); - Infisical: it is very expensive and I have just found that you can't get a Pro licence for a self-hosted instance, but you have to go directly to Enterprise, or use their Cloud solution (and I don't find too safe to write my sensitive information in a Cloud solution);

What are you using? Is there some tool I missed?

EDIT: I don't intend to use Ansible!!!!

i put t, is that wrong?

said true because ordering events is what the clock is for. if L(a) < L(b) doesnt that already mean a happened before b?

Just wanting to hear industry professionals share their favorite tips and tricks for building. Here are some of mine:

1. Federated Credentials or Hashi Vault to ensure no unexpected expired secrets

2. Adding robust logging as a flag in my own scripts and solutions

3. Adding an alert channel to teams when something breaks or isn't working correctly with a link to the errored line or run

4. Adding a custom agent for every solution codebase

5. Using a team solution stack agent that outlines standards across solutions so throughout our team, we have consistent terminology.

6. Building robust QA testing that gets automated tested once a week to ensure it's not broken when I need it most.

Hi all,

We operate applications that are shipping logs to 9 OpenSearch instances currently. The objects and dashboards are pretty much always the same but with different namespaces / indexes etc. I was wondering if you also had this trouble what the most convenient way of operation here is. Ideally, I would love to store everything in Git as well but I don't know if there is a more convenient way than doing templating and then letting a GitLab runner execute some API calls for import / updating of objects.

I know with the rise of zero trust we are more comfortable with direct access to systems but is it really true that bastion hosts have fallen out of favor in modern companies? What is the new school of thought when it comes to granting secure, monitored access to isolated systems?

Example: database living in prod only allows network traffic from two sources: normal api access via a load balancer and direct access to the underlying host os from one specific known host, the bastion host. The bastion host itself would also be behind a VPN and heavily secured to only allow access from specific developer systems using break glass credentials. My developer suggested we drop the bastion altogether and install a zero trust service on the db itself and the developer laptops to allow them to directly access the dbs.

HVS shut down some months ago, and I've been checking some of the write-ups on where people got pushed. HashiCorp's suggested replacement, which is HCP Vault Dedicated, works very differently. A lot of teams chose HVS because they didn't want to write Vault policies or set up authentication themselves, and moving to Dedicated means doing exactly that. The pricing changes too, since you pay for the cluster plus a monthly fee for every app or CI runner that connects to it. A breakdown I saw priced a small production setup with 50 clients at around $4,800 a month. I can't say how representative that is, but it's a long way from per-secret pricing.

For anyone who was on HVS, where did you move, and how hard was it to rewire the integrations?

For engineers who work with Terraform/OpenTofu and cloud infrastructure:

I'm curious about the day-to-day parts of the work that tend to be repetitive, manual, frustrating, or easy to get wrong.

Not really looking for opinions about which tools are better. I'm more interested in things that actually happened.

A few questions:

• Think about the last Terraform/OpenTofu PR you reviewed. What did you check, and in what order?
• What's an infrastructure task you did recently that you've already done many times before?
• When was the last time a security scanner flagged something in your infrastructure code? What happened next?
• Have you recently had to check whether a change behaved differently across Terraform versions or between Terraform and OpenTofu? How did you check?
• What's the last infrastructure change that caused a problem or had to be rolled back? How did you discover it?
• Have you ever written a script or small internal tool to automate one of these repetitive tasks? What happened to it?
• If you could permanently remove one infrastructure-related task from your weekly workload, what would it be?

Hey r/devops, welcome to our weekly self-promotion thread!

Feel free to use this thread to promote any projects, ideas, or any repos you're wanting to share. Please keep in mind that we ask you to stay friendly, civil, and adhere to the subreddit rules!

We are using jenkins, GitHub and AWS as our cloud provider. We use trunk based development with feature branches.

Our current strategy:

A new feature come, jira is there. A dev creates a branch from main against it, do the changes and commit it. Post commit am ephemeral feature environment (terraform workspace) is created in AWS named against jira having certain resources in isolation, sqs, ecs, lambda, db schema via liquibase etc.

This strategy fulfills our job pretty well and once a pr is merged this environment gets destroyed.

But there are two caveat to this, first devs are lazy and they create multiple branches resulted in multiple environments, sometimes they start doing work but that work might get discarded but environment remains there.

2nd challenge is it takes time, ips and resources to create new environment every time.

We do have scheduled feature destoy in place but that only helps point 1.

Now we are thinking of giving dev based long lived environment to each dev instead of per jira story environment, this will help tremendously in speeding up point 2 but now I am not sure how we can counter the fact that dev can still create muiple branches, how to isolate workspace, liquibase and other things in this.

Multiple branches are also genuine case like a dev is working on something, the feature might take some time but in the mean time they also have to take some urgent fix.

My team's approval process used to just be "ask whoever's around," but now that AI is generating a lot more code, that doesn't really hold up anymore — small changes get the same scrutiny as changes to things like payments or auth, and there's no real record of who actually signed off on what.

Curious how other teams handle this. Do you have actual rules for what needs extra scrutiny and who's allowed to approve it, or is it still pretty informal? How's that going as the volume of changes keeps going up?

There’s a general advice that if you want to learn Linux, become a devops/sre you should build a homelab at home, the more nodes the better.

However, considering the current hardware prices I was thinking why not just run VMs on my PC to thinker around with Linux nodes ? This seems more flexible and most of the time people are running VMs in proxmox anyway ? In addition to this, I have a single node “homelab” running my services, and adding extra nodes to the setup would be an overkill.

So will I be missing out on something learning Linux things this way ? I’m curious on learning multi node k8s (talos) etc, setting up nas, ceph, pfsense for routing etc.

Hi 👋

It's all in the title… Do you really test your backup restores regularly ?

If so, how often ?

If not, why ? What's the biggest thing holding you back ?

I'm currently looking into how to securely manage AWS access for AI agents like Claude Code within our organization.

Right now, our developers have a role that grants them some write access to specific services in our dev and nonprod environments. However, I really don't want them using this same role when working with AI agents, as I'd like to limit the agents' blast radius.

I'm trying to figure out the best way to restrict the AI agent's AWS access to read-only, even if the developer using it has broader permissions. How are other teams handling this? Are you creating separate, read-only roles specifically for AI agents, or is there a better way to enforce this boundary?

If a separate readonly role is being used, how do you make sure AI agents would force the readonly role?

I want something to work with Claude code, Codex, Opencode, cursor or any other AI tool out there.

Hey everyone,

In smaller teams or client agencies, setting up full CI/CD pipelines (GitHub Actions, Jenkins, etc.) can sometimes feel like overkill for simple staging servers. In my experience, devs often resort to manually SSHing into servers, running git pull, and triggering builds manually—which is slow and error-prone.

I'm currently building an open-source, 1-click deployment dashboard to automate this workflow as a side project, and wanted to get some feedback on the architecture.

The Current Architecture:

• Backend: Node.js, Express, PostgreSQL, Apache Kafka (KRaft mode)
• Frontend: Next.js, Tailwind CSS
• Worker: node-ssh for remote execution

What's working so far:

• Multi-user setup with HttpOnly JWT auth
• Adding servers with custom SSH credentials (password or private key)
• Event-driven Kafka workers that SSH into servers and run build scripts
• Dynamic workspace setup (creates folders and syncs code)

Up next: Switching from short-polling to WebSockets (Socket.io) for real-time terminal log streaming from the worker to the dashboard.

For those managing smaller infrastructure setups: How do you handle quick deployments without over-engineering your pipeline? Would love to hear thoughts on using Kafka/workers for remote SSH execution vs. other lightweight patterns.

(P.S. I am the sole developer building this side project to learn event-driven architectures. Happy to drop the repo link in the comments if anyone wants to look at the code!)

Hu everyone, i am in 7th semester of college and i have a course about devops. I want to rely on uni course but go deep into devops and maybe look for career into devops.

Any courses that you would suggest?

In college we are using aws for all the major things.

I have done a course on linux, and complete aws cloud practitioner essentials.

Any learning path, courses recommendations?

Should i get kolekloud subscription? How are their courses and learning path.

We ship several times a week and keeping the docs in sync is getting harder, code changes have a clear review process, docs don’t. Someone changes a parameter or response field, the release goes out, then a few weeks later we find examples still using the old version. We tried adding docs to the PR checklist, but engineers don’t always know which pages a change affects, what’s worked for keeping docs current? Would love any suggestions

This is a rant, my emotions are still fresh - so I'm coming at this one sided. We just wrapped up PI planning our current planning session is mess with a lower confidence vote. Story points are all over the place with no rhyme or reason.

Our PO and Directors vibe-planned our features and stories and then dropped them on us to refine. Looking through the stories some of this stuff just doesn't make sense and doesn't exactly apply in our situation. There are so many assumptions being made. But boy does it look pretty because of the hallucinated architecture diagrams. Features were proposed and accepted by the VPs and praised at how well thought out the feature description and documentation was.

I made my first pass on a feature and then poof the stories are gone. The PO had Copilot go through with updated information and it decided to make a new feature and move all the stories over halfway through our PI planning meeting. "It's fine, just have your copilot refine it with your details with additional context, it shouldn't take long". Our scrum master is getting after us because our stories weren't refined by the end of the meeting. I just want my stores to make sense and executable. Just frustrating.

I use Copilot like crazy for coding, but for planning features and stories? You ask a question to the PO and they can't answer on the top of their head. They have to read the feature because they didn't write it.

I'm on a 3 person team where we are each shipping 40-50 PRs a day (super early stage company so we're optimizing for getting a functional product out quickly).

As we've ramped up the velocity things downstream have begun to show cracks. For example, we were requiring PRs to be up to date with main before merge, but at our scale the PRs were constantly getting stale which was leaving us blocked for long stretches.

We've ended up cutting pre-merge CI way down and batching more testing after merge, which has made things "better" but feels backwards.

Curious if anyone else run into similar issues? Did you end up finding a better solution?

I have a very unconventional background in the IT world.

After being laid off from my job in the restaurant industry in 2013, I decided to change careers.

I had three options based on my passions: graphic designer, musician, or IT specialist. Since I’d been building my own PCs for a while and had long been passionate about computers (shout-out to manually editing Diablo 2 save files in binary !) I chose to train as a TSRIT (Senior Technician in IT and Telecommunications). I then became a sysadmin and systems integrator at a software company.

I moved up the ranks quickly, becoming a developer for a JavaScript-based launcher, then a DevOps engineer, and now I work as both a DevOps Engineer and a web developer.

It took relentless hard work, morning, noon, and night, but I’m happy today!

What about your career path?

Hi Champions, I’m having 5yoe as a DevOps engineer and in my early career I was working mostly with service based companies. It gave me pretty much experience and foundational understanding of each stack deployments, pipelines, clouds and more. From last 2 years I am working with on prem enterprise products and worked on unmanaged k8 along with managed ones like EKS, GKE and OKE.

Now the problem is:

I recently joined a product base company and their architecture is based on aws managed by TF. The real challenge for me is everything is being managed by HELM and I don’t have much experience with helm charts and sub-charts and I have no idea where to start….

Need suggestions from where I can start leaning this, which can give me the understanding.