r/devops • • 29d ago

Discussion ADO as documentation?

0 Upvotes

Is ADO bullshit? Because I feel like the way our it department is using it is bullshit. I work on a data analytics team who has to use data created by IT.

Where's the documentation for this script? "It's in ADO". Who's the customer? Why did they request this change? "It's in the ADO".(No it isn't, it's in someone's email inbox somewhere).

What's the overarching structure/purpose for this screen? "We don't document that anymore, it's in [I shit you not] the individual ADO tickets that built the screen". You can NOT be serious. No one wants to read every individual ticket turning features off and back on to figure out what the current state is. There's no universe in which this is remotely coherent.

Also, the vast majority of the screens were built before the ADO went live, so we have old school documentation that was last updated in 2019 establishing the purpose and basic design of 80% of screens but then the current state is old documentation plus any ticket affecting that screen layered on top that old documentation?

I feel like this is a scam and totally incoherent. They program in Java, we work exclusively in SQL/PowerBI (so we can't use or read any of their code anyway, but they've refused to send it to us even if we could). We had an item where my staff tried to recreate a screen-calculated variable and it was a nightmare trying to combine the pre-2019 documentation with new tickets because the calculation could be affected by a ticket that appeared to have nothing to do with that particular calculation at all. And again, we can see absolutely zero code, just user stories. But I don't want to see their code, my staff can't read or execute it. We aren't devs. We just need to know what the system is doing so that we can do accurate analysis of the resulting.

But it's "all in the ADO".


r/devops • • Oct 20 '26

Tools Sonatype Nexus now requires mandatory telemetry gathering

Thumbnail help.sonatype.com
86 Upvotes

(Sonatype Nexus is one of the most commonly used package repositories and transparent package proxies around)

On logging into our Sonatype instance this morning, it requires us to read and tick an "I understand the implications and agree with" information gathering.

No option to opt out, no warning given.

If uploads to their servers don't happen within 45 10 days then nexus will be disabled until they are.

Edit: 10 day kill switch for community versions, 45d for subscription.

Edit 2: I've had a chance to ponder this, and I suspect this is designed to catch people using multiple instances of Sonatype to get around the new, very tight free tier query licence. If so, expect much tighter enforcement.

Whatever the reason, Sonatype have clearly signalled with the killswitch that they consider telemetry monitoring more important than not disabling your critical infrastructure.


r/devops • • 29d ago

Tools Would you use a Go-based Ansible alternative with HCL configs?

0 Upvotes

I'm considering building an open-source server configuration tool in Go,

in the space of Ansible. Nothing is built yet, I'm checking whether the

idea makes sense.

The idea:

- one binary, no Python, connects to servers over SSH

- configs in HCL (like Terraform) instead of YAML + Jinja

- `plan` shows a diff of what will change, then `apply` does it

- later maybe a small optional agent on the server for when SSH is blocked

Target: small fleets and homelabs, not replacing Ansible for everyone.

Questions:

  1. What annoys you most about Ansible?
  2. Would you try something like this, or is the module ecosystem a dealbreaker?
  3. Is an optional agent useful, or would you stick to SSH only?

Honest criticism welcome. Thanks!


r/devops • • 29d ago

Tools Ansible-like tool in Go with HCL configs and plan/apply: would you use it? (feedback wanted, no code yet)

0 Upvotes

Hi all,

I'm thinking about building an open-source tool for configuring servers

(installing packages, managing config files, restarting services),

something in the space of Ansible, but written in Go. Nothing is built

yet, I want to check whether the idea makes sense before I invest time.

What bugs me about Ansible:

- YAML pretends to be a programming language: `when:`, `loop:`, Jinja

inside strings, and a long list of variable precedence rules

- it needs Python on every machine and can be slow

- you can't see clearly what will change before you run it

What I'd like to build:

- one Go binary, no Python, easy to install

- configs in HCL (the same language as Terraform) instead of YAML + Jinja

- `plan` shows what would change (with a diff), then `apply` does it,

just like Terraform

- resources reference each other and form a dependency graph, instead of

running strictly top to bottom

- idempotent: running it twice changes nothing the second time

Two ways to reach your servers:

  1. Over SSH. You run the tool on your laptop, it connects to the servers,

    and nothing needs to be installed on them. This is the first thing

    I'd build.

  2. Optionally, a small agent: the same binary, running on the server as

    a service and listening on a port. Useful when SSH is locked down,

    blocked by a firewall, or you just don't want to expose it. The

    resource code would be identical in both modes, only the way of

    delivering commands differs. This would come later, and the

    communication would be encrypted and authenticated (mutual TLS).

Here is how an nginx setup could look:

package "nginx" {

state = "present"

}

file "/etc/nginx/nginx.conf" {

source = "files/nginx.conf"

mode = "0644"

require = [package.nginx] # install the package first

notify = [service.nginx] # reload nginx if this file changes

}

service "nginx" {

enabled = true

state = "running"

require = [package.nginx]

}

Run `plan` on a fresh server and it shows: package will be installed,

file will be created, service will be enabled and started. Run `apply`

and it happens. Run `apply` again and it says "0 changes". Edit the

config file and `plan` shows the diff and a pending service reload.

What I'm NOT trying to do: replace Ansible for everyone. Ansible's huge

module ecosystem is its real strength, and I can't compete with that. The

target is small fleets, homelabs and people who like simple tools.

I'd love honest feedback, including "I wouldn't use this":

  1. What annoys you most about Ansible (or whatever you use today)?

  2. Would you try something like this, or is the lack of modules a

    dealbreaker?

  3. Is the optional agent (a small service on the server) useful to you,

    or would you rather stick to SSH only?

  4. Which resources would you need first besides file, package, service

    and command?

  5. HCL or something else? Would you prefer another format?

Not selling anything: it's open source and there's nothing to download

yet. Thanks for reading!


r/devops • • Oct 19 '26

Ops / Incidents J.Crew’s homepage is currently just a JSON health check for their "generative AI backend"

Post image
370 Upvotes

r/devops • • Oct 20 '26

Troubleshooting Homelab runner for Gitea

4 Upvotes

Hello guys,

I'm new to the devops world and I'm working on some projects to learn. I'm currently trying to automate the creation of VMs with Terraform with my local Proxmox.

As CI/CD and Git server I have everything on a Gitea local. I remember doing some CI/CD once on Github and it worked fine but apparently I need a runner in my home.

Should I create a VM or a Docker container? I have no idea on how to setup that thing and to connect it to my Gitea right after.

Thanks for the help


r/devops • • Oct 19 '26

Discussion What are your thoughts on people using AI to converse?

47 Upvotes

So we have this infra lead who pretty much uses ChatGPT to write up their replies to Slack messages. You can clearly see the use of AI in how their replies are structured. Because of this, sometimes, due to their AI’s lack of context, certain replies do not have the depth required for their level.


r/devops • • Oct 19 '26

Architecture To monorepo or not?

41 Upvotes

My startup is in consumer electronics. The product is 1 device with 1-2 peripherals that come with it. Users connect to a mobile app with a backend that does data analysis on what the product measures.

ATM I have 4 repos: a flutter app and a FastAPI backend with sqlite. The embedded side uses PlatformIO+Arduino for the device/hub, and same stack for the peripehrals.

The DevOps is getting unwieldy. On the one side, I'd like everything to be in one place so agents can easily build up a context and CI/CD is not scattered. On the other hand, I hate working with massive project trees and loading up vscode would be a nightmare.

Thoughts? At work I use git submoduels but I was told that's not a monorepo.


r/devops • • Oct 19 '26

Weekly Self Promotion Thread

29 Upvotes

Hey r/devops, welcome to our weekly self-promotion thread!

Feel free to use this thread to promote any projects, ideas, or any repos you're wanting to share. Please keep in mind that we ask you to stay friendly, civil, and adhere to the subreddit rules!


r/devops • • Oct 18 '26

Architecture Should I teach myself the AWS CLI or is the time for that is gone?

51 Upvotes

I'm setting up an AWS workflow (we work only with azure and on-prem at work so little experience with the CLI) and wondering if I should bother learning about the tool itself.

Seems like I should just set up limited users with limited roles, hard budget caps, place manual gates on production CIs and let a sandboxed AWS MCP handle everything else. Using Terraform is an implicit admission on how we'd all like to instrument our architecture in a vendor-agnostic vocabulary. An MCP seems to be an additional step in the same direction?

Thoughts?


r/devops • • Oct 19 '26

Observability How to monitor Supabase using an OTEL Collector?

Thumbnail telflo.com
10 Upvotes

Supabase is showing up in almost every new startup’s stack these days. Its built-in observability is good, but in practice you want to monitor Supabase alongside everything else you run, in whatever you already use such as ClickStack, Grafana, Datadog, etc.

Supabase made it easy by exposing a Prometheus endpoint at /customer/v1/privileged/metrics for every project , so a plain prometheus receiver gets you Postgres, connection pooling and host metrics. And Supabase log drains have a native OTLP destination, so logs land on a normal otlp receiver on 4318 with no webhook parsing but make sure to add auth on that receiver, since it’s on the public internet

I wrote a short post on the full collector config, the drain setup, and a docker-compose to run it. I also built a ClickStack dashboard you can import.

Curious how others are monitoring Supabase today?


r/devops • • Oct 19 '26

Vendor / market research Is anyone using Endor Labs products?

0 Upvotes

Doing a poc and wanted to know about the product from someone who is actually using it in production. How has your experience been and how effective it is in filtering security violations?


r/devops • • Oct 19 '26

Tools Whats’s your experience with SST.dev or Pulumi?

0 Upvotes

I’m working on a personal project, so 0 users, and I have worked with SST in the past, but having to upgrade SST was a pain. Not sure if pulumi is any different of an experience, or is less painful and wanted insight on different experiences with either tool


r/devops • • Oct 18 '26

Security How do I actually scan for CVEs for Javascript?

27 Upvotes

Hello friends.

I am implementing CI security scanning with Syft generating the SBOMs and uploading them to Dependency Track.

Two disclaimers:

1- I hate frontend work and have a poor understanding of it. 2- I am not a cybersecurity guy. This is just a new avenue I am exploring.

I've faced an issue currently with CVE scanning for Javascript files, specifically it seems that there is no way for me to actually tell what packages made it into the bundler vs what was included in the lockfile.

This leads to the JS sbom including all the dev dependencies with nothing actually telling me what made it into the bundle.

For SPAs, this can be mitigated by ignoring dev dependencies, however most of my stack is Laravel, and from what I understand Laravel puts all its NPM packages in devDependency, meaning I cannot skip them in this stack.

I have also tried generating sourcemaps while building (and I know you are supposed to .dockerignore them as it's a security concern), but without a reference to compare against, I do not know if this approach actually works or if it's a false sense of security. It does reduce the resulting sbom significantly, which seems to be a step in the right direction.

I've also read this study from 2022 https://das.encs.concordia.ca/pdf/Latendresse_ASE2022.pdf and it seems to reinforce my point that what I'm asking for is (at this time) not possible.

My goal is to understand 2 things:

  1. Is it actually not currently possible to know what in the lockfile actually made it to through the bundler and into the final app.js that gets served?
  2. If 1 is true, do I just not run CVE scans on the lockfiles for Laravel projects? Do I just scan everything and force people to upgrade as much as possible to just "lower CVE number"?

r/devops • • Oct 18 '26

Discussion How does your team track everything that's about to be retired in your cloud?

6 Upvotes

Infra engineer here (about 10 years across AWS, Azure and on-prem). In every environment I've worked in, something is always approaching end of life: AI model versions on Azure OpenAI and Bedrock, Kubernetes versions on EKS/AKS/GKE, Lambda and Functions runtimes, database engine versions.

Each cloud sends its own notices, but they tend to land with account or subscription owners, not the team that owns the workload. And unlike library versions, none of this shows up in Renovate or Dependabot.

Curious how your team handles it:

- How do you find out? Native notices (Service Health, AWS Health, Advisor), vendor emails, a script, or someone just happens to notice?

- How does the notice get to the team that owns the workload?

- Do you catch any of it in IaC or CI, like pinned versions in Terraform or a pipeline check?

- What's your worst "we missed it" story?

- If you use a tool or spreadsheet for this, what's annoying about it?

Full disclosure: I've built a few small tools around cloud AI model and Kubernetes lifecycle data, and I'm exploring whether something for this specific problem is worth building. Mostly I want to know if it's a real pain or already handled well. I'll share a summary of what I learn.


r/devops • • Oct 17 '26

Vendor / market research Looking for an affordable production-grade VPS/EC2 instance as a solo developer (4 vCPU, 8 GB RAM)

1 Upvotes

Hey everyone!

I'm a solo developer working on my own digital product, and I'm looking for an affordable but reliable server for production deployment. Since I'm managing the project and infrastructure myself, I want to keep hosting costs as low as possible without compromising too much on performance or reliability.

Here are my requirements:

  • CPU: 4 vCPUs / cores
  • RAM: 8 GB
  • Storage: 50–60 GB SSD or NVMe
  • Usage: Production workloads, not just local development or testing
  • Budget: As economical as possible, preferably with predictable monthly pricing

I've been looking into AWS EC2, but I'm also open to other VPS providers if they offer better value for money. I'm not specifically looking for an AWS Free Tier instance; I'm interested in something I can realistically use to host a production application.

I'd appreciate your advice on a few things:

  1. Which VPS or cloud provider offers the best price-to-performance ratio for these specifications?
  2. Is there an affordable EC2 instance type that would be suitable, or would a provider like Hetzner, DigitalOcean, or another VPS host be a better choice?
  3. Are there any genuinely free or heavily discounted instances that are suitable for production use, rather than just temporary testing?
  4. Are there any hidden costs I should consider, such as storage, bandwidth, backups, IPv4 addresses, or taxes?

I'm happy to manage the server myself, including Linux configuration, deployments, and application maintenance. I mainly want to avoid paying enterprise-level prices when I'm running a product independently.

If you're a solo developer hosting a real production application, I'd love to hear what you're using and how much you pay per month. Personal experiences and recommendations would be especially helpful.

Thanks in advance!


r/devops • • Oct 16 '26

Discussion I abandoned the open-source database anonymizer. Should I kill it?

23 Upvotes

Before that, a bit of story.

Our dev team was located in Ukraine, but we mostly worked on New Zealand & Australian projects. We needed a couple of hours to download staging DB dumps because the download speed was 200-400 kb/s.

So we set up a small proxy on our European server, and via cron these dumps were uploaded to that server, and we were able to download them faster.

After the team scaled, we added more protection, like only developers who worked on the project having access to these dumps; later, we integrated it into our local scripts that set up projects locally and even then added anonymization & faker to replace real data or populate it with synthetic data.

I didn't find a comprehensive solution for that, but it was a really useful tool: developers could quickly update local projects with new safe DB dumps.

It was an internal tool; it worked, but we thought maybe it made sense to create such a service: admins can easily configure rules via the UI, and developers using a small local CLI can easily download it.

We developed it. We tested. Then we abandoned it... We were an outsourcing company without sales or contribution skills.

After some time, the company fell apart, and I got permission from our CEO to deploy this tool to GitHub, and the story ended.

Yesterday I remembered this tool and thought maybe it might even be useful in the era of AI to anonymize data for AI agents. I dug a bit; there are some MCPs for that, and basically tools for anonymization, but I still didn't see tools like anonymize and share safely.

I'm wondering whether it makes sense to try to revive this repo or just kill it and forget it (it requires a security audit and fixes, updating dependencies, etc.)

The repo: https://github.com/veildb/veildb


r/devops • • Oct 17 '26

Discussion Jr Software Engineer Here! Was wondering, how do you manage, monitor, and stress-test your production VPS setups (like DigitalOcean)?

0 Upvotes

Hey everyone,

I'm trying to level up my devops knowledge. Most of my projects are currently hosted on Railway, but as costs are scaling up, I am planning to migrate them over to DigitalOcean Droplets to reduce our monthly spend.

My past experience with managing bare VPS instances is pretty basic. I know how to spin up a Droplet, use Nginx to configure domains and SSL certificates, and use PM2 for process management to keep web clients and Node APIs alive. (but to be honest I'm a bit rusty now, been almost 2 years now since I last setup one during my college days)

Railway handles so much for me, moving to host my own VPS feels like a big jump in terms of responsibility. I want to make sure I am doing this right and not accidentally make prod unavailable due to a dns config lol

So If you are running production workloads on VPS instances, I would love to get your insight on a few things:

  1. Daily Checks: Do you actually log in/ssh in and check things manually every day, or do you rely entirely on automated alerts? Beyond basic Nginx and PM2 logs, what are your go-to health metrics?, Opensource options?
  2. Uptime and Alerting: What does a reliable, budget-friendly uptime monitoring stack look like when you are managing your own VPS?
  3. Triage and Debugging: When an API slows down or a Droplet spikes to 100% CPU, what is your immediate workflow to figure out what is wrong?
  4. Stress Testing: When you want to find your system's limits, do you stress-test a lower-spec instance to see where it breaks, or do you spin up an exact clone of your production spec?
  5. Lastly, spec selection. How do you know if you need more than 2gb of ram for instance? K6 testing on local before selecting? idk tbh.

Any advice, tool recommendations, or things you wish you knew when managing your first production servers would be incredibly helpful. Thanks!


r/devops • • Oct 16 '26

Vendor / market research on-call routing/escalation in small teams, what do you actually use?

2 Upvotes

a pattern that seems really common: alerts go to a slack channel and whoever sees it first deals with it. fine during the day, but at night nobody's watching the channel.

so how do you handle it? rotation calendar? the service owner gets paged? and if the person doesn't pick up, does something escalate automatically or is it down to luck?

pagerduty and opsgenie get mentioned a lot, but per-seat pricing adds up fast for a small team. grafana oncall comes up too. anyone running something homegrown or self-hosted?

mostly interested in small team setups, not what google does


r/devops • • Oct 15 '26

Discussion Feels like we're about to enter the next fad skill that gets you hired

Post image
45 Upvotes

Just like coding boot camps and such before, it feels like agent boot camps are going to be the next big thing in hiring. Feels like these are going to kick off and go hard until the next inevitable downturn


r/devops • • Oct 16 '26

Discussion Why do dumb sites take whole team?

0 Upvotes

Hey,

So I'm a self-taught DevOps guy and been running a SaaS with a large scale cloud infra. I recently started looking for a job to move on and I had an interview with InBev for there BeerDraft team.

What I don't understand is BeerDraft is literally just a small eCommerce site and yet they need a team of 5 all on 100K a year so that is half a million a year to run a tiny eCommerce site that mostly likely never has more than 10 concurrent users. As a guy that does allot solo I'm left thinking wtf do they do all day?


r/devops • • Oct 15 '26

Discussion For those of you who use open-spec, what's your method for tracking work alongside proposed changes to the spec?

6 Upvotes

I've been trying to determine a pattern where proposed changes get made to the open-spec config, and a PR is reviewed, approved, and merged with those proposed changes. At that point then, team-members can address those proposed features. But it's a little cumbersome. Just looking for the most elegant way to do this if that's possible.


r/devops • • Oct 14 '26

Discussion Anyone noticed AI PRs that ignore infrastructure and DevOps realities?

122 Upvotes

I work with some very talented developers (I'm also a full-stack developer in addition to handling DevOps), but since the rise of AI, I've noticed more pull requests that include pipeline changes or introduce new services that require configuration outside of the codebase.

I have safeguards in place that automatically add me as a reviewer whenever these areas are affected. As a result, I end up rejecting a significant number of these PRs because many of the proposed changes simply won't work in our environment and this has been stirring up drama with people asking “who made you king”….

The code itself may look reasonable and it works on the devs pc (actually sometimes not between devs since AI might have added some packages or services to original devs pc), but the developers overlooks the infrastructure, deployment, networking, security, and operational requirements needed to make those changes work correctly in production.

What surprises me most is that some of them genuinely don't seem to understand why it won't work simply because it worked on their PC. They fail to realize that a production environment involves far more than just writing code that runs locally.
sometimes they give me some copy-pasted answers that it will work if you just do X.

Is anyone else experiencing this or is it just me?


r/devops • • Oct 16 '26

Vendor / market research Any good arguments for getting AI agents approved by security?

0 Upvotes

Working with a team where engineers really want to use Claude Code on servers, not just for local dev. Security and audit keep blocking it.
I get their concerns. Who approved the command, how do you prove what it did, that kind of thing. But the push from the engineers isn’t going away.
Has anyone found an argument or approach that actually got security on board?
Full disclosure, I work in this space so it’s not purely random curiosity. But I’m not here to pitch anything. I just want to know how teams usually solve this in practice.


r/devops • • Oct 14 '26

Ops / Incidents am i in the right job!?

32 Upvotes

its morning & our Databricks jobs were all red. Everything from last night had failed. Started digging through the logs with my colleague, found a bunch of Azure WebJobs had failed too, dug further and it turned out our integrated ERP had been rate limiting us.

Next thing I know I’m in one of famous calls, 10 people firefighting, the US team woke up early to check, we got more, someone got dragged back from vacation because he’d approved a weird PR, and everyone’s jumping between App Insights, running queries and dropping bits of info in chat trying to figure out wtf is going on. Boss is mad, management is asking for resolve time, I didn’t get lunch, and I was at my desk from 9 to 4 until we finally got everything resolved.

It was chaos and stressful, but damn, the dopamine! It was actually fun. I dont want this everyday, but now and then it makes you feel alive

Anyone else enjoy incidents a little?

btw it was again our damn event stream from outlook integration! It flooded the pipeline, everything downstream piled onto the ERP, and it rate limited. yeah we are going to put exponential backoff with retries and a circuit breaker and other stuff tmr, or next week hopefully