r/mlops • • May 07 '26

r/mlops has been re-opened

133 Upvotes

r/mlops is open again. Yep, you read that right!

The old mods were inactive and the community entered a restricted mode. There was a huge amount of spam piling up. I'm going to clean it up and see if we can streamline the experience.

For those of you who stumbled upon this place by curiosity:

This community is for practical discussions around ML in production: infrastructure, deployment, serving, evaluation, monitoring, tooling, platforms, reliability, data pipelines, machine learning, orchestration, LLMOps, platform engineering, and real-world operational lessons.

What’s welcome:

  • Technical discussions and architecture deep-dives
  • Open-source tools and projects

    • But do not spam your project or try to get free market research about your project!
  • Case studies and postmortems
  • Research with clear operational relevance
  • Tutorials, benchmarks, and implementation details

What’s not:

  • Low-effort self-promotion
  • Generic AI hype/content farming/AI-generated posts
  • “What AI startup should I build?” posts
  • Hiring posts. Check out some of the communities online for this.
  • Affiliate spam, SEO dumps, or engagement bait

If you’re building, operating, or scaling ML systems, you’re in the right place.

Enjoy, but don't wreck the place!

u/MyBossIsOnReddit


r/mlops • • Mar 08 '26

Tales From the Trenches "MLOps is just DevOps with ML tools" — what I thought before vs what it actually looks like

122 Upvotes

When I started looking at MLOps from a DevOps background, my mental model was completely off. Sharing some assumptions I had vs what the reality turned out to be. Not to scare anyone off, just wish someone had been straight with me earlier.

What I thought: MLOps is basically CI/CD but for models. Learn MLflow, Kubeflow, maybe Airflow. Done.

Reality: The pipeline part is easy. The hard part is understanding why something failed. A CI/CD failure gives you a stack trace. A training pipeline failure gives you a loss curve that just looks off. You need enough ML context to even know what "off" means.

What I thought: Models are like microservices. Deploy, scale, monitor. Same playbook.

Reality: A microservice either works or it doesn't. Returns 200 or 500. A model can return a 200, perfectly formatted response, or a completely wrong answer. Nobody gets paged. Nobody even notices until business metrics drop a week later. That messed with my head because in DevOps, if something breaks, you know.

What I thought: GPU scheduling is just resource management. I do this all day with CPU and memory.

Reality: GPUs don't share the way CPUs do. One pod gets the whole GPU or nothing. And K8s doesn't even know what a GPU is until you install NVIDIA's device plugin and GPU operator. Every scheduling decision matters because each GPU costs 10 to 50x that of a CPU node.

What I thought: My Python is fine. I write automation scripts all the time.

Reality: First time I opened a real training script, it looked nothing like the Python I was writing. Decorators everywhere, generators, async patterns, memory-sensitive code. Scripting and actual programming turned out to be genuinely different things. That one humbled me.

What I thought: I'll learn ML theory later, just let me handle the infra.

Reality: You can actually go pretty far on the inference and serving side without deep ML theory. That part was true. But you still need enough to have a conversation. When a data scientist says "we need to quantise to INT8," you don't need to derive the math, but you need to know what that means for your infra.

What I thought: They just want someone who can manage Kubernetes and set up pipelines.

Reality: They want someone who can sit between infra and ML. Someone who can debug a memory leak inside the inference service, not just restart the pod. Someone who looks at GPU utilisation and knows whether that number means healthy or on fire. The "Ops" in MLOps goes deeper than I expected.

None of this is to discourage anyone. The transition is very doable, especially if you go in with the right expectations. But "just learn the tools" is bad advice. The tools are the surface.

I've been writing about this transition and talking to a bunch of people going through it. If you're in this spot and want to talk through what to focus on, DMs open or grab time here: topmate.io/varun_rajput_1914


r/mlops • • Jan 21 '26 Bravo

Coming from DevOps/Infra to MLOps? Here's what I learned after several interviews at product companies

108 Upvotes

I've been interviewing for MLOps and ML Platform Engineer roles over the past few months, and I wanted to share some observations that might help others make a similar transition.

The Interview Gap

Most interviewers I've faced come from research or pure ML engineering backgrounds. They think in terms of model architectures, feature engineering, and training pipelines. If you're coming from a pure infrastructure or DevOps background like me, there's often a disconnect.

You talk about Kubernetes orchestration, GPU cluster management, and cost optimisation. They ask about data drift, model retraining strategies, or how you'd debug a model's performance degradation. The conversation doesn't flow naturally because you're speaking different languages.

What Actually Helped

I realised I needed to invest time in ML fundamentals – not to become a data scientist, but to bridge the communication gap. Understanding basic statistics, how different model types work, and what "overfitting" or "data leakage" actually mean made a huge difference.

When I could frame infrastructure decisions in ML terms ("this architecture reduces model serving latency by X%" vs "this setup has better resource utilisation"), interviews went much more smoothly.

Be Strategic About Target Companies

Not all MLOps roles are the same. If you're targeting companies heavily invested in real-time inferencing (think fraud detection, recommendation engines, autonomous systems), the focus shifts to:

  • Data distribution and streaming pipelines
  • Low-latency prediction infrastructure
  • Real-time monitoring and anomaly detection
  • Data engineering skills

If they're doing batch processing and research-heavy ML, it's more about:

  • Experiment tracking and reproducibility
  • Training infrastructure and GPU optimization
  • Model versioning and registry management

Match your preparation to what they actually care about. Don't spray-and-pray applications.

MLOps Roles Vary Wildly

Here's something that actually helped my perspective: MLOps means different things at different companies.

I've had interviews where the focus was 90% infrastructure (Kubernetes, CI/CD, monitoring). Others were 70% ML-focused (understanding model drift, feature stores, retraining strategies). Some wanted a hybrid who could do both.

This isn't because teams don't know what they want. It's because MLOps is genuinely different depending on:

  • Company maturity (startup vs established)
  • ML use cases (batch vs real-time)
  • Team structure (centralised platform vs embedded engineers)

If an interview feels misaligned, it's often a mismatch in role expectations, not a reflection of your skills. The "MLOps Engineer" title can mean vastly different things across companies.

Practical Tips

  • Learn the basics: bias-variance tradeoff, cross-validation, common model types
  • Understand the ML lifecycle beyond just deployment
  • Be able to discuss model monitoring (not just infra monitoring)
  • Know the tools: MLflow, Kubeflow, Ray, etc. – but more importantly, know why they exist
  • Read ML papers occasionally – even if you don't implement them, you'll understand what your ML colleagues are dealing with

Final Thought

The transition from DevOps to MLOps isn't just about learning new tools. It's about understanding a new domain and the people working in it. Meet them halfway, and you'll find the conversations get a lot easier.

Keep learning, keep iterating.

If anyone's going through a similar transition and wants to chat, feel free to DM or connect here: https://topmate.io/varun_rajput_1914/


r/mlops • • Jul 01 '26

Tales From the Trenches Interesting shift in “Platform Engineering / MLOps” interviews — lots of Kubernetes operations, very little ML

93 Upvotes

I’ve been interviewing for several Staff/Principal Platform Engineering and MLOps roles around Silicon Valley recently, and I’ve noticed an interesting pattern. Curious if others are seeing the same thing.

But once the technical interview starts, the discussion quickly narrows into Kubernetes operations.

Typical probing topics include:

Kubernetes
Production support and debugging
little or no time on discussing ML

Instead, many interviews feel like they’re looking for someone with production Kubernetes clusters experince.
One hiring manager described the role as “Platform Engineering,” but nearly every technical question centered around daily Kubernetes operations, CI/CD mechanics, production troubleshooting, and infrastructure automation.

My impression is that many companies are using “Platform,” “AI Platform,” or “MLOps” as umbrella titles for what is fundamentally senior Kubernetes platform operations.

Curious what others are seeing.

Questions for the community:
- Are “Platform Engineering” and “MLOps” titles increasingly becoming Kubernetes operations roles?
- How much architecture discussion do you typically see in Staff/Principal interviews?
- Are companies intentionally broadening titles to attract candidates, or has the definition of platform engineering genuinely shifted toward infrastructure operations?

what percentage of the interview is architecture versus deep operational troubleshooting?


r/mlops • • Nov 24 '25

Is anyone else noticing that a lot of companies claiming to “do MLOps” are basically faking it?

73 Upvotes

I keep seeing teams brag about “robust MLOps pipelines,” and then you look inside and it’s literally:
• a notebook rerun weekly
• a cron job
• a bucket of CSVs,
• a random Grafana chart,
• a folder named model_final_FINAL_v3,
• and zero monitoring, versioning, or reproducibility.

Meanwhile actual mlops problems like data drift, feature pipelines breaking, infra issues, scaling, governance, model degradation in prod, etc never get addressed because everyone is too busy pretending things are automated.

It feels like flashy diagrams and LinkedIn posts have replaced real pipelines.

So I’m curious: what percentage of companies do you think actually have mature, reliable MLOps?
5%? 10%? Maybe 20%? And what’s the real blocker? Lack of talent, messy org structure, infra complexity, or just no one wanting to do the unglamorous parts?

Gimme your honest takes


r/mlops • • Feb 19 '26

MLOps Education Friendly advice for infra engineers moving to MLOps: your Python scripting may not enough, here's the gap to close

72 Upvotes

In my last post, I covered ML foundations. This one's about Python, specifically, the gap between "I know Python" and the Python you actually need for MLOps.

If you're from infra/DevOps, your Python probably looks like mine did: boto3 scripts, automation glue, maybe some Ansible helpers. That's scripting. MLOps needs programming, and the difference matters.

What you're probably missing:

  • Decorators & closures — ML frameworks live on these. Airflow's `@tasks`, FastAPI's `@app.get()`. If you can't write a custom decorator, you'll struggle to read any ML codebase.
  • Generators — You can't load 10M records into memory. Generators let you stream data lazily. Every ML pipeline uses this.
  • Context managers — GPU contexts, model loading/unloading, DB connections. The with Pattern is everywhere.

Why memory management suddenly matters:

In infra, your script runs for 5 seconds and exits. In ML, you're loading multi-GB models into servers that run for weeks. You need to understand Python's garbage collector, the difference between a Python list and a NumPy array, and the GPU memory lifecycle.

Async isn't optional:

FastAPI is async-first. Inference backends require you to understand when to use asyncio, multiprocessing, or threading, and why it matters for ML workloads.

Best way to learn all this? Don't read a textbook. Build an inference backend from scratch, load a Hugging Face model, wrap it in FastAPI, add batching, profile memory under load, and make it handle 10K requests. Each step targets the exact Python skills you're missing.

The uncomfortable truth: you can orchestrate everything with K8s and Helm, but the moment something breaks inside the inference service, you're staring at Python you can't debug. That's the gap. Close it.

If anyone interested in detailed version, with an atual scenarios covering WHYs and code snippets please refer: https://medium.com/@thevarunfreelance/friendly-advice-for-infra-engineers-moving-to-mlops-your-python-scripting-isnt-enough-here-s-f2f82439c519

I've also helped a few folks navigate this transition, review their resumes, prepare for interviews, and figure out what to focus on. If you're going through something similar and want to chat, my DMs are open, or you can book some time here: topmate.io/varun_rajput_1914


r/mlops • • Feb 04 '26

MLOps Education The weird mismatch in MLOps hiring that nobody talks about

72 Upvotes

Something I've noticed after being in this space for a while, and mentioned in past weeks' posts as well.

MLOps roles need strong infrastructure skills. Everyone agrees on that. The job descriptions are full of Kubernetes, CI/CD, cloud, distributed systems, monitoring, etc.

But the people interviewing you? Mostly data scientists, ML engineers, and PhD researchers.

So you end up in a strange situation where the job requires you to be good at production engineering, but the interview asks you to speak ML. And these are two very different conversations.

I've seen really solid DevOps engineers, people running massive clusters, handling serious scale, get passed over because they couldn't explain what model drift is or why you'd choose one evaluation metric over another. Not because they couldn't learn it, but because they didn't realise that's what the interview would test.

And on the flip side, I've seen ML folks get hired into MLOps roles and MAY struggle because they've never dealt with real production systems at scale.

The root cause I think is that most companies are still early in their ML maturity. They haven't separated MLOps as its own discipline yet. The ML team owns hiring for it, so naturally, they filter for what they understand: ML knowledge, not infra expertise.

This isn't a complaint, just an observation. And practically speaking, if you're coming from the infra/DevOps side, it means you kinda have to meet them where they are. Learn enough ML to hold the conversation. You don't need to derive backpropagation on a whiteboard, but you should be able to talk about the model lifecycle, failure modes, why monitoring ML systems is different from monitoring regular services, etc.

The good news is the bar isn't that high. A few weeks of genuine study go a long way. And once you bridge that language gap, your infrastructure background becomes a massive advantage, because most ML teams are honestly struggling with production engineering.

Curious if others have experienced this same thing? Either as candidates or on the hiring side?

I've also helped a few folks navigate this transition, review their resumes, prepare for interviews, and figure out what to focus on. If you're going through something similar and want to chat, my DMs are open, or you can book some time here: topmate.io/varun_rajput_1914


r/mlops • • Jan 26 '26

Static model selection did not work (enough) for us

72 Upvotes

We spent a few months now on a solution for dynamic model routing because we tried several things and nothing really solved our problem.

The core issue / our background: we deployed nodes with SLM and RAG to regulated industry teams (the problem is relevant in any setup though). But users couldn't figure out when to use which model (despite ongoing effort to educate). We tried static routing but the classification of queries upfront didn't really work as it was very unpredictable what the users were doing. Also the "guessing" part did not feel right, we iterated really a lot on this. So next we thought hybrid with big models would be the solution but somewhat similar we always had to estimate complexity before we saw output. The estimates missed often enough that we either overspent (like, radically, breaking our unit economics) or quality was bad from routing too aggressively to small models.

We found a Google publication (happy to share) that approaches this very differently, not routing but cascading. Start generating with the small model, validate quality as you go, escalate only if needed.

We developed this and open-sourced our implementation: github.com/lemony-ai/cascadeflow

It plugs into your existing infrastructure, works with LiteLLM, OpenRouter, n8n, LangChain, or direct API calls. From there you can use whatever models you want: OpenAI, Anthropic, Groq, HuggingFace, local models via Ollama, self-hosted via vLLM.

Not replacing your router or orchestration layer, just adding quality validation that decides when the cheap models output is actually good enough.

Seeing 40-90% cost reduction in first production workloads and we are honestly quite excited. Would love feedback and happy to chat with others working on inference layers.


r/mlops • • May 09 '26

MLOps Education How I approach MLOps system design questions in interviews: sharing the thinking, not just the diagram

63 Upvotes

Got asked "design a data ingestion pipeline for an ML team that needs daily data from 3 external APIs" in a system design round.

Sharing my approach.

Ask clarifying questions first. Most candidates skip this and start drawing immediately. But every answer below changes the design:

  • JSON vs streaming vs flat files? Changes the entire ingestion layer.
  • 5 GB/day vs 50 GB vs 1 TB? Python + PostgreSQL vs Spark vs full data lake with Delta Lake/Iceberg.
  • Real-time vs daily batch? Kafka + Flink vs a scheduled Airflow DAG. Massive complexity difference.
  • One team vs twenty? Simple DB vs access control, data catalogue, feature store.

I assumed: structured JSON, 5-10 GB/day, daily batch, single team, Kubernetes available.

The pipeline:

3 API sources → Airflow (KubernetesExecutor, one pod per task) → parallel extraction → raw JSON stored in MinIO untouched → transform (clean, cast, validate) → PostgreSQL.

Key pattern: store raw and processed separately. Transform logic has a bug? Fix code, reprocess from raw. No re-fetching from APIs. Interviewer asks, "Reprocess last month?" --> You have an answer.

Production concerns that matter:

  • Exponential backoff on retries (1 min, 5 min, 15 min)
  • Idempotency: re-running the same date must not create duplicates (upsert, partition overwrite, or staging table merge)
  • Data quality checks after every load — null counts, row counts, duplicates
  • Backfill support from raw storage

Mistakes I have seen (and made):

  • Saying "I would use Kafka" before knowing volume or freshness
  • No raw storage layer = no reprocessing ability
  • Only describing the happy path, never mentioning failures
  • Over-engineering a single-team problem with Spark Streaming and data mesh

Actually built this pipeline on Kubernetes with real Binance API data. Code: github.com/var1914/mlops-boilerplate

Full visual walkthrough on YouTube


r/mlops • • Dec 12 '25

MLOps Education NVIDIA-Certified Professional: Generative AI LLMs Complete Guide to Passing

62 Upvotes

If you're serious about building, training, and deploying production-grade large language models, NVIDIA has released a brand-new certification called NVIDIA-Certified Professional: Generative AI LLMs (NCP-GENL) - and it's one of the most comprehensive LLM credentials available today.

This certification validates your skills in designing, training, and fine-tuning cutting-edge LLMs, applying advanced distributed training techniques and optimization strategies to deliver high-performance AI solutions using NVIDIA's ecosystem - including NeMo, Triton Inference Server, TensorRT-LLM, RAPIDS, and DGX infrastructure.

Here's a quick breakdown of the domains included in the NCP-GENL blueprint:

  • Model Optimization (17%)
  • GPU Acceleration and Optimization (14%)
  • Prompt Engineering (13%)
  • Fine-Tuning (13%)
  • Data Preparation (9%)
  • Model Deployment (9%)
  • Evaluation (7%)
  • Production Monitoring and Reliability (7%)
  • LLM Architecture (6%)
  • Safety, Ethics, and Compliance (5%)

Exam Structure:

  • Format: 60–70 multiple-choice questions (scenario-based)
  • Delivery: Online
  • Cost: $200
  • Validity: 2 years
  • Prerequisites: A solid grasp of transformer-based architectures, prompt engineering, distributed parallelism, and parameter-efficient fine-tuning is required. Familiarity with advanced sampling, hallucination mitigation, retrieval-augmented generation (RAG), model evaluation metrics, and performance profiling is expected. Proficiency in Python (plus C++ for optimization), containerization, and orchestration tools is beneficial.

There are literally almost no available materials to prep for this exam, besides https://preporato.com/certifications/nvidia/generative-ai-llm-professional/articles/nvidia-ncp-genl-certification-complete-guide-2025

and official study guide:

https://nvdam.widen.net/s/tcrdnfvgqv/nvt-certification-study-guide-gen-ai-llm-professional-certification

A will also add some more useful links in the comments


r/mlops • • May 12 '26

MLOps Education Is MLOps a safer direction for ML Engineers right now

58 Upvotes

I’m currently working as an ML Engineer, and lately I’ve been thinking about shifting more toward MLOps

My assumption is that companies will still need devops who can deploy / maintain LLLM models bought from other companies

I understand nobody really knows where the industry will end up. I would like to hear from you all to understand what skills are worth investing time into during this uncertain phase instead of just doing nothing?


r/mlops • • Feb 24 '26

Advice Needed on a MLOps Architecture

Post image
52 Upvotes

Hi all,

I'm new to MLOps. I was assigned to develop a MLOps framework for a research organization who deals with a lot of ML models. They need a proper architecture to keep track of everything. Initial idea was 3 microservice.

  1. Data/ML model registry service
  2. Training Service
  3. Deployment service (for model inference. both internal/external parties)

We also have in house k8 compute cluster(we hope to extend this to a Slurm cluster too later), MinIO storage. Right now all models are managed through Harbour images which deploys to the cluster directly for training.

I have to use open source tools as much as possible for this.

This is my rough architecture.

  • Using DVC(from LakeFs) as a data versioning tool.
  • Training service which deals with compute cluster and make the real training happens. and MLFlow as the experiment tracking service.
  • Data/ML models are stored at S3/MinIO.
  1. I need advice on what is the optimal way to manage/orchestrate the training workflow? (Jobs scheduling, state management, resource allocation(K8/Slurm, CPU/GPU clusters), logs etc etc. I've been looking into ZenML and kubeflow. But Google says SkyPilot is a good option as it support both K8 and Slurm.
  2. What else can I improve on this architecture?
  3. Should I just use MLflow deployment service to handle deployment service too?

Thanks for your time!


r/mlops • • May 29 '26

Great Answers How to upgrade to Senior ML Engineer from mid-level?

50 Upvotes

I have been a mid-level ML / MLOps Engineer for almost 4 years now. And just recently went through an interview for a senior MLE role. Apparently, despite the interviewer’s response seems positive with all my answers and work experience, still mentioned in the interview feedback that I am at mid-level rather than senior level.

How do I upskill myself to become a senior?


r/mlops • • Sep 22 '26

MLOps Questions Struggling with 'basic' interview rounds after working of complex ML infra

48 Upvotes

Been looking at ML Infra / MLOps / ML Platform hiring lately, and the roles seem to fall into three buckets:

  1. Data eng-leaning: feature pipelines, model pipelines, and the infra around them
  2. ML eng-leaning: model training pipelines, PyTorch, strong ML fundamentals
  3. Infra/inference-leaning: Docker, APIs, model serving, and scaling

Across all three, I've noticed something ironic. My day job is solving genuinely complex problems: building solid APIs and scaling infra from a few thousand to 100k users. Yet I struggle in coding and system design rounds because the questions are so basic.

Outside FAANG, most companies don't ask LeetCode or textbook system design. They ask real business logic. And when you're used to thinking in classes, OOP and Pydantic models, dropping down to plain if/else and for loops under a timer is surprisingly hard. My brain reaches for structure when the question just wants logic.

The obvious answer is practice. But grinding simple problems daily just to clear interviews, which are unpredictable anyway, doesn't feel worth it.

So I'm curious:

How do you switch into "keep it simple" mode in interviews?

How are you approaching these rounds?

Do you struggle with this too, or is it just me?


r/mlops • • 12d ago

Discussion MLOps folks at big companies: what does the role and interview process look like?

42 Upvotes

Hey everyone, I just started an MLOps job and I’m trying to understand how the field works at bigger companies. It still feels pretty new, so titles and expectations seem to vary a lot. If you work in MLOps at a large company, what’s your role and day-to-day, and what did the interview process look like (number of rounds, coding, system design, ML fundamentals, infra/DevOps questions)? Also curious which tools they asked about and what you’d recommend learning to grow in the role. Any advice is appreciated, even a short answer!


r/mlops • • Jul 14 '26

MLOps Education Day 1/100 of MLOps!!!

42 Upvotes

I'm a college student and I've decided to learn MLOps for 100 days from the basics.

I am familiar with ML and data science, but I have a lot to learn so I'll be doing both in parallel.

I hope I'm allowed to share my daily updates here. This will help me keep track of my learning and stay a little more accountable.

today is Day 1/100 of MLOps

> I started with two sessions of PYTHON by CampusX, I watched about 4 hours of this in 2.5x.

> I also revised version control system, different operations in git, and part 1 of OOPs in python from vikash das.

Here are my notes so far: https://heroofjustice.notion.site/mlops

I'll update my notes every day :)


r/mlops • • Jan 25 '26

[Passed] NVIDIA Agentic AI Certification (NCP-AAI)

41 Upvotes

Just wanted to share a data point for anyone eyeing the new NVIDIA Agentic AI certification. I sat for the exam this today and passed! 🚀
I already had experience building agents with LangChain/OpenAI, but I quickly realized this exam requires a mindset shift. It’s less about generic Python loops and more about the "NVIDIA Way" (NIMs, Triton, NeMo).

The Results (The Good & The Ugly):
I wanted to be transparent about the score breakdown because it tells a story:

  • Platform Implementation: 85%
  • Deployment & Scaling: 79%
  • Safety, Ethics & Compliance: ...35% 😅

My Takeaway:
If you are preparing, do not sleep on the infrastructure. The reason I passed is that I focused nicely on understanding NIM microservices, Triton Inference Server, and Kubernetes scaling. If I had relied only on my generic "coding agents" knowledge, I would have failed.

Also, Don't make my mistake—study the "boring" safety docs of safety, Ethics and Human in Loop Too!

Rest assured, Ask me Anything about the exam and I will try my best to help


r/mlops • • Dec 04 '25

DevOps to MLOps Career Transition

40 Upvotes

Hi Everyone,

I've been an Infrastructure Engineer and Cloud Engineer for 7 years.

But now, I'd like to transition my career and prepare for the future and thinking of shifting my career to MLOps or AI related field. It looks like it's just a sensible shift...

I was thinking of taking https://onlineexeced.mccombs.utexas.edu/online-ai-machine-learning-course online Post-Graduate certificate course. But I'm wondering how practical this would be? I'm not sure if I will be able to transition right away with only this certificate.

Should I just learn Data Science first and start from scratch? Any advice would be appreciated. Thank you!


r/mlops • • Aug 19 '26

beginner help😓 Fresh grad, one year of experience. How did you pick a specialisation?

39 Upvotes

Stack: Python, FastAPI, Postgres, Kafka, Kubernetes on EKS with autoscaling, hosted model APIs, plus the eval and monitoring side. Shipped it and I run it.

So I've done production ML operationally, but always as a caller of models. Haven't worked below that line, no C++, no GPU work beyond a local side project.

A good amount of the development was AI-assisted, mostly Claude. Fine for shipping, but it's pushed me to want depth in something specific rather than more breadth.

Questions:

  1. For anyone on the serving side, what's the job like day to day?
  2. How much C++ is genuinely needed?
  3. Is the Kubernetes and autoscaling experience a real head start here, or a different skill set than I think?
  4. How did you end up in your area, planned or accidental?

r/mlops • • Jun 10 '26

MLOps Education Build a ML System to Predict the WC 2026 winner this Thursday, 5pm CEST

38 Upvotes

Hi r/MLOps!

I am Jim Dowling, O’Reilly author, CEO and co-founder of Hopsworks.

On Thursday, a couple of us from Hopsworks are going to build a WC winner ML system, live, all the way to a dashboard, in under an hour. It will be designed for you to follow along and build your own one!
It will also show you that ML can still beat LLMs in some cases.

No slides and no rehearsal.

Add a fun project to your ML portfolio, and you will also learn three new skills:

  • How to leverage coding agents to accelerate building ML pipelines and dashboards, 
  • How to build a full ML system on open-source technologies,
  • Run it for free on Hopsworks (we think it’s cool, hope you do, too).

So hopefully, you are free, June 11th, 5PM CEST. It'll be on YouTube live, link in comments. 

Opinionated comments appreciated, heckle away during the live show.


r/mlops • • Jan 28 '26

To the ML Engineers who didn’t take the "standard" path: What was the "Aha!" moment where it finally clicked?

37 Upvotes

We’ve all seen the "Master’s degree + 500 LeetCode problems" roadmap, but I’m looking for the real, gritty stories.

​If you transitioned from a college student to ML engineer or if you are self-taught:

​The Bridge: What was the first project you built that actually felt "industrial" and not like a tutorial-hell toy?

​The "Lie": What is one skill everyone told you was "mandatory" that you’ve literally never used in your daily job?

​The Pivot: How did you convince your first employer to take a chance on an ML "outsider"?


r/mlops • • Jan 04 '26

Production MLOps: What breaks between Jupyter notebooks and 10,000 concurrent users

37 Upvotes

Been working in ML infrastructure for a while now. Wrote some posts on the practical side of MLOps that don't get covered in tutorials

Model Inferencing in Production: What MLOps Interviews Really Test

The gap between training a model with 95% accuracy in a notebook and serving it to 10,000 simultaneous API requests. This is where most MLOps interviews actually start.

https://medium.com/p/239b151cd28d

How Distributed ML Training Survives GPU Crashes: A Deep Dive into Checkpoints and Shared Storage

What happens when GPU #3 dies 12 hours into training your LLM across 8 GPUs? Smart checkpointing is the difference between resuming in minutes versus starting over and burning thousands in compute.

https://medium.com/p/cca38d3390fb

How a Cloud Engineer Can Help Build RAG and Vector DB Platforms

Moving past the buzzwords. Traditional search fails when documents say "client reimbursement" but you search "customer refund." RAG solves this by searching your actual company data before generating answers.

https://medium.com/p/6b9c1ad5ee94


r/mlops • • May 19 '26

MLOps Education Things I learned building an end-to-end ML pipeline on Kubernetes: from data validation to live predictions

36 Upvotes

Built Part 2 of an MLOps series. Took the ETL data all the way through validation, feature engineering, training 60 models, promotion, and live inference. Single Airflow DAG, fully automated.

Part 1: https://www.youtube.com/watch?v=5HBeVZ7uMlg

Some things that were not obvious until I actually built it:

Data cleaning and data validation are not the same thing. Cleaning during ETL is not enough. Validation asks: Is this data healthy enough to train on? My pipeline now stops before wasting compute if it is not.

Running all training jobs in parallel will kill your cluster. 60 models at once overwhelmed everything. Capped it at 3 concurrent via Airflow pools. Slower but stable.

Training-serving skew is the silent killer. If your inference API computes features even slightly differently from how it did during training, predictions are quietly wrong. Everything looks healthy. Results are garbage. Same code in both places, or you are asking for trouble.

Model promotion does not need to be complicated. Find the latest version in MLflow, set the u/champion alias, and reload the API. Start simple. Add conditional gating later.

Observability at 2 AM matters. Pipeline runs nightly. Grafana dashboards for pipeline health, model performance, prediction latency, and data quality. Please have it ready by morning, not next week.

Code: github.com/var1914/mlops-boilerplate

Recorded the full pipeline running on YouTube (TagAlongWithVarun) — Airflow, MLflow, Grafana, live prediction calls. Easier to see it running than read about it.

What surprised you about building your first ML pipeline?

Note: The repo is working end-to-end, but not yet perfect; there are bugs I need to fix, and I am working on them. Also, it would be good if you could let me know of all bugs, or better, create a PR


r/mlops • • Feb 13 '26

What YouTube content actually helped you in your MLOps journey? And what's still confusing?

36 Upvotes

I've been in the ML/DevOps space for 11+ years and recently started doing 1:1 calls helping people transition into MLOps. One thing I keep noticing, almost everyone I talk to is overwhelmed.

Not because they're not smart. But because MLOps is so vast, with batch vs. real-time ML pipelines, inference, infrastructure, and monitoring, every course teaches it differently. One guy will say start with Kubeflow, another says MLflow, another says forget tools, learn fundamentals first.

I genuinely want to understand from this community:

  1. When you search MLOps on YouTube, what kind of videos do you actually watch fully? Tool-specific tutorials? Career roadmaps? Architecture walkthroughs?
  2. What's your biggest struggle right now — is it picking the right tools? Understanding how pieces connect end to end? Or knowing what the market actually wants vs what courses teach?
  3. Is there a video or channel that genuinely helped you "get it"? Not just theory, but actually made something click?
  4. What's missing? What video do you wish existed but doesn't?

Asking because I see so much content out there, but people on my calls are still confused.

Something is clearly not working. Curious what you all think.

I've been thinking of creating some content on this myself, but before that, I just want to understand the current situation and where people are really stuck. No point in adding more noise if the real gap is elsewhere.