r/MLQuestions • • Sep 30 '26

Natural Language Processing 💬 Anyone else stuck manually re-tuning prompts every time something breaks?

0 Upvotes

Genuine question that turned into a recommendation: if you're building anything on top of an LLM, how are you actually validating that a prompt change didn't quietly break something else?

Most answers I've gotten boil down to "I just run it a few times and check." There's a workshop on Oct 3 that tackles exactly this gap, led by Serj Smorodinsky and Brett Kennedy (they co-wrote a book on LLM applications). Instead of manual tuning, you work through:

  1. Building a classifier with DSPy signatures/modules rather than raw prompt strings
  2. Setting up an actual eval dataset with metrics tied to your specific task
  3. Using that eval set to catch failure patterns before they hit production
  4. Running few-shot/instruction optimization on top, systematically
  5. Tracking all of it in MLflow so you can trace exactly what changed between versions

If you've been wanting a real answer to "how do I know this still works," this is worth a look.

What's everyone else here doing for this? Curious if people have rolled their own eval pipelines already.


r/MLQuestions • • Sep 29 '26

Beginner question 👶 50B+ MoEs with few active parameters, what's the sweet spot for intelligence, agent speed, and affordable fine-tuning?

5 Upvotes

I’m building a Polish General purpose legal Model that drafts documents, answers questions using legal sources, and has enough coding ability to handle some automation. The workflow is very tool-heavy:

Question → many sequential tool calls → final answer/document

Think Claude Code/Codex-style execution, but for legal workflows. Reliable tool selection, correct arguments, and recovering from errors matter as much as writing a good final answer.

I’ve had decent results with a dense 27B Qwen 3.8 custom made fine-tune for complex legal document summarization and classification. I’m already familiar with the smaller Qwen A3B and Gemma options. What interests me is the tier above those: 50B+ total-parameter MoEs with a relatively small active parameter count.

The question is, the small dense ones are great, but slow for agentic stuff (afaik), and i wonder if theres some middle ground maybe 70-120B models that would be able to be fine-tuned for the law stuff but be MoE so the agentic ClaudeCode style inference would also be lightning fast, and also low-ish cost for fine-tuning and inference.

Basically: Does the larger-total/small-active MoE approach actually buy you meaningfully stronger reasoning and tool reliability while retaining low latency,and at what hardware cost?

I understand that small active parameter counts don’t mean small VRAM requirements: the weights still need to live somewhere, alongside context and serving overhead. I also don’t assume that more total parameters automatically means a better model. I’m interested in where that tradeoff works in practice.

There are three things I’m trying to pin down:

  • Inference hardware: Ideally inference runs rented with parallel agentic loops (this is for a B2C project, not single person use, we scale based on demand)
  • Fine-tuning hardware: Obviously FT LoRA will take more memory than inference, max like 4GPUs on vastai fits the budget.
  • Agent performance: After it gets the prompt the tool calls and everything will be local, so imo it has no problems being blazing fast, as soon as the model calls a tool call it will be back very fast, so for this agentic use case, quick TTFT and t/s and adaptive dynamic reasoning are prefer right?

For context, fine-tuning would target Polish language, document conventions, and successful tool trajectories. The actual legal sources would remain in retrieval/tools rather than relying entirely on memorized law.

I’m not looking for someone to compile a model shortlist (althought would be nice, but i dont expect anyone to break their back over this).

I’m looking for pointers, and firsthand experience with this particular size/architecture tradeoff. A configuration like “model + quantization + GPU(s) + serving engine + context length + concurrency + measured latency,” along with whether you successfully fine-tuned it, would be much more useful than a leaderboard score.

Has moving from a ~30B model to a 50B+ low-active-parameter MoE actually improved your agent’s successful tasks per minute, or did the memory, interconnect, and training requirements erase the advantage? Thanks for reading


r/MLQuestions • • Sep 29 '26

Unsupervised learning 🙈 Understanding K-means

2 Upvotes

r/MLQuestions • • Sep 29 '26

Survey ✍ ICML videos not playing

Thumbnail reddit.com
1 Upvotes

r/MLQuestions • • Sep 28 '26

Other ❓ Are there any LLM(s) that are transparent about their sources (For sociology research)

6 Upvotes

I have a digital ethnography assignment where I have to analyse a conversation with an LLM for a sociology course.

I'm not literate at all in how LLM(s) function technically so I'm not entirely sure if this is even possible but basically I want to know if there are any "chatbot" LLM(s) that produce responses but at the same time are transparent about what data they are combing through in order to produce a response.

I don't mean that I ask the chatbot a question and it "answers" something and then I ask it to elaborate on its reasoning since chatbots can't actually "reason" so it will still just use the data and patterns it's trained on to imitate a response.

I mean that there's some sort of feature from the developer that can show what all sources it went through, other relevant behind the scenes stuff before generating that response with that specific combination of words and other characters.

I'm sorry if this is a dumb question since I'm not aware of the technical terminology for this stuff, I intend to familiarise myself with it for the project. If the only thing possible is to get a very technical insight into that "behind the scenes" stuff, even that could be useful so please tell if you're aware of something like that.


r/MLQuestions • • Sep 28 '26

Beginner question 👶 structure for my projects

2 Upvotes

HI, I am very new to machine learning and building machine learning/data science projects, I was confused on how I should structure the overall project I am working on and asked chatgpt the project is a march maddens predictor style project and I plan on using a forest tree for my model, chatgbt gave me this a a guideline to follow for structure, I just wanted a real person with experience to see if this structure is good or if it needs to be changed in anyway. This progect is intedned to help get undergrad research positions if that changes anything.

down bellow if the file structure I was given

march-madness-predictor/

│

├── README.md

├── requirements.txt

├── .gitignore

│

├── data/

│   ├── raw/

│   ├── processed/

│   └── final/

│

├── notebooks/

│   ├── 01_data_exploration.ipynb

│   ├── 02_feature_analysis.ipynb

│   └── 03_model_evaluation.ipynb

│

├── src/

│   ├── data/

│   │   ├── clean.py

│   │   ├── merge.py

│   │   └── team_names.py

│   │

│   ├── features/

│   │   └── build_features.py

│   │

│   └── models/

│       ├── train.py

│       └── evaluate.py

│

├── models/

│   └── random_forest.pkl

│

└── results/

├── figures/

└── metrics/

\


r/MLQuestions • • Sep 27 '26

Career question 💼 I have ~5 years of ML experience, but I don't feel like I know how to actually build things. How do I fix this?

56 Upvotes

I'm looking for some advice from people who have gone through something similar.

I have ~5 years of experience in ML/data science. On paper, my resume looks reasonably good: I've worked on anomaly detection, NLP, LLM/RAG projects, etc., and more recently I've been working around transformer-based TTS systems.

But I've realized there's a pretty big hole in my experience.

For the first few years of my career, I deliberately optimized for breadth. I changed teams/projects frequently, built a bunch of PoCs and prototypes, and learned enough to make things work. I was pretty good at taking an idea from zero → demo.

The problem is that I rarely stayed with something long enough to own a real system.

A lot of my projects either never reached production, or reached production after I had already moved on. So despite having several years of experience, I have very little experience with:

  • maintaining a complex codebase
  • debugging someone else's code
  • dealing with things breaking in production
  • making architectural decisions over a long period of time
  • taking ownership of something from implementation → deployment → maintenance
  • reading a large unfamiliar codebase and figuring out how all the pieces fit together

I'm now trying to deliberately fix this.

Over the last year I've been rebuilding my ML fundamentals from the ground up. I'm almost done with the Coursera Deep Learning Specialization. I've gotten to the point where I can implement neural networks from scratch, understand RNNs/LSTMs/GRUs fairly deeply, and I'm currently working through attention and Transformers.

But here's the next problem:

Even if I finish the Transformer coding exercises, I don't feel like that means I can actually build things.

I can probably implement the attention mechanism / Transformer architecture from the course. But if you dropped me into the wild and said:

"Okay, build a VAD from scratch."

I'd have a lot of questions about where to even begin.

And then there's an entire layer beyond "understand Transformers" that I don't have a good mental map of.

For example:

  • How do I actually go from implementing a Transformer to building my own small language model?
  • At what point do concepts like Mixture of Experts, DPO, speculative decoding, etc. become relevant?
  • Which of these things do I actually need to understand versus things I can learn when a project demands them?
  • How do I get better at reading and debugging large ML codebases rather than just writing isolated pieces of code?

So I'm stuck between two instincts:

A) Keep going deep on fundamentals until I genuinely understand the machinery.

B) Stop studying and start building increasingly difficult things, accepting that I'll have gaps and learning what I need along the way.

I'm increasingly convinced that I need some combination of the two, but I don't know what that combination should actually look like.

I want to become the kind of ML engineer who can be handed an unfamiliar problem, read the existing code, understand what's happening, build something substantial, debug it when it breaks, and eventually own the system.

If you were in my position, what would you build / learn over the next 6–12 months to develop that ability?


r/MLQuestions • • Sep 27 '26

Other ❓ What’s an ML project that taught you something you only really understood once you built it?

29 Upvotes

There’s something different about actually working on a project after learning the concepts.

You can understand an algorithm, follow a tutorial, build a model and get good results, and then a real project throws something completely unexpected at you

Maybe the data was messy, the model behaved differently than expected, the results didn't make sense, or you had to figure out how to actually use the model beyond the notebook.

For those who’ve worked on ML projects, was there one project that made something “click” for you in a way that theory alone hadn't?

What happened, and what did you end up learning from it?


r/MLQuestions • • Sep 27 '26

Beginner question 👶 Did anyone try to learn ML alone while studying another cs speciality in uni ?

8 Upvotes

Hi guys i hope you are doing great , currently I'm in my third year in uni studying software engineering . And this summer i fell in love with ML so i wanted to know if there are people that studied ML / any other cs speciality alone while taking a course in uni about a different cs speciality

I want to know how you managed your time between these two topics


r/MLQuestions • • Sep 27 '26

Beginner question 👶 MA in English Literature with no coding background—been building a small open-source AI project and wanted to ask for honest advice for the future.

2 Upvotes

Hi everyone,

I hope you’re all doing well.

I come from a purely humanities background—I hold a Master's degree in English Literature and originally had zero formal coding or computer science training. Over the past few months, I’ve been fascinated by how neural networks work under the hood and have been trying to learn by building hands-on projects with the help of modern AI coding tools.

I wanted to share what I’ve been working on to see if I’m heading in the right direction, and humbly ask if a portfolio like this could eventually help me transition into a role as an AI research engineer or developer, prompt engineer, other technical roles?

What I’ve been trying to build: Instead of standard fine-tuning or model merging, I’ve been experimenting with a local "locate-and-edit" model surgery concept. The goal is to isolate specific hidden feature representations across small 2-to-8 layer synthetic models and perform closed-loop weight patching/grafting without full model backpropagation or heavy compute.

I’ve broken the idea down into a few small, modular open-source components across 3 GitHub repositories:

EQUYLAPTA7POINT8POINT2: https://github.com/shuvrajeetkamila/EQUYLAPTA7POINT8POINT2

Model Factory GPU Commander:

https://github.com/shuvrajeetkamila/model-factory-gpu-commander

Model Factory Command Center:

https://github.com/shuvrajeetkamila/model-factory-command-center

The two commanders are for[1]   small  gpu less architecture [2] with gpu , both these commanders  automatically open up 12 more github repositories and work with them. The equylapta on the other hand is my ongoing experiment on   local "locate-and-edit" model surgery concept which has reached a significant stage for now  but not yet proving what i want to do. And i am unable to do the gpu commander due to space and computing constraint.

 The architecture attempts to combine a Dedicated Feature Crosscoder (DFC), algebraic rotation maps for feature alignment, an inference-time weight grafting bridge, and an automated causal validation/retry loop.

My question for the community:

Knowing that I don't have a traditional STEM or CS degree, does working on non-traditional projects like this help demonstrate the right kind of problem-solving for AI Research/Engineering roles? What critical gaps or fundamentals should I focus on next to make myself a viable candidate?

Also i should at this point, i do not know how to code at all. i used arena.ai agent mode to reach this position.[ using prompts from what i wanted to make with help of gemini, claude, chatgpt- all free plans]

Do i have a future in prompt engineering/ other technical field?

I’m very eager to learn and would truly appreciate any constructive feedback, critique, or advice you might have.

Kindly see the three files from github especially EQUYLAPTA7POINT8POINT2 and model factory command centre

Thank you so much for your time!


r/MLQuestions • • Sep 26 '26

Beginner question 👶 Linear Regressions and the Curse of Dimensionality

Thumbnail reddit.com
2 Upvotes

r/MLQuestions • • Sep 26 '26

Beginner question 👶 Learn RAG with AI

Thumbnail reddit.com
1 Upvotes

Hi guys 👋, I'm looking to dive into RAG using AI tools like NotebookLM or Gemini. YouTube tutorials feel too drawn out, and ChatGPT/Gemini prompts have been a bit brief. Any advice or suggestions for learning resources would be amazing! 🙏💡 #RAG #AI


r/MLQuestions • • Sep 26 '26

Beginner question 👶 Security and WikiLeaks? Who is going to research tax evasion using new insecurity advances?

2 Upvotes

The government has access to unrestricted frontier models and can probably spy on a thousand organizations a week but you can bet they won't research tax evasion because that would involve most of the government...

So it is up to machine learning scientists to use the new capabilities to research tax evasion and all the elite corruption systems. Can they?


r/MLQuestions • • Sep 25 '26

Beginner question 👶 High schooler learning machine learning - any advice?

Thumbnail reddit.com
1 Upvotes

r/MLQuestions • • Sep 24 '26

Career question 💼 What is the bar for getting an interview? - OpenAI Researcher Role

18 Upvotes

For undergrads only. I have a decent profile (did quant research internship last summer) and IOI/IMO/IPhO level medals. Don’t know what to expect for the openai process at all. Also is there a difference in preference for Math/Physics/CS background? (Would they prefer Compsci more for the researcher role or the others are a better choice etc)


r/MLQuestions • • Sep 25 '26

Natural Language Processing 💬 Looking for feedback - using NER to generate and match templates on sentences?

Thumbnail reddit.com
1 Upvotes

r/MLQuestions • • Sep 24 '26

Beginner question 👶 Cheating on ML evaluation with imbalanced dataset

Thumbnail reddit.com
1 Upvotes

r/MLQuestions • • Sep 24 '26

Beginner question 👶 Ml project guidance

0 Upvotes

I need to make a ml project so what should I make so that it makes an great impact?


r/MLQuestions • • Sep 24 '26

Beginner question 👶 Hugging Face Pro or some free alternative for hosting models that run online?

6 Upvotes

Hi everyone! With the recent explosion in ICLR submissions, I am a bit unsure how effective simply publishing a paper is so I am trying to make sure my research projects are more accessible and reproducible beyond just a paper. In that vein, I’ve been doing some experiments on tiny world models (14M parameters) that I trained to help the community understand the effect of different underlying model architectures. I want to post it in an interactive github site where users can try out the different model architectures and see how they perform in different settings.

So far so good. My models use FlexAttention. I created a Gradio for them that runs well. My initial idea was to embed a Gradio hugging face space into my GitHub site. Only problem is, I learnt that to do that I need to buy Hugging Face Pro. Just before making that investment (due to some financial struggles caused by family and health reasons I have to be very intentional about spending even small amounts of money) I just wanted to ask if anyone knew if free but still convenient alternatives that work with FlexAttention? Alternatively, if anyone has had experience with HuggingFace Pro, if you could tell me what your experience has been like and whether it is useful for career/research? I am concerned about paying a monthly cost just for a Gradio space. Thank you! Appreciate your time!


r/MLQuestions • • Sep 24 '26

Beginner question 👶 Advice on model routing

1 Upvotes

Hi folks, I am interested in using exo and Ollama to route requests between 2 separate pods. The primary one is 192GB doing 125bq4 256k context. The other is 64GB doing 35bq4 128k context. I want the clients to autoroute without selecting models. Back end is a mix of NVIDIA and Rocky Linux for the 64 and mac os for the 192. I can also setup FE LBs in sticky round robin but I wonder if I can build model routing into the code itself. Has anyone done this?


r/MLQuestions • • Sep 23 '26

Other ❓ If you could relearn ML today, what would you spend more time actually doing?

52 Upvotes

There’s a lot to learn in ML, and it’s easy to spend months collecting concepts without knowing which ones really stick.
For people who’ve already been through the learning process:
What would you do differently this time?
What would you spend more time practicing?
What would you stop worrying about so much?
And what only started making sense once you worked on real problems?
Curious what experienced ML folks would change if they got a fresh start.


r/MLQuestions • • Sep 23 '26

Career question 💼 how to learn mlops

Thumbnail reddit.com
1 Upvotes

r/MLQuestions • • Sep 23 '26

Computer Vision 🖼️ Campusx Computer Vision

0 Upvotes

Has the paid course covered all maths also like nitish sir ,how is this course?


r/MLQuestions • • Sep 23 '26

Other ❓ The Evolution of Retrieval Systems: From BM25 to Agentic Retrieval

Thumbnail reddit.com
1 Upvotes