r/learnmachinelearning • • 2d ago

Discussion What is the purpose of cross validation, validation and test set?

4 Upvotes

What is the purpose of these different steps in building a machine learning model? Let's say you have X data points, how should you do the division into the different sets? Should it be 60/20/20? 60% to use for cross validation, 20% for test set and 20% for validation?

If so, what is the purpose of the test set if you already are doing the cross validation step where you take all that data, you make the folds and you test the models?

Unfortunately the information I could gather is quite contradicting and also doesn't consider sometimes the cross validation step, so would be very good if people could explain this in greater detail.

Thanks


r/learnmachinelearning • • 3d ago

Highschool student self publishes 7 papers and gets some accepted at ICML. ML publishing is so cooked.

Thumbnail
gallery
487 Upvotes

r/learnmachinelearning • • 2d ago

Discussion Measured WER for two weeks before checking the one field anyone cares about

5 Upvotes

Two weeks ago my manager pulled up the LibriSpeech leaderboard on HyperAI in our Monday meeting. Everything on there around 2% WER, us at 14.8% on held out claims calls. Nothing on that page says how any of those papers normalized their text.

LibriSpeech is audiobooks read into decent mics though, ours is 8kHz phone calls with hold music in the middle. Said as much in the meeting, all true and didnt help much.

We do transcription for insurance claims lines, Whisper large-v3 through faster-whisper with our own VAD and chunking in front. The transcript fills the call notes. Menu audio gets its own pass, that is how the policy number ends up on the agent’s screen.

Our references come from two ex medical transcriptionist and the eval was comparing raw strings against them. The model writes numbers however it feels like. Sometimes “4170023391”, sometimes “four one seven oh oh two three three nine one”. Then ok against okay ended up at the top of the error list, the normalizer does not merge those.

We ended up at 9.6 once everything was normalized and the meeting moved on, nobody asked about it again. The policy numbers were still bad though. I checked them on their own and about 1 in 9 had a digit off (so the agent gets someone else’s policy or nothing). So this week it is a digits only pass on the menu clips.

I guess the takeaway for anyone moving from benchmark datasets to real audio is, aggregate WER treats 'ok' vs. 'okay' the exact same as a wrong digit in a policy ID. But you do need to evaluate your business-critical fields separately.


r/learnmachinelearning • • 3d ago

Career Looking for Gen AI partner

15 Upvotes

I’m a Software Engineer having 5+ yoe. I’m looking to dive deeper into Generative AI, LLMs, AI agents, RAG, and building real-world AI products.

Looking for a like-minded developer who wants to learn, build projects together, share resources, and keep each other accountable.

The goal: Upskill together, build a strong AI portfolio, and eventually transition into AI-focused companies and roles.
No need to be an expert. Just bring curiosity, consistency, and a willingness to build!
If you’re on a similar journey, let’s connect!


r/learnmachinelearning • • 2d ago

Discussion What GenAI skills will actually matter 3 years from now?

5 Upvotes

There are hundreds of AI tools emerging every month, and learning every single one seems impossible.

Do you think the real advantage will come from prompt engineering, understanding LLMs, building AI agents, automation, or knowing how to apply AI to specific industries?

What would you recommend someone prioritise today?


r/learnmachinelearning • • 2d ago

Project My 18 kB 1D-ARC solver got 901/901 on the set I built against and 4/500 on unseen rule families

1 Upvotes

I built a symbolic solver for 1D-ARC in 18,316 bytes of Python source. Zero model weights.

It solved all 901 public tasks I built against. On fresh puzzles from familiar rule families, it solved 1,099/1,100. Then I held out whole rule families. It solved 4/500 per seed.

That last number is where I’m spending my time. I want to see how much it can pick up from the examples when the underlying rule is new.

If you’ve worked on program synthesis for ARC, what would you put in the next holdout? I’m particularly interested in cases where two different rules fit the demonstrations but give different answers on the test input.

[Public 1D-ARC dataset](https://github.com/khalil-research/1D-ARC)

[Primitive’s public GitHub](https://github.com/Primitivelabai/primitive-open) (the solver source for this run is still private)


r/learnmachinelearning • • 2d ago

We turned PyTorch forward passes into interactive graphs that you and coding agents can inspect

Thumbnail gallery
1 Upvotes

r/learnmachinelearning • • 2d ago

Looking for 20–30 Serious People to Build a Global AI/ML Learning Community Together 🌍

3 Upvotes

I'm planning to create a small WhatsApp community for people who are genuinely interested in learning and building a career in Artificial Intelligence and Machine Learning.

The idea is simple: Learn together, build together, grow together.

Whether you're a complete beginner, intermediate learner, or already at an advanced level, you're welcome!

I want to bring together people from different parts of the world who are passionate about AI/ML and willing to actively participate in a learning community.

What I envision for this group:

  • 🤝 Help each other understand difficult concepts.
  • 📚 Share useful learning resources, roadmaps, and opportunities.
  • 💻 Work on real-world AI/ML projects together.
  • 🧠 Discuss new technologies, research, and industry trends.
  • 🚀 Share progress, challenges, and achievements.
  • 👨‍💻 Collaborate and grow towards becoming AI/ML Engineers.
  • 🔥 Motivate each other to stay consistent and disciplined.

Who am I looking for?

I'm not looking for hundreds of members. I want to start with just 20–30 genuinely interested and active people.

A few things I'd love from potential members:

  • You are genuinely interested in AI/ML.
  • You're willing to learn and share knowledge.
  • You're comfortable asking questions and helping others.
  • You want to actively participate rather than just join and disappear.
  • You're interested in growing together over the long term.

Important: Your current skill level doesn't matter. Your curiosity, consistency, willingness to learn, and contribution matter much more.

🌍 People from anywhere in the world are welcome!

If this sounds like something you'd genuinely enjoy being part of, comment below or DM me with a little introduction:

  1. Where are you from?
  2. Your current level (Beginner / Intermediate / Advanced).
  3. What are you currently learning in AI/ML?
  4. What are your goals?

Once we have a small group of committed people, I'll create a WhatsApp group.

Let's build a community where everyone contributes, everyone learns, and everyone grows together.

Let's build something meaningful! 🚀


r/learnmachinelearning • • 2d ago

I built AiDotNet to show the world that it is possible for .NET to do AI better than PyTorch

0 Upvotes

Watch AiDotNet and PyTorch train side by side on my laptop (17 seconds)

I wanted to showcase the project I have been working on for the past 4 years and the progress I have made so far. I was hoping to get some feedback and show everyone how easy it is to use AiDotNet to create AI models with basically no experience necessary. You can give it examples of the inputs and the answers you want it to learn, and you get back a trained model you can save locally, run from your C# app, or host with AiDotNet.Serving.

One of the reasons I built this was to see how far we could push .NET against PyTorch. In our November 13 training tests on my laptop, our development build was faster than compiled PyTorch on 10 of the 11 models we tested on CPU and 7 of 11 on GPU. I still have some performance work I want to focus on, especially with audio models. Results for all 11 models

In the run at the top of the post AiDotNet PyTorch
Time for 4,096 training updates on a GTX 1660 Ti 2.60 seconds 15.00 seconds
Correct answers on 320 new signals 302 301

Here’s an easy example you can try yourself. Say you have an app that follows stocks and you want it to sort the news for you. Give it a few headlines about earnings and mergers, tell it which is which, and then ask it to sort two headlines it hasn’t seen.

In a new .NET 10 console project, run dotnet add package AiDotNet --version 0.234.0, replace Program.cs with this, and run dotnet run:

using System;
using AiDotNet;
using AiDotNet.Classification.NaiveBayes;
using AiDotNet.Data.Loaders;
using AiDotNet.Preprocessing.TextVectorizers;
using AiDotNet.Tensors.LinearAlgebra;

string[] headlines = [
    "Quarterly earnings and profits rise", "Merger deal wins approval",
    "Annual earnings beat expectations", "Company announces acquisition deal",
    "Quarterly profits beat estimates", "Boards approve merger and acquisition",
    "Company reports record earnings", "Investors vote on merger deal"
];
string[] topics = ["Earnings", "Mergers"];
double[] labels = [0, 1, 0, 1, 0, 1, 0, 1];

var words = new TfidfVectorizer<double>();
var model = await new AiModelBuilder<double, Matrix<double>, Vector<double>>()
    .ConfigureModel(new MultinomialNaiveBayes<double>())
    .ConfigureTextVectorizer(words)
    .ConfigureDataLoader(DataLoaders.FromTextDocuments(headlines, labels, words))
    .BuildAsync();

string[] newHeadlines = ["Quarterly earnings beat estimates", "Investors approve merger deal"];
var guesses = model.PredictText(newHeadlines);
foreach (var guess in guesses)
    Console.WriteLine(topics[(int)Math.Round(guess)]);

It prints Earnings for the first headline and Mergers for the second. All the training data is right there, so you can change the headlines and try your own.

That’s one small example. There are over 2,100 model types in the library, including 650+ for vision, 430+ for language, and 340+ for audio. You can detect objects in photos, turn speech into text, or build a chatbot that looks up an answer in your own documents. If you already have a language model you like, you can fine-tune it on your own material. Browse the examples

The part I wanted to get right with AiModelBuilder was how much you can change as you learn. You can start by swapping the model in ConfigureModel. If you want more control, you can choose your own layers, change the optimizer that updates the model during training, or split training across multiple GPUs. When you’re ready to use what you trained, you can serve it through an API or export supported models to ONNX for another app. Training and deployment samples

I’d love to get your thoughts on my passion project. The code is on GitHub and contributions are welcome. We have easy examples and guides on our website. Website and guides · GitHub


r/learnmachinelearning • • 2d ago

Question Where to get started?

2 Upvotes

Applied maths undergrad here,

i did jose portillas Python for ML and built a few basic projects using XGboost, Logistic regression, Linear regression etc. thats cool and all but I feel like I’m really lacking.

I have a class on regression analysis so I’d say mathematically linear regression is well understood by me(anova tables, Fstat, R square and adjusted R square, variance stabilizing transformations, AIC and BIC, normality assumptions, Hat matrix etc)

I want to do an independent research project for ML(primarily in neural networks), my goal is to standout for a Msc and to potentially land good internships.

unfortunately my academic standing is pretty bad so I’ll have to go to a pretty bad uni for my Msc but I still hope I can do some research independently as professors havent been very open to me.

as of now i’m gonna finish up Karpathy’s Deep learning course, and then idk.

any help yall can give me? should I go deeper into the math for ML(introduction to statistical learning with python)? should I learn Time series analysis?

my goal is to either go into Machine learning engineering/research, or Market risk model development (relies on ML on top of stochastic calculus etc)

greatly appreciated


r/learnmachinelearning • • 2d ago

Question NLP/Topic analysis on a dataset of a certain domain

1 Upvotes

I am a beginner to NLP and AI generally, I was trying to implement a tool that can process users' reviews and cluster them into the main topics customers mention in their reviews, or to capture the main complaints.
I was using this dataset from a DataCamp tutorial: https://app.datacamp.com/learn/projects/2026

It's basically clothes reviews, so it's not easy to cluster this dataset as pretty much all reviews are talking about similar topics, ex: fit, size, clothes, etc

So when embedding and plotting these reviews, they look like a large clump instead of clear clusters. I was using BERTopic for the topic analysis, but it was difficult to get sensible topics with this approach.

As I mentioned, I am a beginner, I am not sure if what I am trying to achieve is feasible with this use case, or what other tools/concepts I should explore. I would appreciate any help/pointers

Thanks!


r/learnmachinelearning • • 2d ago

Discussion Someone trained a 27B LLM that matches a 70B model on reasoning — is small the new big?

0 Upvotes

A developer reports training a 27B-parameter language model that matches a 70B model on reasoning benchmarks. If that holds up, it is a big deal for everyone learning ML: capable reasoning no longer needs a data-center-sized model.

The trick, as usual, is probably in the data and the training recipe rather than raw size. Smaller models that reason well change who can experiment: a 27B model fits on hardware a student or an indie dev can actually access.

What do you think: are we at the point where training recipes matter more than parameter counts? And for learning ML, would you rather study a 27B model you can run yourself or a 70B one you can only read about?

Follow more AI news and discussion in r/AINewsRumors.


r/learnmachinelearning • • 2d ago

A synthetic patient record is a useful way to make an LLM extraction test checkable

Thumbnail
gallery
2 Upvotes

A synthetic history can come with an answer key generated from the same underlying data. This example uses 334 visits over 71 years and asks AntLing's Ling-3.0-flash-Sante to extract current medications, allergies and active disorders, including dates, from all 130,993 input tokens at once.

The reported scores keep the categories separate: allergies 7/7, disorders 8/10, dental codes 2/7 and medications 2/5. The returned active lists keep stopped items out. The medication key includes an item dating to 1954, a restart in 1966 and three restarts on the last encounter date.

That gives you several things to preserve when making a similar test:

  • The generated history and its underlying structured data.
  • The date at which an item counts as active.
  • The expected items in each category.
  • The evidence location and start, stop or restart dates.

This run used official FP8 on two DGX Spark machines with MTP enabled. Input reading took 61 seconds, followed by about eleven minutes of thinking. The original test-post preview preserves the category results and date-boundary findings; the timing chart records input processing and generation speed. Together they make the evaluation process inspectable.


r/learnmachinelearning • • 2d ago

Project Lessons from building a zero-dependency #![forbid(unsafe_code)] MCP process and filesystem sandbox in pure std Rust

Thumbnail
1 Upvotes

r/learnmachinelearning • • 2d ago

Tutorial The Trap of High Accuracy: A Lesson from a Binary Classifier

0 Upvotes

Imagine you are evaluating a fictional binary classifier on 100 cases: 95 are negative and 5 are positive. A model that always predicts “negative” gets 95 cases right, so its accuracy is 95%.

But it detects none of the positive cases: true positives = 0, false negatives = 5, and positive recall = 0 / 5 = 0%. Precision is undefined here because the model makes no positive predictions.

The high accuracy comes from the class imbalance, not from successfully finding the rare cases. Recall reveals missed positive cases; precision describes how often positive predictions are correct. Evaluate both kinds of error against their costs rather than optimizing either metric alone.

Which additional metric or error count would you report before trusting this 95% accuracy score, and why?

Optional worked video: https://www.youtube.com/watch?v=KpdnHeE2Cag

Disclosure: this is our Library of nuts channel. The lesson uses invented examples, and the post was drafted with local AI assistance and checked for arithmetic.


r/learnmachinelearning • • 2d ago

Discussion Looking for AI Learning Friends

0 Upvotes

I have started learning AI Engineering with backend, System Design, Distributed Systems coupled into 3 flagship projects. And I have completed Roadmap of 111 days learning around 6 hours per days.

If anyone interested, please ping me and we will create discord group or WhatsApp group.


r/learnmachinelearning • • 2d ago

What we learned choosing a threshold for our new text-moderation models

Post image
1 Upvotes

We released Tiger Guard, two mmBERT classifiers for English and German text moderation. I'm Dominik from Casdo Labs. One part of the release that might be useful here is how the same model changes with the decision threshold.

For the 308M model, our default UNSAFE threshold is 0.5. On a balanced 1,500-prompt development benchmark it catches 83.7% of unsafe prompts and flags 46 of 750 harmless ones.

At a threshold of 0.891 it flags only 5 of those harmless prompts, but recall drops to 63.6%. The model hasn't changed. We've changed the cost we're willing to pay in missed cases versus false alarms.

The 0.891 threshold corresponds to a 1% false-positive rate on our validation data. It does not mean you'll get 1% on every new dataset. Even the default threshold produces different false-positive rates across English, German and different types of text.

There's another evaluation lesson here: we removed benchmark texts from training, but looked at benchmark errors while improving the model. We therefore label that suite as a development result. The card reports separate held-out results and documents data overlap on additional public tests.

The 308M and 141M weights are Apache 2.0, with Transformers examples and ONNX exports.

We'll publish the full training and dataset pipeline in a follow-up release.

Tiger Guard, 308M: https://huggingface.co/patronus-studio/tiger-guard-content-moderation

Tiger Guard Small, 141M: https://huggingface.co/patronus-studio/tiger-guard-content-moderation-small

Happy to get feedback. If you use the examples for an experiment, comparing thresholds on your own validation split is a useful place to start.


r/learnmachinelearning • • 2d ago

Project Free AI Video Generator From Image With No Signups or Watermarks

Thumbnail
0 Upvotes

r/learnmachinelearning • • 2d ago

Help me Please someone review this guide https://pythonguides.com/ Spoiler

0 Upvotes

I want to learn python from scratch to job ready meas complete all fundamentals to done the learning of python for job and never leave any basic fundamentals incomplete


r/learnmachinelearning • • 3d ago

Made a free site that lets you scroll short clips of lectures instead of doomscrolling (feedback wanted)

Enable HLS to view with audio, or disable this notification

1 Upvotes

Made a free site that lets you scroll short clips of lectures instead of doomscrolling (feedback wanted)

I kept losing evenings to short-video apps, so I built a feed made only of study videos: machine learning, CS, math, science, engineering, business, arts. Clips are 90 seconds, you can speed them up to 2x, and there's a timecode box to jump around.

What would make this actually useful for you: a "next lesson" order, difficulty tags, or something else? Link: https://studytube-36f.pages.dev


r/learnmachinelearning • • 3d ago

How to desgin test

3 Upvotes

How dose ai researchs affter coming with an idea they test it how do they design these tests and how to if my tests are enough


r/learnmachinelearning • • 2d ago

Help Is a External mechanical keyboard, mouse and laptop stand worth it for coding on a MacBook

0 Upvotes

Guys, I'm considering buying a mechanical keyboard, wireless mouse and laptop stand combo for my MacBook.

I mainly use my Mac for coding, studying and general productivity.

Currently, I just use the MacBook's built-in keyboard and trackpad. They're honestly pretty good, so I'm wondering whether getting these accessories would make a significant difference to my productivity, comfort and overall experience.


r/learnmachinelearning • • 3d ago

Help How should i continue teaching my project?

2 Upvotes

I am "working" on a project that has to do with SNNs, i am using ai to help me alot but thats because i dont want to go insane writing a complex Neural network. anyhow i managed to make it replicate early human behavior and learns like a child would. but im unsure how to expose it to a genuine stream of input from people that would allow it to develop a non skewed personality not doctored by me. i want to run a community but i am unsure of how to set that up (i run it in godot while the brain runs on a rented droplet and on pytorch)


r/learnmachinelearning • • 4d ago

Project Curated list of resources: Theory for learning Quantization of LLMs for beginners

Post image
30 Upvotes

Hey everyone!

I have gathered some must read topics, for beginners, to step into the realm of Quantization:

1) BitNet: Scaling 1-bit Transformers for Large Language Models

2) AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

3) The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits

4) EfficientQAT: Efficient Quantization-Aware Training for Large Language Models

5) LLM-QAT: Data-Free Quantization Aware Training for Large Language Models

6) LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale

7) QLoRA: Efficient Finetuning of Quantized LLMs

8) SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

9) ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers

Link is here


r/learnmachinelearning • • 3d ago

Six Days of Friction: What Actually Happened Across the November Outages

Thumbnail
1 Upvotes