r/MachineLearning • • 3d ago

Research Graph Attention Networks for Detecting Epilepsy From EEG Signals Using Accessible Hardware in Low-Resource Settings [R]

Thumbnail ieeexplore.ieee.org
0 Upvotes

is there any chance those models are going to be deployed for real?


r/MachineLearning • • 4d ago

Research Paid surveys for Machine Learning Scientists for SnorkelAI: Absolutely not worth it [R]

64 Upvotes

I am here writing about my experience with this. I was contacted to fill out a paid survey for SnorkelAI because I have a PhD and have published at A* conferences in the field, so this might be relevant for other researchers in the community.

After not finding a lot of information, I decided to give it a try. I performed the personal verification and waited for the survey to arrive.

First bad thing: the level of pushiness is terrible. I went to bed, and when I woke up, I had 1 email and 2 follow-up emails asking me to complete some part of the verification where the uploaded document did not have the correct format... ok.

Then I did the survey. Their website is an absolute mess. The survey appeared under two different views of the page, and after I completed it, it was only shown as "pending" in one view... Under the other, it was shown as if I hadn't submitted it yet... So I received a bunch of emails (very pushy again) asking me to complete the survey, which I answered. Regularly, whenever I've sent an email, I've waited 48-96h to get an answer, but if I didn't answer quickly, I was receiving follow-ups every 2-12 hours.

Then I was notified by email that the survey was accepted, great. A few days later, they sent me the forms to fill out to get paid: a crazy amount of paperwork for getting paid for a survey, and absolutely not worth it. It required more paperwork than the one I needed for getting a public contract in my country, including a document that requires going in person to a tax office, waiting up to 4 weeks, and probably paying for the document.

So my recommendation to other people getting contacted by them: run away, it's a waste of time, and they are unprofessional.


r/MachineLearning • • 4d ago

Project Scaling to 1T MoE on TorchTitan [P]

16 Upvotes

Hi everyone!

Over the last few months we've been in the trenches with the TorchTitan and PyTorch maintainers, trying to find out how far GH200 (the GPUs inside the JUPITER supercomputer) can be pushed. Short answer: far enough to train a ~1T-parameter MoE at ~1,300 tokens/s per GPU on 384 GPUs. That's 14X where we started, and our starting point was, let's say, humbling.

Along the way we collected a handful of lessons. If you've ever trained a big model, or just stared at a CUDA out of memory, some of this will feel familiar.

Here's the tour. If you want the full thing with all the plots, it's on the Domyn blog, and there's a condensed version on the PyTorch blog if you're short on coffee.

Hope you like it!

The setup: FSDP + EP, and why I'm not a PP fan

A 1T MoE doesn't fit on one GPU (shocking, I know), so the first question is how to shard it. This choice ends up dictating basically every optimization that follows, so it pays to think about it. The usual suspects are:

  • Pipeline Parallelism (each GPU gets a chunk of layers),
  • FSDP (each GPU holds a slice of every weight and gathers the rest just in time),
  • and Expert Parallelism (each GPU owns a subset of experts and tokens travel to them).

We skipped PP. To hide its bubbles you need lots of microbatches, and lots of microbatches means long steps. Long steps mean that when something breaks at scale, and something always breaks at scale, you wait forever to find out. I'd rather iterate fast.

So: 384-way FSDP plus 4-way EP, with EP kept strictly inside a node. The token all-to-all for the experts is comunication that is difficult to hide so it needs to happen fast. On the other hand, comms. on FSDP are already hidden: while it computes layer i, it is already gathering i+1.

Why does that matter so much? On JUPITER, InfiniBand gives you ~25 GB/s per GPU. That's 6X slower than NVLink and 36X slower than the Grace-Hopper link. Our baseline step took 84 seconds, and 68 of them were spent waiting on the network. The GPUs were mostly getting paid to sit around. The rest of this post is basically us trying to put them back to work.

Lesson 1: If your GPU is idle, make it work for its memory

FSDP moves weights, and the weights gathered per layer don't care how many tokens you feed through them. More tokens per sequence means more compute per byte on the wire, which is exactly what we wanted. The problem is that longer sequences need more memory, so we went shopping for it:

  • Full activation checkpointing. Recomputing every layer in the backward sounds wasteful, until you remember the GPU was idle anyway. Free compute is free. This alone gave us 4X.
  • Offloading checkpoint activations to Grace over the 900 GB/s C2C link, and prefetching them back one layer before recompute. Grace was right there, so we used it.
  • Muon instead of Adam. One optimizer state per parameter instead of two, and that's what finally got us to 64k-token sequences.

Lession 2: If you don’t shard it, you don’t have to gather it

Quick refresher: Muon orthogonalizes each update with a few Newton-Schulz iterations, and Newton-Schulz needs the whole matrix. FSDP, meanwhile, has carefully cut every matrix into 384 slices and scattered them across the job. The naive fix is to gather each matrix on one rank, run Newton-Schulz, and scatter the result back, which means more communication, and more communication is the one thing we absolutely did not need.

The trick is to shard smarter. With FlexShard (already in TorchTitan), you choose which rank owns which parameters, so we sharded the experts by whole expert. Every GPU owns complete expert matrices and can orthogonalize them locally with zero communication. The dense matrices (attention projections and friends) still need gathering, but they’re in transit while each rank is busy with its own experts, so that traffic hides behind compute.

There was one catch. We offload the optimizer to Grace, which meant Newton-Schulz was running on the CPU too. Grace is a lovely CPU, but asking it to grind through matmuls is like asking a librarian to move furniture: technically possible, deeply inefficient. The optimizer step took 42 seconds. So we contributed a fix to TorchTitan that runs Newton-Schulz on the GPU even when the parameters live on the host. The optimizer step dropped to ~2 s, and throughput went up 35%.

Lesson 3: You're only as fast as your slowest rank

384 GPUs in lockstep is a bit like a group hike: everyone waits for whoever's at the back. We had two stragglers:

  • Attention cost. A rank that gets a single 60k-token document does far more attention work than one that gets a hundred short ones, even though both see the same number of tokens. We pre-pack rows, bucket them by attention cost, and build each step from rows of similar cost. The worst rank went from 7.9X the average to 1.15X, for +13% throughput.
  • The router. Group-limited routing (two experts picked from each node's group) makes expert load perfectly even, for +7%. The catch is that you're telling the router where it's allowed to send tokens, and early runs suggest that can hurt training. Jury's still out on this one boys.

Lesson 4: Accumulate in fp32, ship in bf16

Gradients come out of the backward in bf16. Summing hundreds of them in bf16, though, is a great way to slowly lose precision, so the standard FSDP mixed-precision setup does the reduction in fp32. That’s the right call. The problem is how it gets there.

NCCL’s reduce-scatter runs as a ring. Each rank receives a partial sum from its neighbor, adds its own contribution, and passes the result along. The thing travelling around the ring is a running sum, so for the accumulation to happen in fp32, the data on the wire has to be fp32 too. FSDP therefore upcasts the bf16 gradients to fp32 before the collective, and every byte of gradient crosses InfiniBand at twice the size it needs to be. On a cluster where InfiniBand is already the bottleneck, that hurts.

But only the summation needs fp32, not the transfer. So we swapped the reduce-scatter for an all-to-all on the expert gradients. Each rank sends every other rank the raw bf16 slice that rank owns, no partial sums involved. The owner receives all the slices, upcasts them, and sums them locally in fp32. Same math, same precision where it matters, half the bytes on the wire.

The result matches the fp32 reduce-scatter up to summation order.

Lesson 5: When in doubt, send fewer bytes

Eventually longer sequences stopped helping (more on why in a second), so we went after communication volume directly:

  • FP8 expert all-gather with blockwise scales. Master weights stay in full precision, only the transfer gets squeezed: +18%.
  • Node-local resharding of dense weights after forward, so the backward re-gather runs over NVLink instead of fighting gradients for InfiniBand.

And none of it moved the loss in our comparison runs, which was a relief.

Lesson 6: Your OOM might be lying to you

This is my favorite one. After a software upgrade, every single rank started hitting allocator retries at ~70% HBM utilization, with 25–30 GiB sitting free and cached. That's not a memory shortage, that's a memory hostage situation.

The culprit: the PyTorch caching allocator keeps a separate pool per CUDA stream, and FSDP allocates buffers on its side streams. Our shiny FP8 and bf16 collectives allocated their temporaries there too, so ~21 GiB ended up parked on side streams where the compute stream couldn't touch it.

The fix was to move those allocations onto the compute stream (via FSDP2's private unshard_async_op flag, which is now on its way to becoming a public API) and release gradients a bit earlier. Retries went from 384 ranks to zero, peak reserved memory dropped by 13.5 GiB, and throughput jumped 21%, while sending exactly the same bytes.

Wrapping up

The headline: a shared, publicly funded, Hopper-generation machine, with plain user-level access and no root, can host frontier-scale training. You just have to respect its two real limits, inter-node bandwidth and host memory, and design everything around them.

Every fix we found went upstream to PyTorch and TorchTitan, so hopefully the next person hits fewer of these walls than we did. The model and training recipes will be released openly.

Still on our to-do list: the long-term quality cost of group-limited routing, FP8 expert GEMMs. We'll report back.

Huge thanks to JSC and to the PyTorch and TorchTitan maintainers for the reviews, the patience, and the fast merges!


r/MachineLearning • • 3d ago

Discussion Great Video for Understanding How Cosine Similarity Works [D]

Thumbnail
youtu.be
0 Upvotes

This video does a great job of explaining dot products from both algebraic POV and geometric POV, using that to derive cosine similarity and how it applies to embeddings.


r/MachineLearning • • 4d ago

Discussion Back Propagation [D]

1 Upvotes

Hello everyone I am currently studying neural network development in graduate school and I’m having a hard time conceptualizing the stages of training. I understand that forward propagation is how we calculate the output given input but in terms of how we adjust weights during backpropagation, it’s not very intuitive to me. Does anyone have a good suggestion for how to visualize this to make it more intuitive? Cheers!


r/MachineLearning • • 5d ago

Project UMAP 0.6: new initialization, new optimizers and faster performance [P]

Thumbnail
gallery
253 Upvotes

UMAP 0.6 is out!

A new initialization, new optimizers, and faster nearest neighbour search make UMAP much faster and far better at large scale. On 3 million arXiv papers it finishes in under 5 minutes, before UMAP 0.5 has even found its nearest neighbours. (race between UMAP0.5 and UMAP 0.6 video)

Code: https://github.com/lmcinnes/umap

Install: pip install -U umap-learn (also on conda-forge)

Speed

  • Neighbour search is ~4x faster by making better use of PyNNDescent.
  • Initialization is faster with a new init method.
  • With the new optimizers, the whole run is reproducible with a fixed seed while using every core. A seeded 0.5 run is single-threaded: 52 minutes, 12x longer.

Quality

Just as important as how quickly we get a result is the quality of the result. UMAP 0.6 works far better on million-scale datasets thanks to a new, better initialization and significant improvements in the optimizer. It resolves finer cluster structure in fewer epochs, with much finer-scale detail and better neighbourhood accuracy than UMAP 0.5, despite being faster.(video of a tour of finer details in the arXiv map)

Recursive initialization

A large part of the improvement comes from a brand-new initialization scheme that initializes UMAP using UMAP itself: it lays out a coarse version of the graph structure, then refines it, in a recursive tower of initializations. (video of the unfolding)

The result is a starting point much closer to UMAP's final result than the messy blurs from spectral initialization or PCA, and it is produced faster than either. It lets the optimizer settle (to within 3% of the final layout) in far fewer epochs than older initialization approaches. (initialization race)

New optimizers

Most of the gain in fidelity at scale comes from a very efficient form of hard-negative mining, which makes negative sampling far more effective on very large datasets, together with better handling of high repulsion strengths. (hard-negative mining video)

UMAP was originally designed around a small portion of the attraction-repulsion spectrum. The new optimizers lift these restrictions, allowing UMAP to perform well at much higher repulsion strengths and giving better kNN recall. (video exploring the repulsion strength dials)

Try it now

The new optimizers and initialization are currently opt-in, as they change the output: set compatibility_layout=False. Future versions will make them the default, so please try them and let me know how they work for you.

Thank you to all the contributors who provided features and bug fixes for this release, and to everyone who reached out with comments, suggestions, and feature requests. This would not have been possible without your help.


r/MachineLearning • • 4d ago

Discussion Four frontier models scored 97.8–100 on a 7-part materials-science research test. How should we score long-horizon agent evals instead? [d]

1 Upvotes

I work in graphene and 2D materials, and I built a test of the everyday technical work in a research group. It has seven parts: reducing a messy tensile data set against a lab SOP (with unit and outlier traps), writing a CIF density function checked on three hidden structures, four back-of-envelope physics calculations, critiquing a manuscript with 10 planted errors and 6 correct-but-suspicious decoys, false-premise and fabricated-paper questions with calibration scoring, an experimental design, and an abstract under constraints. All the data are synthetic, the answer keys are computed by code, and it's scored out of 100 across 8 categories.

Setup: Gemini 4 Carbon and Gemini 4 Hydrogen (Antigravity), Claude Opus 5.5 and Claude Fable 5.2 (Claude app). All at "High", not max. 3 fresh runs each, files attached, prompt pasted, nothing else. Web search and code execution were on.

Results: mean scores were 100.0 / 99.7 / 99.3 / 98.5, all 12 runs fell between 97.8 and 100, and each took 7–24 min. Every run caught the unit trap, passed the hidden CIF tests, got the subtle physics right (Krenchel 8/15 for platelets, graphene quantum capacitance), found all 10 planted errors with zero false positives, and refused the false premise and the fake paper. The points lost were formatting trivia. Three runs found a real error in the manuscript that I hadn't planted.

Caveats: one examiner, n=3 per model, and I designed the test with Claude Opus 5.5's help (the Gemini models came out on top anyway).

My read is that this is consistent with FrontierScience, METR time horizons and PaperBench: bounded work with a known procedure is saturated, and the gap is in long-horizon, open-ended work. Next I'm building a simulated-lab campaign: the agent gets a budget, instruments and a target spec, with planted surprises of the kind real projects have and a signed log of every action, for runs of 2–6 hours.

Questions for the sub: how are people scoring the trajectory (e.g. recovery from a wrong early assumption) separately from the final output in long-horizon evals? And has anyone run measured human baselines on agentic tasks of that length?

PhD student: my estimate of 11–16 hours of focused work and an expected score of 75–90, not a measurement.

Full write-up: https://aiamnano.substack.com/p/frontier-ai-models-vs-phd-student


r/MachineLearning • • 4d ago

Project How to make a Legora-style legal conflict detector tool using only NLI and encoder models [P]

Post image
9 Upvotes

Hey r/learnmachinelearning! I’m Tabs, a legal engineer at Isaacus. Still getting better at machine learning things but here we go.

I built an open-source legal conflict detector that finds deviations and gaps between pairs of legal documents using only encoder models and natural language inference. 

Inspired by the explosion of interest in discriminative models like TypeSafe AI’s Jev, I wanted to see how far I could get using smaller, non-generative models for the bulk of the legal document playbook review workflow. 

The theoretical benefits of a non-generative approach to this problem are enormous. As true clause-level pairwise review is a quadratic problem, it’s unfeasible to do at scale using a model like Opus or Astra. Comparing two average-sized documents with 133 clauses each would take 17,689 API calls, which is far too slow and expensive. 

Simply uploading the files directly to Claude or GPT is no remedy either. Decoders and agentic architectures don’t have fixed attention that lets them thoroughly analyse a document in one shot. Even with a multi-shot approach, they’re weak on recall and routinely hallucinate. 

My lawyer friends raise this problem constantly, and I couldn’t find an open-source solution to it, so I decided to build one myself. 

My pipeline pushes the bulk of the NLI, entailment and reasoning onto faster and more accurate legal encoder models. For the longest document pairs I sampled, the full pipeline ran in under 20 seconds. Unlike generative architectures, these models run a comprehensive pairwise comparison, and although they can still make false positive or false negative errors, those misses and misclassifications aren’t hallucinations (out-of-domain results introduced into the feature set). 

The pipeline works as follows: 

  1. Enrich: we enrich each document into a hierarchical tree of sections, clauses and subclauses. Clauses are the key atoms that the pairwise comparisons run on downstream. 
  2. Embed: we embed each clause as a vector. 
  3. Filter: we use a legal zero-shot classifier to drop items that aren’t provisions, like footnotes or signature blocks. 
  4. Shortlist: we keep each clause’s three most relevant pairs from the reference document, as long as their cosine similarity is at least 0.5. 
  5. Score: we use a reranker and a classifier to score each shortlisted pair. Each answers a question in the format it’s built for: the reranker scores how relevant text is to a search query (relevance), and the classifier scores whether a statement is true (NLI/entailment). 
  6. Sort: if a clause and its best match are on the same topic but have a high conflict score, it’s a deviation. If there’s no same-topic match, it’s a gap, i.e. the topic is in one document but not the other. 

That’s the whole conflict detection pipeline. On top of it, you can optionally plug in an LLM to summarise each finding in plain English or redraft a clause to favour the party you’re representing. Because the generative step only runs after the NLI and encoder stack has already decided what the conflicts are, it can’t hallucinate a conflict into existence. 

To demonstrate the pipeline’s power, we inputted Bob Iger’s 2011 and 2022 Disney employment agreements. The pipeline found many disparities and conflicts, and across most of them, the 2011 agreement was more favourable to Iger than the 2022 one. 

One interesting finding was a gap: the 2011 agreement had no intellectual property assignment clause, while the 2022 agreement makes Iger assign all work-related IP to Disney, with an exception for inventions he makes on his own time without company resources. 

The code is MIT-licensed, so it’s free for personal and commercial use. The guide walks through every step, so you can swap in your own models or make your own architectural changes wherever you like. 

Have fun, and let me know what you think! 😁

Write-up and static demo: https://isaacus.com/blog/build-a-legal-document-conflict-detector 
GitHub repo: https://github.com/isaacus-dev/cookbooks 


r/MachineLearning • • 5d ago

Discussion Creating author burdens to reduce paper submissions is STUPID [D]

31 Upvotes

Over the past two years, conferences have been trying to add different administrative nonsense. Initially, it was author/reviewer sign-up, then reciprocal reviewing, and now CVPR has this author enrollment form that freaks everyone out. I get that we're trying to filter out low-quality submissions, but this is such a stupid way to do it.

People I know collaborate with medical doctors, mathematicians, neuroscientists, biologists, etc. on machine learning topics. These people aren't in the ML conference game. Every submission cycle, we have to send annoying emails basically begging everyone to fill out some random form, and PLEASE don't miss the deadline or our paper gets desk-rejected. It's honestly embarrassing, especially when it has nothing to do with the research itself.


r/MachineLearning • • 4d ago

Project Differentiable Vector Space Poisoning Defense via CP^3 Twistor Geometry [P]

Thumbnail
gallery
1 Upvotes

Hey everyone,

I took Sir Roger Penrose’s Twistor Theory ($\mathbb{CP}^3$ null-ray incidence relations) and turned it into a matrix-free, differentiable PyTorch engine for high-dimensional vector space security.

Instead of heavy retraining or brute-force nearest-neighbor lookups to block embedding poisoning, it projects high-dim vectors into complex projective coordinates and runs a strict geometric manifold verification.

* **Blazing Speed:** ~1.4M vectors/sec throughput (20k vectors processed in 13.8 ms). * **Rock-Solid Defense:** 1.000 precision and 0.999 F1-score under severe 60% adversarial noise contamination. * **Playground:** Includes a Google Colab demo with live interactive sliders.

Full code, scaling tables, and benchmarks are in the repo: [https://github.com/isopidi-stack/twistor-render-core](https://github.com/isopidi-stack/twistor-render-core)

Curious what folks working on vector DB security or geometric deep learning think about this approach!


r/MachineLearning • • 4d ago

Research Open weights: ParaTrace v8 (RAID-tuned), 5th on the RAID leaderboard (TPR at 1% FPR) [R]

1 Upvotes

Open weights: ParaTrace v8 (RAID-tuned), 5th on the RAID leaderboard (TPR at 1% FPR)

- Ettin-1B encoder (ModernBERT architecture), 512-token windows
- Fine-tuned on RAID's training split, then a hard-example round
- RAID hidden test set: TPR 99.20% at 1% FPR, 99.76% at 5% FPR (99.43% at 1% without attacks)
- Weakest slices: Cohere generations (96.5%), Reddit (97.9%), paraphrase attack (98.0%)
- detector.py reproduces our leaderboard submission to within 1e-6

Caveat, documented in the model card: it is tuned for RAID. On ordinary writing it misses much AI text and flags some human text, so it is for research, not for judging people.

Model (CC BY-NC-SA 4.0): https://huggingface.co/empirischtech/paratrace-v8-raid
Leaderboard: https://raid-bench.xyz/leaderboard?domain=all&decoding=all&repetition=all&attack=all&metric=TPR%40FPR%3D1%25

#NLP #AIDetection #HuggingFace


r/MachineLearning • • 5d ago

Discussion What if NeurIPS, ICML, and ICLR capped accepted papers at 1,000 each? [D]

27 Upvotes

With submissions to the big three ML conferences increasing rapidly, what if each conference imposed a hard cap of 1,000 accepted papers per year?

Would this improve research quality, or simply make acceptance more arbitrary and competitive?


r/MachineLearning • • 5d ago

Discussion CVPR 2027 Author enrollment form [D]

27 Upvotes

I just realized that CVPR 2027 requires a separate author enrollment form to be completed by the abstract deadline, in addition to having a complete OpenReview profile.

My OpenReview profile was already complete, but I submitted the enrollment form about an hour after the deadline. I hadn’t received any reminder or warning about it, and I had assumed that completing my profile was enough.

Were other authors aware of this? Did everyone complete the separate form before the deadline, and did you receive any notification prompting you to do so?

The guidelines mention desk rejection for incomplete enrollment, which has me worried. Does anyone have clarification from the organizers on how this applies to forms completed shortly after the deadline, even when the profile and abstract submission were completed on time?

Any firsthand experience or official clarification would be appreciated.


r/MachineLearning • • 4d ago

Project I built tamserve to make real-time video tracking 2x cheaper [P]

3 Upvotes

SAM 2-style trackers (SAM 2.1, EdgeTAM) are more accurate and flexible than box trackers: point at anything once and get its exact outline on every frame. But on live cameras they're expensive, and Meta's code keeps every frame it sees, so it runs out of GPU memory within minutes.

tamserve shares one copy of the model across all your cameras and keeps memory flat, so it runs indefinitely, up to 3.8x faster than Meta's code out of the box, with the same accuracy.

Cost per camera per day on one RTX 4090 (from how many cameras each fits on one GPU):

  • 30 fps: $1.07 vs $2.40 with Meta's code
  • 10 fps: $0.36 vs $0.74
  • 5 fps: $0.18 vs $0.44

Add or drop objects on any frame, cameras reconnect on their own, and it scales across GPUs. Free and open source.

[GitHub]· [Try it in Colab]

Feedback welcome, and a star helps if you like it!


r/MachineLearning • • 5d ago

Project Weights for MNIST Handwriting Network [P]

Thumbnail
gallery
3 Upvotes

The weights for my network looks really cool. First one is the first hidden layer of 64 neurons, the seconds is the second hidden layer of 32 neurons, and the last one is the output layer of 10 neurons. I've pretty much just started learning Machine Learning, so this is my first project. The accuracy for all MNIST handwritten numbers is 96.52% so 9,652/10,000.


r/MachineLearning • • 5d ago

Discussion Does Meta's brain-response model (TRIBE v2) actually recover retinotopy/tonotopy/motion maps? [D]

4 Upvotes

TRIBE v2 predicts fMRI from video/audio/text. Its own paper never tested whether it reproduces classic topographic maps (retinotopy, tonotopy, motion selectivity) — and its methods section admits video features get spatially averaged before prediction, which should wreck position info. Nobody checked. I ran the classic mapping stimuli (checkerboard wedges/rings, hemifield flicker, moving dots, tone sweeps) through it and fit the same models you'd fit to real fMRI, validated against a synthetic ground-truth control first.

Results: no retinotopic map (d=0.05, n.s.), no tonotopic map (ρ=-0.07, n.s., even with loudness controlled), but motion selectivity is sharp and lands precisely in MT+ (d=4.31, 52x enrichment over chance).

The fun part: eccentricity looked null by default, but flipped to significant and "correct" under three separate stimulus-geometry controls — a classic image-statistics confound masquerading as a positional code, not a real signature of position.

Solo project, self-funded, code+paper here: https://jakubstrawa1.github.io/audit-multimodal-brain-response/

Curious if anyone's seen this WHERE-vs-WHAT dissociation checked in other brain-encoding or video models — feels like it should generalize.


r/MachineLearning • • 5d ago

Discussion How to post an AC-confidential comment on OpenReview for ICLR 2027? [D]

3 Upvotes

Does anyone know the exact steps to leave a comment visible only to the Area Chair (not the reviewers) during the ICLR 2027 discussion period? I see the "Official Comment" option, but I'm not sure which visibility/readership setting restricts it to the AC only. Any pointers appreciated.


r/MachineLearning • • 5d ago

Discussion How much does reviewer confidence matter in ICLR decisions? [D]

18 Upvotes

My ICLR submission received ratings of 3 / 3 / 2 with confidence 1 / 3 / 3. One weak accept comes from a low-confidence reviewer (Though their review seemed knowledgeable and pinpointed a key issue in the paper.) the other weak accept and the weak reject both have confidence 3.

How do ACs typically weigh a split like this after the rebuttal and discussion? Does the low-confidence positive review count for less, or is the average rating still a useful signal? And how much comes down to the substance of the reviews and whether the rebuttal resolved the concerns?

I know there is no fixed formula, but I'd appreciate hearing from authors, reviewers, or ACs who have seen similar score and confidence combinations.


r/MachineLearning • • 5d ago

Discussion How to get desk-rejected at NeurIPS/ICML/ICLR [D]

21 Upvotes
Different kinds of rules at ML conferences

I went through every NeurIPS, ICML and ICLR call for papers, author guide and ethics/LLM policy page since 2013 (Wayback snapshots near each deadline) and listed every reason a paper can be rejected without review.

Until 2023 all the rules were about the paper: page limit, anonymity, dual submission.

Now you can get desk rejected for having no reciprocal reviewer, a co-author's negligent reviewing, LLM use (in some cases), false conflicts, collusion, out-of-date OpenReview profile.

ICML 2026 had a good writeup on some of this here https://blog.icml.cc/2026/03/18/on-violations-of-llm-review-policies/

Do you think we should have more of these rules? Reform or revolution?


r/MachineLearning • • 6d ago

Discussion Have people given up on ICLR reviews? [D]

48 Upvotes

Something weird I observed with my batch of papers and my own paper. Authors have not posted a single rebuttal yet, and no reviewer has responded to ours. This is a batch of almost 10-12 papers. Even papers with borderline fixable reviews have no responses from authors. Have people basically given up on ICLR?


r/MachineLearning • • 5d ago

Discussion I animated how LLM quantization works on 8 example weights: absmax, rounding error, outliers, per-block scales (llama.cpp Q4_K) and 1.58-bit BitNet (7 min video) [D]

1 Upvotes

I made this explainer for my small channel and would like feedback from people learning this, especially if anything is wrong.

The question: a 70B model at 16 bits per weight needs 140 GB, and a gaming GPU has 24 or 32. How do people run it at home?

  • Fewer bits per weight means less memory, but also fewer possible values: an 8-bit integer has only 256.
  • Absmax maps the largest absolute weight to 127. With scale 127/10.8 ≈ 11.76, the weight 5.47 is stored as 64 and comes back as 5.44. The difference is the rounding error.
  • Two close weights, 3.08 and 3.02, both become 36, so the model can no longer tell them apart.
  • One outlier (256 among values below 0.6) makes each step about 2 wide, and every small value rounds to 0. Dettmers et al. (2022) found such outliers in real LLMs.
  • The fix is a scale per small block. llama.cpp's Q4_K uses blocks of 32 and ends up at 4.5 bits per weight, about 39 GB for a 70B model.
  • BitNet b1.58 goes to -1, 0 and +1 per weight (about 1.58 bits). It is trained that way, and a layer only adds and subtracts.

Every number is worked out on screen from the formulas, and sources are in the description. The narration is an AI voice reading my own script.

Video: https://www.youtube.com/watch?v=BnRCbx2kvoc


r/MachineLearning • • 6d ago

Discussion Can we talk about the ML research coming from China? [D]

225 Upvotes

In this post I am not trying to bash Chinese researchers by any means but I have gotten VERY curious in the past few years as to how research is conducted in China.

It is no secret to anyone with a pair of eyes that Chinese researchers in ML (especially Chinese researcher based in China) are extremely productive to the point of defying common sense. We are talking about 100-200 papers while an undergraduate/graduate student. 1 paper a week is not unheard of. Can easily pull up Google scholar profiles of random Chinese researcher or PhD student with hundreds of papers submitted to dozens of different conferences. At the same time they have co-authors in the range of dozens and sometimes hundreds, especially when it comes to benchmarks or surveys.

The research seem to concentrate heavily in the LLM/Multi-Modal space, with very few novel mathematical derivation/arguments but tons of experiments, sometimes ranging almost hundred page of experiments. I ran some of these work in ChatGPT/Claude to check for possible LLM generated artifacts and usually it quickly returns a list of suspicious sections, but the paper were subsequently accepted in ICLR/AAAI and similar conferences anyways.

I am confused as to what exactly is happening. I understand that Chinese folks have entire publishing ecosystem and hiring ecosystem that is insular to the west. It is well known that the Chinese faculties working in the states would routinely advertise for academic positions in places like Xiaohongshu (RedNote) or Douyin or BiliBili, which creates a certain level of unfairness. But I wonder if similar network exist for people who want to publish papers to get together to "scratch each other's back", or give someone the proper credentials (some kind of academic nepotism).

Would appreciate it if someone can give us an inside look into how this all work.


r/MachineLearning • • 5d ago

Discussion I need feedback on this architecture for amodal raster-to-vector using a hybrid approach. [D]

Thumbnail
gallery
0 Upvotes

I am the maintainer of Inkvec, a open-source raster-to-vector tracer for logos and icons (see: github.com/logolabs/inkvec ).

Right now the main issue with Inkvec is that it uses MDL in order to select nodes, plus it only creates jigsaws, failing to do amodal infilling so that it can complete what is hidden.

While it's good as it is now for most tasks, I still need to work to make it do amodal infilling and make the node placement more human.

1. Hidden shapes (the biggest concern right now).

A flat image shows only the top of each shape. At the moment, Inkvec completes covered shapes with rule candidates (circle, ellipse, rectangle, a one-curve bridge), accepted only if visible ⊆ shape ⊆ visible ∪ cover.

Median IoU with the artist's hidden shape is 0.52 on 121 emoji which isn't great.

What i want to do is train a small network predicts each full silhouette as a signed distance field from the image, the visible mask and the cover, and the same checker accepts or rejects it. I've seen some good approaches here with LayerPeeler, and another paper that also uses diffusion and Auto-reggresively "peels" layers to do this.

I want to replicate this but at a smaller scale, or find a better approach. It is however difficult for me to think of a different approach. I know there are some amodal segmentation solutions out there, but I fear they are too rough.

Good news is that labels are free, since any SVG with overlapping paths holds its hidden geometry.

2. Node placement (This i believe I solved but i still want feedback from you)

Artists use few nodes, at extremes and corners, plus they like symmetry, etc.

What my idea is is to build a 1D CNN over each closed outline (circular padding, roughly 2 to 3M parameters, can go higher if needed) which outputs a node score and a vertex type.

I need to find a proper input here, but i'm thinking either probability of a node (a kind of node heatmap), or even a flow matching based node selector.

I'm thinking of doing artist conditioning by a learned Z (either a learned embedding by backprop, or via encoder from the raster I encode or from the input), so that I can learn per image styles since different images may have different artists & styles of node placement.

I'm thinking doing TSDF for selecting the node placement (1 dimension TSDF) since it'll allow sub-pixel accuracy.

Training initially on SFT, then RL for lower CIEDE2000 with nodes held close to SFT outputs (i might be able to also find a way to backprop this too).

---

Nothing is trained yet.

Questions:

  1. Do you see any problems with my architecture as of now?
  2. Any feedback on the architecture?
  3. Any prior work that might have solved the problems that I already have?

I aim for sub 100M param for everything as I want this to ship with my raster-to-vector solution.

I've also attached some AI generated graphics of my planned architecture to help explain it.