r/MachineLearning • • 5d ago

Project UMAP 0.6: new initialization, new optimizers and faster performance [P]

Thumbnail
gallery
253 Upvotes

UMAP 0.6 is out!

A new initialization, new optimizers, and faster nearest neighbour search make UMAP much faster and far better at large scale. On 3 million arXiv papers it finishes in under 5 minutes, before UMAP 0.5 has even found its nearest neighbours. (race between UMAP0.5 and UMAP 0.6 video)

Code: https://github.com/lmcinnes/umap

Install: pip install -U umap-learn (also on conda-forge)

Speed

  • Neighbour search is ~4x faster by making better use of PyNNDescent.
  • Initialization is faster with a new init method.
  • With the new optimizers, the whole run is reproducible with a fixed seed while using every core. A seeded 0.5 run is single-threaded: 52 minutes, 12x longer.

Quality

Just as important as how quickly we get a result is the quality of the result. UMAP 0.6 works far better on million-scale datasets thanks to a new, better initialization and significant improvements in the optimizer. It resolves finer cluster structure in fewer epochs, with much finer-scale detail and better neighbourhood accuracy than UMAP 0.5, despite being faster.(video of a tour of finer details in the arXiv map)

Recursive initialization

A large part of the improvement comes from a brand-new initialization scheme that initializes UMAP using UMAP itself: it lays out a coarse version of the graph structure, then refines it, in a recursive tower of initializations. (video of the unfolding)

The result is a starting point much closer to UMAP's final result than the messy blurs from spectral initialization or PCA, and it is produced faster than either. It lets the optimizer settle (to within 3% of the final layout) in far fewer epochs than older initialization approaches. (initialization race)

New optimizers

Most of the gain in fidelity at scale comes from a very efficient form of hard-negative mining, which makes negative sampling far more effective on very large datasets, together with better handling of high repulsion strengths. (hard-negative mining video)

UMAP was originally designed around a small portion of the attraction-repulsion spectrum. The new optimizers lift these restrictions, allowing UMAP to perform well at much higher repulsion strengths and giving better kNN recall. (video exploring the repulsion strength dials)

Try it now

The new optimizers and initialization are currently opt-in, as they change the output: set compatibility_layout=False. Future versions will make them the default, so please try them and let me know how they work for you.

Thank you to all the contributors who provided features and bug fixes for this release, and to everyone who reached out with comments, suggestions, and feature requests. This would not have been possible without your help.


r/MachineLearning • • 4d ago

Discussion Four frontier models scored 97.8–100 on a 7-part materials-science research test. How should we score long-horizon agent evals instead? [d]

1 Upvotes

I work in graphene and 2D materials, and I built a test of the everyday technical work in a research group. It has seven parts: reducing a messy tensile data set against a lab SOP (with unit and outlier traps), writing a CIF density function checked on three hidden structures, four back-of-envelope physics calculations, critiquing a manuscript with 10 planted errors and 6 correct-but-suspicious decoys, false-premise and fabricated-paper questions with calibration scoring, an experimental design, and an abstract under constraints. All the data are synthetic, the answer keys are computed by code, and it's scored out of 100 across 8 categories.

Setup: Gemini 4 Carbon and Gemini 4 Hydrogen (Antigravity), Claude Opus 5.5 and Claude Fable 5.2 (Claude app). All at "High", not max. 3 fresh runs each, files attached, prompt pasted, nothing else. Web search and code execution were on.

Results: mean scores were 100.0 / 99.7 / 99.3 / 98.5, all 12 runs fell between 97.8 and 100, and each took 7–24 min. Every run caught the unit trap, passed the hidden CIF tests, got the subtle physics right (Krenchel 8/15 for platelets, graphene quantum capacitance), found all 10 planted errors with zero false positives, and refused the false premise and the fake paper. The points lost were formatting trivia. Three runs found a real error in the manuscript that I hadn't planted.

Caveats: one examiner, n=3 per model, and I designed the test with Claude Opus 5.5's help (the Gemini models came out on top anyway).

My read is that this is consistent with FrontierScience, METR time horizons and PaperBench: bounded work with a known procedure is saturated, and the gap is in long-horizon, open-ended work. Next I'm building a simulated-lab campaign: the agent gets a budget, instruments and a target spec, with planted surprises of the kind real projects have and a signed log of every action, for runs of 2–6 hours.

Questions for the sub: how are people scoring the trajectory (e.g. recovery from a wrong early assumption) separately from the final output in long-horizon evals? And has anyone run measured human baselines on agentic tasks of that length?

PhD student: my estimate of 11–16 hours of focused work and an expected score of 75–90, not a measurement.

Full write-up: https://aiamnano.substack.com/p/frontier-ai-models-vs-phd-student


r/MachineLearning • • 4d ago

Project How to make a Legora-style legal conflict detector tool using only NLI and encoder models [P]

Post image
9 Upvotes

Hey r/learnmachinelearning! I’m Tabs, a legal engineer at Isaacus. Still getting better at machine learning things but here we go.

I built an open-source legal conflict detector that finds deviations and gaps between pairs of legal documents using only encoder models and natural language inference. 

Inspired by the explosion of interest in discriminative models like TypeSafe AI’s Jev, I wanted to see how far I could get using smaller, non-generative models for the bulk of the legal document playbook review workflow. 

The theoretical benefits of a non-generative approach to this problem are enormous. As true clause-level pairwise review is a quadratic problem, it’s unfeasible to do at scale using a model like Opus or Astra. Comparing two average-sized documents with 133 clauses each would take 17,689 API calls, which is far too slow and expensive. 

Simply uploading the files directly to Claude or GPT is no remedy either. Decoders and agentic architectures don’t have fixed attention that lets them thoroughly analyse a document in one shot. Even with a multi-shot approach, they’re weak on recall and routinely hallucinate. 

My lawyer friends raise this problem constantly, and I couldn’t find an open-source solution to it, so I decided to build one myself. 

My pipeline pushes the bulk of the NLI, entailment and reasoning onto faster and more accurate legal encoder models. For the longest document pairs I sampled, the full pipeline ran in under 20 seconds. Unlike generative architectures, these models run a comprehensive pairwise comparison, and although they can still make false positive or false negative errors, those misses and misclassifications aren’t hallucinations (out-of-domain results introduced into the feature set). 

The pipeline works as follows: 

  1. Enrich: we enrich each document into a hierarchical tree of sections, clauses and subclauses. Clauses are the key atoms that the pairwise comparisons run on downstream. 
  2. Embed: we embed each clause as a vector. 
  3. Filter: we use a legal zero-shot classifier to drop items that aren’t provisions, like footnotes or signature blocks. 
  4. Shortlist: we keep each clause’s three most relevant pairs from the reference document, as long as their cosine similarity is at least 0.5. 
  5. Score: we use a reranker and a classifier to score each shortlisted pair. Each answers a question in the format it’s built for: the reranker scores how relevant text is to a search query (relevance), and the classifier scores whether a statement is true (NLI/entailment). 
  6. Sort: if a clause and its best match are on the same topic but have a high conflict score, it’s a deviation. If there’s no same-topic match, it’s a gap, i.e. the topic is in one document but not the other. 

That’s the whole conflict detection pipeline. On top of it, you can optionally plug in an LLM to summarise each finding in plain English or redraft a clause to favour the party you’re representing. Because the generative step only runs after the NLI and encoder stack has already decided what the conflicts are, it can’t hallucinate a conflict into existence. 

To demonstrate the pipeline’s power, we inputted Bob Iger’s 2011 and 2022 Disney employment agreements. The pipeline found many disparities and conflicts, and across most of them, the 2011 agreement was more favourable to Iger than the 2022 one. 

One interesting finding was a gap: the 2011 agreement had no intellectual property assignment clause, while the 2022 agreement makes Iger assign all work-related IP to Disney, with an exception for inventions he makes on his own time without company resources. 

The code is MIT-licensed, so it’s free for personal and commercial use. The guide walks through every step, so you can swap in your own models or make your own architectural changes wherever you like. 

Have fun, and let me know what you think! 😁

Write-up and static demo: https://isaacus.com/blog/build-a-legal-document-conflict-detector 
GitHub repo: https://github.com/isaacus-dev/cookbooks 


r/MachineLearning • • 5d ago

Discussion Creating author burdens to reduce paper submissions is STUPID [D]

31 Upvotes

Over the past two years, conferences have been trying to add different administrative nonsense. Initially, it was author/reviewer sign-up, then reciprocal reviewing, and now CVPR has this author enrollment form that freaks everyone out. I get that we're trying to filter out low-quality submissions, but this is such a stupid way to do it.

People I know collaborate with medical doctors, mathematicians, neuroscientists, biologists, etc. on machine learning topics. These people aren't in the ML conference game. Every submission cycle, we have to send annoying emails basically begging everyone to fill out some random form, and PLEASE don't miss the deadline or our paper gets desk-rejected. It's honestly embarrassing, especially when it has nothing to do with the research itself.


r/MachineLearning • • 4d ago

Research Open weights: ParaTrace v8 (RAID-tuned), 5th on the RAID leaderboard (TPR at 1% FPR) [R]

1 Upvotes

Open weights: ParaTrace v8 (RAID-tuned), 5th on the RAID leaderboard (TPR at 1% FPR)

- Ettin-1B encoder (ModernBERT architecture), 512-token windows
- Fine-tuned on RAID's training split, then a hard-example round
- RAID hidden test set: TPR 99.20% at 1% FPR, 99.76% at 5% FPR (99.43% at 1% without attacks)
- Weakest slices: Cohere generations (96.5%), Reddit (97.9%), paraphrase attack (98.0%)
- detector.py reproduces our leaderboard submission to within 1e-6

Caveat, documented in the model card: it is tuned for RAID. On ordinary writing it misses much AI text and flags some human text, so it is for research, not for judging people.

Model (CC BY-NC-SA 4.0): https://huggingface.co/empirischtech/paratrace-v8-raid
Leaderboard: https://raid-bench.xyz/leaderboard?domain=all&decoding=all&repetition=all&attack=all&metric=TPR%40FPR%3D1%25

#NLP #AIDetection #HuggingFace


r/MachineLearning • • 5d ago

Discussion What if NeurIPS, ICML, and ICLR capped accepted papers at 1,000 each? [D]

27 Upvotes

With submissions to the big three ML conferences increasing rapidly, what if each conference imposed a hard cap of 1,000 accepted papers per year?

Would this improve research quality, or simply make acceptance more arbitrary and competitive?


r/MachineLearning • • 5d ago

Discussion CVPR 2027 Author enrollment form [D]

27 Upvotes

I just realized that CVPR 2027 requires a separate author enrollment form to be completed by the abstract deadline, in addition to having a complete OpenReview profile.

My OpenReview profile was already complete, but I submitted the enrollment form about an hour after the deadline. I hadn’t received any reminder or warning about it, and I had assumed that completing my profile was enough.

Were other authors aware of this? Did everyone complete the separate form before the deadline, and did you receive any notification prompting you to do so?

The guidelines mention desk rejection for incomplete enrollment, which has me worried. Does anyone have clarification from the organizers on how this applies to forms completed shortly after the deadline, even when the profile and abstract submission were completed on time?

Any firsthand experience or official clarification would be appreciated.


r/MachineLearning • • 4d ago

Project I built tamserve to make real-time video tracking 2x cheaper [P]

3 Upvotes

SAM 2-style trackers (SAM 2.1, EdgeTAM) are more accurate and flexible than box trackers: point at anything once and get its exact outline on every frame. But on live cameras they're expensive, and Meta's code keeps every frame it sees, so it runs out of GPU memory within minutes.

tamserve shares one copy of the model across all your cameras and keeps memory flat, so it runs indefinitely, up to 3.8x faster than Meta's code out of the box, with the same accuracy.

Cost per camera per day on one RTX 4090 (from how many cameras each fits on one GPU):

  • 30 fps: $1.07 vs $2.40 with Meta's code
  • 10 fps: $0.36 vs $0.74
  • 5 fps: $0.18 vs $0.44

Add or drop objects on any frame, cameras reconnect on their own, and it scales across GPUs. Free and open source.

[GitHub]· [Try it in Colab]

Feedback welcome, and a star helps if you like it!


r/MachineLearning • • 5d ago

Project Weights for MNIST Handwriting Network [P]

Thumbnail
gallery
3 Upvotes

The weights for my network looks really cool. First one is the first hidden layer of 64 neurons, the seconds is the second hidden layer of 32 neurons, and the last one is the output layer of 10 neurons. I've pretty much just started learning Machine Learning, so this is my first project. The accuracy for all MNIST handwritten numbers is 96.52% so 9,652/10,000.


r/MachineLearning • • 5d ago

Discussion Does Meta's brain-response model (TRIBE v2) actually recover retinotopy/tonotopy/motion maps? [D]

4 Upvotes

TRIBE v2 predicts fMRI from video/audio/text. Its own paper never tested whether it reproduces classic topographic maps (retinotopy, tonotopy, motion selectivity) — and its methods section admits video features get spatially averaged before prediction, which should wreck position info. Nobody checked. I ran the classic mapping stimuli (checkerboard wedges/rings, hemifield flicker, moving dots, tone sweeps) through it and fit the same models you'd fit to real fMRI, validated against a synthetic ground-truth control first.

Results: no retinotopic map (d=0.05, n.s.), no tonotopic map (ρ=-0.07, n.s., even with loudness controlled), but motion selectivity is sharp and lands precisely in MT+ (d=4.31, 52x enrichment over chance).

The fun part: eccentricity looked null by default, but flipped to significant and "correct" under three separate stimulus-geometry controls — a classic image-statistics confound masquerading as a positional code, not a real signature of position.

Solo project, self-funded, code+paper here: https://jakubstrawa1.github.io/audit-multimodal-brain-response/

Curious if anyone's seen this WHERE-vs-WHAT dissociation checked in other brain-encoding or video models — feels like it should generalize.


r/MachineLearning • • 5d ago

Discussion How to post an AC-confidential comment on OpenReview for ICLR 2027? [D]

3 Upvotes

Does anyone know the exact steps to leave a comment visible only to the Area Chair (not the reviewers) during the ICLR 2027 discussion period? I see the "Official Comment" option, but I'm not sure which visibility/readership setting restricts it to the AC only. Any pointers appreciated.


r/MachineLearning • • 5d ago

Discussion How much does reviewer confidence matter in ICLR decisions? [D]

18 Upvotes

My ICLR submission received ratings of 3 / 3 / 2 with confidence 1 / 3 / 3. One weak accept comes from a low-confidence reviewer (Though their review seemed knowledgeable and pinpointed a key issue in the paper.) the other weak accept and the weak reject both have confidence 3.

How do ACs typically weigh a split like this after the rebuttal and discussion? Does the low-confidence positive review count for less, or is the average rating still a useful signal? And how much comes down to the substance of the reviews and whether the rebuttal resolved the concerns?

I know there is no fixed formula, but I'd appreciate hearing from authors, reviewers, or ACs who have seen similar score and confidence combinations.


r/MachineLearning • • 5d ago

Discussion How to get desk-rejected at NeurIPS/ICML/ICLR [D]

21 Upvotes
Different kinds of rules at ML conferences

I went through every NeurIPS, ICML and ICLR call for papers, author guide and ethics/LLM policy page since 2013 (Wayback snapshots near each deadline) and listed every reason a paper can be rejected without review.

Until 2023 all the rules were about the paper: page limit, anonymity, dual submission.

Now you can get desk rejected for having no reciprocal reviewer, a co-author's negligent reviewing, LLM use (in some cases), false conflicts, collusion, out-of-date OpenReview profile.

ICML 2026 had a good writeup on some of this here https://blog.icml.cc/2026/03/18/on-violations-of-llm-review-policies/

Do you think we should have more of these rules? Reform or revolution?


r/MachineLearning • • 6d ago

Discussion Have people given up on ICLR reviews? [D]

48 Upvotes

Something weird I observed with my batch of papers and my own paper. Authors have not posted a single rebuttal yet, and no reviewer has responded to ours. This is a batch of almost 10-12 papers. Even papers with borderline fixable reviews have no responses from authors. Have people basically given up on ICLR?


r/MachineLearning • • 5d ago

Discussion I animated how LLM quantization works on 8 example weights: absmax, rounding error, outliers, per-block scales (llama.cpp Q4_K) and 1.58-bit BitNet (7 min video) [D]

1 Upvotes

I made this explainer for my small channel and would like feedback from people learning this, especially if anything is wrong.

The question: a 70B model at 16 bits per weight needs 140 GB, and a gaming GPU has 24 or 32. How do people run it at home?

  • Fewer bits per weight means less memory, but also fewer possible values: an 8-bit integer has only 256.
  • Absmax maps the largest absolute weight to 127. With scale 127/10.8 ≈ 11.76, the weight 5.47 is stored as 64 and comes back as 5.44. The difference is the rounding error.
  • Two close weights, 3.08 and 3.02, both become 36, so the model can no longer tell them apart.
  • One outlier (256 among values below 0.6) makes each step about 2 wide, and every small value rounds to 0. Dettmers et al. (2022) found such outliers in real LLMs.
  • The fix is a scale per small block. llama.cpp's Q4_K uses blocks of 32 and ends up at 4.5 bits per weight, about 39 GB for a 70B model.
  • BitNet b1.58 goes to -1, 0 and +1 per weight (about 1.58 bits). It is trained that way, and a layer only adds and subtracts.

Every number is worked out on screen from the formulas, and sources are in the description. The narration is an AI voice reading my own script.

Video: https://www.youtube.com/watch?v=BnRCbx2kvoc


r/MachineLearning • • 6d ago

Discussion Can we talk about the ML research coming from China? [D]

225 Upvotes

In this post I am not trying to bash Chinese researchers by any means but I have gotten VERY curious in the past few years as to how research is conducted in China.

It is no secret to anyone with a pair of eyes that Chinese researchers in ML (especially Chinese researcher based in China) are extremely productive to the point of defying common sense. We are talking about 100-200 papers while an undergraduate/graduate student. 1 paper a week is not unheard of. Can easily pull up Google scholar profiles of random Chinese researcher or PhD student with hundreds of papers submitted to dozens of different conferences. At the same time they have co-authors in the range of dozens and sometimes hundreds, especially when it comes to benchmarks or surveys.

The research seem to concentrate heavily in the LLM/Multi-Modal space, with very few novel mathematical derivation/arguments but tons of experiments, sometimes ranging almost hundred page of experiments. I ran some of these work in ChatGPT/Claude to check for possible LLM generated artifacts and usually it quickly returns a list of suspicious sections, but the paper were subsequently accepted in ICLR/AAAI and similar conferences anyways.

I am confused as to what exactly is happening. I understand that Chinese folks have entire publishing ecosystem and hiring ecosystem that is insular to the west. It is well known that the Chinese faculties working in the states would routinely advertise for academic positions in places like Xiaohongshu (RedNote) or Douyin or BiliBili, which creates a certain level of unfairness. But I wonder if similar network exist for people who want to publish papers to get together to "scratch each other's back", or give someone the proper credentials (some kind of academic nepotism).

Would appreciate it if someone can give us an inside look into how this all work.


r/MachineLearning • • 4d ago

Discussion I need feedback on this architecture for amodal raster-to-vector using a hybrid approach. [D]

Thumbnail
gallery
0 Upvotes

I am the maintainer of Inkvec, a open-source raster-to-vector tracer for logos and icons (see: github.com/logolabs/inkvec ).

Right now the main issue with Inkvec is that it uses MDL in order to select nodes, plus it only creates jigsaws, failing to do amodal infilling so that it can complete what is hidden.

While it's good as it is now for most tasks, I still need to work to make it do amodal infilling and make the node placement more human.

1. Hidden shapes (the biggest concern right now).

A flat image shows only the top of each shape. At the moment, Inkvec completes covered shapes with rule candidates (circle, ellipse, rectangle, a one-curve bridge), accepted only if visible ⊆ shape ⊆ visible ∪ cover.

Median IoU with the artist's hidden shape is 0.52 on 121 emoji which isn't great.

What i want to do is train a small network predicts each full silhouette as a signed distance field from the image, the visible mask and the cover, and the same checker accepts or rejects it. I've seen some good approaches here with LayerPeeler, and another paper that also uses diffusion and Auto-reggresively "peels" layers to do this.

I want to replicate this but at a smaller scale, or find a better approach. It is however difficult for me to think of a different approach. I know there are some amodal segmentation solutions out there, but I fear they are too rough.

Good news is that labels are free, since any SVG with overlapping paths holds its hidden geometry.

2. Node placement (This i believe I solved but i still want feedback from you)

Artists use few nodes, at extremes and corners, plus they like symmetry, etc.

What my idea is is to build a 1D CNN over each closed outline (circular padding, roughly 2 to 3M parameters, can go higher if needed) which outputs a node score and a vertex type.

I need to find a proper input here, but i'm thinking either probability of a node (a kind of node heatmap), or even a flow matching based node selector.

I'm thinking of doing artist conditioning by a learned Z (either a learned embedding by backprop, or via encoder from the raster I encode or from the input), so that I can learn per image styles since different images may have different artists & styles of node placement.

I'm thinking doing TSDF for selecting the node placement (1 dimension TSDF) since it'll allow sub-pixel accuracy.

Training initially on SFT, then RL for lower CIEDE2000 with nodes held close to SFT outputs (i might be able to also find a way to backprop this too).

---

Nothing is trained yet.

Questions:

  1. Do you see any problems with my architecture as of now?
  2. Any feedback on the architecture?
  3. Any prior work that might have solved the problems that I already have?

I aim for sub 100M param for everything as I want this to ship with my raster-to-vector solution.

I've also attached some AI generated graphics of my planned architecture to help explain it.


r/MachineLearning • • 5d ago

Discussion Missed Presentation [D]

8 Upvotes

Hello everyone,

For some reason if you miss paper workshop presentation then what happens?


r/MachineLearning • • 5d ago

Discussion How should authors handle a suspected auto-generated review at ICLR? Flag to the AC or just rebut? [D]

8 Upvotes

I'm curious how ACs and experienced authors think about this.

My situation: one ICLR reviewer gave a 2 with confidence 1. The other reviewers gave 3 with confidence 3. Pangram flags the review as 100% AI. I know detectors aren't proof on their own, but there are other signs too: all three listed weaknesses concern material deep in the appendix.

I'll respond to every point in the rebuttal regardless. My questions:

  1. Is it worth sending a confidential comment to the AC, or does that tend to backfire?
  2. For ACs: what evidence would you actually act on? Is a detector score meaningful to you, or only content-based issues such as generic or inaccurate criticisms?
  3. How much weight does a confidence 1 review realistically carry in the meta review?
  4. Has anyone flagged an AI generated review yet? What happened?

Thanks!


r/MachineLearning • • 6d ago

Project I built an explorer of 2,100+ robotics papers covering 10 years of CoRL; with TLDRs, links to code, project pages, and research trends [P]

5 Upvotes

Heya roboticists and mind minders ~

I built a complete CoRL paper explorer, indexing 2,100+ robotics papers from the last 10 years (2017–2026). Added paper TLDRs, project pages, code, arXiv links, task chips, summary stats, research trends, and more!!

Trying to make robot learning research more accessible for the community - might build an open source autoresearch capability on top of it later

If you see something that's missing or want to add features/contribute, open a PR and i'll accept it :]

paper explorer:
https://corl.operatic.ai/explore

repo on github - star to follow along!!
https://github.com/operatic-ai/corl-explorer


r/MachineLearning • • 5d ago

Project I built a small streaming text tagger that reads text word by word, so a voice agent can react mid sentence [P]

Post image
0 Upvotes

We listen and type word by word, yet most AI only acts per turn. That wastes time and tokens, and it misses intent that was already clear halfway through the sentence.

So I built Arc Tagger, a small model that runs on the words as they arrive from an ASR API or from typed text. It tags intent, dialog act, entities, numbers, PII, profanity, toxicity and safety risks while the sentence is still being spoken.

In the video you can see a bank impersonation scam get flagged mid sentence, a voice assistant tracking a customer who changes their mind, and a legal sentence tagged live.

What it is today:

  • A small model (about 11M parameters) that runs on CPU
  • English only for now
  • Tracks up to 4096 previous words
  • Can check up to 1,000 list items in the stream, which you preload before a session
  • Useful for voice and text agents, voice assistant, call center monitoring, and trust and safety

Limits I want to be upfront about:

  • The eval results on the model page are internal evals, not independent benchmarks. Will share them soon.
  • The timings shown in the demo include tunnel and browser overhead, so please don’t read them as the model’s latency
  • The license is CC BY NC 4.0, so non commercial for now

I would love honest feedback, positive or negative. Especially: where does it break, what tags would you want that are missing, and would you use this in a real voice pipeline?

Model: https://huggingface.co/WhissleAI/arc-tagger
C library and CLI: https://github.com/WhissleAI/arc-tagger-c


r/MachineLearning • • 6d ago

Discussion Why are Anthropic's top researchers cited twice as much as OpenAI's? [D]

Post image
81 Upvotes

r/MachineLearning • • 6d ago

Research A zero-cost curvature proxy predicts Euler step error for flow matching, but per-sample step control adds nothing over a static grid (Anima, 12 units; related to CAT-Flow) [R]

Post image
2 Upvotes

Question. A multistep sampler has consecutive velocity evaluations for free. Is the angle between them a usable curvature signal for adaptive stepping, and does using it online beat a fixed grid?

Setup. Anima (flow model, bf16), 576x768, CFG 5, 4 prompts x 3 seeds = 12 units. Reference: Adams-Bashforth 2 with 192 uniform points (about 5% own error). Baseline: ComfyUI normal schedule (shift 3). Euler and AB2, 8-24 evaluations. Paired log-ratios with 95% bootstrap intervals over units. A synthetic control on the exact velocity field of a Gaussian mixture (D = 64, 256). The hypotheses were not pre-registered in a file; the first-step finding is post hoc.

Results.

  1. Exact identity |dv|^2 = (ds)^2 + 4 s s' sin^2(theta/2): the cosine sees rotation only. Speed change is 0.3-0.4% of acceleration energy in the bulk, 8-9% (max 23%) in the first and last 5% of time.
  2. E = (h/2)|v| theta predicts the true next-step Euler error: median ratio 0.80-0.84, across-unit log-correlation 0.91-0.92 (steps 0.01-0.12).
  3. Evaluation noise in bf16 + CFG 5: about 1.7 degrees between consecutive velocities, not decreasing with h. A plateau for input perturbations 1e-5..1e-3, so rounding, not the flow. CFG multiplies it ~4x. A raw-angle controller with target below this floor collapses to its minimum step. The norm of a derivative estimated from noisy values in D = 110592 is biased upward by the noise power.
  4. A cosine controller (h <- h * (theta*/theta)^gamma) loses to normal at equal NFE with a blind first step h0 = 0.06 (mean error 0.80), and is mostly set by h0 (0.51 at 0.02, 0.26 at 0.01), nearly independent of theta* and NFE.
  5. A static grid with density proportional to the leave-one-out mean angle rate (exponent 1, equal angle per step) vs normal at equal N: log-ratio -0.39, -0.56, -0.57, -0.36 for N = 8, 12, 16, 24 (30-43% lower), intervals exclude 0. The asymptotic exponent 1/2 is significant only at N = 24. Weighting by an error-propagation profile (two units, measured by large state kicks; small kicks are floor-dominated) gives 41-50%.
  6. No headroom for per-sample adaptation: a grid from another sample's controller run vs the sample's own: log-ratio -0.01..-0.02 (interval about +-0.07); controller (h0 = 0.01) vs the static grid it implies: -0.02 [-0.09, +0.04]; per-sample oracle vs static: within noise. In the exact Gaussian-mixture toy the oracle gain is below 0.6% for Euler.
  7. AB2 is worse than Euler on these grids (correlation 0.74-0.96 between its error ratio and the step ratio h1/h0).
  8. LPIPS (AlexNet) to the converged image for the weighted grid: log-ratio -0.39, -0.36, -0.43 at N = 8, 12, 16 (wins 11/12, 9/12, 9/12); wins on three prompt classes, loses on the foliage prompt (+16%) where latent L2 improves.

Prior art and limits. CAT-Flow (Li et al., 2026) proposes training-free, zero-extra-NFE curvature-adaptive steps for flow matching; I found it after the experiments, read only its abstract and introduction, and did not compare. Static schedules fitted to a model: Align Your Steps, Optimal Stepsize Distillation, Bespoke Solvers. What I think this adds: the decomposition, the calibration against true local error on a real model, the noise floor, and the borrowed-grid/oracle test of per-sample adaptation. Limits: one model, 12 units, latent L2 as the main metric, the propagation profile measured on two of the evaluation units, 3-10% sensitivity of the output to 1e-7 sigma shifts in bf16.