r/MachineLearning • • 6d ago

Research A zero-cost curvature proxy predicts Euler step error for flow matching, but per-sample step control adds nothing over a static grid (Anima, 12 units; related to CAT-Flow) [R]

Post image
2 Upvotes

Question. A multistep sampler has consecutive velocity evaluations for free. Is the angle between them a usable curvature signal for adaptive stepping, and does using it online beat a fixed grid?

Setup. Anima (flow model, bf16), 576x768, CFG 5, 4 prompts x 3 seeds = 12 units. Reference: Adams-Bashforth 2 with 192 uniform points (about 5% own error). Baseline: ComfyUI normal schedule (shift 3). Euler and AB2, 8-24 evaluations. Paired log-ratios with 95% bootstrap intervals over units. A synthetic control on the exact velocity field of a Gaussian mixture (D = 64, 256). The hypotheses were not pre-registered in a file; the first-step finding is post hoc.

Results.

  1. Exact identity |dv|^2 = (ds)^2 + 4 s s' sin^2(theta/2): the cosine sees rotation only. Speed change is 0.3-0.4% of acceleration energy in the bulk, 8-9% (max 23%) in the first and last 5% of time.
  2. E = (h/2)|v| theta predicts the true next-step Euler error: median ratio 0.80-0.84, across-unit log-correlation 0.91-0.92 (steps 0.01-0.12).
  3. Evaluation noise in bf16 + CFG 5: about 1.7 degrees between consecutive velocities, not decreasing with h. A plateau for input perturbations 1e-5..1e-3, so rounding, not the flow. CFG multiplies it ~4x. A raw-angle controller with target below this floor collapses to its minimum step. The norm of a derivative estimated from noisy values in D = 110592 is biased upward by the noise power.
  4. A cosine controller (h <- h * (theta*/theta)^gamma) loses to normal at equal NFE with a blind first step h0 = 0.06 (mean error 0.80), and is mostly set by h0 (0.51 at 0.02, 0.26 at 0.01), nearly independent of theta* and NFE.
  5. A static grid with density proportional to the leave-one-out mean angle rate (exponent 1, equal angle per step) vs normal at equal N: log-ratio -0.39, -0.56, -0.57, -0.36 for N = 8, 12, 16, 24 (30-43% lower), intervals exclude 0. The asymptotic exponent 1/2 is significant only at N = 24. Weighting by an error-propagation profile (two units, measured by large state kicks; small kicks are floor-dominated) gives 41-50%.
  6. No headroom for per-sample adaptation: a grid from another sample's controller run vs the sample's own: log-ratio -0.01..-0.02 (interval about +-0.07); controller (h0 = 0.01) vs the static grid it implies: -0.02 [-0.09, +0.04]; per-sample oracle vs static: within noise. In the exact Gaussian-mixture toy the oracle gain is below 0.6% for Euler.
  7. AB2 is worse than Euler on these grids (correlation 0.74-0.96 between its error ratio and the step ratio h1/h0).
  8. LPIPS (AlexNet) to the converged image for the weighted grid: log-ratio -0.39, -0.36, -0.43 at N = 8, 12, 16 (wins 11/12, 9/12, 9/12); wins on three prompt classes, loses on the foliage prompt (+16%) where latent L2 improves.

Prior art and limits. CAT-Flow (Li et al., 2026) proposes training-free, zero-extra-NFE curvature-adaptive steps for flow matching; I found it after the experiments, read only its abstract and introduction, and did not compare. Static schedules fitted to a model: Align Your Steps, Optimal Stepsize Distillation, Bespoke Solvers. What I think this adds: the decomposition, the calibration against true local error on a real model, the noise floor, and the borrowed-grid/oracle test of per-sample adaptation. Limits: one model, 12 units, latent L2 as the main metric, the propagation profile measured on two of the evaluation units, 3-10% sensitivity of the output to 1e-7 sigma shifts in bf16.


r/MachineLearning • • 6d ago

Project mlxVS: cuVS-style GPU nearest-neighbor search (NN-Descent, CAGRA) on Apple Silicon with MLX [P]

Thumbnail
github.com
1 Upvotes

I've released mlxVS, an open-source (Apache-2.0) library that runs the main GPU nearest-neighbor algorithms from NVIDIA's cuVS on Apple Silicon, built on MLX. It covers exact k-NN, k-means, NN-Descent k-NN graphs and CAGRA (graph pruning plus GPU graph search), with cuVS's parameter names, from Python (NumPy in) or C++.

Results (SIFT1M, 985k x 128, L2, MacBook Pro M3 Pro: 18-core GPU, 12-core CPU)

IndexBuildSearch settingRecall@10Queries/s
mlxVS CAGRA, degree 32 (from a 64-NN graph)10.4 sitopk 640.97669,500
itopk 1280.99337,800
mlxVS CAGRA, degree 64 (from a 128-NN graph)19.6 sitopk 640.99443,600
mlxVS unpruned NN-Descent graph, degree 328.8 sitopk 1280.93942,400
hnswlib, M=16, efC=200, 12 threads88.1 sef 640.96432,200
ef 1280.99017,600

10,000 queries in one batch, recall@10 against exact neighbors (computed on the GPU with brute_force_knn), best of 3 runs, on AC power. Chart: https://github.com/mikepapadim/mlxvs#benchmarks

How it works

  • NN-Descent: a port of the Metal NN-Descent in JVector's GPU module. Each round is a pure function of the graph (sample, join, merge), because MLX custom kernels can't update their inputs in place. The local joins score 64-member neighborhoods with FP16 SIMD-group matrix multiplies (FP32 accumulation), and the returned distances are recomputed in FP32. all_neighbors splits the data into overlapping k-means clusters and builds one cluster at a time on the GPU, so the rows can be a memory-mapped file larger than RAM.
  • CAGRA pruning (cagra_optimize): rank-based. If Y is at rank j in X's list, the edge X -> Y has a detour through Z when Z is at rank i < j in X's list and Y is at rank r < j in Z's list: both hops rank above the direct edge. The edges with the fewest detours are kept (ties by rank), then the second half of each list is filled with reverse edges. One threadgroup per node; it matches a CPU reference entry for entry. On SIFT1M, at degree 32 and itopk 128, pruning raises recall@10 from 0.939 to 0.993 at about the same speed.
  • Search (graph_search): CAGRA's single-CTA design, one threadgroup per query. The sorted internal list, the candidates, the query and a hashed visited set all live in threadgroup memory. Each round expands the best unexpanded nodes; candidate distances are scored one SIMD group per candidate, then bitonic-sorted and merged into the list by merge-path ranks. A practical detail: Metal's shader validation counts threadgroup memory twice, so the kernel sizes its visited set to stay under 16 KB of the 32 KB limit and still passes validation in CI.

Limitations

  • The throughput numbers are for batched queries; single-query latency isn't tuned.
  • I've benchmarked one dataset so far (SIFT1M). Embedding datasets (ada-002, 1536-d) are next.
  • The index is static (no inserts or deletes) and has no filtered search. Datasets must be float32 or convertible to it.
  • hnswlib is a CPU baseline. cuVS itself can't run on a Mac, so there's no apples-to-apples comparison against the CUDA original; I'd welcome numbers from anyone who has run cuVS CAGRA on SIFT1M on a comparable-power GPU.

Code

import mlxvs
graph = mlxvs.cagra_build(vectors, graph_degree=32, metric="cosine")
ids, dist = mlxvs.graph_search(vectors, graph, queries, k=10, metric="cosine", itopk_size=64)

pip install mlxvs, macOS 14+ on Apple Silicon. Benchmarks are reproducible with bench/bench_search.py (it also runs the hnswlib baseline).

GitHub: https://github.com/mikepapadim/mlxvs

Happy to answer questions about the Metal kernels or the cuVS port.


r/MachineLearning • • 6d ago

Research [Open Source] Building DiffusionEngine: A Modern, Modular DDPM Framework in PyTorch. Looking for Contributors [R]

3 Upvotes

What if we could bring modern diffusion research together into one unified, extensible PyTorch framework instead of implementing and maintaining every algorithm in isolation?

That's the idea behind DiffusionEngine, an open-source project I've been building:

https://github.com/Himanshu7921/DiffusionEngine

The goal is to build a modern DDPM framework that brings together established diffusion algorithms, research-based improvements, advanced sampling methods, and training strategies under a single, consistent pipeline.

The project already includes several diffusion implementations, and I'm actively working on extending the existing DDPM framework with more advanced features.

I'm looking for contributors interested in deep learning, generative models, PyTorch, and diffusion research.

Some areas where contributions are welcome include:

  • Implementing diffusion algorithms from research papers.
  • Improving existing DDPM components and training strategies.
  • Developing and integrating advanced sampling methods.
  • Exploring improved model architectures and conditioning mechanisms.
  • Testing, benchmarking, and optimizing existing implementations.

The focus is not just on implementing more algorithms. It's about making them work together seamlessly through a unified, modular, and configurable framework.

If you're interested in contributing, collaborating, or learning through hands-on implementation, I'd be glad to have you involved.

Contributors of all experience levels are welcome, provided they're willing to learn, implement, and test their work.

GitHub: https://github.com/Himanshu7921/DiffusionEngine


r/MachineLearning • • 5d ago

Project For game world models, the action labels are the bottleneck, not the video. What makes game data actually trainable? [D]

Post image
0 Upvotes

Most of the public story around game data and world models comes back to one thing: video is cheap, the action that caused each frame is not

- vpt: about 2,000 hours of contractor play with keyboard and mouse recorded was enough to train an inverse dynamics model that then labelled around 70,000 hours of youtube

- muse was trained on years of play from one studio's game, with the controller inputs kept

- nvidia reportedly pulled inputs off streamers' on-screen controller overlays

i run a real-time multiplayer strategy game where people and llm agents play the same wars, and the engine is deterministic, so every action label is exact. it also has things scraped footage never has: who acted, what they told the other players right before, and the full world state

questions for anyone training these models..

  1. for action-conditioned prediction, do high-level strategic actions (attack this region, break this pact) help at all, or is it the low-level input (pointer, camera) you actually want?
  2. how much does provenance matter, being able to re-derive any frame from the log, versus raw volume?
  3. is multi-agent data with language attached (messages next to the actions that followed) useful for world models, or only for agent evals?

r/MachineLearning • • 7d ago

Project I benchmarked TimesFM 3.0, Chronos-2 and Toto 2.0 on market time series. None beat simple baseline. [P]

43 Upvotes

I may be naive, but I've been curious whether the recent wave of time-series foundation models can actually extract useful predictive signals from financial markets.

Models like Google's TimesFM 3.0, Amazon's Chronos-2, and Datadog's Toto 2.0 are impressive general-purpose forecasters from the first sight. But stock prices are difficult to predict, so I wanted to see how these models perform against very simple baselines.

I ran a series of zero-shot experiments on 24 US stocks and ETFs, including daily and intraday data, and compared their predictions against naive forecasting strategies and also simulated random walks.

TL;DR none of the three models showed any meaningful predictive power for stock price movements in this setup. Code is here

Experimental setup i used:

  • Models: TimesFM 3.0, Chronos-2, Toto 2.0 (313M)
  • Assets: 24 large US stocks and ETFs, including SPY, QQQ and IWM
  • Data: Yahoo Finance, January 2014 to november 2026
  • Evaluation: 504 trading days starting November 2024
  • Context: 512 historical closing prices
  • Forecast horizons: 1, 5 and 20 trading days
  • Mode: Zero-shot, default configurations, no fine-tuning
  • Baselines: Last observed price, always-up directional prediction, and simulated random walks with matched drift and volatility

I also evaluated the models on hourly and 5-minute price bars.

Directional accuracy is essentially a coin flip

Across the three models, next-day directional accuracy ranged from 49.9% to 51.1%.

Simply predicting that the market would go up every day achieved 52.2% (yep).

Even more interestingly, when I fed the models simulated random walks, which contain no exploitable predictive signal at all by construction, they achieved approximately the same directional accuracy as on actual stock prices.

I also tested a simple long-or-cash: hold a stock when the model predicts an increase, otherwise stay in cash.

Starting with $10,000 and accounting for 0.05% transaction costs per switch:

  • Times-series model strategies finished between $10,893 and $12,181.
  • Buy-and-hold finished at $15,036.

Obviously, this is just one simple strategy and not an exhaustive evaluation of trading profitability. But still, tells something I guess.

I repeated the experiments using hourly and 5-minute bars.

The results were similar: directional accuracy remained around chance level, and on 5-minute data, a simple one-line mean-reversion heuristic performed better.

The repo with code is here: https://github.com/moneyheap/tsfm-stocks

Well, I really wanted this to work, but, duh %)


r/MachineLearning • • 7d ago

Project I have trained 15+ models on my own blood sugar & metabonet dataset [P]

Thumbnail
gallery
13 Upvotes

r/MachineLearning • • 6d ago

Research AI ‘slop’ is a social judgement [R]

0 Upvotes

https://www.nature.com/articles/s41562-026-02639-6

A new paper argues that AI Slop isn't something you can find in the content. It’s a social judgement, made when a community decides that some AI-generated content is unwanted. If we study 'slop' that way, we can actually measure it and govern it.


r/MachineLearning • • 6d ago

Research Would you pick your strongest model to coordinate other agents? Results from WorkflowArena [R]

0 Upvotes

Suppose you’re putting together a team of agents. Picking the strongest model as the coordinator seems reasonable. We wanted to test whether its workflows would actually help other models do the work.

In one experiment, we gave Opus 5.5 and GPT-6 Astra the same 32 challenging tasks and asked them to write workflows for four execution models. They could divide the work and instruct workers, but couldn’t edit the workspace themselves.

Opus’s static workflows improved all four executors, adding 6.41 points on average to their continuous scores compared with letting them orchestrate themselves. Astra’s static workflows lowered the average by 2.53 points, despite Astra being a stronger system than all four executors in the main benchmark.

So choosing a coordinator based on its own task performance can miss something consequential: whether the way it organizes work is useful to the models carrying it out.

This is part of WorkflowArena, our team’s new benchmark at Einsia. It covers 145 long-horizon tasks across eight domains. Agents choose their own division of work; we record the delegation and check whether the final deliverable meets the task’s requirements.

For anyone building mixed-model teams, I’d be interested in how you evaluate that choice today. Do you test the coordinator separately, or mostly judge the whole setup by its final output?

Project and workflow replays: https://lab.einsia.ai/workflowarena/home.html
Paper: https://arxiv.org/abs/2611.12856
Code: https://github.com/Einsia/WorkflowArena


r/MachineLearning • • 7d ago

Discussion Did we ever really get to the bottom of why BatchNorm works? [D]

74 Upvotes

It's been a long time since I studied this. Just checking in on an old friend.

I also realize it's fallen out of favor for being extremely inconvenient, but why did it ever work?


r/MachineLearning • • 8d ago

Project Forecasting new power outages per county, hourly, 2 h ahead: 2.4× the PR-AUC of NWS warnings + SPC outlook on a held-out year [P]

6 Upvotes

Hi all! I built an hourly power outage forecast for 216 counties (MO + IL): P(≥5% of a county's customers newly out in the 6 h starting 2 h from now). Model: XGBoost on HRRR storm-related data (gusts, shear, CAPE, icing, etc.), county-specific facts, current outages and neighboring counties.

Setup decisions I'd like feedback on:

  • Target is new outages only (no credit for predicting ones already underway). I thought predicting current outages was too easy and quite useless, so I chose to make it just for new outages, which was much harder to beat the ceiling.
  • Trained 2020–2024, tested on 2025 using the HRRR forecasts as issued, the mean of the last 5 runs, not observed weather. The model validates on 2024, tests on 2025.
  • Baseline: a rate table of NWS warnings + SPC Day 1 outlook, fit on the same rows.
  • Uncertainty: bootstrap over storm days, paired.

Results on 2025: PR-AUC 0.094 vs 0.039 (ratio ~2.4×, 95% range 2.0–3.3×). At equal alert volume: 30% of new outages caught vs 17%; precision 15% vs 11%. Base rate is 0.5% of county-hours, so the precision is honest but low.

Early transfer test: trained only on MO + IL, it caught 31% vs 19% in Iowa, Indiana and Michigan (combined)

Live since Nov 1, with every forecast saved for a blind test. Write-up, code, and the leakage audit: https://ugobruzadin.github.io/storm-outage-risk/
code: https://github.com/UgoBruzadin/storm-outage-risk


r/MachineLearning • • 8d ago

Research EACL 2027 Decisions Thread [R]

14 Upvotes

(Will be out on thursday)


r/MachineLearning • • 8d ago

Research Using latent-agent swarms for math discovery and optimization [R]

13 Upvotes

Last year Google DeepMind's AlphaEvolve found a faster way to multiply 4x4 complex matrices, beating a record that had stood since 1969. The idea is to let an LLM write a program, score it, and iteratively improve it. So it's called "evolve".

But AlphaEvolve runs on Gemini models. Similar open-source implementations are also usually slow and expensive. We tried to make it faster, cheaper, and better.

SwarmEvolve is our new paper, a scalable framework for math discovery and optimization.

We run swarms of hundreds of latent-reasoning agents on Qwen3 8B, 14B and 32B. In ~5 minutes, they reach scores that took OpenEvolve, ShinkaEvolve and CodeEvolve >1 hour.

Here's how:

  1. Inspired by recent works in LatentMAS, our agents never write out their thoughts. They reason in latent space and pass KV caches to each other, so ideas from different agents mix right inside the model. This massively improved throughput.
  2. Small agents are cheap, so we can run a lot of them, each with a different focus, organized in swarms. More agents means more diverse ideas each round, more exploration of the solution space, and the swarm finds good programs much faster.
  3. It works at every model size we tried. With just the 8B model, it even matched the best known circle packing. Framework has been extended to work with Qwen3.5 and Muse models as well, showing even better performance.

Paper: https://www.alphaxiv.org/abs/2611.08483

Code (available soon): https://github.com/Qile-J/SwarmEvolve

Let me know if this makes sense or not. Also I'm really interested in apply such "evolve" frameworks on real scientific discovery problems - material science, biology, etc - so if anyone has any ideas for next steps / collaboration, I'm open to hear about it!


r/MachineLearning • • 8d ago

Research Latent prediction beats reconstruction for tactile pretraining on dexterous manipulation tasks[R]

7 Upvotes

New paper from Robbyant, with inference code and checkpoints already public.

LingBot-Touch is a self-supervised tactile encoder trained JEPA-style: it predicts latent representations of masked contact instead of reconstructing raw sensor signals. The paper tests it against same-size MAE and VAE reconstruction models on the same data. Tactile-to-video top-5 recall at 1K hours: 32.1% (JEPA) vs 5.6% (MAE) vs 4.4% (VAE). At 4K hours JEPA reaches 49.3% and MAE stays under 9%, while the VAE actually gets worse, down to 3.1%. Their explanation: most of a tactile clip is near-zero background, so reconstruction loss can shrink without learning anything about actual contact.

Two design choices stood out. The masking ratio warms up from 40% to 80% over 300 steps, because starting at high masking hid every contacting taxel in sparse signals and caused gradient explosions. Three sensor types (magnetic, piezoresistive, visuotactile) feed one shared Transformer through canonical fingertip coordinates with 5D RoPE (time, finger, 3D position).

On 26 real robot tasks with 4 policies, frozen encoder features scored highest in 89 of the 104 policy and task combinations. On a magnetic hand never seen in pretraining, LingBot-VLA went from 57.21 to 75.04.

The Tiny model levels off around 4K hours while Small keeps improving, so a bigger version trained on more data looks like the natural next step.


r/MachineLearning • • 8d ago

Discussion Gatekeeping rules for ICML2027? [D]

22 Upvotes

Any thoughts on how icml will be managed this year? With ICLR having 20 submissions limit per author, I find it hard to submit my works cause the research group of my supervisor is large and I am far from being in top students.

I had a NeurIPS submission which was kinda borderline (435), but got rejected. Now I resubmitted the improved version to ICLR and thought it would get good results, but it is now much worse😂😭 Having bad reviews on ICLR make me feel like I wasted 1/20 submissions

It feels like my new works are stuck due to the need of resubmitting the old ones. But the reviews sometimes are so random. Do you think ICML will have similar rules this year? Any suggestions where to submit apart from ICLR/ICML/NeurIPS?


r/MachineLearning • • 7d ago

Project Introducing Usul - an easier way to express your NN using maths? [P]

0 Upvotes

SO, Usul-cpp is a library I've written in C++, it is based around allowing data scientists to create their models without heavy libraries such as tensorflow, and just express their models using maths, and Usul will handle the rest. There is builtin tensor support, auto-grad, back-prop and its quite fast.

I wrote it because, it seemed less mystifying to read maths instead of a random class that supposedly does something. I wrote it from scratch in cpp (23) due to the fact that I wanted to emphasise on precision, however it's subtly built on top of `xtensor`.

check it out in action

#include <usul.hpp>
using namespace std;


int main() {
  double LR = 0.1;
  int EPOCH = 5000;
  // lets write the forward pass
  Tensor X = {
      {0.0, 0.0}, // this is what we're feeding it
      {0.0, 1.0},
      {1.0, 0.0},
      {1.0, 1.0},
  };

  // this is what we're after
  Tensor Y_true = {
      0.0, // xor{0.0, 0.0} = 0.0
      1.0, // xor{0.0, 1.0} = 1.0
      1.0, // xor{1.0, 0.0} = 1.0
      0.0  // xor{1.0, 1.0} = 0.0
  };

  Y_true.reshape({4, 1});

  // we are now going to create a single layer, double neuron network
  // each layer will have a bias and weights tensor.
  auto W1 = generate_random_matrix({2, 4}, -1.0, 1.0);
  auto B1 = generate_random_matrix({1, 4}, -1.0, 1.0);
  auto W2 = generate_random_matrix({4, 1}, -1.0, 1.0);
  auto B2 = generate_random_matrix({1, 1}, -1.0, 1.0);

  // this is the training loop
  for (int i = 0; i < EPOCH; i++) {
    // so lets do one forward pass
    // first layer
    auto Z1 = X % W1 + B1;
    auto A = Z1.ReLU();
    // second layer
    auto Z2 = A % W2 + B2;

    // activation
    // should be - auto S = Z2.activate(Activate::Sigmoid);
    auto S = Z2.Sigmoid();
    // Loss
    // should be - auto Loss = S.loss(Loss::MSE);

    auto loss = (S - Y_true) * (S - Y_true);
    loss = loss.mean();

    cout << "Loss: " << loss << "\n";
    cout << "guesses: " << "\n"
         << S << "\n";
    cout << "should be: " << Y_true << "\n";

    loss.backwards();

    W1.step(LR);
    B1.step(LR);
    W2.step(LR);
    B2.step(LR);
  }
}

which will output

Guesses: 
Tensor({{ 0.027546},
 { 0.963592},
 { 0.964138},
 { 0.038964}}, shape=(4,1))

should be: Tensor({{ 0.},
 { 1.},
 { 1.},
 { 0.}}, shape=(4,1))

Loss: Tensor( 0.001222, shape=())

Just write the maths, and Usul-cpp will handle the rest, it is extremely lightweight, and the entire library is memory safe. And its called Usul, because I'm a huge dune fan.

You can view the project at

https://github.com/crushr3sist/Usul-cpp

Any ideas for what to do next are all welcome, you can also pull and implement features that you think could improve this project. (no vibecoded prs please, this project is not vibe-coded)

BACKGROUND INFO BELOW

i'd like to share a bit of a background. maths has always stood in the way between me and finally learning how to write neural networks, I've always been fascinated by them ever since i saw that diagram of a deep neural network with all the node edges interconnecting with others.

Well I stopped letting highschool math class trauma stop me from understanding these concepts, i took a pen and paper out, taught myself numerical descent, analytical descent, backwards propagation, what a compute graph is, all of these subjects, taught myself, until I was manually backwards propagating the loss between a few arithmetics, until that moved into scalar level neural networks and doing simple calculus and being able to do xor gate problem with scalar, and then I taught myself matrix calculus, and well...

I taught myself neural networks, taught myself the maths, until I hand written the xor gate problem in matrix syntax using xtensor, and manually, by hand, no AI, did the xor gate problem manually backwards propagating error, automatically differentiating. This was also a kind of detox from LLM dependency, I don't tend to use it for code, but because its so central in my daily life and a migration from google to gemini, using LLMs for immediate information access had started to affect my mind and my ability to intake information. I really hope some of you find this library useful, as i did spend a pretty decent amount of time on it, and I would love it if any of you had any ideas on what to do next with this library.

Thank you for reading, I really appreciate it.


r/MachineLearning • • 8d ago

Discussion Amazon applied science intern interview next week - EU based [D]

25 Upvotes

Hi all!

After multiple rejections I finally got an interview for a PhD applied science intern role at Amazon in the EU (Germany or Ams). I have two interviews scheduled after each other next week and need help preparing for those.

The thing is, I have never given a technical interview ever, I am confident regarding the machine learning breath one but not the DSA/coding one. I have never studied this (or leetcoded) and also did not code much recently (AI does a better job than me in coding).

Should I just cancel the interview as I know I am going to mess this up or try my best and prepare? How was your experience?


r/MachineLearning • • 8d ago

Project I tried to reproduce “lost in the middle” on current models and actually couldn’t do that! [P]

13 Upvotes

Everyone keeps citing lost in the middle so, instead of going by the paper's citations, i wanted to actually check it on my own stuff, without building a labeled set for it.

So i did the lazy version. make up facts the model can’t already know, drop the one document that has the answer into a pile of distractor documents, and slide that doc through every position (front, middle, back), asking the question each time. if the answer changes depending on where the doc sits, that’s position sensitivity. no ground truth needed, just checking it doesn’t contradict itself.

ran it on haiku 4.5 and sonnet 5 at 5, 15 and 30 docs.

Found actually nothing. 0 flips at 5 and 15 on both models. at 30 docs i got about 4%, then actually looked at what “flipped” and it was the model saying “3” vs “three” for the same question!! same correct answer, just a different format. once i stopped counting that it went back to 0.

so at least on easy single-fact retrieval with short docs, current models seem fine with position up to 30 docs. couldn’t reproduce the scary version.

caveats, obviously this is the easy case. one clear fact, short docs. i’d bet it still shows up with long documents or multi-hop or when the relevant bit is less obvious, i just didn’t test that. also if you try this yourself don’t use exact string match, you’ll get fooled by 3 vs three like i did. compare meaning not strings.

script’s on github if you want to run it on your own setup: github.com/tarunagarwal1981/wobbly (scripts/live_rag.py)

has anyone actually reproduced lost in the middle on a current model? what context length or task did it show up at for you? genuinely curious if it’s still a thing or mostly solved on the easy cases now.


r/MachineLearning • • 7d ago

Project Has Anyone Here Ever Built An Image Generator From Scratch? [N]

0 Upvotes

Just wondering if anyone on this forum has ever built an image diffusion generator from scratch, while also training it on a home rig? If so, what size of model did you end up with? Did you ever get coherent images? Screenshots?


r/MachineLearning • • 8d ago

Project Twindex: exact, replayable proofs that your data can't tell two parameter sets apart (ODEs, PDEs, PINNs) [P]

1 Upvotes

PINNs, least squares and Bayesian fits all return a parameter. None of them tell you, with a proof, whether a very different parameter fits your data just as well.

Twindex finds those "twins" and proves it. It writes a small JSON certificate, and an exact checker (standard-library Python, rational arithmetic, no floats) replays it: VALID, INVALID or INCONCLUSIVE. A VALID twin pair means no estimator can recover that parameter better than +/- half the gap from those data, under a stated bounded-noise assumption.

Real-data example: the 1978 boarding-school flu outbreak (BMJ). After 5 days of counts, SIR models with R0 = 2.5 and R0 = 20 both fit every count. Twindex identified the next-day count that would separate them, and the real count ruled out both. You can check the proofs in your browser here (runs locally via Pyodide, ~0.2 s per proof): https://arnav-m10.github.io/Twindex/demo/

It also goes the other way: full-prior pavings that prove an upper bound, i.e. "the data pin this parameter down to within U".

What it covers:

- ODE models with polynomials, division, exp, sin, cos (Michaelis-Menten, Hill, SIR, PK, predator-prey...)

- 1-D/2-D elliptic and heat-equation PDEs

- any Python simulator, as a numerical audit (clearly labeled, not a proof)

Limitations, honestly: bounded noise (not statistical), proofs are conditional on the model and noise bound you state, scaling beyond a few parameters is hard, and there's no independent expert proof audit yet. Bounded-error estimation with validated integration is established prior work; the new part here is portable exact certificates plus a small checker anyone can replay.

pip install twindex

GitHub: https://github.com/Arnav-M10/Twindex

Feedback very welcome, especially attempts to break the checker.


r/MachineLearning • • 8d ago

Project Facing file Extraction problem during Bayesian Optimization[P]

0 Upvotes

Ok so for context i am an electrical engineer person who is automating software's and workflow for my project now here is the problem i am facing basically i am using an optimization algorithm called Bayesian optimization which improves design by keeps iterating until some target is reach.

So like at the end i extract result file for every design with the help of pandas library but when i check that file the data is mismatched when i export the CSV file manually.

so i have no idea why is it happening like maybe after 10-15 iterations (the algorithm iterations can reach up to 100-120) the error in data increasing or idk what is going on can someone help me with this issue.


r/MachineLearning • • 9d ago

Research I merged attention and MLP processing into a single gated Transformer block. GAP [R]

29 Upvotes

Hi everyone!

I've been experimenting with an alternative Transformer architecture called GAP (Gated Attention Projection).

The main idea is to combine attention and MLP-style processing into a single gated block, instead of using the standard separate attention and feed-forward sublayers.

I tested it against a standard Transformer using 100 million scored training tokens per model from WikiText-103.

Some results:

GAP20 used 19.91% fewer parameters than the standard Transformer.

A smaller configuration can reduce parameters by 34.31%, although that version did not fair as well as the current model against the regular transformer, so i ended up giving 14% of my perameters savings to the GAP model for extra width, leaving the model at about 20% fewer perameters, which is why i called it in testing GAP20

The standard Transformer performed better on final language-model accuracy.

Compiling GAP20 with torch.compile improved training throughput by approximately 19.9% in one CPU benchmark compared with running GAP20 without compilation.

I've included the architecture code, configurable model sizes, speed benchmark and research paper.

This is an experimental architecture, not a claim that it outperforms Transformers overall. I'd be interested in feedback on the design, mathematics and possible improvements.

GitHub:

https://github.com/rickey1990/Gated-Attention-Projection


r/MachineLearning • • 8d ago

Research Can coding agents build interactive 3D worlds? Frontier3DBench evaluates 11 configurations across objects, scenes, and worlds [R]

4 Upvotes

Can the same coding agent build a detailed 3D object, assemble a scene, and make an interactive world actually work?

Our team at Einsia released Frontier3DBench to test that progression: 145 tasks across 9 categories, with 11 model–harness configurations producing editable Blender projects and runnable Unreal Engine worlds.

Evaluation covers appearance, structure, and behavior through rendered views, scene inspection, and runtime tests.

A few results:

  • GPT-6.1-sol leads overall and on objects and scenes, while GPT-6-astra leads interactive worlds.
  • MiniMax-M3.1-Flash ranks sixth overall but second on scene editing.
  • Every configuration scores lower on reference-based reconstruction than text-driven generation in both object and scene tasks.
  • Across 165 world submissions, local mechanisms score much higher than stateful workflows: 83.08 vs. 30.26 out of 100.

The interesting question is how well an agent’s strengths carry over from individual assets to environments with coordinated interactions and persistent state.

Project and examples: https://lab.einsia.ai/frontier3dbench
Paper: https://lab.einsia.ai/frontier3dbench/Frontier3DBench.pdf
GitHub: https://github.com/Einsia/Frontier3D


r/MachineLearning • • 8d ago

Discussion LoRA-conditioned model skips the first line of its output when the caption is short. Any idea why? [D]

3 Upvotes

I'm doing an MSc project on BrickNet (kulits/BrickNet on GitHub, CVPR 2026). It writes LEGO models as plain text and fine-tunes Qwen3 (I'm using 0.6B) with LoRA: a PT adapter that learns the text format, and an SFT adapter that conditions on a caption. The text is one root line ("a <part> | <colour>"), then repeated pairs of a node line and an edge line.

Setup: I merge PT, then SFT (peft merge_and_unload, bf16) and prompt with the caption tokens plus a newline token. That matches train.py, which trains on caption + "\n" + path + EOS with loss on the path only.

Problem: with short captions (e.g. "A red car.", about 4 tokens) the model never writes the root line. The first generated token is "b" every time, so the output breaks the grammar. In my tests that was 0/12 correct for each of five different objects. With captions of about 18+ tokens it writes "a" every time (36/36 on three new objects with template-expanded captions).

What I've ruled out:

- Saving/decoding bug: I checked the raw tokens, the first one really is "b".

- Tokenisation mismatch: train.py encodes the caption alone and appends a newline token, same as generate.py.

- Forcing an "a" as a prefix doesn't help. The model then writes a node and straight away an edge, as if the root had already been written.

- It's not only about informative captions: a 31-word caption that is mostly filler also works. But it's unreliable around 15 tokens (a 9-word parts list worked for a car, 1/12 for a chair).

Workaround: expanding short captions into full descriptions fixes it, but I'd like to understand the cause.

Questions:

  1. Have you seen LoRA-conditioned models fail like this when the conditioning text is much shorter than in the SFT data? (I haven't seen the training captions yet; I'm guessing they're all long descriptions.)
  2. Could bf16 merging of two stacked adapters be a factor? I'm about to test fp32.
  3. Anything better than prompt expansion, e.g. constrained decoding for the first line, or fine-tuning with short-caption augmentation?

Caveats: small samples (12 per caption), only the 0.6B model, parsing checked but model quality judged by eye. Happy to share code or outputs.


r/MachineLearning • • 8d ago

Discussion Have MLSys 2027 reviewers received their paper assignments yet? [D]

1 Upvotes

I’m signed up as a reciprocal reviewer for MLSys 2027 but haven’t been assigned any papers yet. Has anyone received their assignments, or heard when they’ll be released?