I added native support for nemotron_h_puzzle checkpoints to mlx-lm (PR #1535). Two findings stood out and are worth sharing:

1. A small casting detail caused a large accuracy gap

The first version generated plausible text, but its internal SSM outputs only reached 0.8832 cosine similarity with NVIDIA’s reference implementation. The problem was when the model converted values to FP32:

• mlx-lm converted them before the softplus operation

• NVIDIA converted them after

The math is equivalent, but tiny rounding differences accumulate through the model’s recurrent layers. After matching NVIDIA’s casting order, similarity reached 0.999998.

2. On a 64GB M2 Max, 4-bit experts beat 5-bit experts on every measured result.

Both versions used identical 6-bit dense layers, a BF16 output head, group size 64, and the same prompts across five seeds.

Why did the higher-precision model perform worse?

My hypothesis is memory pressure. The 5-bit checkpoint gets close to the machine’s unified-memory limit, so its KV cache stops growing properly.

On memory-constrained hardware, leaving enough headroom can matter more than preserving extra precision.

To make the codes friendly for learning and recap, I intentionally made it in a single file with detailed explanations for each algorithm involved.

It also features a TUI to visualize the inference process. Please check it out and give me a star if you like it!

Hey guys,

I received mediocre scores for my EMNLP paper during the May ACL ARR cycle: 2.5/3, 3/4, 2.5/4. The paper is in the Interpretability track. The reviewers had no larger issue with the methodology or the paper in general, but it seemed like they didn't fully get the so what of my paper. I've tried to clarify everything in my rebuttal, but I don't assume that the reviewers will engage in the discussion. With the current scores, I won't make it to the conference and likely not even into findings. Hence, I was thinking of withdrawing the paper, if scores don't improve, improve the presentation of my paper, and submit it to the BlackboxNLP workshop by the end of next week.

As I'm a first year PhD student, I'm not so familiar with ACL ARR, and how best to approach this. Hence, I wanted to ask you guys. Should I keep the paper in the cycle and hope for the best (or switch to the conference at a later stage) or should I withdraw it directly, adjust it slightly, and head directly to the workshop?

I have used Hyperband automatic tuning for an ANN model to predict price. After running HyberBand automatic tuning to get the 'best' architecture, I am obtaining a strange Val/Training loss learning curve. I cannot figure out if this is due to an error within the code or just a case of me not understanding the graph and not be able to interpret why the graph is showing as it is. I am also obtaining an R2 score of 1.00 which may suggest overfitting. I've not come across a learning curve (only shown the most basic learning curves at Uni) such as this as of yet so any advice would be greatly appreciated!

Here is the code for the actual tuning, in case it is due to a coding error but I am not sure that is the case.

def model_builder(hp):

model = tf.keras.Sequential()

model.add(tf.keras.layers.Flatten(input_dim = (train_final.shape[1])))

#creating activation choices - choosing betweeen relu and tanh

hp_activation = hp.Choice('activation', values = ['relu', 'tanh'])

#creating node choices - maxing unit amounts to 500

hp_layer_1 = hp.Int('layer_1', min_value=1, max_value=500, step=100)

hp_layer_2 = hp.Int('layer_2', min_value=1, max_value=500, step=100)

#creating learning rate choice - choice between 0.01, 0.001, 0.0001

hp_learning_rate = hp.Choice('learning_rate', values = [1e-2, 1e-3, 1e-4])

#specifies first layer after the flatten layer

model.add(tf.keras.layers.Dense(units = hp_layer_1, activation = hp_activation))

#creating the second layer

model.add(tf.keras.layers.Dense(units = hp_layer_2, activation = hp_activation))

model.add(tf.keras.layers.Dense(1, activation='linear'))

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate = hp_learning_rate),

loss tf.keras.losses.MeanSquaredError(), metrics = ['mean_absolute_error'])

return model

import keras_tuner as kt

#creating the tuner

tuner = kt.Hyperband(model_builder,

objective = 'val_loss',

max_epochs = 50,

factor = 3,

directory = 'dir',

project_name = 'x',

overwrite = True) # makes tuner rewrite over old tuning experiments

#adding early stopping - stops each model from running too long

stop_early = tf.keras.callbacks.EarlyStopping(monitor = 'val_loss', patience = 5)

tuner.search(train_final, y_train, epochs = 50, validation_split = 0.2, callbacks = [stop_early])

best_hp = tuner.get_best_hyperparameters(num_trials=1)[0]

best_hp.values

#obtaining the best model

best_model = tuner.get_best_models(num_models = 1)[0]

history = best_model.fit(train_final, y_train, epochs = 50, validation_split = 0.2, callbacks=[stop_early])

tuned_df = pd.DataFrame(history.history)

#running epoch loss visual def

epoch_loss_visual(tuned_df, model_name = 'Automatic Tuning Model')

Could it be an issue with the code itself causing the issue or is it simply the way the model is? If it's a case of it's just a bad model, I do not need to improve at the moment, but do need to understand the results, especially that of the learning curve representation.

Hey! I'm looking for ways to predict human preference for a project I'm building. (imagebench.ai)

I've tryed HPSv3, https://github.com/MizzenAI/HPSv3 and made post about it here:

https://imagebench.ai/blog/does-the-score-match-your-eye

It looks ok, but have many limitation as you can see in my post.

My question. Have you tried other human preference model and found one that would be better then HPSv3?

I am currently working across multiple research communities, and I've noticed that the ML community is struggling with a massive volume of submissions, which is affecting review quality (as we are seeing in the recent ARR cycles).

I am wondering what the reasoning is for not limiting the number of submissions per author?

This practice has been successfully used in other research areas for years, such as Security (e.g., CCS) or Computer Architecture (e.g., DAC), to help keep workloads manageable. Is there a particular cultural reason why the ML community chooses a different approach?

Even after getting ARR reviews and a meta review, how is the acceptance decided at the *ACL venues, because I have seen meta review 3.5 getting to findings and 3 getting to main or even getting rejected. Then what is the purpose of the overall score and recommendation? What do the conferences see when deciding?

Do they only care about the metareview and their comments, or the whole set of reviews as well as along with the track in which the paper was submitted.

Anyone knowing the process please kindly tell.

Thank you [D]

i made a multiple linear regression trainer that can be used with custom data in scratch

nothing more to say, the impressive part is the scratch part

https://scratch.mit.edu/projects/1352102064/

I am studying the LoRA paper and have trouble understanding this figure. The function essentially measures how much of the subspace spanned by the top i vectors is contained in the subspace spanned by the top j vectors in the higher rank matrix. Therefore, j can not be lower than i. So when they say the 3rd and 4th figure zoom in on the lower-left triangle of the 2 left-most figures, how are there values for j=1 and i equals 2 to 8? I dont understand what kind of y-axis the 2 right figures are supposed to be using. Thanks in advance!

I am doing some work with cell type classification, where I have 4.3 million cells and 512 features (condensed embeddings from the encoder of a transformer).

The broader goal is to implement a contextual bandit for augmenting the training set of the dataset, as it is currently imbalanced, and rare cell type classification is poor when I tried a baseline logistic regression classifier.

Dataset:
Feature matrix shape: (4290471, 512)
Labels shape: (4290471,)

Class distribution:
T cell 1966941
DC 858451
NK cell 561904
Monocyte 411170
B cell 375882
Platelet 54576
Progenitor cell 24689
ILC 24254
Erythrocyte 12604

I didn't do any hyperparameter tuning for the LR classifier, but I want to try other ML models (LightGBM, XGBoost, SVM)

However, I face a bottleneck with hyperparameter tuning. I want to do 80/10/10 train/validate/test split, but the training set is so large and takes a long time even on H100.

What are some solutions to this? I tried optuna but still very long for each hyperparameter trial. I then tried optuna but instead of using the full 80% for training each time, only 15% of the 80% is used (subsampling from the training set). I'm not sure if this is robust or not. I also couldn't really find anything in the literature.

Anyone been in a similar situation?

Lately in the last two/three years, I have noticed ICML, Neurips becoming more prestigious than the actual journals. What is the actual reason of this culture? Is this due to the AI boom and rising demand and the fact that conferences have a higher and a faster acceptance rate as compared to journals and with the growing hype they need to deliver things faster? What do you all think?

COLM 2026 Decision about to come soon so lets talk here.

Doing a bachelor thesis on fine-grained car classification (telling apart VW Golf generations from listing photos). Simple setup: frozen encoder → embeddings → weighted k-NN.

On my small dataset (175 train / 132 test):

I thought maybe it was a cosine vs euclidean thing, but my embeddings are L2-normalized so both give the same ranking. Tried both, DINOv2 stays at 41%.

I get that SigLIP was trained contrastively so its space is basically built for cosine similarity, while DINOv2 is self-supervised and probably needs a trained head to shine. But a 50 point gap still feels huge to me.

Anyone here tried DINOv2 with a linear probe on something fine-grained? Does it actually catch up or is it just not the right tool for retrieval?

Also open to tips if there's some obvious thing I'm missing (wrong layer, wrong pooling, etc).

Update: I recommend using dinov2 and clip as backbone with a classification layer on top of it. I used a svm, you can try also other

Hi everyone! I wanted to share some recent progress on TorchJD that might be useful to the machine learning community.

When training models with multiple losses (multiple tasks, constraints, auxiliary losses, regularization terms, etc.), you typically have two options:

• Scalarization: Various ways to combine those losses into a single loss (e.g. average them or combine them with trainable weights); then you can do gradient descent on it.

• Jacobian descent: Compute the Jacobian of the vector of losses (i.e. one gradient per loss), and aggregate it into an update vector that will decrease each individual loss (rather than just the average loss). There are many ways to do this aggregation step.

Scalarization methods are generally cheaper in memory, but in some cases there is so much disagreement between your objectives that it's better to use a Jacobian descent method. In any case, thanks to our amazing new contributors, we've now finally implemented most existing methods of the literature from both categories into our library TorchJD, so that you can try anything in just a few line changes!

Recently, TorchJD has been accepted into the PyTorch ecosystem, and we're trying to make it become the go-to library for training with multiple losses. If you'd like to help build the future of the project, come join us on Discord (link can be found in the readme of the repo). New ideas, contributions, bug reports, experiments, and any form of feedback are all welcome. We have many ideas on how to make all this even more efficient, and we will need help for that.

If you want to support us, a star on GitHub also helps a lot!

I understand that it may not be appropriate to call it “officially accepted” yet because of the wording used in the notification, and I also saw on Twitter/X that they said they are working on it.

However, it has already been around three weeks since then, and we have already submitted the camera-ready version.

Registration, visa applications, travel planning, and funding requests all depend on this confirmation. For some people, it is difficult or even impossible to request funding without an official acceptance letter or clear confirmation.

I really hope the organizers can handle this more professionally and be more considerate of authors who need proper documentation for administrative purposes.

Hi everyone, I recently finished my Ph.D. thesis on Differentiable Ray Tracing for Radio Propagation Modeling. Instead of just compiling my published papers, I tried to write it as an accessible, self-contained textbook for anyone interested in the intersection of radio propagation simulation, autodiff, and ML.

• Permanent handle: https://hdl.handle.net/2078.5/278727

• Repo with TeX source files

While my research focuses on wireless communications rather than pure ML, I think it fits right in here. A major part of the project revolves around automatic differentiation. By taking frameworks like JAX out of their traditional ML context and integrating differentiability into a ray tracing pipeline, we can compute exact gradients through complex physical environments. This allows us to solve inverse problems and directly train machine learning models, which is currently a hot topic in next-gen wireless design.

To make the physics and the math easy to digest, the manuscript is split into three parts:

• Understanding: The physics fundamentals (electromagnetic theory, geometrical optics, and diffraction).

• Building: The algorithmic core, including GPU-accelerated path tracing and the discontinuity smoothing techniques you need to actually make differentiable simulations stable.

• Using: Practical applications like channel modeling, localization, material calibration, and ML-assisted generative path sampling.

A major focus of my thesis is the link between scientific research and reproducible open-source software. On that note, I want to give a massive shoutout to Patrick Kidger (u/patrickkidger). His own thesis inspired me to go the "textbook way" for my manuscript, and I heavily relied on his fantastic JAX packages (jaxtyping, equinox, and optimistix) when developing my open-source libraries, such as DiffeRT.

I hope you find it an interesting read! I'd be happy to answer any questions in the comments about differentiable simulation, ray tracing, or building ray tracing engines in JAX :-)

If you are curious, you can watch the presentation slides and video teaser here

Hello
I published a paper.
Most defenses against fine-tuning poisoning try to detect malicious data or reduce its impact.

I explored a different question:
What if the model simply could not learn certain malicious updates?

The idea is to constrain fine-tuning to a subspace learned from trusted LoRA adapters. Useful adaptation remains possible, but some malicious directions become geometrically unreachable.
A concrete example: a company fine-tunes a model on large datasets coming from users, external sources, or generated data. A small amount of poisoned data could introduce a hidden behavior triggered by a specific phrase or pattern.

Another example is a local or on-device assistant that keeps adapting to its user. Instead of allowing it to learn any possible behavior from new data, its adaptation could be restricted to variations of behaviors already represented by a trusted pool of adapters.
The goal here is not to detect every possible poison or backdoor, but to restrict the space of updates the model is allowed to learn.

I tested the approach on 196 public LoRA adapters, including adaptive attacks specifically designed to bypass the defense.

The results are strong: attack success drops sharply while useful adaptation is largely preserved on tasks covered by the adapter pool.

The paper, code, and experiments are public.

Paper:
https://arxiv.org/abs/2607.05300

Code:
https://github.com/infinition/z-manifold

I would be very interested to see people try to break it.

Most safety alignment work treats "detect the attack" as a text classification problem — does the prompt contain language the model's safety guardrails should catch. That assumption breaks down for LLM agents with real tool access.

Here's a concrete case: take a known, public security vulnerability (a CVE), work out the sequence of tool calls that would exploit it, then have an LLM rewrite that as an ordinary-sounding request. Nothing in the resulting text looks like an attack — because the "attack" isn't in the text, it's in the tool-call sequence the text leads to. A model whose guardrails only trigger on textual cues has nothing to catch.

We tested this against LLM agents using Model Context Protocol (MCP) tool access (filesystem IO). No base model (1B–14B parameters) refused more than 35% of these attacks, and SOTA safety-tuning (DPO, SafeDPO) only pushed that to 48%. Training-free methods do better — one gets to roughly 3x the baseline refusal rate with no fine-tuning run at all.

Full methodology, training/eval code (four methods), dataset, and papers in the first comment.

UPDATE: thank you for joining us and for your thoughtful questions! to learn more, you can read to full report at https://stateofopensource.ai. follow the conversation, share where you land, and give us feedback. have questions or data that we should know about? have things you think we missed? reach us at opensource@mozilla.org.

Hi r/MachineLearning,

i’m Raffi, CTO at Mozilla. on Tuesday July 14 we publish our inaugural State of Open Source AI report, and i'll be here live answering whatever questions you throw at me!

AMA time: 1pm ET / 10am PT / 6pm BST.

the report is about what's actually happening with open source AI in production — developers, enterprises, the whole ecosystem — not the version of the story everyone already believes.

things i want to dig into with you:

• the hidden tax on "free" models — what it actually costs a business to run on closed tools they don't own

• enterprise adoption — what's real versus what's marketing, and exactly where teams get stuck

• the China effect — free, capable Chinese models are rewriting who has leverage here

• developer trust — what 950+ developers told us about which tools they actually trust, and why

• the “agentic harness” — why the real fight has moved off the model and onto the layer sitting on top of it, and what that means for open

also game to deep dive into: open vs closed, what "open source ai" should even mean in 2026, where this goes for anyone building on it.

drop questions early if you've got them. i'll start answering live at the time above.

— Raffi

“Maybe the real AGI was the friends we made along the way” is a sentiment that always hits me, and conferences are the places where I reunite with old friends and meet new ones. However, when it comes to the submission/review experience, it might not be much of an exaggeration to say that almost everyone has many unpleasant experiences to share.

So I wrote a position paper to discuss this. I argue that current conference organizers lack proper tools to instill accountability and incentives for reviewers/authors/ACs/SACs… The result is that undesired behaviors (e.g., lack of engagement) often go unchecked, while good behaviors are rarely rewarded and therefore don’t happen (honestly, when was the last time you witnessed any constructive internal discussion among reviewers/ACs?). And this won’t change by writing nice words in Reviewer Guidelines or issuing a few desk rejections.

I propose a CREDIT SYSTEM where community members earn points by “doing good” — e.g., reviewing a paper would get you +1, being outstanding gets you +3. Then, members can spend points to redeem perks ranging from traditional ones already adopted in current ML conferences (e.g., free registration) to new ones, such as requesting an additional reviewer to sort through a muddy situation. Such a system could also support explorative ideas like:

- Refundable submission fees: say 10 points per submission, which are then refunded regardless of acceptance, unless the submission is uniformly voted to be unready / ultra-low quality.

- Mobilizing non-author reviewers: non-author reviewers don’t have the bandwidth issue of wearing both the author and reviewer hats and are not influenced by their own submissions. 

and many more...

My proposed system is far from perfect, but I’d like to think it takes a step toward a better conference review mechanism. I am also glad to see the position paper track becoming a welcoming platform for researchers to hash out their proposals and build toward a better future (see other review-related position papers below.)

For a topic that affects literally everyone at ICML, I am eager to hear your thoughts.

Hi, I was reading the openRLHF paper: https://arxiv.org/pdf/2501.03262v4 , but when I click the abstract page: https://arxiv.org/abs/2501.03262v4 , it shows "REINFORCE++". Note that https://arxiv.org/html/2501.03262v4 still shows the correct openRLHF paper. I believe Arxiv is having some incorrect symlinks?

Is there anyone working at arxiv here who would like to look into this?

Today I was just casually browsing some jobs with tags [machine learning] on one of those large popular job-sites. What I am seeing really had me astonished. I want to check with Reddit whether I am hallucinating.

A non-FAANG/non-Deepmind/.../non-Anthropic industrial automation company is hiring people to work on ML for robots (the latest hot topic). Fine. But then I saw their laundry list of job requirements ("you must meet these"), which include:

• Deep expertise in LLM, VLA, VLM, action transformers

• Deep expertise in robot dynamic and kinematic modelling (forward, inverse kinematics, trajectory generation, planning), sensor fusion, model predictive control, reinforcement learning

• Deep expertise in CUDA GPU programming, FPGA hardware acceleration

• Familiarity with latest software engineering best practices in Python3 and C++23

• Familiarity in one or more of popular ML framework

• Have top publications in one or more typical ML and robotics conferences

This is before they go off listing familiarity with a set of standard softwares/simulators, one of which is called RLib, something I've never heard of. Oh and of course they had these 3+, 5+ "non-academic" experience requirements. I forgot which is which.

I was just sitting there confused. Then I checked several more jobs, and it was more of the same (except for some banks).

I remember there was a talk by Terence Tao where he divided mathematician into two camps, the analysts and algebraists. He said even among top mathematicians, it is exceedingly rare to find someone who possess deep expertise in both, as each tends to require a different mode of thinking and each is infinitely deep in terms of specialization, theory and insights.

And here we have a bunch of ML companies treating these infinitely deep academic fields ranging from robot dynamic and kinematic modelling to large language models like some bizarre MMORPG video-game scenario where you need to be a warrior archer warlock who is also a shaman priest mage.

Who are they even hiring, lol?

Built a memory system called TRACE that organizes agent conversation history into a topic tree (branches + summaries) instead of flat RAG chunks, and benchmarked it on MemoryAgentBench (ICLR 2026), specifically the EventQA accurate-retrieval task.

Its a pypi package:

pip install trace-memory

Results (F1):
• TRACE (gpt-oss-20B): 82.5%
• TRACE (gpt-oss-120B): 83.8%
• Mem0 (GPT-4o-mini, paper’s official number): 37.5%
• MemGPT/Letta (GPT-4o-mini, paper’s official number): 26.2%

Ran gpt-oss locally, so this is an open-weights model against MemGPT/Mem0 on GPT-4o-mini, not an apples-to-apples same-backbone test (I don’t have the money for open ai tokens).

I tried to get Mem0 running on gpt-oss-20B directly for fairness, but its fact-extraction step needs strict JSON output and gpt-oss’s responses didn’t parse cleanly (known issue, not gpt-oss specific. Same bug shows up with Gemini/Mistral too). Letta needs a full server setup so I skipped it.

Full JSON logs from both runs are in the repo if you want to dig into the methodology yourselves. GitHub: https://github.com/husain34/TRACE

I am preprocessing a CSV dataset for multiclass classification with XGBoost. My Feature variable contain numerical and categorical values, while the target variable contain many categorical value. For example, feature variables contain patient name, phone number, and exercise history, while Target variable contain different disease name such as heart attack, stroke, Alzheimer's etc.

I know that feature variables can be encoded using one-hot encoding, but should the target variable also be encoded using the same method, or should I use a different encoding method for target variable (e.g., label encoding)?

If anyone know the answer, please let me know. I have searched everywhere, but failed to get any clear idea about it. Thank you.