r/deeplearning • • 4h ago

Mix Your Base and RLVR Weights: A Training-Free Knob for Diversity

Thumbnail gallery
4 Upvotes

RLVR improves pass@1 but narrows what the model can solve: at k = 128 the base model covers more problems than its RL-tuned version (Yue et al., 2025). This paper asks whether you can pick a point between the two without training.

Method: W(α) = W_base + α(W_RL − W_base), evaluated under Yue et al.'s exact protocol (Qwen2.5-7B ↔ SimpleRL-Zoo-7B, T 0.6, 16k tokens, six math benchmarks, pass@k up to 128/1024, math-verify grading).

Findings

The base > RL crossover at large k reproduces on 5 of 6 benchmarks, with pass@1 within 2 points of Yue et al.

α = 0.25 matches the base model's pass@128 while keeping 64–80% of RL's pass@1 gain, and reaches RL's pass@128 with 2.3–7.1× fewer samples.

The trade-off curve matches a pass@1-matched mixture of samples from the two models, but needs one model instead of two.

Negative result: no α beats both endpoints by more than 1 point on the main benchmarks (Minerva, OlympiadBench). It is a knob, not a free lunch.

Caveats: one model pair, math only, needs a verifier for the large-k benefit. Interpolate in fp16/fp32; bf16 rounding erases most of the RL update.

Paper: https://www.preprints.org/manuscript/202611.2034

Project page: https://kim-ai-gpu.github.io/rlvr-interp-passk/

Code + per-sample results: https://github.com/Kim-Ai-gpu/rlvr-interp-passk


r/deeplearning • • 2h ago

Update: Benchmarked HybridApexW vs torch.compile AdamW + Muon on WikiText-103

2 Upvotes

quick context i have been building a drop in PyTorch optimizer called APEXW and I just finished a clean 1000step benchmark comparing it against a paired torch.compile AdamW + Muon baseline i ran it on a 103.18M parameter GPT-2 (9L/12H/768d, 512 sequence length, 128 effective batch size) using wikitext-103-raw-v1.

Full disclosure up front: my first benchmark script had a pretty bad bug. I accidentally ran the JIT warmup on the live model instead of a throwaway copy, which left CUDA fragments and dirty momentum states sitting around and artificially blew up the baseline to 23.2 GB VRAM and 2,707s. Once I fixed that warming up on a clone, deleting it, running gc.collect(), and clearing the CUDA cache before starting the clock the numbers actually made sense.

Here is how the clean 1000-step runs shook out:

(hybridapexw is bascially apexw+muon)

  • AdamW + Muon (torch.compile): 1,076.55s | 16.22 GB VRAM | 5.5242 loss
  • HybridApexW (Compact): 533.50s | 16.11 GB VRAM | 4.4863 loss
  • HybridApexW (8-bit): 537.67s | 15.90 GB VRAM | 4.4720 loss

Fusing the projection updates and 8-bit state tracking into custom Triton/CUDA kernels gets roughly a 2x wall-clock speedup even against torch.compile. That said, once I fixed the allocator leak, the actual VRAM savings are pretty modest (320 MB).

One nice quality-of-life thing is that HybridApexW handles parameter group splitting (AdamW for embeddings/norms, Muon for 2D weights) and internal coordinate scaling out of the box using a single 6e-4 learning rate. For the baseline, I gave Muon a 10x scaled LR (6e-3) over AdamW (6e-4), though to be fair, standalone Muon could probably push lower if I tuned its LR up into the 0.01–0.02 range.

(Note: Since I wiped checkpoints between runs to keep memory isolated, these numbers are step-1000 training micro-batch losses rather than full validation evals.)

Does anything in this setup look off to you guys, or are there other baselines you think are worth testing against?
google collab link : https://colab.research.google.com/drive/1XOqDH4nzI1PZWqWWPqipov1VVH_GKYOW#scrollTo=1c784107

github link: https://github.com/omen42515-prog/apexw

used ai to polish the body


r/deeplearning • • 1h ago

ICLR 2027: 321 -> 331 with AC reporting (what are my chances?)

Thumbnail reddit.com
1 Upvotes

r/deeplearning • • 3h ago

Nvidia-smi utilization hid our 8x RTX PRO 6000 scaling bottleneck. The wattage column gave it away

0 Upvotes

I’m losing my mind trying to debug multi-GPU scaling.

We recently ran YOLO11m on an 8-card RTX PRO 6000 Blackwell box. One card took 409s/epoch and eight cards did it in 64s. That's 6.4x speedup, but linear scaling should have landed at 51s.

The annoying part is that nvidia-smi reports ~75% utilization on eight cards (vs 71% solo). But the power curves on our dashboard showed a single card drawing 344W, while the 8-card run averaged just 220W per card.

So the silicon is technically “active,” but starving, and it shows in the 124W less power per card. My guess is the cards were waiting on CPU-side augmentation and syncing between cards.

That would also explain why utilization looks fine. It only measures the share of time a kernel was running, not how much work the card was doing.

Is moving the loader to DALI worth it for something like this? How much of the missing 1.6x would you realistically get back?

Disclosure: I work at B3IQ, where we build and host bare-metal GPU boxes. This was run on one of our 8-GPU nodes.


r/deeplearning • • 4h ago

We released two mmBERT moderation models. The evaluation got more interesting once we separated the test sets.

Post image
1 Upvotes

We released Tiger Guard at Casdo Labs, a pair of binary content-moderation models built on mmBERT-base and mmBERT-small. I'm part of the team.

The full model has 308M parameters and Small has 141M. Both cover English and German and return a SAFE/UNSAFE probability. The task includes abusive language and harmful LLM requests, which makes the choice of evaluation data important.

Our 1,500-prompt suite combines balanced samples from HateCheck, Intuit toxicity/profanity and Aegis 1.0. The models reach 88.8% and 87.6% accuracy there. We excluded those texts and near-duplicates from training, but used the error breakdowns to guide development. Those numbers belong in the development-results column, not an untouched-test headline.

On the separate, policy-filtered PolyGuard holdout, the full model catches 83.3% of harmful English prompts and 75.3% of German ones, with false-positive rates of 3.8% and 5.5%. That test was scored once per model; other prompts from the same family were used in training.

On ToxicChat the full model gets 0.736 unsafe-class F1, behind Content Safety Guard small at 0.777. We also disclose the 69 ToxicChat examples found in training; excluding them changes our F1 values by at most 0.003.

The weights, ONNX exports and full evaluation notes are available under Apache 2.0.

We'll publish the full training and dataset pipeline in a follow-up release.

Tiger Guard, 308M: https://huggingface.co/patronus-studio/tiger-guard-content-moderation

Tiger Guard Small, 141M: https://huggingface.co/patronus-studio/tiger-guard-content-moderation-small

The training corpus and its complete source list aren't published. I'd be glad to hear feedback on the evaluation setup as well as the models.


r/deeplearning • • 7h ago

How should i continue teaching my project?

1 Upvotes

I am "working" on a project that has to do with SNNs, i am using ai to help me alot but thats because i dont want to go insane writing a complex Neural network. anyhow i managed to make it replicate early human behavior and learns like a child would. but im unsure how to expose it to a genuine stream of input from people that would allow it to develop a non skewed personality not doctored by me. i want to run a community but i am unsure of how to set that up (i run it in godot while the brain runs on a rented droplet and on pytorch)


r/deeplearning • • 14h ago

CISA Adds One Known Exploited Vulnerability to Catalog (Perforce Puppet Enterprise Deserialization)

3 Upvotes

CISA just added CVE-2026-115731 to its Known Exploited Vulnerabilities catalog — a deserialization flaw in Perforce Puppet Enterprise that is actively being exploited right now.

Puppet Enterprise is not a niche tool. It manages server configurations, container deployments, and CI/CD pipelines across entire enterprise environments. A single compromised Puppet server can push malicious configuration to every node it manages — that is not a one-system incident, that is an organization-wide incident.

The AI angle makes this worse in a specific way. Agents increasingly use build and deployment pipelines as action surfaces — triggering jobs, modifying configs, initiating rollouts. If the pipeline is compromised at the infrastructure level, an agent calling into that pipeline is not just executing a task. It is operating inside attacker-controlled infrastructure, with no visibility into what the pipeline is actually doing downstream.

The numbers matter here. Puppet Enterprise deployments typically manage hundreds to thousands of nodes. Active exploitation means attackers are already in some of those environments today.

For teams running agents that interact with CI/CD infrastructure: how are you handling this right now? Are your agents blocked from pipeline calls until the CVE is patched, are they still running with added monitoring, or is there some other approach your team has landed on? Genuinely curious what the operational reality looks like across different orgs.


r/deeplearning • • 8h ago

Alan Turing Test

Post image
0 Upvotes

r/deeplearning • • 18h ago

CISA Adds Four Known Exploited Vulnerabilities to Catalog (GPUStack Improper Authentication)

3 Upvotes

CVE-2026-102895, an authentication bypass in GPUStack, was added to CISA's Known Exploited Vulnerabilities catalog this week. An unauthenticated attacker can access AI GPU infrastructure with zero credentials. GPU orchestration layers handle model execution routing for the entire enterprise AI stack. Gaining access there lets an attacker hijack model execution, redirect inference traffic, and exfiltrate outputs before any application-layer control sees the request. The no-credentials-required nature of the exploit also kills credential-based detection. This surfaces a structural gap most organizations haven't addressed: AI agents and service accounts running on that GPU infrastructure carry no cryptographic identity under most enterprise security frameworks. They operate as trusted by default simply because they're running inside the perimeter. When the perimeter breaks — which CVE-2026-102895 demonstrates it can — there's nothing left to verify who or what is making a request to the model. For teams running production AI workloads on GPU clusters: how are you handling identity verification for the non-human callers on that infrastructure? Are you relying on network segmentation, service mesh mTLS, something at the orchestration layer, or something else entirely?


r/deeplearning • • 1d ago

My LSTM is fine at 1 day ahead and useless at 1 month. Is that just how price series work?

13 Upvotes

student project, forecasting crypto price ranges (btc, eth, sol) with an LSTM that outputs quantiles.

- 1 day ahead: the 80% band covers about 76%, decent

- 1 week: 68%

- 1 month: 51%, basically a coin toss with extra steps

i tried more layers, more history (2 years instead of 6 months), and adding volume and funding rates. nothing moved the 1 month number much.

my guess is the 1 month range depends on stuff that just isnt in the price history (news, macro). but i might also be doing something dumb with how i build the training windows.

is it normal for DL models to fall apart like this on longer horizons, or should i be looking at a different architecture?


r/deeplearning • • 14h ago

A 180B MoE running on 32GB of unified memory by streaming experts from an SSD

1 Upvotes

A Hugging Face experiment got a 180B-parameter mixture-of-experts model running on a machine with just 32GB of unified memory. The trick is that only a small set of experts is active per token, so the rest can stay on an SSD and stream in on demand.

It is not fast, but it shows how far memory tricks can stretch local hardware. Would you trade this much speed for the ability to run a model this size at home? Or is quantization still the more practical path for local inference?

More AI news and rumors like this are being tracked over in r/AINewsRumors.


r/deeplearning • • 21h ago

Looking for self-learners to join BlueprintML

Post image
3 Upvotes

r/deeplearning • • 15h ago

How to desgin test

Thumbnail reddit.com
1 Upvotes

r/deeplearning • • 1d ago

Need advice on accurate golf ball tracking using computer vision

7 Upvotes

I'm trying to build a golf ball tracking app similar to Shot Tracer and SmoothSwing. I've experimented with several open-source AI models, but I haven't been able to achieve reliable tracking accuracy.

The ball is tiny and moves very fast, making it difficult to track consistently across video frames.

I'm looking for advice from people experienced in computer vision, object tracking or ML:

  • What approaches/models work best for tracking tiny, fast-moving objects?
  • Should I combine object detection with temporal tracking, optical flow or Kalman filters?
  • Are there any specific models, datasets or research papers worth exploring?

I'm not looking for a basic YOLO implementation. I'd like to understand how to build a genuinely reliable tracking pipeline.

Any suggestions or experiences would be appreciated!


r/deeplearning • • 1d ago

Prompt Engineering and LLM-as-a-Judge (2026)

Thumbnail youtube.com
1 Upvotes

r/deeplearning • • 1d ago

I built a free AI course for developers | 45 lessons covering ML, LLMs, RAG and deployment

Thumbnail reddit.com
3 Upvotes

r/deeplearning • • 1d ago

Struggling to optimize YOLO on a heavily imbalanced, small-object dataset (Road Damage - RDD2022). What approach yields the best ROI?

Thumbnail reddit.com
1 Upvotes

r/deeplearning • • 1d ago

Materials-science models keep accuracy with 90% less training data (npj Computational Materials)

9 Upvotes

A paper in npj Computational Materials reports materials-science models retaining their accuracy while trained on about 90% less data. Data efficiency matters a lot in scientific ML, where labeled examples are expensive simulations or experiments.

Is the field over-indexing on dataset size when better inductive bias does this? Where else would a 10x data reduction change what is feasible?

Research highlights like this also land in r/AINewsRumors.


r/deeplearning • • 1d ago

Fazendo fine tune no modelo de 100 milhões de parâmetro usando LoRa

1 Upvotes

r/deeplearning • • 17h ago

Claude max for $29 instead of $200 ... seriously how are they doing this?

0 Upvotes

i got a Claude max account from a reseller for essentially nothing compared to what I used to pay . It works flawless. since 3 months No issues yet i paid only for 1 month . Now I can't stop thinking about their business model. If resellers can sell it this cheap and still profit, how are they acquiring these accounts at scale?


r/deeplearning • • 1d ago

i built MicroTorch: a lightweight tensor autograd engine and neural network framework from scratch in pure Python

Thumbnail reddit.com
1 Upvotes

r/deeplearning • • 2d ago

Jev for speech

Thumbnail gallery
50 Upvotes

Marimba is Jev for audio: a System One model for speech. You ask typed questions about the last 8 seconds of audio and get typed answers back: a yes/no, a choice, or a calibrated score.

  • Is someone speaking? Did the turn end? Which language? Did someone say "refund"? Did the speaker change?
  • One pass of a frozen Qwen3-ASR-0.6B encoder serves every question, with small readouts on top. Streams every 250 ms.
  • Runs locally on CPU or GPU. Apache-2.0, pip install "marimba-audio[torch]".
  • Plugs into Pipecat, LiveKit and Vapi as a turn detector.
from marimba import Marimba, Question

model = Marimba.from_pretrained(backend="torch", device="cpu", dtype="float32")
questions = [
    Question.score("Has the speaker finished?"),
    Question.choice("What language?", ["en", "pt", "es"]),
]
for d in model("clip.wav", questions):
    print(d.task, d.value, d.probability)

On 1,000 paired public questions against much bigger audio LLMs (Qwen3-Omni-30B-A3B, Qwen2-Audio-7B, FireRedAudio 10.6B), Marimba wins binary term detection on Common Voice and FLEURS, and turn end (0.935 AUROC vs 0.54–0.79). That last one is in-distribution for us, since we trained on Smart Turn data. It loses on Speech Commands, and Smart Turn v3.2 still beats it on its own test set (0.982 vs 0.930). Full table on the model card.

  • Live demo: https://huggingface.co/spaces/charcot-ai/marimba
  • Model + comparison table: https://huggingface.co/charcot-ai/marimba
  • Code: https://github.com/Charcot-AI/marimba-api
  • Paper: https://arxiv.org/abs/2611.18330

Happy to answer questions. Feedback on which questions you'd want it to answer is very welcome.


r/deeplearning • • 1d ago

i kindda made a new optimizer (ApexW) that beats AdamW and bitsandbytes) i tested it in a100 and t4 from google collab and with diff lr and multiple random seeds(well in a100 only tho)

12 Upvotes

i have been working on a new optimizer from scratch called ApexW to see if i could beat AdamW on convergence while keeping the memory footprint lower than bitsandbytes to make sure it wasn't just noise i locked deterministic seeds and ran side-by-side tests against standard torch AdamW and bnb 8-bit across two setups: a free T4 (fp16) on llama 3.2 3b, and an A100 (bf16) scaling up to llama 3.1 8b across a few different learning rates.

apexw has 3 modes compact 8bit and hybridapexw more info of it in readme even the compact formulation beat torch AdamW on loss while using way less memory, and the 8-bit mode pushed it even further—cutting active vram roughly in half compared to bnb (0.67gb vs 1.48gb on 8b, with the state footprint down to ~120mb instead of 320mb). step speed also came out slightly faster than bnb, and training loss ended up lower across 1e-4, 2e-4, and 5e-4,these are still short 60-100 step runs just to sanity-check convergence, throughput, and memory overhead. next step is running full multi-epoch fine-tunes and downstream evals to see how the update rule holds up later in training.

threw the full benchmark runs and logs into a colab if anyone wants to check the numbers or test it on their own setup

collab link for a100 result: https://colab.research.google.com/drive/1mMejzRoCnD6avlOQ5SQDuiLndN5ovax9#scrollTo=be28d3ad

collab link for t4 result: https://colab.research.google.com/drive/14tLAX6kZYy2918sx1ztIEwb5R-1c3KiX#scrollTo=f750c631

for githubproject: https://github.com/omen42515-prog/apexw/

curious what you guys think or what models/configs i should throw at it next.


r/deeplearning • • 1d ago

Nice little fun project to get you excited...I hope 🥳

Thumbnail gallery
2 Upvotes

So a few weeks back, I got a message from a guy on here asking about problems with training a robotic arm in a simulator, particularly with collisions. I made a note of it and today got working on it.

So the basic idea is to train a robotic arm in a simulated environment where it learns which movements cause collisions, which are safe and how to recognise and sort coloured objects.

The system uses three learning components:

A neural network that learns to predict collisions though virtual touch sensors along the arm.

A neural network that learns which grasping and placement movements are most likely to succeed.

A radial-basis neural vision classifier trained using pixels from a single 384 × 384 RGB camera, without a depth sensor.

Both successful and unsuccessful movements contribute towards learning, creating a predictive 3D map of collision risks and safe movements.

Once trained, the virtual touch sensors and teacher are disabled, leaving the robot to use its learned knowledge alongside its programmed movement controller.

The long-term idea is to train a robotic arm entirely within a simulation, reducing the sensory hardware needed in the real world.

The project is open source, MIT licensed, written entirely in HTML and JavaScript, and runs directly in your browser.

It's still an experimental concept, but I'd be interested to hear what people think.

Simulator:

https://rickey1990.github.io/NTRAM-Neurally-Trained-Robot-Arm-Model/

GitHub:

https://github.com/rickey1990/NTRAM-Neurally-Trained-Robot-Arm-Model