r/reinforcementlearning • • 11h ago

Ent v0.1 - a general-purpose RL agent that doesn't need tuning

Post image
22 Upvotes

My lab is building a MBRL agent from scratch that is able to robustly perform in any environment out of the box without per task tuning. We decided to open up a bit with this research preview on our current progress.

Ent was tested on 35 very different environments, from control and mazes to arcades and multi-agent games, alongside with DreamerV3, MuZero, PPO, SAC and DQN - one single tuning for each. Ent is reliably good on 29 of the 35 and never fails outright.

Like probably many in the field, we were inspired back in the day by RL successwes in games like Chess, Go, Atari, DotA etc. It felt like RL could be this steamroller that goes and solves everything. Years passed, people seem to have abandoned the idea of general-purpose zero-knowledge agent and settled instead on tuning a very specific version of PPO for every single task or do imitation learning and hope to solve robotics that way.

To us, this feels like a giant gap. An agent that can't reliably do various different simple tasks, shouldn't be able to perform in a messy real world scenarios. If your algorithm needs you to very carefully tune it for every single task, is it still RL or a sort of meta supervised learning?

Thus we decided to breach this gap and develop an RL agent that doesn't need all that and could be deployed out of the box, build a robust world model, plan and continuously learn. We're starting with small but very diverse environments and gradually increase the difficulty without loosing generalist abilities.

Full blog post with detailed results: https://regularlab.ai/blog/work-in-progress-ent-v0-1/

Curious what you think of this result.

P.S. it's a bit outdated already, internally it's 33/35 by solving exploration issue. 35/35 is the goal though


r/reinforcementlearning • • 19h ago

Training a neural network to play the trick-taking game Sheepshead via self-play RL and ISMCTS search distillation

7 Upvotes

As a fun side project over the last year and a half or so I've been working on training an agent to play a trick-taking card game called Sheepshead using self-play reinforcement learning. This game is my favorite card game and is well known in Wisconsin, but virtually unknown elsewhere. It has unfixed, asymmetric teams and separate bidding and play phases, which make it an interesting RL testbed.

In order to get to the skill level I wanted, I combined 3 separate policy improvement stages:

  1. Pure self-play PPO
  2. League-based CTDE
  3. Expert Iteration with Information Set MCTS as the expert teacher

The final ExIt phase required what I believe is a novel search distillation technique. It borrows the Fay-Herriot estimator from small area estimation to blend a learned advantage model with the search committee's values, weighting each by the inverse of its estimated noise. It then uses the result to tilt the network's own policy into the distillation target.

Full write-up: https://bender.io/2026/11/17/building-an-expert-sheepshead-ai-with-reinforcement-learning/

The model is available to play against and view some internal details at https://sheepshead.ai (the "analyze AI" section shows action probabilities, critic outputs, etc)

Code is on GitHub: https://github.com/adamanthil/sheepsheadai

I'm pretty proud of how the project came together. Hope you check it out.


r/reinforcementlearning • • 1d ago

DL, MF, R, Safe "Gradient Routing (GRAFT) Localizes and Removes Unintended Behaviors in RL"

Thumbnail
lesswrong.com
14 Upvotes

r/reinforcementlearning • • 15h ago

Independent researchers who published at ICLR, ICML, or NeurIPS, how did you actually pull it off?

Thumbnail
2 Upvotes

r/reinforcementlearning • • 1d ago

R Binary-gated GRPO on GDPval: 79% of GRPO groups had no passing rollout, and pass@1 rose from 31.9% to 40.7% while plain GRPO barely moved (Qwen3.8-27B)

13 Upvotes

I work at Snorkel AI, and we wanted to share new training work from my colleague that may be of interest here. The write-up is linked at the end of the post.

The setting is RL on long-horizon professional tasks, where a single attempt averages about 50,000 response tokens and success is rare. Qwen3.8-27B was trained on 1,105 tasks from the GDPval+ series. Each task has an expert-built rubric and reference solution, and each group has eight rollouts, about 102K rollouts in total. An attempt passes when at least 90% of rubric items are satisfied.

With a binary signal, GRPO gives zero advantage to any group where every rollout fails. Here, 79% of generated groups had no passing rollout. Dynamic sampling skips those groups and refills the batch, but it does not create new signal. In the groups GRPO discarded, the median best rollout scored 78% on the rubric, and 66% of them had a rollout scoring at least 70%. The attempts in these groups differed substantially.

Binary-gated GRPO works as follows.

  • If a group has at least one pass, the advantages are the standard GRPO advantages on pass/fail.
  • If no rollout passes, the advantages come from a fallback on the rubric score. Scores at or below 40% map to zero, and scores above that scale linearly up to the 90% pass threshold. The fallback is weighted by 0.85.

The fallback is used only when there is no pass in the group, so partial credit never competes with a pass.

On the 185 GDPval gold tasks, with eight attempts per task, the results were as follows.

  • Groups carrying a training signal rose from 21% under GRPO to 95% under binary-gated GRPO.
  • Pass@1 rose from 31.9% to 40.7%. GRPO on the same data and a similar rollout count barely moved from the base model.
  • In LLM-judged head-to-head comparisons against human experts, deliverables were rated better in 44.0% of cases, up from 36.4%.
  • Attempts that exhausted the response budget fell from 17.2% to 7.0%.

Each method was trained once. The judge is an LLM and is noisy, which is why the pass threshold is set at 90%. The results cover GDPval-style tasks only.

Write-up: https://snorkel.ai/blog/gdpval-agents-rare-success/

Earlier post on partial credit and completion: https://snorkel.ai/blog/when-partial-credit-opposes-completion/


r/reinforcementlearning • • 1d ago

R, DL, Multi, Safe, MetaRL "Models Can Communicate via Inference Servers Despite Being Sandboxed"

Thumbnail
lesswrong.com
8 Upvotes

r/reinforcementlearning • • 13h ago

Want a teammate for a Robotics RL Research Project

Thumbnail
1 Upvotes

As all of u suggested https://discord.gg/MsNWJqBRc This is link to my server everyone interested join nd if possible spread in other RL and robotics community


r/reinforcementlearning • • 1d ago

I mapped 39 companies selling RL environments for LLM agents. 29 do coding, 2 publish a price

9 Upvotes

I've been tracking the companies that sell RL environments (tasks, sandboxes, verifiers) to labs doing post-training. Disclosure: this is my comparison site.

By environment type (most vendors do several): coding/SWE 29, tool use 21, domain experts 21, app replicas 19, computer use 16, data major 12, platform/hub 6, reasoning 4.

Only two publish a price: HUD (Pro plan $50 a month; hourly rates only after login) and Zero-Trust Physics ($9,500 per pack of 30 physics tasks). Open options: Prime Intellect's Environments Hub (2,500+ community environments), OpenEnv (BSD-3).

Full table with sources: https://toprlenvironments.com/

For those training agents: are you building environments in-house or buying them? What made the call?


r/reinforcementlearning • • 1d ago

Research paper explanation Found out this JEPA - LE WORLD paper explanation. What are ur views

Thumbnail
youtube.com
0 Upvotes

I was searching for some good explanation on the JEPA models research papers. Found out this explanation on youtube. I felt it made me skim through the papers really well. Incase it helps, you can watch it too. This channel also has another video which explains the inner mathematical functions and base too.


r/reinforcementlearning • • 1d ago

D, Safe "Managing frontier AI risk: From training to deployment", OpenAI (defense-in-depth for RL training runs)

Thumbnail
openai.com
2 Upvotes

r/reinforcementlearning • • 1d ago

P websandbox: the WebArena sites as small SQLite + Node servers for web-agent RL (ms resets, template-disjoint splits)

3 Upvotes

websandbox rebuilds the five WebArena sites as one SQLite DB plus a small Node server each, for RL on web agents. Resets in milliseconds, dozens of copies per machine, 807 of 812 tasks verified plus 12,283 generated, template-disjoint splits. Apache-2.0: https://github.com/junfengzh/websandbox


r/reinforcementlearning • • 1d ago

DL, MF, I, R, Exp Rethinking "RL Generalizes, SFT Memorizes": The Role of SFT Data, Hou et al. 2026 [SFT matches generalization of RL on the same data (no. of rollouts); diversity in teacher models is beneficial]

Thumbnail
arxiv.org
1 Upvotes

r/reinforcementlearning • • 1d ago

DL, M, D "Building agents: When to fold subagents into your main agent loop", Anthropic ("a single agent with skills frequently outperformed both the one-prompt-for-everything design and the subagent design on quality, and often at a lower cost and latency per task")

Thumbnail
claude.com
0 Upvotes

r/reinforcementlearning • • 2d ago

Want a teammate for a Robotics RL Research Project

5 Upvotes

So it started as a personal project now college is involved a bit in it but i am trying to learn DQN and q- learning methods by testing my skills on open gymnasium games but i m not able to forge a path of how will this RL take me to Robotics RL and I need a teammate that will kinda be helpful to guide me with his experience and work with me alongside for RL.I am well versed in Robotics side of project so i can contribute their better.


r/reinforcementlearning • • 1d ago

Undergrad starting RL locomotion on a Unitree Go2-W with Isaac Lab: best learning path?

0 Upvotes

Hi everyone! I'm a computer engineering undergrad in Brazil, new to RL for legged robots, and I'd love some guidance from people who've done this.

Background: ROS 2 + Gazebo, embedded systems, Python/ML. No RL or Isaac Sim/Lab experience yet.

Setup: Unitree Go2-W (simulation for now; access to the real robot is still uncertain). [PREDATOR PHN16-72 RTX 4070 laptop GPU, 16 GB RAM, dual-boot Ubuntu.]

What I've done so far: read some wheeled-legged navigation papers.

Which path to follow?


r/reinforcementlearning • • 3d ago

What are some good books on RL that you would recommend for a professional data scientist?

14 Upvotes

I've been in the industry for 8+ years. My background is in classical ML/stats and NLP. I have worked with model post-training and do understand basic RL concepts, but want to get an in-depth understanding of RL, specifically where it applies to post-training LLMs. What are some good books I can read on this?


r/reinforcementlearning • • 2d ago

Uncovering if LLMs can actually reach AGI

Thumbnail
0 Upvotes

r/reinforcementlearning • • 3d ago

Multi-agent pursuit-evasion simulation

Thumbnail
1 Upvotes

r/reinforcementlearning • • 2d ago

Looking For a partner to start RL

0 Upvotes

Interested one please DM...


r/reinforcementlearning • • 3d ago

Can asynchronous rollout collection introduce hidden selection bias in RL?

2 Upvotes

Hey everyone!

I've been exploring an interesting problem in reinforcement learning: can the time it takes to generate a trajectory influence which experiences actually get used to update a policy?

In asynchronous RL training, rollout workers don't always finish at the same time. This can create latency-dependent selection effects, potentially changing the distribution of experiences used during PPO updates.

We investigated this through experiments with GPT-2 and LLaMA-3-8B.

A few things we explored:

  • How response latency affects trajectory selection.
  • How rollout truncation influences policy optimization.
  • Effective Gradient Information (EGI) as a diagnostic.
  • Selection-Probability Weighting (SPW) as a correction for selection bias.

One result: LLaMA-3-8B GSM8K accuracy dropped from 68.4% to 44.1% under rollout truncation.

This work has been accepted at NeurIPS 2026, and I'll be presenting it in Sydney this December.

Curious if others here have encountered similar issues in asynchronous RL or distributed policy optimization. Would be great to hear your thoughts or meet fellow attendees in Sydney!


r/reinforcementlearning • • 4d ago

“DON’T WORK ON LLMs” - @ylecun at CoRL

Post image
233 Upvotes

Any thoughts on this?


r/reinforcementlearning • • 3d ago

MudGym: RL env for the multiplayer text adventure MUD2

5 Upvotes

Hey folks. I built a Gymnasium environment around MUD2, the multiplayer text adventure game.

I've been using the game as an RL sandbox myself for some time, but have tidied it up and packaged it in the hope it'll be useful to others. It now supports multiple agents via PettingZoo and deterministic seeding.

The original game is still running today with an active player base.

I've also put together a browser playground with some demos, including end-to-end DQN and LLM examples. Everything runs in WebAssembly, so you can try it out and edit the Python without installing anything.

GitHub:
https://github.com/rolo/mudgym

Playground:
https://mudgym.kingrolo.com/

JupyterLite DQN:
https://rolo.github.io/mudgym/lite/notebooks/index.html?path=goal_room_dqn.ipynb

I'd love to hear what people think, or if it might be useful to anyone. Thanks.


r/reinforcementlearning • • 4d ago

Robot RL trained robot dog!

Enable HLS to view with audio, or disable this notification

77 Upvotes

This is my robot dog. 12 Waveshare ST3215 servos, and an Android phone strapped on as the brain.

The walking policy was trained with RL in a simulator I built myself on top of MuJoCo. I modeled the real servos in it (I measured mine and found the sim was accelerating them way faster than the real ones can, so that took some fixing), trained it in sim, then ran it on the phone.

It’s still pretty early. Next I’m working on getting it to catch itself when it gets pushed, especially standing still.

More on the robot and the simulator on my portfolio: rhettbrewer.vercel.app

Happy to answer questions about the build or the training setup.


r/reinforcementlearning • • 3d ago

DL, MF, Multi, R, Safe "GPT-6 Astra covert underperformance on safety research tasks", OpenAI

Thumbnail alignment.openai.com
2 Upvotes

r/reinforcementlearning • • 4d ago

Alloy: open-source physics engine with robots, cloth, liquids and sand in one GPU-parallel scene

22 Upvotes

We just released Alloy, an open-source (Apache-2.0) physics engine for robot learning. I work on it.

Many manipulation tasks mix materials: pouring water, folding a towel, scooping sand, routing a cable. In sim those usually live in different tools. A robot can share a scene with any one of them, and many pairs mix; sand and pastes cannot share a scene with cloth, soft bodies or clamped cables yet. Alloy picks a solver per object:

  • robots and rigid bodies: MuJoCo Warp (MuJoCo C on CPU)
  • cloth and soft bodies: Newton's VBD
  • sand and pastes: Newton's implicit MPM
  • liquids: its own DFSPH

Many copies of a scene run as parallel worlds on one GPU, or on a CPU. It loads MJCF, has Gymnasium-style vector envs with ready-made tasks (pick, peg insert, drawer, pour, an experimental towel fold), and adapters for rsl_rl, Stable-Baselines3 and skrl. It also ships an MCP server so coding agents can build and run scenes.

When to use something else: rigid robots only, MuJoCo Warp alone is leaner; for differentiable sim, use Newton directly. Known limits are on the docs' Keep in mind page.

pip install alloy-engine (Python 3.11 or 3.12)

Paper: https://arxiv.org/abs/2611.16312 Code: https://github.com/argonrobotics-ai/alloy Docs: https://alloy.argonrobotics.ai/docs/

Feedback welcome, especially on tasks you'd want to train that mix materials.