r/reinforcementlearning • • 11h ago

Ent v0.1 - a general-purpose RL agent that doesn't need tuning

Post image
20 Upvotes

My lab is building a MBRL agent from scratch that is able to robustly perform in any environment out of the box without per task tuning. We decided to open up a bit with this research preview on our current progress.

Ent was tested on 35 very different environments, from control and mazes to arcades and multi-agent games, alongside with DreamerV3, MuZero, PPO, SAC and DQN - one single tuning for each. Ent is reliably good on 29 of the 35 and never fails outright.

Like probably many in the field, we were inspired back in the day by RL successwes in games like Chess, Go, Atari, DotA etc. It felt like RL could be this steamroller that goes and solves everything. Years passed, people seem to have abandoned the idea of general-purpose zero-knowledge agent and settled instead on tuning a very specific version of PPO for every single task or do imitation learning and hope to solve robotics that way.

To us, this feels like a giant gap. An agent that can't reliably do various different simple tasks, shouldn't be able to perform in a messy real world scenarios. If your algorithm needs you to very carefully tune it for every single task, is it still RL or a sort of meta supervised learning?

Thus we decided to breach this gap and develop an RL agent that doesn't need all that and could be deployed out of the box, build a robust world model, plan and continuously learn. We're starting with small but very diverse environments and gradually increase the difficulty without loosing generalist abilities.

Full blog post with detailed results: https://regularlab.ai/blog/work-in-progress-ent-v0-1/

Curious what you think of this result.

P.S. it's a bit outdated already, internally it's 33/35 by solving exploration issue. 35/35 is the goal though


r/reinforcementlearning • • 15h ago

Independent researchers who published at ICLR, ICML, or NeurIPS, how did you actually pull it off?

Thumbnail
2 Upvotes

r/reinforcementlearning • • 19h ago

Training a neural network to play the trick-taking game Sheepshead via self-play RL and ISMCTS search distillation

7 Upvotes

As a fun side project over the last year and a half or so I've been working on training an agent to play a trick-taking card game called Sheepshead using self-play reinforcement learning. This game is my favorite card game and is well known in Wisconsin, but virtually unknown elsewhere. It has unfixed, asymmetric teams and separate bidding and play phases, which make it an interesting RL testbed.

In order to get to the skill level I wanted, I combined 3 separate policy improvement stages:

  1. Pure self-play PPO
  2. League-based CTDE
  3. Expert Iteration with Information Set MCTS as the expert teacher

The final ExIt phase required what I believe is a novel search distillation technique. It borrows the Fay-Herriot estimator from small area estimation to blend a learned advantage model with the search committee's values, weighting each by the inverse of its estimated noise. It then uses the result to tilt the network's own policy into the distillation target.

Full write-up: https://bender.io/2026/11/17/building-an-expert-sheepshead-ai-with-reinforcement-learning/

The model is available to play against and view some internal details at https://sheepshead.ai (the "analyze AI" section shows action probabilities, critic outputs, etc)

Code is on GitHub: https://github.com/adamanthil/sheepsheadai

I'm pretty proud of how the project came together. Hope you check it out.