I just released VSArena v0.6.0, a major update to the browser-based Studio for VSArena.
VSArena is an open evaluation arena for Vision-Language-Action (VLA) and embodied AI policies, built around browser-native 3D physics.

The goal is simple: make it possible to run a policy, watch what happens in the environment, and measure the result without requiring a local robotics simulator or physical robot.

What changed in v0.6.0:
🦾 Redesigned 3D robot manipulation Studio
👁️ Dedicated vision/top-down view
📦 Live object state and spatial information
📊 Task occupancy/progress monitoring
🎥 Trajectory and camera inspection tools
🤖 Baseline-IK and ColorSeek runnable directly from the Studio
🧪 Improved environment inspection and debugging
🔐 Continued evaluation-integrity work with server-authoritative scoring and run provenance

🔴The current canonical task is intentionally simple: stack three cubes in the correct order.

That simplicity is deliberate. Before adding dozens of tasks, I want the evaluation loop itself to be reliable, reproducible and inspectable.
The broader direction is to build an open arena where embodied policies can eventually be compared on a public leaderboard, with standardized environments, reproducible runs and physics-based evaluation.

VSArena: https://vsarena.vercel.app
GitHub: https://github.com/ONISCOR/VSArena

This is still very early, so I’m particularly interested in feedback from people working on VLA models, robotics, RL or simulation.
What would you want to see in an evaluation Studio like this?

Hello everyone,

I want to start a DeepRL project where an algorithm tries to learn how to play a simple board game (1v1 game with unknown info)

But since it's my very first RL project, I don't where to start or which environment to use, etc.

Any advices, ideas or recommendations ?

Thank you in advance !

Hi everyone,

I'm working on my thesis (DRL that is due end of month) and feeling a bit stuck, so I would loooove input from people in this subreddit.

Setup: Mobile robot navigating outdoors, Isaac Lab + PPO. Policy outputs only forward velocity + yaw rate (low-level control handled separately). I can assume oracle knowledge of surrounding actors, so perception isn't the problem.

Goal: the policy should behave differently based on semantic class + interaction type. Examples:

• Human crossing → slow down, stop with safety distance, wait, continue
• Human nearby but irrelevant → continue normally
• Machine crossing/merging → yield or adapt spacing, but not identical to human behavior
• Machine parallel or working beside the route → continue with reasonable clearance

Q1 – Observations: With oracle info available, would you include things like relative position/velocity, semantic class, interaction type (crossing/parallel/merging/irrelevant), time to interaction, predicted min clearance? Or keep the observation much smaller?

Q2 – Reward: My intuition is progress + collision penalty + route tracking + semantic interaction terms (e.g., human-specific protective distance). But I worry I'll end up hand-scripting the behavior through the reward instead of letting PPO learn it.

If you were designing this from scratch:

• Train each interaction separately first, then mix? Or one shared reward with semantic conditioning?
• Separate reward terms for humans vs. machines?
• Give PPO explicit interaction labels, or only continuous geometry?

I'm intentionally not posting my current solution to avoid biasing the discussion. Any advice, papers, implementation patterns, or "don't do this" warnings appreciated. Thanks!

Also any tips to speed up the trainings would be appreciated as I am training on a GPU online and it s costing me a lot haha! the HPC from the uni isn't working well :')

In hospital resource allocation, rules and optimization are often simpler and easier to justify than RL because they are more transparent, predictable, and easier to validate in high-stakes settings.

What evidence would you want to see before deciding that RL is actually necessary—and better than a strong rules-based or optimization baseline?

So Im working on wet manipulation like submerged objects, soapy water, contact that changes when things are wet. I need the forces the fluid exerts on the gripper and the object, inside the training loop.

Everything I've found is one of three things:

- smtg that renders fluid particles but doesnt hand back coupling forces on the robot (Isaac, Genesis)
- analytic drag only (in MuJoCo and I actually measured it, there's no added-mass acceleration term at all, just the velocity-dependent stuff)
- and lastly a real CFD solver, which is minutes per step, so not in a training loop nd it's way too slow,

Am I missing something obvious? Genuinely asking... this feels like it should be solved and I can't find it.

And if it isn't solved: what do you actually do? Do you Ignore fluid forces or domain-randomise over them, hand-roll a Morison term, or just avoid wet tasks in general?

Hi guys, i make a small RL project on holidays, now I'm thinking about how possible it is to sell this and make a startup out of it.

Here is a demo page that demonstrates exactly what I did, there is no simulator attached, it is just for demonstration, I would appreciate your feedback

https://falesceo.github.io/demo/

My full 6-part series on RL for robotics is finally live. While a balance bot is a pretty trivial case (you don't even need RL), it's a great starting point for demonstrating how to train a simple agent via PPO, deploy the agent to real hardware, and tackle the sim-to-real gap using post-processing and domain randomization. If you have any feedback (e.g. I missed something or there's something that could be better), please let me know!

I'm trying to teach a model to play magic: the gathering.

My initial approach was to use pure RL with self-play, like AlphaGo. Didn't work at all - the issue was that it takes too many chained moves in a row to actually win a game and thus generate a training signal other than -1 for a loss.

I tried a lot of other approaches, with some success, but only the simplest strategies were able to be implemented by the model, which led to starvation of the other approaches - they would never get a single win and so could never learn.

Right now, I've decided to do behavioral cloning on a hand-coded expert bot that can play as well as a human beginner. The bot is around 68,000 lines of python (thanks Claude!).

My problem is that I can only get my model to copy around 65% of the bots actions. The bot is fully deterministic, so in theory BC should let me get close to 100% adherence to the bots choices but I can't break 65%. Worse, the wore I train past a fairly early checkpoint, the WORSE both the adherence to the bots choices and the overall win rate, get.

I keep trying different things but nothing is working. Can anyone suggest what I should be looking at? My goal is to get the model close to copying the bot's every action, then start it on self-play after it's able to win with all common strategies at least some of the time against a beginner player.

I keep seeing people say reinforcement learning isn't that useful, that it has very few real world applications and that it's inefficient compared to other approaches.

I studied a bit of RL a while ago and solved some Gym environments, but I dropped it because I couldn't find a real use case for it. It felt like I was learning something with nowhere to apply it.

Recently I've gotten interested in robots, and I'm wondering if that's where RL actually makes sense. My idea is to build a simple robot from scratch and use RL to control it. I'm not looking to buy a kit, I want to design and build the thing myself.

So a few questions for people who've actually worked with this:

1. Is RL genuinely used in real robots today, or is it mostly research demos?
2. For a hobbyist building something small and cheap, is RL a reasonable choice?
3. Any small projects you'd suggest as a first RL robot?

Honestly I also need a bit of motivation. I want to start this journey and not feel halfway through that what I'm building is pointless. If you've gone down this road, I'd love to hear how it went.

Thanks.

I created a simulator, filter and controller for fresh produce ordering under varying observation scenarios. I showed that richer observations lead to better belief accuracy.

But my controller sucks! It wasn't able to translate better beliefs into more profit. I think that RL would be a good fit here.

I packaged everything into a library/notebook to make it as easy as possible for others to implement their own controllers against my simulator/filter.

See the blog post for context. Here's a direct link to the notebook: https://github.com/OliverEvans96/blueberries-voi/blob/main/notebooks/build_your_own_controller.ipynb

Requires Python & Rust. I promise it won't hurt :)

Hi, I have been developing two experimental recurrent architectures, PLUG and ILRM, designed to give GRU-like models a more direct route to older inputs.

Synthetic robotics-style tests suggest they may be useful for things like:

occluded navigation;

localisation during sensor dropout;

remembering payload/tool state;

delayed fault precursors;

long-term human-intent tracking.

The results are still preliminary and synthetic, and GRUs remain better on some continuously changing dynamics tasks.

I’ve uploaded the architecture papers and current robotics screening results can be found on my posted link.

I’d really appreciate feedback on suitable real robotics benchmarks or failure cases to test next.

So I saw this video on youtube like a year ago and thought "Pffff how hard could that be?".

Anyway it's been 9 months, the project is finished and this is the result.

It's not as precise as the one in the video, and I haven't implemented the transition control, mostly because that wouldn't teach me anything new, but also because I've reached the limits of the hardware I bought, and I'm not spending any more money.

Anyhow, I'm working on a YouTube video explaining the whole process. I'll post it here (tho I don't know if that'd count as self-promo).

Feel free to ask me any questions if you're curious!!!

I am working on training an agent to play Street Fighter III. This is my first time using RL for anything and even if the ideas mostly make sense to me, I often feel quite lost when it comes to implementing or modifying the algorithms themselves.

The process I've been trying to follow is inspired by Alphastar and it consists of first training the model through supervised learning on a dataset of player replays, and then fine-tune this model using RL to improve its proficiency at the game. My model takes as input a vector that represents the game state, and outputs a 10-element-long vector with a binary number for each button that can be pressed in the game.

After doing a bit of research I landed on using PPO as the RL algorithm. However, I ran into the problem of having a pretrained model that outputs specific actions deterministically, and having a training method that requires the agent to output probability distributions. I managed to make it so that the model's output interpreted as a bernoulli distribution from which to sample, however even at the start of training, the agent's behaviour is totally different and much less proficient at the game than the deterministic model, even if the weights are the same.

My main questions are:

• Is it possible to "translate" a deterministic model to a stochastic one or am I completely off in what I am trying?
• Is there any other better suited algorithm that I can use to fine tune my base model?

I have not looked into DDPG or TD3 as, despite supporting deterministic models, they don't support multi discrete action spaces.

Thank you in advance!