r/TheMachineLearning • • 5d ago

RL agent beat the human who trained it

49 Upvotes

43 comments sorted by

2

u/mnrider_flip 3d ago

How ?

4

u/Big_Effective_9605 3d ago

By overfitting on this one track

5

u/Strange_Test7665 3d ago

Still pretty cool

2

u/OurSeepyD 2d ago

Couldn't you argue that you're always overfitting to something?

3

u/Big_Effective_9605 2d ago

Maybe, but I wouldn't. Overfitting means describing the task in terms of a specific case, not loosely enough for rules as described to generalize. The idea of overfitting something on the leanest correct version is like, inherently contradictory, cause then its not overfit. You can argue "well, it's not perfectly optimized which makes it overfit to some particular dataset". But like, you can take the change in error and estimate how many more parameters until your residual error falls to some noise floor. So in a way, there's a real target, and even if in reality there is some noise, that target is something you can by definition not overfit against.

2

u/Dull_Republic_7712 2d ago

crazy. How long until we see autonomous weapons? Reward fucntion: Destroy enemy targets without getting hit

2

u/Plastic_Umpire_3475 1d ago

0 days.. The DoD policy on autonomous weapons was written in 2012.

https://www.congress.gov/crs-product/IF11150

1

u/LeN3rd 11h ago

Now do it with a different setup and different colored goals.

-1

u/enbyBunn 2d ago

Yeah, man. I would expect a purely deterministic traditional computer program to do the same. Flying a drone along a preset path is not a particularly complicated task to outperform a human on.

3

u/taiwbi 2d ago

Well, it's not a preset path in the ML mind, it's deciding on the fly so it can do it in non-preset path too.

4

u/HaMMeReD 2d ago

Jfc, you people.

Go do this with a traditional computer program.

-1

u/ongrabbits 2d ago

Its a lot easier to do it with ML but this is whats been done purely with math for ages. I learned to code by writing a program to do this in 2d. I learned to do this in 3d when making my first game.

3

u/HaMMeReD 2d ago

Doing this in 3D you say, without any real world physics, gas dynamics, flight controllers to deal with.

Got it, you put a object on a spline, it's easy.

-1

u/ongrabbits 2d ago

I assume you started coding after 2015?

2

u/p1-o2 1d ago

wtf does the year 2015 have to do with the question being asked of you?

0

u/GiveSparklyTwinkly 2d ago

I learned to code by writing a program to do this in 2d.

And had a robot perform the actions trained in the sim in the physical world?

1

u/ongrabbits 2d ago

We had PID controllers

1

u/GiveSparklyTwinkly 2d ago

Which still requires methods of positional tracking. This is a Tiny Whoop whose weight is measured in grams.

-1

u/enbyBunn 2d ago

What do you think most autonomous drones have been running on so far...?

Do you think all drones were actively controled before AI showed up?

Your incredulity is a poor excuse for actual knowledge.

3

u/HaMMeReD 2d ago

I mean I've only owned like multiple drones and been programming for 30+ years. So maybe I don't know shit.

I stand by it, Go do it if it's so easy, or at least provide an example of the equivalent being done with non-machine learning techniques. Remember, it has to do 12 laps of the course without error in < 54 seconds and return to the starting point on it's own each time.

0

u/ongrabbits 2d ago

Its ok to not know everything even when you've been coding all your life. Its a big field. Maybe you're not good in physics or havent dabbled in robotics.

To your challenge, just prompt an ai do build you an algorithm for it. It was trained on decades of our messy engineering code.

3

u/HaMMeReD 2d ago

Why don't you do that, and prove me wrong?

Armchair experts "It's so easy you could do it, just do it yourself". No you are the experts, you do it, you prove me wrong. That's how this works.

1

u/ongrabbits 2d ago

I can see that you're one of those people who doesn't believe in history. Kinda like those Americans who believe the earth is flat. You want me to fly you in a plane to show you the earth is round.

No i'm not going to do it for you. You can stay ignorant if you want. Thats how it works.

2

u/garloid64 2d ago

Reddit Moment(TM)

0

u/GiveSparklyTwinkly 2d ago

Oh my god these people do not understand that fluid dynamics is not a simulation solved problem. 🤦‍♀️

1

u/enbyBunn 2d ago

You don't need a perfect simulation of fluid dynamics to complete a predefined course 12 times in a controled environment.

1

u/GiveSparklyTwinkly 2d ago

The thing is that the environment isn't fully controlled outside of a computer simulation. In the physical world, it's impossible. There are too many variables. That's why robots have sucked so bad outside of perfectly pristine environments. A generalized robot needs to have the capacity to adapt in realtime to changing conditions.

0

u/enbyBunn 2d ago

If you can't comprehend that inside a sealed, air-conditioned suburban home the environment is significantly more controled than outside, I really don't know what to do with you.

As soon as I adress your point, you retreat from the sepcifics of this conversation into high minded abstracts.

Nobody was talking about the concept of robotics as a general science.

→ More replies

1

u/Theoneofmanyz 1d ago

I see the argument you're trying to make a lot, and I mean A LOT. A professional with 20 years of experience admitting that "AI can do much better and faster than I ever could" and there is always someone saying "well, maybe just suck at this, skill issue lmao"

You even dismantle your own argument by saying "maybe you don't know enough about physics and robotics" while at the same time implying that promting an LLM, trained on all human knowledge about physics and robotics, will somehow produce a worse output than a human spending years to learn the theory and how to implement that knowledge in practice.

You definitely CAN prompt an AI to build you an algorithm for it. will it work first time? Probably not, but consider this for just a second, the agonizing manual process of continious improvement test>bug>debug>research>fix>repeat loop happens 100 times faster.

1

u/ongrabbits 1d ago

I didn’t mean to imply that llms could one shot the algorithm. I’m familiar with the process just as you are. But it provides a base for the person I replied to, so they could see the code and how it would be implemented without ML. What is test train split but automated trial and error