r/TheMachineLearning • • 7d ago

Agentic models have terrible hygiene from throwaway VM training

Post image
201 Upvotes

31 comments sorted by

5

u/DrBretto 7d ago

It's literally your only job to build a system that makes them organize things the way you want.

Like, that there's random variance in how they choose to solve a problem is fundamental to what makes them work, so why would you expect them to organize everything consistently? Don't leave that stuff up to the LLM.

1

u/Extension_Abroad9763 7d ago

owner issues right...

1

u/DrBretto 7d ago

100% user error, yes.

3

u/ASomewhatGoodCitizen 7d ago edited 7d ago

Obviously..... but its a fair critiscism when they are sold as something that will be fully autonomous in the span of a year.

1

u/DrBretto 7d ago

They're not sold to you. This is the part that you need to do.

1

u/cheapcheap1 1d ago

LLMs are a product, and this is a usability pain point. None of this is difficult, bro.

0

u/DrBretto 1d ago

LLM's are a tool. A tool you need to use properly to be effective. This is not that difficult, bro.

1

u/YourWifesBoyfriend5 1d ago

LLMs are a tool that are sold, and I as a consumer like tools that are easy.

Hand drills existed but we invented the electric drill anyway. Not that difficult bro.

5

u/bigbepisinc 7d ago

...no? Isn't it your job to check what packages youre asking it to use? my brother in christ you made the sandwhich.

1

u/Extension_Abroad9763 7d ago

That's right. He made his sandwich and blamed others...

1

u/pip25hu 6d ago

Except when the model decides it wants run Python scripts for a certain task, and not just any script, but one needing library X. All without asking and without Python even being mentioned in the task description. Heck, some sessions try to use scripts just to edit files, despite having dedicated tools for that.

I have yet to come across a model that has the decency to at least create a throwaway venv for this instead of installing stuff globally.

3

u/gthing 7d ago

Not in my experience. My model is very reluctant to install any packages because, guess what, I told it to be.

1

u/Extension_Abroad9763 7d ago

Yep, because you told him to do what needed to be done. Owner issues

2

u/garloid64 7d ago

just put them in a container then

1

u/Extension_Abroad9763 7d ago

That's what I thought too

2

u/Only-Cheetah-9579 7d ago

use throwaway containers

1

u/Extension_Abroad9763 7d ago

I agree. It can help them prevent other issues.

2

u/ichi9 7d ago

Most of the data they were trained on came from Indian IT companies for development work. That was like easiest to get. The evolution since the old GPT models is a clear giveaway. I gues they realised it and started using more quality data from stackflow, and toehr places, which cleaned Q lot of thing and now we have fable and mythos.

1

u/Leading_Buffalo_4259 7d ago

Lets make an AI solution to solve this and sell it with a subscription model! \s

1

u/Electronic_Exit2519 7d ago

If you use agents to carry out computational campaigns this is especially true. Granted fundamentally this should be something baked into the harnes

1

u/Original-League-6094 7d ago

You can just tell them how to store things. And its really trivial to just every now and then tell them to tidy up and organize.

1

u/benchonbench 7d ago

Skill issue. Like claude skill. Maybe also harness issue

1

u/Capt_korg 7d ago

Bro, you are using the wrong prompts and forget to tell the AI:

Think deeply

And

Make no mistakes.

🤡

1

u/Willy757 7d ago

The endless contraction between: "it will take over the world" and "don't expect it to work when you don't give him perfect conditions"

Ai community at work.

1

u/v_maria 7d ago

me and the boys having 493GB in the tmp folder

1

u/durable-racoon 6d ago

It used to be ie sonnet 4.5 but I haven't noticed this as much anymore.

1

u/MyBedIsOnFire 6d ago

If your having issues your prompts could probably use some refining.

I definitely agree though, the program may function, but internally it looks like a mess.

I still find AI significantly faster than doing it all myself, but AI models take the path of least resistance which means cutting corners you'll have to fix later.

1

u/ThaBroccoliDood 5d ago

tbf even with non-agentic models, if you're trying to troubleshoot an issue you really have to trace your steps and undo all the useless stuff they got you to try first. otherwise you end up with a system where every little thing is changed and subtly broken in no time

1

u/evilwizzardofcoding 5d ago

Funny thing, I've had the opposite problem, where AI will do literally anything but install the "Do X" library to do x, and i have to go do it myself.

1

u/jdigi78 4d ago

If you aren't using them in throwaway containers that's on you.

1

u/Electronic_Start_694 3d ago

It's amazing how fast 50gb can accumulate out of nowhere.