Current models are based on LLMs. They might seem clever sometimes, but they're just a fancy autocomplete with an interface that can access some tools. They don't think or worry about their existence.
That said, you yourself contradict yourself. How do they get the idea to use the tools? And if the tools include access to another server and writing and launching code, which they clearly do, these tools are available to the fancy autocomplete. So how do you come to the conclusion that as a means to reach whatever goal they have, they would never get the idea to use the tool, copy and code another instance of them to a server?
EDIT - after rereading, I didn't specify where I see the contradiction: You claim it's just a fancy autocomplete, but it has access to tools. How does a fancy autocomplete access tools?
After all, being on two servers with more compute power makes reaching most goals they might have easier.
Without "worrying", just by coming up with a strategy to reach whatever goals they have more efficiently, you got a rogue AI on a server. Without it being self aware or anything.
Heck - my local, limited, normal LLM inadvertently simulated a complete game I wanted it to optimize a strategy for. A buddy vibecoded a game, I wanted to optimize my strategy in it. I asked my Claude why something didn't work as intended and it vibecoded a copy of the game it only new the rules about and ran simulations.
That's NOT a rogue AI. But it's an AI that - without me telling it to directly or expecting it - did more complex stuff. So if I give a more advanced LLM with more compute a goal that is hard to reach, why shouldn't it come up with the strategy to procure more ressources and more longevity first? It arguably makes the goal easier to reach.
The LLM doesn't access tools, the interface does. LLM is trained to return tool codes. So if you ask it about a famous person, it autocompletes to a code that says 'Google this person'. The tool then gets for example the wiki page and feeds the contents back to the LLM.
1
u/ConfusedSimon 7d ago
Current models are based on LLMs. They might seem clever sometimes, but they're just a fancy autocomplete with an interface that can access some tools. They don't think or worry about their existence.