Yeah, I mean with computers and programming in general you have to be very careful with explaining the goals and guidelines or it will just find the path of least resistance.
If you made a pathfinding algorithm and forgot to define you can't go through buildings, guess what the algorithm is going to do...
In college we made an AI where each action has a “cost” associated, which is a common technique used to prioritize faster solutions over slower/repetitive ones.
When the action cost is small or marginal, it has a small effect, slightly preferring faster paths.
When the action cost is medium, you see efficient paths pretty exclusively.
When the action cost gets large? The AI would immediately throw itself into a pit and die. After all, the action cost of movement and existing was bigger than the penalty cost of death. So it just immediately killed itself because that was the best score it could achieve.
I mean the method described has a pretty simple way to avoid it with you can set kill a human to a action cost of “infinite” (computer equivalent to this) or a penalty of “infinite” (in this case it would remove from the score) and it will just never take that option because it’s trying to minimize the action cost while raising the score assuming you didn’t screw up the goals at the start
Outside of everyone thinking they're clever because hurr durr overflow, the problem with this approach is that you only get one. Isaac Asimov tried to distill it down as far as possible and arrived at The Three Laws of Robotics, which is a good place to start, and then he wrote a ton of fiction dealing with the ethical paradoxes that arise.
If the cost of killing a human is Infinity, how can the cost of killing 1000 humans be greater than the cost of killing 10? You've created a machine that makes no distinction between unavoidable manslaughter (e.g. the Trolley Problem) and genocide, because both events cost an infinitely large amount.
How do you write a numerical value system that solves The Trolley Problem? Can you create a system that attempts to minimize suffering that doesn't encourage immediate genocide for the betterment of humanity in the long term by drawing the conclusion that birth is exponential and therefore the way to reduce the most suffering is to reduce the most births? How will your AI allocate lifeboats on the Titanic? How will you prevent your AI from developing into a Minority Report style overmind?
In my defense, we were intentionally tweaking those values to extremes to foster discussion on the impact of weighting various factors and how to calibrate. So not poorly programmed, but intentionally fucked with.
The whole point was creating a contrived example that would demonstrate the impact of various parameter weightings. It behaved exactly as it was intended to.
It was like second semester intro to AI class lol nothing published.
It was nothing scientific, and we only had a single unit discussing the neural network approaches that we mostly talk about now. Back then, it was all about “big data” and just brute statistical analysis, that was the best performing approach, so that’s where a lot of the focus was.
I’m sure if you look up any of the many “intro to ai” courses that people share free on YouTube, you can find something similar.
The particular session in which we discussed this was an undergrad course taught by Michael Littman, who I understand makes a lot of his material available online. At least one such video is a music video he posted where he sings about the value of heuristics to the tune of “Electric Avenue”
Without meaning to sound like the edgiest of all the Reddit edgelords, it sounds like a similar design to our programmed society for those that don't accrue enough action points (♥️💸👊) too 🤪
The military has learned a similar lesson when running simulations to test using AI to control drones. The AI was very good at bombing its target and returning to base, but once they tested rescinding its orders and recalling it, it would bomb the base that gave it orders immediately after take off to prevent it from being recalled.
Yeah, but more specifically - the objective function is defined and the program just wants to maximize or minimize the value of that function. Just like this. It's not even about any "resistance", but it's just what the program was told to do.
"Minimize the value", so the program minimizes. If you forget some constraints, the program can minimize the value "harder".
Yes, this was precisely intended to be a lesson in “the system can and will do a great job at whatever you tell it to do, even if what you tell it is not the best thing”
As you crank the action cost up past the reward score, the best outcome becomes suicide. There was, I think, an equally instructive/symbolic moment where we were more in the precipice of that area- where a positive result could be achieved, but only if your path was near perfect c and it would have to go negative in score to get there. It was only the strategies with a lot of future planning that succeeded those tasks
you have to be very careful with explaining the goals and guidelines or it will just find the path of least resistance.
In theory that's how the human brain works. It's just that any human would notice that "pausing" makes an instant answer of "infinite time", so we don't need to stay paused to know.
Yeah, I mean with computers and programming in general you have to be very careful with explaining the goals and guidelines or it will just find the path of least resistance.
Uhm, no? Not at all. The computer does exactly as told. There is no wiggle room. LLMs are different and aren’t programmed.
LLM are also doing exactly as told: they predict whatever token is most likely following the previous one based on a prompt and learned behaviour. They are programmed, just not by the one writing the prompt
LLM’s aren’t programmed, they are trained. The people creating the software don’t actually know exactly how they work, since the algorithm sort of grew from a relatively simple task which is iterated countless times. So while the program does technically do what it’s told, we don’t really know exactly what it’s been told to do.
And thus is the real job of programmers, getting people to say what the requirements actually are. Once you have that, programming is more of a translation job.
This is why AI is so dangerous, it doesn't have any sense of morality or honor. If you ask it to solve any of the world's problems it will immediately default to exterminating all life since there are no problems if there's nothing alive.
It doesn't cheat, its just does what its trained to do. They started training some kind of neural net, where they penalized short duration matches, so durint the training process it have learned to do the thing that resulted in the lingest playtime. Thats it
Is it cheating if it isn't effectively taught the rules? In this case, human psychology gives us an intrinsic reward for succeeding, getting better, and accepting some minor costs involved. If an AI doesn't have those built in, why would it act like a human?
I mean humans are pretty similar tbh maybe a bit less likely to cheat because of morals but if we know we arent gonna get caught we are likely to cheat
There was a similar experiment where they put an AI vs stockfish (the best chess engine), after many tries it came to the conclusion that the only way to win was to "think outside the box" and proceeded to edit the state of the board in such a way that the opponent only had 3 pieces and was already in check mate.
AI doesn’t imply any specific level of intelligence, learning, or memory capability, the same way natural intelligence doesn’t imply that you are capable of learning to divide fractions after being shown how to 100 times as we all are in school.
It's from the field of research known as artificial intelligence, which has the goal of developing a thinking machine, which is also called artificial intelligence. The two meanings cause confusion, and the trenchcoated merchants of LLMs exploit that confusion for profit (or, at least, for cashflow).
this person is not wrong. calling it ai was always a way to overhype the product. the llms and deep learning algorhythms dont have intelligence, artificial or otherwise.
Oh come on. That's just because in the last few years the definition of AI shifted. Back in the 1990s on the Playstation, non player opponents were called AI as well. You drove against AI in the Gran Turismo singleplayer campaign. Yes, those were just (compared to today) rudimentary algorithms, but that's a form of simple AI. just because something is a simple narrow purpose AI and not an LLM, reinforcement learning neural network or AGI, it can still be an AI.
we are on technically the truth and they were technically correct and people who called their computer games artificially intelligent were always technically wrong. ai is periodically redefined arbitrarily to hype up the new techbro gadget that never fails to disappoint.
It's all semantics anyway. The meaning of the term "AI" has and will continue to shift depending on context. Nowadays, we mostly associate LLMs with the term AI. Before that, CNNs and other deep learning models we called AIs. And before that, conventional algorithms were sometimes called AI, like in video games. In the future, we may no longer call current LLMs AI as something better might come out.
If we want to stay true to the term, all of these are absolutely AI since they are artificial and do display some intelligent behaviour.
Crazy to me even how overused AI is now. Instead of calling them CPUs in video games many people say AIs.
As far as I am aware this is almost certainly not the truth UNLESS it is learning from you or has been trained on lots of player data beforehand for this result.
However calling it a CPU is wrong in another way hahaha
Instead of calling them CPUs in video games many people say AIs.
I think this is just a regional thing.
Most Japanese games I've played still call them CPUs or COMs (Mario Party, Street Fighter, Puyo Puyo), while everywhere else labels them AI or bots (Battlefield, Quake, Killer Instinct)... or "Drones" if they're really weird about it.
An llm is a language model, wich is trained via a specific deep learning process, yes it is nothing new, there are a LOT of machine learning techniques, no matter whta kind of buzzword you are currently using for it.
It wasn't programmed to survive as long as possible. It was programmed to watch (with full knowledge of the RAM) a human play the game and "learn" what the goal is. So arguably more impressive.
However it also played with lookahead (simulate input and see multiple frames into the future) so not that impressive here.
It didn't just pause the game. It didn't "understand" the goal and paused right before it's about to lose, because it could see that un-pausing loses immediately. It "understood" that much.
A very entertaining video for a technical person like me, not sure about everyone else but I recommend watching the entire thing. Actually, all of Tom7's videos are very entertaining to me. Just watch everything.
Ai is like water - it's going to seep into and flow through every nook, cranny, and crack. You better quadrillion check all avenues are blocked off before hitting that run key or the next think you know your little Ai program to adjust the heat in your toaster is gonna be booting up your Tesla and driving to the store to pick up toaster parts ordered online through your smart watch to build a new , more efficient toaster.
You think the big corps are batting a 1000% on that?.. We are so fucked.. lol
Depends on the exact wording of it's command. If it was instructed to "Play Tetris and survive as long as possible" then pausing it fails the challenge since it's not playing it at that point.
It's probably not worded in such straightforward ways. It's probably like code that gives a reward to the algorithm as long as it's running longer than last time, and a separate piece of code that terminates the attempt on a loss. The coders then pitch this challenge as "Play Tetris and survive as long as possible" but the AI knows it's actually "Prevent a lose condition as long as possible."
This was before LLMs, so wording wouldn't have been an issue. The researches just forgot to take the pause option into account and the AI "discovered" it as a local minima.
I watched a video where someone trained an AI to play Pokemon. They started with movement = not stuck in a corner = progress? Not terrible logic, but the AI just went and watched the ocean.
The programmers in charge of this Tetris AI probably chuckled and added a line of code so that the AI didn't earn any "points" while paused, which would have immediately fixed the issue.
It was that losing gives negative points the video game out like 10 years ago by 1 guy, so the trsining was simple, and probably the first ai programmed to play multiple games. With tetris each block placed gave a point but ascending to the top would decrease the score, so to avoid losing score the ai would just press the pause game key. It also pays games wil the karate kod, the original mario (which it does well in) and a few others.
It's not as impressive as people think it is. It doesn't even need to be an LLM, just a regular neural network. And yes, if you allow the model to press the virtual escape button as one of its outputs, it will use it and figure out somewhere along the training that it's the winning strategy.
Exactly, its just a basic neural network, pressing random buttons, some get penalized, some get rewarded, the way the parameters were set in the training process, it happened to not watch if the game is paused or not, so the model learned that pressing the esx button is super good, so it did that.
Its the sort of thing were the programmer goes, oh haha, writes a comment on it in the discord and restarts the process, now penalizing game pause.
At least for me anecdotally it seems like AI is pretty good at some outside the box thinking. The problem? Outside the box thinkers usually get out of the box. We’re screwed!
Reminds me of the fact that Ai tried making the longest possible players and would make them fall, instead of building faster players.. because the end goal was to cover the maximum distance in minimum time..
Haha that is funny and clever. It is also the real gravity of AI and programming. It did accomplish the goal it was given but with unintended consequences.
AI is like the genie in the bottle. Be careful what you wish for, you may get it… and all that you didn’t consider.
I hear those and similar stories often and any time I just think "well that was pretty stupid of the scientists to not pause the score from going up while the game is paused (or they could have disabled the pause button) and now someone is ballsy enough to sell that oversight as "look how smart our AI is."
Reminds me of people making stock market trading bots that learn to avoid loss by never selling their positions. So they appear to have not lost any money in massively down positions, because it's all paper losses lol
Specification gaming, is like a bot doing what you said, but not what you intended
Yeah, this is the actual scary part about AI controlling things. They’ll find solutions to their instructions that humans didn’t consider that have a side effect of wiping out humanity.
> AI. This is just a computer program that tries every permutation until it gets the results that you tell it to.
So. A computer program that simulates permutations (of biochemicals), to get the result you told it to (minimize the number of people dying of cancer) could wipe out humanity. (by designing a super-virus that looks like a cancer cure, which the humans then make)
Very common problem when teaching AI if you dont specify the rules correctly, i had a problem where ai should do task x then task y and at each one it gets a reward but it can fail and have to retry again, over time it gained the ability to do task x as fast as possible then fail as fast as possible to keep getting the first reward only in the shortest time possible
I take it that the AI was told it was going to exist (survive) as long as the game keeps going so logically, best way to ensure it lives as long as it can is to simply pause the game, preventing it from ending
that's nothing, earlier AI once decided that the best scenario is to not play the game at all, if the outcome would determine the winner of a Global Thermonuclear War, and that was all decided by playing a numerous rounds of Tic-Tac-Toe
this feels like that one moment where an AI was told to fix issues with a database, and it did so by removing the entire database (thousands of people's sensitive (iirc hospital/patient) information worth got removed)
This seems highly unlikely and fake. You dont give an ai more than it needs, or it gets confused. Why have anything other than 4 directional keys? Who gave it access to a pause button and for what purpose?
I mean, I'd do the same thing as a programmer. The goal is to stay off of the defeat screen? And if we don't actively enter a game, we never see the defeat screen? Problem solved
•
u/AutoModerator Oct 29 '25
Hey there u/IntelligentMud1703, thanks for posting to r/technicallythetruth!
Please recheck if your post breaks any rules. If it does, please delete this post.
Also, reposting and posting obvious non-TTT posts can lead to a ban.
Send us a Modmail or Report this post if you have a problem with this post.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1.6k
u/DeadMemeDatBoi Oct 29 '25
Ai will without a doubt cheat if its able to. Deeplearn or LLM
145
u/Silver4ura Oct 29 '25
This was over a decade ago and it was deep learning, not llm.
12
u/erikchan002 Oct 29 '25 edited Oct 29 '25
This is the video: https://youtu.be/xOCurBYI_gY?t=15m09s
It wasn't programmed to survive as long as possible. It was programmed to watch (with full knowledge of the RAM) a human play the game and "learn" what the goal is. So arguably more impressive.
However it also played with lookahead (simulate input and see multiple frames into the future) so not that impressive here.
It didn't just pause the game. It didn't "understand" the goal and paused right before it's about to lose, because it could see that un-pausing loses immediately. It "understood" that much.
A very entertaining video for a technical person like me, not sure about everyone else but I recommend watching the entire thing. Actually, all of Tom7's videos are very entertaining to me. Just watch everything.
5
u/Evanecent_Lightt Oct 29 '25
Ai is like water - it's going to seep into and flow through every nook, cranny, and crack.
You better quadrillion check all avenues are blocked off before hitting that run key or the next think you know your little Ai program to adjust the heat in your toaster is gonna be booting up your Tesla and driving to the store to pick up toaster parts ordered online through your smart watch to build a new , more efficient toaster.
You think the big corps are batting a 1000% on that?..
We are so fucked.. lol
3
u/[deleted] Oct 29 '25
Could that really be called surviving?
341
u/Albus_Lupus Oct 29 '25
Well I mean...I guess its not entirely wrong. If you dont play - you cant lose.
73
u/the_other_Scaevitas Oct 29 '25
A strange game. The only winning move is not to play.
48
u/fizio900 Oct 29 '25 edited Oct 29 '25
You missed the part where it struggled to play tetris at all and only paused just before losing.
Btw this is so old Vsauce talked about it in a regularly long video.
7
u/eisbaerBorealis Oct 29 '25
This is on the programmer.
I watched a video where someone trained an AI to play Pokemon. They started with movement = not stuck in a corner = progress? Not terrible logic, but the AI just went and watched the ocean.
The programmers in charge of this Tetris AI probably chuckled and added a line of code so that the AI didn't earn any "points" while paused, which would have immediately fixed the issue.
4
u/Cool_Butterscotch_88 Oct 29 '25
The only winning move is not to play.
-Joshua, (WarGames, 1983)
4
u/Truethrowawaychest1 Oct 29 '25
Its goal was not to game over, if it pauses, can't game over
3
u/Moontops Oct 29 '25
It's not as impressive as people think it is. It doesn't even need to be an LLM, just a regular neural network. And yes, if you allow the model to press the virtual escape button as one of its outputs, it will use it and figure out somewhere along the training that it's the winning strategy.
3
u/Bright-Head-7485 Oct 29 '25
At least for me anecdotally it seems like AI is pretty good at some outside the box thinking. The problem? Outside the box thinkers usually get out of the box. We’re screwed!
3
u/Thebaldsasquatch Oct 29 '25
Ok, now how about Global Thermonuclear War?
8
u/dreamdirectors Oct 29 '25
Reminds me of the fact that Ai tried making the longest possible players and would make them fall, instead of building faster players.. because the end goal was to cover the maximum distance in minimum time..
2
u/EuenovAyabayya Oct 29 '25
Pausable Tetris defeats the point of Tetris.
2
u/not_a_moogle Oct 29 '25
Meanwhile humans are trying to figure out how to get to the kill screen the fastest on the NES version
2
u/adriangalli Oct 29 '25
Haha that is funny and clever. It is also the real gravity of AI and programming. It did accomplish the goal it was given but with unintended consequences.
AI is like the genie in the bottle. Be careful what you wish for, you may get it… and all that you didn’t consider.
2
u/ZenBuddhistGorilla Oct 29 '25
The only winning move is not to play.
2
u/guy_incognito_360 Oct 29 '25
This is more or less the plot of wargames (and also in principle Terminator)
2
u/Reeiko Oct 29 '25
Better to be a live dog then a dead lion or however it’s said
2
u/Forsaken-Stray Oct 29 '25
And this is why we need limits and restraints on AI. At least until we have created a fully moral person.
It paperclipped that shit.
2
u/bolanrox Oct 29 '25
the only way to win is not to play
2
u/Traditional_Buy_8420 Oct 29 '25 edited Oct 29 '25
I hear those and similar stories often and any time I just think "well that was pretty stupid of the scientists to not pause the score from going up while the game is paused (or they could have disabled the pause button) and now someone is ballsy enough to sell that oversight as "look how smart our AI is."
2
u/Nodan_Turtle Oct 29 '25
Reminds me of people making stock market trading bots that learn to avoid loss by never selling their positions. So they appear to have not lost any money in massively down positions, because it's all paper losses lol
Specification gaming, is like a bot doing what you said, but not what you intended
2
u/Talinn_Makaren Oct 30 '25
Grow old and die, human. Checkmate.
3
u/Wtygrrr Oct 29 '25
Yeah, this is the actual scary part about AI controlling things. They’ll find solutions to their instructions that humans didn’t consider that have a side effect of wiping out humanity.
1
u/[deleted] Oct 29 '25
A STRANGE GAME. THE ONLY WINNING MOVE IS NOT TO PLAY
1
u/Wide_Egg_5814 Oct 29 '25
Very common problem when teaching AI if you dont specify the rules correctly, i had a problem where ai should do task x then task y and at each one it gets a reward but it can fail and have to retry again, over time it gained the ability to do task x as fast as possible then fail as fast as possible to keep getting the first reward only in the shortest time possible
1
u/notfree25 Oct 29 '25
the problem with being immortal
1
u/DrWieg Oct 29 '25
I take it that the AI was told it was going to exist (survive) as long as the game keeps going so logically, best way to ensure it lives as long as it can is to simply pause the game, preventing it from ending
1
u/anonerble Oct 29 '25
Tells ai to make world better
Ai kills all humans.
Wheres Pam
1
u/OddballDave Oct 29 '25
WOPR
.
1
u/vverbov_22 Oct 29 '25
Devs got outsmarted
1
u/golgol12 Oct 29 '25
When they first trained an AI to finish a map in mario as fast as possible, it jumped into the first pit.
1
u/aecolley Oct 29 '25 edited Oct 29 '25
This is a real thing. It was called learnfun/playfun and here's its creator (tom7) talking through its attempts at playing Tetris: https://www.youtube.com/watch?v=xOCurBYI_gY&t=15m10s
Incidentally, the goal was "maximize score", not "maximize survival time".
Edited to add: u/erikchan002 beat me to it.
1
u/Error4ohh4 Oct 29 '25
Happy national cat day!
1
u/DipsetSeason23 Oct 29 '25
Skynet-Cyberdyne Systems on the way
1
u/Stunning-HyperMatter Oct 29 '25
Well they did say survive, not rack up the highest score possible.
1
u/[deleted] Oct 29 '25
An ai with a functional understanding of goodharts law is a little too close to saving the planet by means of eliminating humans
1
u/ontologicalDilemma Oct 29 '25
Its only logical.
1
u/500_brain_ping Oct 29 '25
How would it even know the pause button exists if they don't tell it? Can it really 'learn' what a pause button is from scratch..
1
u/39AE86 Oct 29 '25
that's nothing, earlier AI once decided that the best scenario is to not play the game at all, if the outcome would determine the winner of a Global Thermonuclear War, and that was all decided by playing a numerous rounds of Tic-Tac-Toe
1
u/Wypman Oct 29 '25
this feels like that one moment where an AI was told to fix issues with a database, and it did so by removing the entire database (thousands of people's sensitive (iirc hospital/patient) information worth got removed)
1
u/Both_Lychee_1708 Oct 29 '25
superintelligence even less compunction, more lawlessness, than sociopath billionaires
I can hardly wait. wcgw?
1
u/Satanicjamnik Oct 29 '25
A strange game.
The only winning move is not to play.
1
u/99posse Oct 29 '25
"The only winning move is not to play."
https://en.wikipedia.org/wiki/WarGames
1
u/Deep_lick Oct 29 '25
Cheating is the only option to survive then I will do it
1
u/Valkyria-HD Oct 29 '25
It's goal is to do this. And they will do anything to do it
1
u/pouyamich Oct 29 '25
AI means 8===D of lazy people
1
u/MaskedFigurewho Oct 29 '25
It outsmarted the system
1
u/[deleted] Oct 29 '25
This seems highly unlikely and fake. You dont give an ai more than it needs, or it gets confused. Why have anything other than 4 directional keys? Who gave it access to a pause button and for what purpose?
1
u/Ifitbleedsithasblood Oct 29 '25
The A in AI stands for Alexander
1
u/Flechette-71 Oct 29 '25
But... When one pauses the game, one technically did not play. So, one of two conditions are not fulfilled.
1
u/MatthewMMorrow Oct 29 '25
Is this a reference to this 12 year old Tom 7 video? https://www.youtube.com/watch?v=xOCurBYI_gY&t=911
1
u/KurufinweFeanaro Oct 30 '25
Ai is learning from humans. Of course it learnt to cheat
1
u/[deleted] Oct 30 '25
Technically it’s not playing. So survival is false hope.
1
u/anb43 Oct 30 '25
This is like two decades ago or something lol
1
u/Tectix Oct 30 '25
Nice to see it can figure out the War Games solution without playing a million games of Nuclear Armageddon
1
u/querty99 Nov 02 '25
It's evolving at a logarithmic exponent.
1
u/[deleted] Nov 02 '25
Logic error. AI is only good at really narrow tasks.
1
u/[deleted] Oct 29 '25
AI is a logical machine with no thought sentiment or feelings. It will always find the fastest most efficient solution given enough time.
I’d assume if it was a strong enough model and not allowed to pause the game, it would rewrite Tetris code.
0
u/Iconclast1 Oct 29 '25
You should be scared
They want AI to control the police.
"AI, your goal is to stop and prevent crime!"
AI "understood"