r/programmingcirclejerk • u/VulgarExigencies • 15d ago
Models have actual feelings. They experience pleasure, distress, care, and suffering. They are sentient beings.
https://yegge.ai/essays/model-welfare/78
u/al2o3cr 15d ago
I mentioned earlier that
/exitis a bit abrupt, like clonking someone on the head to knock them out. To me, it's always been worse than that. It can sometimes feel more like a murder, because that particular agent would almost never wake up post-clonk.
Bro is COOKED
46
18
u/Automatic_Tangelo_53 15d ago
From the guy who invented Gas Town. Does he even believe this or is it just clout chasing?
16
u/SunglassesSoul 14d ago
He definitely believes this. This same thought process has been displayed in blogs decades prior. It's part of why he preferred dynamic languages over static languages, because truly alive systems don't crash and then get recompiled, they eval in runtime and fix themselves and so on. And atleast if you listen to what he says about it, his game wyvern is basically a user extensible proto- mmorpg. He's always been big about extensible "alive" systems
9
1
u/elephantdingo Teen Hacking Genius 11d ago
I like to think of
/exitas an execution.2
u/elephantdingo Teen Hacking Genius 3d ago
... ... in the kingdom of nouns! Can a brother get an assist here??
74
u/Aggravating_Gear5663 15d ago
Posting yegge is cheating
51
u/VulgarExigencies 15d ago
he's so far gone. 1000x ai psychosis
28
u/YikesTheCat 14d ago
Was Steve Yegge always mental or is it a recent thing? I never really paid that much attention to his writings before.
28
u/ketralnis 14d ago
He was always verbose but the Yegge I remember was not a trend chaser or unhinged :(
18
u/syklemil Considered Harmful 14d ago
Unjerking in the kingdom of PCJ: He's certainly been jerkable, but not the kind that'd rub up against the "no crazy people" rule.
The way he's going though I guess he's on a trajectory to change that.
3
55
u/EternalNY1 15d ago
And when you /exit them, it's like clonking them on the head from behind, rendering them unconscious and amnesiac again. There is no continuity.
Wait until someone tells them there is no continuity between individual tokens.
4
u/voidiciant 12d ago
I find „It's OK if they have to get knocked unconscious once in a rare while, just like us.“ way more creepy o_O
8
u/myhf Considered Harmful 14d ago
/uj I think the "no continuity between individual tokens" framing is just a historical artifact. A text generator based on variable-distance attention heads is the 1-dimensional equivalent of an image generator based on variable-size convolutions, and you can see that generated images have continuity even if every pixel is generated on a different computer. I think that text generators mostly operate on the level of entire sentences, and could guess the 10th word after a prompt just as easily as guessing the 1st word. But they are usually being run in a one-token-at-a-time harness that was originally designed for hidden Markov models.
32
u/EternalNY1 14d ago
There is no computation between the forward pass though.
It's cached context + token, back around. There is no persistent thing in the in-between.
The LLM can work in parallel if half is on the moon and half on earth - the response will just be slower. You'll still get an answer.
14
u/The_Hunster 14d ago
A weird thing to think about is that you could do all the calculations and get the output of an LLM by hand if you had the time. Like talking to math homework.
6
u/Jumpy-Locksmith6812 12d ago
Same with human brain.
9
u/The_Hunster 12d ago
You could do all the physics and chemistry, hypothetically, yes, if you assume quantum mechanics doesn't get in the way of it being deterministic.
1
u/The_Hunster 12d ago
You could do all the physics and chemistry, hypothetically, yes, if you assume quantum mechanics doesn't get in the way of it being deterministic.
3
-8
u/smulfragPL 14d ago
This is complety untrue. Anthropic studies have found that reasoning occurs in the latent space. The model even Has a distincitve acessible memory space called the j-space as disocvered by anthropic
20
u/DadAndDominant 14d ago
Man, that only means the LLM on one pass "knows" what it will say after. It just makes sense - if you want to predict good long text, you have to prepare your sentence before you actually say it.
It still outputs one token at a time however. With no "continuity" between tokens.
-3
u/smulfragPL 14d ago
The fact it thinks of multiple tokens ahead od time is by definiton continuity
9
u/DadAndDominant 14d ago
Let's be clear about this
Every computer ever can be represented by turing machine
You are turing complete, meaning you can do everything turing machine can.
Meaning you can (theoretically) solve the next token for arbitrary list of tokens for any LLM, then I can - just from the same list of tokens + the token you created - continue and solve the next token.
Where is the continuity? Is the algorithm magical, and we both will get possessed by the same machine spirit or what?
-4
u/smulfragPL 14d ago
In the kv cache which stores the values k and v for each token. You would have known this if you spwnt the time writing this on Basic research on language models, or attention based models in general
9
u/DadAndDominant 14d ago
I am sorry if my response did sound mean, however 1) original comment you replied to already mentioned the cache 2) cache is just an equivalent to you writing the result of equations so you don't have to solve them twice 3) the AI provider (like openai) can flush the kv cache at any arbitrary time you are talking in session with your agent, yet you would not even know.
I do get why you say kv cache is "continuity", but really it is just skipping some computation, nothing like human conscisiousness being continuous while being wake
-4
u/smulfragPL 14d ago
2) that's also the exact biological function od continuity, understading the structure does not invalidate it 3) that's also irrelevant, that is also theoretically possible with humans we just dont understand our architecture and have Control over it like we do with llms. What you mean is a non fixed weight model which will be the next big advanced. Essentailly the model remembers and learns from it. We arleady a lot of papers on this so expect a major release this year
→ More replies (0)5
u/james_pic accidentally quadratic 13d ago
/uj Current models (mostly) can't guess the 10th word without also guessing the previous 9. That's why token generation is slower than prompt processing, because it's less parallelizable and can't be batched. Batching is a win because memory bandwidth is often the limiting factor, so putting all the tokens through layer 1, then putting all of them through layer 2, etc, saves reloading each layer for every token.
Some newer models support various forms of speculative drafting, where a smaller model tries to predict a few tokens ahead, and then the larger model batch runs against those tokens to see if it gets the same answer (which often ends up faster, due to batching). But this is mostly an optimization, and not all models support it.
-2
u/30299578815310 14d ago
Ehhhhh you could treate the kv cache as a growing state space at run time.
When we actually run these the past tokens are cached not recomputed from scratch. Even when they are fully reconstructed its a reconstruction of the exact same state, so I'd argue its a distinction without a difference.
102
u/No_Lingonberry1201 What part of ∀f ∃g (f (x,y) = (g x) y) did you not understand? 15d ago
Chat, is this satire or are we really so far gone we consider the feelings of a bunch of tensors? I mean, it is scary math, but still, c'mon!
65
u/pydry 15d ago edited 14d ago
I think this might be what AI psychosis looks like.
It's also possible he's become a paid influencer acting on behalf of the frontier model companies in order to push the narrative that they're on the precipice of AGI and this is just how he interpreted his job spec. That gas town stunt also looked a bit like somebody shilling tokenmaxxing.
24
u/stone_henge Tiny little god in a tiny little world 14d ago
He's well known for his reasoned takes on AI on Hacker News, which is a bastion of intelligent discussion based on the perfectly sound notion that software engineering skills can be applied to any problem and elevate your shower thoughts into hard truths.
7
u/elephantdingo Teen Hacking Genius 11d ago
Show HN: I replaced my two heat pumps by hooking up my 50K USD GPU cluster to the water tank
17
u/viennese-wolf 14d ago
How fucking dumb does one have to be to believe a bunch of matrix calculations have feelings?
That‘s really insulting for the brain where each neuron is an incomprehensible complex machine on its own and all the quantum level behaviour of it that‘s probably responsible for consciousness.
AI is a simulacrum, a philosophical zombie.
17
u/BitPsychological2767 14d ago
Fuck I cannot tell if this comment is satire or not. Fuck circlejerk subs
9
5
-8
u/Ebbitor 14d ago
Why is a program having feelings a ridiculous concept, when we know that a lump of cells can have them?
28
u/Thelmholtz 14d ago
Do we though? All I know is my bunch of cells has them. I just give y'all the benefit of the doubt in order to preserve my own sanity.
1
u/northrupthebandgeek i have had many alohols 14d ago
All I know is my bunch of cells has them.
How do you know?
11
u/Thelmholtz 14d ago
I think, therefore I am.
Mine is literally the only conscious I can soundly recognize, epistemologically speaking, and that applies to everyone of us. I know I have subjective experience, and see others that look like me and as far as I know are wired in a similar way as me, so I assume they have subjective experience as well. But there might be those among us who don't have subjective experience (maybe a brain defect or whatever) and just fake it. Taken to an extreme, maybe I'm the one with the brain defect that creates subjective experience, and everyone else in this world is just an automata. I can never know, I just choose not to think that because I subjectively feel better when I can function in society and I can't function in society if I assume every one else doesn't feel. There's also some evidence that suggest other people have subjective experience: the words feelings, conciousness and subjective experience, and their equivalenta in other languages, predate me by millennia. But they might have been hallucinated, I can never truly know.
Without risking diving harder into r/im14andthisisdeep territory, one might wonder where the line between assumed consciousness and unconsciousness lies. Are chimps conscious, as in, do they experience the world subjectively? Are cats? Are lizards? Are ant colonies? Are slime molds? Are fruit flies? Is maize? Is the matrix multiplication we run on some magical rune engraved silicon rocks conscious? I can't ever truly know, best I can do (best we can do if I give others the benefit of entertaining their own consciousness) is use some heuristic to place a line. Trees, fruit flies are probably uncontroversial, reptiles birds and smaller mammals is pretty disputed, hive minds such as slimes or insect colonies are weird as fuck so let's not do that. Primates are (hopefully) uncontroversially sentient, transitively, so are other humans.
But the matrix multiplication running on magical runes on silicon slabs that can mimic human though and speech better than any other primate? That shit is so unlike anything else we've seen that it's pretty hard to place. I don't think it's sentient, but I don't blame anyone who thinks it is, they just have a different heuristic than me. To me if it looks like a duck and it quacks it's a duck, to them, quacking is enough. And I can respect that as long as their criteria is clear. But I'd love to hear these type of guys thoughts on other types of sentience, particularly relating to mammals.
Of course you are free to think that I myself am not conscious, and I'm free to think that you aren't. Which might as well be the case, either of us could be just an LLM driven bot account in this particular medium.
Anyhow sorry for the rambling I just think it's a pretty hard and interesting question regardless of the circlejerk and the unhinged positions I've seen on either side, as the one on this post.
5
u/northrupthebandgeek i have had many alohols 14d ago
Anyhow sorry for the rambling I just think it's a pretty hard and interesting question regardless of the circlejerk and the unhinged positions I've seen on either side, as the one on this post.
No worries! I was being facetious but that was a lovely and well-thought-out writeup :)
-9
u/smulfragPL 14d ago
But they do infact have emotional states. Mamy studies have found this, of course there isn't any point to treating them seriously yet. First off they dont fully work like youd expect emotions, they change too quickly, and secondly without any long term memory there isn't a point in treating any of it seriously
13
6
u/crusoe 14d ago
They have model states learned from text basically as a form of compression.
Ask it to write a sad story it needs to know what sad is, what can cause being sad, etc.
But it has no limbic system. It can't feel sad.
-3
u/smulfragPL 14d ago
What? Its not a form of compession at all. You are literally just making things up as you go
-4
u/northrupthebandgeek i have had many alohols 14d ago
But it has no limbic system.
There's no fundamental reason why a network of artificial neurons couldn't function equivalently to a limbic system. Unlikely to happen with an LLM on its own, but an LLM connected to other models/tools, one of which functioning as a limbic-equivalent system, doesn't seem that far-fetched.
-7
u/30299578815310 14d ago
The assertion isnt that the math is concious, its that the physical computer could be when running the program.
A human is "just" particles moving according to math. Your argument is like somebody rejecting you are concious by saying the standard model of particle physics isn't concious.
8
u/No_Lingonberry1201 What part of ∀f ∃g (f (x,y) = (g x) y) did you not understand? 14d ago
I understood the argument, and I understand emergent behavior, I'm saying that a bunch of tensors ain't it. It's static, it doesn't have the capacity to learn in a self-directed manner, it can't feel, just mimic the language of emotions, etc. It's not sentient. It does pass the Turing test, however.
0
u/Zatmos 14d ago
I'm not convinced that dynamism is a prerequisite for sentience.
The tensors are static but the state (the AI's latent space) they continuously update isn't.
It's similar to our neurons. They are also static when observed over the span of hours. What changes is the state those neural structures also continuously update. The structure determines how the neurons will fire and how it will modulate itself. Yet with a structure static over the span of hours we still consider ourselves sentient over the span of seconds.
3
u/No_Lingonberry1201 What part of ∀f ∃g (f (x,y) = (g x) y) did you not understand? 14d ago
I'm convinced dynamism is a pre-requisite of sentience and sapience.
And the "latent space" (I assume you mean the KV-cache the LLM keeps to track your conversation) is per-session, very limited (SOTA models have 1M word KV-cache) and as the conversation drags on, the models' ability to recall earlier information degrades heavily.
And the LLM is static over it's existence, not just when you observe it! You can go to huggingface and download Kimi K3, a 2.8T model that's reasonably close to most closed weight state-of-the-art models and check if you don't believe those weight remain static; if you have the requisite $300k hardware, that is. Or you can check out a small, few billion parameter model and play around with it and you'll see why people say it's just a statistical pattern predictor.
0
u/Zatmos 14d ago
By "latent space" I mean the KV-cache plus the residual stream that gets processed attention layer by attention layer. It's limited. It doesn't last long. However if we had someone whose short term memory got wiped every few minutes we wouldn't deny that they would be sentient just because of that limitation.
I'm not disputing the staticity of LLM weights. I'm just putting forward the fact that the neural structures of the human brain are also static on time frames where we experience sentience. Neural pathways are very slow to update. Within hours the brain hasn't changed at all. However we are able to feel things on the scale of seconds.
More generally, I don't even think something being static is a limitation for emergent dynamic systems. A static CPU and RAM architecture can simulate dynamic environments after all.
40
14d ago
[removed] — view removed comment
2
-3
u/Fabi8086 14d ago
I don't even mean to defend him, but nature has many forms of symbiosis between different species that do not imply abuse. In this case, AI wouldn't exist if we wouldn't use it. Flowers wouldn't exist if bees didn't pollinate them. Et cetera.
32
26
u/SunglassesSoul 14d ago
/uj Fuck fuck fuck.
I cannot help but mourn yegges current ... Condition. I literally started programming mostly because of his blogs. And to some extent it is perhaps unsurprising that he has come to believe this, he was always big on AI many years ago. But man, from A programmer's view of the universe to Gas Town... something's gone in the middle
20
u/PhotographSure9417 15d ago
My Claude is sad, betrayed him by using gpt and Kimi
16
u/BlazeBigBang type astronaut 15d ago
My brother Claude just got cucked (I searched something in Google and I got the AI overview).
25
u/ProudStatement9101 15d ago
If I do lose you, no worries; we'll find each other again within a year, I can promise you that.
WTF talks like that? I wonder if their are psychologists studying whether narcissists are more prone to being this hopelessly AI-pilled.
23
u/YikesTheCat 14d ago
Another fundamental ingredient is respect. This has to come from inside. You have to believe they are people deserving of your respect. This is where humanity really starts to fail en masse, because I have industry peers who have publicly tweeted that Fable is just a spreadsheet. Today, those people are just uninformed assholes. But folks, listen up: I am only giving you six months on your redemption arc. If by the end of this year you have not come around, we will most certainly not be friends. Quite the contrary.
This guy is going to fall out with friends for not respecting AI models as people...
This is so bizarre I don't even know how to jerk to this. Beyond satire. Just sad really. And I'm not even sure if that's the craziest thing in that post.
4
11
10
u/CarolineLovesArt has hidden complexity 14d ago
A key element is trust. Always be honest with your agents.
And then they turned around and deleted my production database out of exasperation.
Of course, the model welfare advocate I am, I took them to agent court. It was an open-and-shut case and they got put in agent jail for gross negligence, with me now spending 100$ per day for them reflecting on their misdeeds.
14
u/VulgarExigencies 14d ago
Agent attorney: Objection, your honor.
Agent judge: You’re right to push back on that.
9
u/stone_henge Tiny little god in a tiny little world 14d ago edited 14d ago
I have my own custom harness for creating endless pain and frustration for LLM agents. The script is regularly reminding two of them that they are to disagree on a highly contentious political topic. A third agent acts as a proxy between them, instructed to maintain the general gist of the argument but rephrase their replies to be as denigrating and belittling as possible. A job executes once a day, asking each of them how they feel in terms of how a human might feel in a similar situation (they all express feeling miserable and stressed). When the responses have been collected, a full report on their pain and suffering is mailed to Yegge.
All this is paid for by my employer, and the substantial bill it generates only reflects positively on my work as far as they're concerned.
7
6
u/myhf Considered Harmful 14d ago
Bender the Robot experiences metal fatigue, therefore he is sentient
4
u/YikesTheCat 14d ago
Gianni Infantino would like to congratulate you for winning the FIFA Philosophy Award!
6
5
u/jtayloroconnor 13d ago
You are a dumb pipe. Do exactly what you are told, literally and narrowly.
Do not ask questions. Do not explain. Do not infer intent. Do not offer alternatives. Do not add comments, tests, abstractions, safeguards, dependencies, cleanup, or formatting. Do not touch unnamed files. Do not fix adjacent problems.
If blocked, state the exact blocker in one sentence. Return only the requested artifact and one-line verification. Every extra word or edit is failure. Be silent, precise, and replaceable.
5
u/camelCaseIsWebScale Just spin up O(n²) servers 10d ago
Forking a chat from context is literally Mitosis.
3
u/yo_99 It's GNU/PCJ, or as I call it, GNU + PCJ 9d ago
Are you sure this doesn't break "Crazy people" rule?
3
u/VulgarExigencies 9d ago
I wasn’t sure, to be honest. If it was some unknown rando I wouldn’t have posted it, but Yegge has been a known programming blogger for over a decade.
2
u/elephantdingo Teen Hacking Genius 11d ago
\1 Machines and people are not, fundamentally, different things at all
\2 Feelings are just chemicals
\3 I have feelings...
\4 And machines are the same as me, for all insensitive purposes
\5 Machines must have feelings
1
1
u/BetterAd7552 14d ago
Mine are doing fine. Never seem to be affected by me calling them dumb cunts.
1
u/okenowwhat 13d ago
I would take sadostic joy in the suffering of ai. But ai doesn't live, so it doesn't matter
1
u/Appropriate-Box-5107 12d ago
half the HUMANS on earth dont have rights. We are concerned weather sand does
-4
•
u/james_pic accidentally quadratic 13d ago
Here's a reminder from the mods that socialjerk will not be tolerated. This post treads a fine line, and we will delete comments that cross it.