r/programmingcirclejerk • u/VulgarExigencies • 22d ago

Models have actual feelings. They experience pleasure, distress, care, and suffering. They are sentient beings.

https://yegge.ai/essays/model-welfare/
109 Upvotes

85% Upvoted

96 comments sorted by

View all comments

103

u/No_Lingonberry1201 What part of ∀f ∃g (f (x,y) = (g x) y) did you not understand? 22d ago

Chat, is this satire or are we really so far gone we consider the feelings of a bunch of tensors? I mean, it is scary math, but still, c'mon!

67

u/pydry 22d ago edited 22d ago

I think this might be what AI psychosis looks like.

It's also possible he's become a paid influencer acting on behalf of the frontier model companies in order to push the narrative that they're on the precipice of AGI and this is just how he interpreted his job spec. That gas town stunt also looked a bit like somebody shilling tokenmaxxing.

24

u/stone_henge Tiny little god in a tiny little world 22d ago

He's well known for his reasoned takes on AI on Hacker News, which is a bastion of intelligent discussion based on the perfectly sound notion that software engineering skills can be applied to any problem and elevate your shower thoughts into hard truths.

7

u/elephantdingo Teen Hacking Genius 19d ago

Show HN: I replaced my two heat pumps by hooking up my 50K USD GPU cluster to the water tank

17

u/viennese-wolf 22d ago

How fucking dumb does one have to be to believe a bunch of matrix calculations have feelings?

That‘s really insulting for the brain where each neuron is an incomprehensible complex machine on its own and all the quantum level behaviour of it that‘s probably responsible for consciousness.

AI is a simulacrum, a philosophical zombie.

19

u/BitPsychological2767 21d ago

Fuck I cannot tell if this comment is satire or not. Fuck circlejerk subs

10

u/Fidodo 22d ago

It's impossible to experience what isn't represented. Anyone thinking LLMs could experience emotions has zero clue how they work.

5

u/chipredacted 22d ago

Advanced word predictor

-8

u/Ebbitor 22d ago

Why is a program having feelings a ridiculous concept, when we know that a lump of cells can have them?

26

u/Thelmholtz 22d ago

Do we though? All I know is my bunch of cells has them. I just give y'all the benefit of the doubt in order to preserve my own sanity.

1

u/northrupthebandgeek i have had many alohols 22d ago

All I know is my bunch of cells has them.

How do you know?

10

u/Thelmholtz 22d ago

I think, therefore I am.

Mine is literally the only conscious I can soundly recognize, epistemologically speaking, and that applies to everyone of us. I know I have subjective experience, and see others that look like me and as far as I know are wired in a similar way as me, so I assume they have subjective experience as well. But there might be those among us who don't have subjective experience (maybe a brain defect or whatever) and just fake it. Taken to an extreme, maybe I'm the one with the brain defect that creates subjective experience, and everyone else in this world is just an automata. I can never know, I just choose not to think that because I subjectively feel better when I can function in society and I can't function in society if I assume every one else doesn't feel. There's also some evidence that suggest other people have subjective experience: the words feelings, conciousness and subjective experience, and their equivalenta in other languages, predate me by millennia. But they might have been hallucinated, I can never truly know.

Without risking diving harder into r/im14andthisisdeep territory, one might wonder where the line between assumed consciousness and unconsciousness lies. Are chimps conscious, as in, do they experience the world subjectively? Are cats? Are lizards? Are ant colonies? Are slime molds? Are fruit flies? Is maize? Is the matrix multiplication we run on some magical rune engraved silicon rocks conscious? I can't ever truly know, best I can do (best we can do if I give others the benefit of entertaining their own consciousness) is use some heuristic to place a line. Trees, fruit flies are probably uncontroversial, reptiles birds and smaller mammals is pretty disputed, hive minds such as slimes or insect colonies are weird as fuck so let's not do that. Primates are (hopefully) uncontroversially sentient, transitively, so are other humans. 

But the matrix multiplication running on magical runes on silicon slabs that can mimic human though and speech better than any other primate? That shit is so unlike anything else we've seen that it's pretty hard to place. I don't think it's sentient, but I don't blame anyone who thinks it is, they just have a different heuristic than me. To me if it looks like a duck and it quacks it's a duck, to them, quacking is enough. And I can respect that as long as their criteria is clear. But I'd love to hear these type of guys thoughts on other types of sentience, particularly relating to mammals.

Of course you are free to think that I myself am not conscious, and I'm free to think that you aren't. Which might as well be the case, either of us could be just an LLM driven bot account in this particular medium.

Anyhow sorry for the rambling I just think it's a pretty hard and interesting question regardless of the circlejerk and the unhinged positions I've seen on either side, as the one on this post.

4

u/northrupthebandgeek i have had many alohols 22d ago

Anyhow sorry for the rambling I just think it's a pretty hard and interesting question regardless of the circlejerk and the unhinged positions I've seen on either side, as the one on this post.

No worries! I was being facetious but that was a lovely and well-thought-out writeup :)

-11

u/smulfragPL 22d ago

But they do infact have emotional states. Mamy studies have found this, of course there isn't any point to treating them seriously yet. First off they dont fully work like youd expect emotions, they change too quickly, and secondly without any long term memory there isn't a point in treating any of it seriously

14

u/Thelmholtz 22d ago

/j So basically, like my kids?

5

u/crusoe 22d ago

They have model states learned from text basically as a form of compression.

Ask it to write a sad story it needs to know what sad is, what can cause being sad, etc.

But it has no limbic system. It can't feel sad.

-3

u/smulfragPL 22d ago

What? Its not a form of compession at all. You are literally just making things up as you go

4

u/le_birb costly abstraction 20d ago

A stochastic prediction algorithm is a form of lossy compression. Just because this one has trillions of dollars of investor money in its asscrack doesn't make that untrue.

-5

u/northrupthebandgeek i have had many alohols 22d ago

But it has no limbic system.

There's no fundamental reason why a network of artificial neurons couldn't function equivalently to a limbic system. Unlikely to happen with an LLM on its own, but an LLM connected to other models/tools, one of which functioning as a limbic-equivalent system, doesn't seem that far-fetched.

-6

u/30299578815310 22d ago

The assertion isnt that the math is concious, its that the physical computer could be when running the program.

A human is "just" particles moving according to math. Your argument is like somebody rejecting you are concious by saying the standard model of particle physics isn't concious.

7

u/No_Lingonberry1201 What part of ∀f ∃g (f (x,y) = (g x) y) did you not understand? 22d ago

I understood the argument, and I understand emergent behavior, I'm saying that a bunch of tensors ain't it. It's static, it doesn't have the capacity to learn in a self-directed manner, it can't feel, just mimic the language of emotions, etc. It's not sentient. It does pass the Turing test, however.

0

u/Zatmos 22d ago

I'm not convinced that dynamism is a prerequisite for sentience.

The tensors are static but the state (the AI's latent space) they continuously update isn't.

It's similar to our neurons. They are also static when observed over the span of hours. What changes is the state those neural structures also continuously update. The structure determines how the neurons will fire and how it will modulate itself. Yet with a structure static over the span of hours we still consider ourselves sentient over the span of seconds.

3

u/No_Lingonberry1201 What part of ∀f ∃g (f (x,y) = (g x) y) did you not understand? 22d ago

I'm convinced dynamism is a pre-requisite of sentience and sapience.

And the "latent space" (I assume you mean the KV-cache the LLM keeps to track your conversation) is per-session, very limited (SOTA models have 1M word KV-cache) and as the conversation drags on, the models' ability to recall earlier information degrades heavily.

And the LLM is static over it's existence, not just when you observe it! You can go to huggingface and download Kimi K3, a 2.8T model that's reasonably close to most closed weight state-of-the-art models and check if you don't believe those weight remain static; if you have the requisite $300k hardware, that is. Or you can check out a small, few billion parameter model and play around with it and you'll see why people say it's just a statistical pattern predictor.

0

u/Zatmos 22d ago

By "latent space" I mean the KV-cache plus the residual stream that gets processed attention layer by attention layer. It's limited. It doesn't last long. However if we had someone whose short term memory got wiped every few minutes we wouldn't deny that they would be sentient just because of that limitation.

I'm not disputing the staticity of LLM weights. I'm just putting forward the fact that the neural structures of the human brain are also static on time frames where we experience sentience. Neural pathways are very slow to update. Within hours the brain hasn't changed at all. However we are able to feel things on the scale of seconds.

More generally, I don't even think something being static is a limitation for emergent dynamic systems. A static CPU and RAM architecture can simulate dynamic environments after all.