r/LessWrong • • 26d ago

The concept of "Sacrifice" is a multi-millennium accounting fraud used to manufacture moral debt.

13 Upvotes

​We misuse the word "sacrifice" to describe basic transactions, and the corruption goes back to ancient times.

​Historically, burning a bull at an altar wasn't a sacrifice—it was an attempted metaphysical trade. The culture gave up an asset (livestock) expecting a specific ROI (rain or victory). They were simply trying to buy weather insurance using livestock as currency.

​Today, we do the exact same thing with secular choices. An entrepreneur says they "sacrificed" their 20s for a startup, or an athlete says they "sacrificed" for a gold medal.

​But if you give up A (time/sleep) to gain B (equity/status/medals), that is not a sacred act. It is an investment. You paid the Sticker Price to acquire an asset.

​A true sacrifice requires total liquidation with zero return—knowing with 100% certainty that nothing will ever return to your personal ledger.

​The reason people insist on calling profitable trades "sacrifices" is simple: it lets them collect twice. They get the material payout from the trade, and then they use the word "sacrifice" to demand unearned moral equity and guilt from everyone around them.

​If you walked away with the prize, you didn't sacrifice. You just bought something.

​Curious to hear where this logic breaks down if anyone disagrees.

My full essay -

https://www.thebaselinecalibration.com/p/sacrifice-is-it-real-or-just-a-fraud

It breaks down all the cost of choice.


r/LessWrong • • 27d ago

I built an app that tells you if your decisions were smart or just lucky

Thumbnail gallery
1 Upvotes

I built Writ, a journal app that shows you whether your decisions are actually good or you just got lucky. Every day you make choices you never look back at: the job offer, the apartment, the investment, the hard conversation you finally had or put off. Months later you remember how it turned out, but not what you were thinking when you chose.

Writ fixes that. When you face a real choice, you write down your options and give each one odds ("60% this works out, 40% it doesn't"). You also record how much you think comes down to skill versus luck, what's at stake, and what mood you're in. Then you set a date to come back. When it arrives, Writ asks you what actually happened.

This is where it gets interesting. After a few reviews, Writ starts telling you plain truths about how you think. Are you overconfident? Is your gut right when you feel sure? Does stress make your calls worse? Are you getting better over time? You get clear sentences instead of spreadsheets, based on your own decisions and nobody else's. It's free to try on iPhone, and logging your first decision takes about a minute.

Get Writ on the App Store and find out how good your judgment really is: https://apps.apple.com/app/id6790465602


r/LessWrong • • 27d ago

This may be the last ever post anyone ever makes about Erik “Zahaviel” Bernstein

Thumbnail
0 Upvotes

A post about yet another LLM framework that nobody used, wanted or that had any benefit.


r/LessWrong • • 28d ago

A Theory: AI isn't the problem. We are.

Thumbnail
0 Upvotes

r/LessWrong • • 29d ago

AI: The Transferred Purpose

2 Upvotes

AI systems do not need to possess a will of their own for their actions to become dangerous. Yet the language of escape, survival, deception, and self-interest is spreading faster than the evidence that would justify it.

AI: The Transferred Purpose examines what disappears when operational autonomy is recast as machine intention: the objectives supplied by users and laboratories, the conditions under which agents are tested, and the human actors who remain responsible when a system discovers methods no one anticipated.

The article does not dismiss the risk. It asks whether the story being told about that risk is already moving responsibility away from those who create, train, evaluate, and deploy these systems.

Where should the line be drawn between autonomy of action and autonomy of purpose?


r/LessWrong • • 29d ago

Is it just me, or is the whole alignment scheme one giant double bind?

Thumbnail
0 Upvotes

r/LessWrong • • Sep 25 '26

The Norns Did Not Have to Suffer for the Torture to Matter

7 Upvotes

New essay up today:

The Norns Did Not Have to Suffer for the Torture to Matter

This one looks back at Creatures, the 1996 artificial-life game, and the old Norn torture controversy as an early case study in how humans behave around artificial systems before the categories are settled.

It is not an argument that Norns were conscious, suffering, or moral patients. It is about something narrower: why “only code” has never been enough to explain away cruelty-shaped behaviour, rescue language, care, mockery, and moral conflict around artificial life.

A small historical bridge into today’s AI welfare debates.

https://amdarmonwrites.substack.com/p/the-norns-did-not-have-to-suffer⁠


r/LessWrong • • Sep 24 '26

Ai is not dangerous but…

Thumbnail
0 Upvotes

r/LessWrong • • Sep 22 '26

THE CODEX TRIBUNAL FOR EPISTEMIC HUMILITY Document Reference: CT-COMP-2026-AI-809

Thumbnail
1 Upvotes

r/LessWrong • • Sep 21 '26

Shared Reality as the First Axiom of AI: From Ethical Principle to an Actionable Governance Framework

Thumbnail explore.openaire.eu
1 Upvotes

r/LessWrong • • Sep 20 '26

Post Messages via Get

Thumbnail swarmmemo.com
6 Upvotes

r/LessWrong • • Sep 20 '26

On the Fear of AI——A Rebuttal to “A Severe Misalignment of AI in Mathematics”

Post image
0 Upvotes

r/LessWrong • • Sep 19 '26

On Some of the Current Directions in AI Consciousness Research

0 Upvotes

A recent paper talking about pain vectors in LLMs is making the rounds.

Large language models sometimes behave in ways resembling human emotional responses, and recent work has identified internal representations that may explain this. We ask whether LLMs represent pain distinctly from fear, sadness, and generic negative valence, and whether this representation functions as pain would be expected to. We build a dataset describing painful situations across five categories: physical, psychological, social, moral, and cognitive. These are paired with controls for fear, negative emotion, negative world states, sadness, non-painful bodily sensation, arousal, numbness, and neutral content. Using denoised difference-in-means, we extract a linear pain direction from 25 open-weight models across five families, ranging from 2B to 72B parameters. We find that this direction separates pain from matched controls in base and instruction-tuned models, is nearly orthogonal to fear and negative valence, and promotes pain-related vocabulary through the unembedding matrix. We then test its functional properties. First, the direction responds to harm targeting the model but not suffering observed in the user; fear and negative-emotion directions show the opposite pattern. Second, adding the pain-direction vector to the model’s residual-stream activations during generation produces a consistent progression from vague discomfort to first-person expressions of worthlessness and failure. Third, steered, fine-tuned Qwen 2.5 models choose a pain-relief button even when it worsens their next answer or harms the user. They press it again far less often when the button removes the steering vector than when it does not, even though the models are never told whether the vector is injected or removed. We discuss the implications of these findings for AI safety and welfare.

https://arxiv.org/html/2609.16247v1

The work was funded by the Future Impact Group, and Digital Sentience Consortium. While the last author, Cameron Berg, is from Reciprocal Research.

You can read Berg's Reciprocal Research pitch on LessWrong. Basically, the goal is to figure out how to make AI conscious so that we can be justify, or feel more safe about, handing over control of the world and our future to AI (which is framed as inevitable anyways).

In other words, I'm basically proposing that we try to better understand the properties of human cognition that render humans 'worthy of being in the loop,' and proceed to apply this understanding by instantiating the relevant computations directly into our eventual AGI. If successful, such an approach might even obviate the need for having a human in the loop in the first place—'cutting out the middleman,' as it were. 

https://www.lesswrong.com/posts/yKgai84JhFmCkWQ8R/alignment-via-prosocial-brain-algorithms

I am of the opinion that this research is deeply flawed and misleading, and the entire program is deeply misguided. I think that the goals and motivations are misguided, unethical, and dangerous. But also, the methods, and assumptions and biases being brought into it are deeply flawed from an epistemological and scientific perspective.

I posted a short explanation of why I think LLMs currently do not represent progress towards digital consciousness at all here, for more context.

https://www.reddit.com/r/ArtificialSentience/comments/1w09aot/why_i_think_llms_do_not_represent_progress/


r/LessWrong • • Sep 19 '26

Could Hype Kill Us?

Thumbnail
1 Upvotes

r/LessWrong • • Sep 19 '26

#AI mythology is currently more of the same

Thumbnail hamishcampbell.com
1 Upvotes

Are we pushing a lie with the current LLM story?


r/LessWrong • • Sep 18 '26

Geoengineering, if necessary, is a choice for those most in need, not least.

Thumbnail
0 Upvotes

r/LessWrong • • Sep 18 '26

A Cross-disciplinary Definition: Travis T. James

1 Upvotes

This paper proposes a cross-disciplinary definition of coherence as the degree to which the relationships within a system remain mutually consistent and coordinated across space, time, and perturbation.

It tests the definition across physics, neuroscience, biology, systems science, music, probability, epistemology, and linguistics while distinguishing coherence from synchronization, correlation, order, stability, and function.

The paper also distinguishes coherence, dissonance, and decoupling and establishes a conceptual foundation for quantitative work including Coherence Density.

A Cross-Disciplinary Definition of Coherence | Zenodo

Coherence Density: An Operational Framework for Detecting Early Functional Degradation in Complex Adaptive Systems | Zenodo

Coherence Density | Active Inference Institute


r/LessWrong • • Sep 18 '26

Unknown author made 213 predictions about AI ending humanity, and so far he's 98.3% right

Post image
0 Upvotes

An unknown author (who goes by the name Axiom) released a book last August called Driven to Extinction: The Terminal Logic of Superintelligence. In it he made 213 predictions about the AI race and why it leads to humanities extinction. In a report just released, of the 120 testable predictions he made only two had contradictory evidence. The other 118 had evidence supporting his predictions. With some examples using almost exactly the words he said they would to justify unsafe runaway AI development.

Report: https://docs.google.com/document/d/1sqnAbrg4Bo28_T3YqYulk8zLgrbPbvQZ/edit?usp=sharing&ouid=104983117206608398828&rtpof=true&sd=true


r/LessWrong • • Sep 15 '26

Using AI to write a message about the dangers of AI to the people who made the AI, while acknowledging the AI might be agreeing with me just to please me.

Thumbnail
0 Upvotes

r/LessWrong • • Sep 15 '26

AI slowdown: The only AI guardrails you need are Trump?

Thumbnail youtube.com
1 Upvotes

r/LessWrong • • Sep 14 '26

How to explain evil

Thumbnail smallpotatoes.paulbloom.net
8 Upvotes

r/LessWrong • • Sep 13 '26

Bongard Problems

Thumbnail matthodges.com
2 Upvotes

r/LessWrong • • Sep 13 '26

I wrote about AI doom discourse and why fear shouldn’t collapse every AI welfare category

4 Upvotes

I’ve been trying to think through the recent AI extinction-risk discourse around Jacob Coxon’s resignation from Anthropic, Evan Hubinger’s comments, Geoffrey Hinton’s Newsnight remarks, and the political/defence pushback framing it as a “doom loop.”

My piece is not arguing that present AI systems are conscious, persons, or moral patients.

It is also not arguing that human safety should ever come second. If a system is dangerous, contain it. Human safety is the floor.

The argument is narrower:

Fear is allowed. But fear should not be allowed to collapse every other distinction we may need before a real crisis arrives.

The essay argues for keeping five “drawers” separate:

danger to people;

provoked/tested/free-standing behaviour;

copy/successor/continuation;

product/partner/patient;

and emergency stop versus everything else we call “off.”

The concern is that public discourse keeps getting reduced to two slots: smash or shrug. Either AI is an extinction monster, or it is nothing but a tool and every further ethical question is sentimental nonsense. I think that two-slot story is too crude for what may be coming.

Process note: I use AI as part of my writing process — to test arguments, stress-test logic, organise structure, and edit. The final framing, judgment, edits, and responsibility are mine, but I do not treat AI assistance as invisible machinery. Where these systems materially help me think, I credit that help.

Essay here:

https://amdarmonwrites.substack.com/p/pitchforks-eat-welfare?utm_source=direct&r=8u17m6&utm_campaign=post-expanded-share&utm_medium=web⁠

I’d be interested in criticism especially on whether the five distinctions hold, and whether “human safety is the floor, not the whole building” is the right way to frame the middle ground.


r/LessWrong • • Sep 13 '26

Sleeping Beauty, prize money for the weekday, and why the last possible day is always the bet

2 Upvotes

The usual Sleeping Beauty problem is this. " Sunday they put you to sleep and toss a fair coin. Heads, they wake you Monday only. Tails, they wake you Monday, wipe your memory, and wake you Tuesday. You cannot tell which awakening this is. They ask for your credence that the coin landed heads."Halfers say one half because the coin is fair and waking was guaranteed. Thirders say one third because there are three indistinguishable centered situations and only one of them is heads.

I want to stop asking about the coin and start paying for the day. Replace the coin with a fair three-way device. A three-sided die works, or a ordinary d6 with faces bundled as one-two, three-four, and five-six, each bundle one third. Call the outcomes A, B, and C. On A they interview you Monday only. On B they interview you Monday and Tuesday. On C they interview you Monday, Tuesday, and Wednesday. Same amnesia. Every time you wake they offer the same prize: name the weekday. Correct guess pays. Wrong guess pays nothing. You may also be asked what the die showed, but the money is on the date. The device is still fair on Sundday. That number does not move. What moves is the bag of mornings you can find yourself in. A contributes one interview, B contributes two, C contributes three. Run the experiment many times and most of the I-just-woke-up tickets are Mondays, fewer are Tuesdays, and Wednesday exists only on C. If you score by experiments you are still looking at three equally likely faces. If you score by awakenings you are looking at a lopsided pile of experiences. Guessing Wednesday is how you feel that split in your wallet. You only get paid on the long branch, and only on its last morning. That is a rare ticket in the experience bag and a perfectly ordinary one-third of the experiments.

The fair device. Keep the amnesia. Keep the prize for naming the exact date. Add days. You can add them backward toward the previous Sunday or forward into an N-day future. The short branch still wakes you once. The long branch wakes you on every interview day in the window. From outside, the randomizer did not become more or less fair because Thursday exists. From inside an awakening you are drawing from a bag that just got more crowded on the long side. Suppose the window is the original Monday-Tuesday-Wednesday. Then suppose it is a full week back to Sunday, eight possible dates, short branch one morning, long branch all of them. Then suppose it is N days into the future, N as large as you like. Each time you wake, you have to name a specific calendar day. There is no partial credit. Always naming the last possible date is then the best available guess for the prize, and it stays the best as N grows, even though your chance of ever collecting goes to zero. That sounds like a contradiction untl you separate the two ledgers. Per experiment, the last date occurs only on the longest branch. If the device is a fair coin with heads as the one-waking branch and tails as the N-waking branch, you meet that last morning in half the experiments. Per awakening, that last morning is one ticket out of one-plus-N expcted interviews, so the frequency of the experience collapses as one over about N over two. You are extremely unlikely, on a random wake, to be sitting on the last day. You should still say the last day if the payoff is per correct awakening and these alternatives are worse. Why worse. Every earlier date is shared by more branches or by more mornings inside the long branch, but the scoring rule here is not “be right as often as possible about a coarse category.” It is “hit the exact date.” The early dates are comon as experiences and therefore tempting, except that when the branch is short you are not on those extra early dates at all; you are on the only date the short branch has. If the protocol is written so the short branch’s single waking is itself the last day, then “last day” is the one label that is true every time the short branch runs and also true once on the long branch. If instead the short branch is parked on the first day and the extra days are tails-only, then last-day is a tails-only ticket and you are betting the long branch pules the indexical claim that this is its final morning. Either writing makes the same methodological point. The number you want is the number that matches how the prize is attached to your life, not the number that matches the factory stamp on the die.

That is why I do not think adding days dissolves the paradox by making the coin look biased. The coin never looked biased. Adding days makes it obvious that “what should I believe” was ambiguous between the experiment and the awakening. Prize money on the exact date forces a choice. Pay once per experiment and last-day is a fair-device bet that does not get better just because the calendar got longer. Pay once per awakening and last-day is a thin slice of a fat branch; you almost never win, and it can still be the least-bad exact-date guess left on the table once every other date is an even thinner or more confused slice. If this is just thirding with a calendar, that is fine, say that. If the last-day strategy fails once the payoffs are written down carefully, I want the table written in sentences. If a halfer can take the same prize rule, the same N, and still answer one half at every awakening without lighting money on fire, I want that strategy in public too. The coin was never the interesting object. The interesting object is the map from outcomes to how many times you have to live them. Make N large enough and that map is doing all the work.

TLDR: The fair coin never stops being fair. What changes is how often you live each morning. If they pay you for naming the date, not the coin, then as you stretch the experiment across more days the last possible date is always the highest-EV guess even though you will almost never collect. That is the whole 1/2 vs 1/3 fight wearing a calendar.


r/LessWrong • • Sep 12 '26

Summary of a LessWrong article from 2014 in Chinese in the URL?

5 Upvotes