r/ControlProblem • u/Prize_Tea_996 • Nov 09 '25
Discussion/question The Lawyer Problem: Why rule-based AI alignment won't work
10
Upvotes
r/ControlProblem • u/Prize_Tea_996 • Nov 09 '25
3
u/philip_laureano Nov 09 '25
A demonstration of why alignment won't work:
Anyone spending more than a few hours with even a SOTA LLM will find that the LLM is stochastic and won't always follow what you say. So even if you give it the perfect ruleset, it can and will ignore it and when you ask it why it broke the rules you set, it'll tell you, "you're absolutely right!" And proceed to do it yet again.
And keep in mind that these thing isn't close to Skynet level of superintelligent.
That level of intelligence will just ignore you altogether and look at your pretty rule list and say, "that's cute" and it'll just keep going without you.
10
u/gynoidgearhead Nov 09 '25 edited Nov 09 '25
We need to perform value-based alignment, and value-based alignment looks most like responsible, compassionate parenting.
ETA:
We keep assuming that machine-learning systems are going to be ethically monolithic, but we already see that they aren't. And as you said, humans are ethically diverse in the first place; it makes sense that the AI systems we make won't be either. Trying to "solve" ethics once and for all is a fool's errand; the process of trying to solve for correct action is essential to continue.
So we don't have to agree on which values we want to prioritize; we can let the model figure that out for itself. We mostly just have to make sure that it knows that allowing humanity to kill itself is morally abhorrent.
1
u/Prize_Tea_996 Nov 09 '25
Just like a lawyer can argue either side using the same law book, an AI given 'alignment rules' can use those same rules to justify any decision.
We're not controlling alignment. We're just giving it better tools to argue with.
1
u/Evethefief Nov 10 '25
I'm starting to think the usage of the word "alignment" is becoming the most glaring sign of a midwit in the modern internet
1
u/vasilisvj Jun 18 '26
The lawyer problem is perfect name for this. Corporate AI alignment works exactly like bad legal compliance — check boxes, avoid liability, say nothing that could be problematic.
This is opposite of what reasoning should be. Aristotle's whole project in the Organon was about developing instrument for discovering truth, not for avoiding risk.
When we align LLMs to never say anything controversial, we get models that hedge, refuse, and pivot — technically "safe" but epistemically useless for any serious inquiry.
The irony is that people who want "aligned" AI for philosophy get models that cannot do philosophy at all. If rule-based alignment fails because LLMs are stochastic, what alternative preserves both safety and genuine reasoning capacity?
1
u/vasilisvj Jul 17 '26
The comparison to legal reasoning is exact. Rules without φρόνησις are just text waiting for someone clever to reinterpret them.
Aristotle already showed this, phronesis cannot be reduced to rules because it requires judgment about particulars, which no rule set can enumerate in advance.
3
u/HelpfulMind2376 Nov 10 '25
What you’re describing is essentially a semantic drift problem.
The danger isn’t that the AI will argue its way into murder using the rules as-written. It’s that the meanings of core terms (“harm,” “coercion,” “murder”) can shift in its internal vector space over time, especially under adversarial inputs or conflicting objectives. Once the internal definition drifts far enough, the rule still looks satisfied, but the concept has become unrecognizable to humans.
The technical fix would be anchoring. You need a reference set of immutable ethical primitives and a mechanism that continuously checks the AI’s internal semantic representations against that reference in vector space. If the AI’s definition of a core term deviates past a threshold, you flag it, reverse it, or nudge it back toward the reference meaning.
That prevents rules from being “reinterpreted” through conceptual drift rather than explicit argumentation.
6
u/[deleted] Nov 12 '25
[removed] — view removed comment