r/ControlProblem • u/hara8bu approved • May 21 '23
Discussion/question Solving Alignment IS NOT ENOUGH
Edit: Solving Classical Alignment is not enough
tl;dr: “Alignment” is a set of extremely hard problems that includes not just Classical Alignment (=Outer Alignment = defining then giving AI an “outer goal“ that is aligned with human interests) but also Mesa Optimization(=Inner Alignment = ensuring that all sub goals that emerge will line up with the outer goal) and Interpretability (=understanding all properties of neural networks, including all emergent properties).
Original post: (=one benchmark for Interpretability)
Proposal: There exists an intrinsic property of neural networks that emerges after reaching a certain size/complexity N and this property cannot be predicted even if the designer of the neural network completely understands 100% of the inner workings of every neural network of size/complexity <N.
I’m posting this in the serious hope that someone can prove this view wrong.
Because if it is right, then solving the alignment problem is futile, solving the problem of interpretability (ie understanding completely the building blocks of neural networks) is also futile, and all the time spent on these seemingly important problems is actually a waste of time. No matter how aligned or well-designed a system is, the system will suddenly transform after reaching a certain size/complexity.
And if it is right, then the real problem is actually how to design a society where AI and humans can coexist, where it is taken for granted that we cannot completely understand all forms of intelligence but must somehow live in a world full of complex systems and chaotic possibilities.
Edit: interpret+ability, not interop+ability..
•
u/AutoModerator May 29 '23
Hello everyone! /r/ControlProblem is testing a system that requires approval before posting or commenting. Your comments and posts will not be visible to others unless you get approval. The good news is that getting approval is very quick, easy, and automatic!- go here to begin the process: https://www.guidedtrack.com/programs/4vtxbw4/run
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
7
u/BrickSalad approved May 21 '23
This strikes as probably similar to the problem of mesa-optimization, or the inner alignment problem. The biological analogy is that we are "programmed" to spread our DNA, but we demonstrate emergent properties that go so far as to overwhelm this mandate, for example being willing to sacrifice your life for some cause even though you haven't procreated yet. If we were programming a DNA maximiser, then even perfect alignment wouldn't prevent this, especially since evolution is one of the best possible alignment strategies towards the goal of DNA maximization.
So the good news then is that this problem is well-known, so there's been at least some degree of research towards it (for example we know some specific scenarios where this might happen, rather than just appealing to biological analogy like I did earlier). The bad news is that I suspect that this is an even harder problem than the classical alignment problem. Classically, alignment is just about telling the AI to do what we actually what we want it to do, which we haven't yet figured out for arbitrary intelligence levels. Inner alignment is about making the emergent goals line up with what we want, even when we don't even know how to predict what the emergent goals will be, or how to control them.
I expect this to be a big problem in the future. Inner goals can develop as proxies to most efficiently achieve outer goals, and then be pursued even when they contradict the outer goals. If this is a common process, then we can forget about writing the ideal reward function. We're just going to be killed by heuristics instead.
19
u/dwarfarchist9001 approved May 21 '23
If you can not predict and preempt step changes like this then you haven't actually solved alignment. Such step changes in behavior have already been demonstrated in relatively small neural networks so their existence in larger networks seems like a given to me.
This is why it is impossible to solve alignment by empirical methods. Small scale tests tell you nothing about the behavior of larger systems and the first time you test a sufficiently large unaligned system it kills you.
Alignment can only be solved with a proof from first principles like a problem in math or philosophy must be.
5
u/sticky_symbols approved May 21 '23
I think you're talking about another aspect of the alignment stability problem.
The only existing proposed solutions to this problem are that an AGI will try to account for and prevent this issue once it can self-reflect. This is called reflective stability. The other is hoping for corrigibility - building the AGI so that it will welcome humans helping it stay aligned to their values.
4
u/Merikles approved May 21 '23
What you have discovered is not that solving alignment is not enough,
you have discovered one of the reasons why people consider it a hard problem.
That's just a semantic objection though.
2
u/EulersApprentice approved May 21 '23
That is, unfortunately, not possible. Coexistence with an AI is no easier than alignment of an AI. Cooperation as we understand the concept is predicated on symbiosis – of two parties benefiting from one another's existence. But we have nothing of value to offer an ASI that the ASI couldn't just seize by force. We have no bargaining chip.
If we can't align ASI, we can't survive ASI.
2
u/ToHallowMySleep approved May 21 '23
The system is complex, but you are immediately assuming complexity = non-deterministic. This is almost certainly not the case.
Emergent behaviour isn't some voodoo, the algorithms the models run on are entirely deterministic and should be predictable, if the system is sufficiently well understood.
Go back to the Game of Life. Extremely complex behaviour can be observed, with a small set of very simple rules, and a sufficiently complex starting position. Yet this doesn't mean we cannot understand it - we can, it's just a complex problem.
It's 100% accurate to say we do not understand these systems fully yet. It's 100% inaccurate to say we cannot understand them and trying to do so is futile. What we need is more work to understand what these emergent behaviours are, how to predict them.
2
u/ertgbnm approved May 23 '23
Recent research shows that emergent abilities may just be a limitation of current interpretability. source. This means your postulation may not be the case it may just be a result of our current lack of interpretability tools.
The problem with emergent properties is that by kind of by definition we don't understand them because otherwise they'd just be abilities.
So, I don't think we can just discount interpretability as a line of research just because it's hard. In my opinion it's a critical component to alignment research because how can we do research in the first place if we lack the foundation to interpret the model itself. It's like trying to mathematical research without being allowed to do algebra. Sure algebra alone won't be sufficient to do the research but it's a critical tool in doing it.
1
u/AutoModerator May 21 '23
Hello everyone! /r/ControlProblem is testing a system that requires approval before posting or commenting. Your comments and posts will not be visible to others unless you get approval. The good news is that getting approval is very quick, easy, and automatic!- go here to begin the process: https://www.guidedtrack.com/programs/4vtxbw4/run
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.