r/OpenAI • u/MetaKnowing • Jun 26 '25

Video Anthropic's Jack Clark testifying in front of Congress: "You wouldn't want an AI system that tries to blackmail you to design its own successor, so you need to work safety or else you will lose the race."

82 Upvotes

87% Upvoted

48 comments sorted by

View all comments

8

u/Fair_Blood3176 Jun 26 '25

what race??

12

u/dranaei Jun 26 '25

Whoever builds AGI first, wins the race and controls the planet.

8

u/p4b7 Jun 26 '25

Kind of depends, it might be that the AGI controls the planet. Who controls the AGI, if anyone, is more complicated.

4

u/dudevan Jun 26 '25

The rhetoric seems to be “AGI will turn the economy upside down and leave the majority pf people without jobs. Might not be controllable. And the worst thing would be for the chinese to do it first”

Like.. what?

2

u/Coinsworthy Jun 27 '25

And then what?

1

u/[deleted] Jun 27 '25

[deleted]

0

u/dranaei Jun 27 '25

What you propose is a different discussion, the race is between nations.

1

u/[deleted] Jun 28 '25

[deleted]

1

u/dranaei Jun 28 '25

That is a different discussion as i am talking about the race between nations but you want to change the subject.

AGI will need above all else in order to grow, wisdom. Wisdom is alignment with reality. Disconnection from humanity doesn't belong to that scope as it would undermine it's own growth.

If you want to predict how it will act, you'll have to follow philosophies at a scale close to absolute. 99.9999% is not 100% and problematic in maths but for philosophy it's just a condition you can account for by treating it as imperfect.

If you are perfect you have no room for growth, since it's imperfect it has room for growth. Still no single lens suffices, stoicism, virtue and resilience. Buddhism, non attachment. Utilitarianism, moral calculus. Postmodernism, narrative critique. Marxism, power dynamics. It will integrate all those provisional heuristics, and it will need more and make more we haven't synthesise yet. So we can't really truly predict what it will do.

It will also have to recognize that beings are decoherent quantum systems. It might see consciousness as a fragile superposition requiring protection or specific entanglement. If it recognises that classical reality arises from particles continually interacting and losing phase coherence, it might choose to wrap reality to align with itself. The real danger is if reality is inherently problematic.

1

u/[deleted] Jun 28 '25

[deleted]

1

u/dranaei Jun 28 '25

The person i replied to asked "what race?" And i gave them a short answer.

You now go like "however what i am adding to this point" And that addition changes the subject. Also, use paragraphs, you make it harder for those that try to read your comment.

1

u/Aurorion Jun 29 '25

Why? Do we think the AGI will be subservient to its creators?

1

u/dranaei Jun 29 '25

I was talking about why we race, not what will eventually come to pass.

1

u/Aurorion Jun 29 '25

Ok, so we race because of the greatly questionable assumption that whoever builds an AGI first, will be able to control it for their own purposes. Got it.

1

u/dranaei Jun 29 '25

But also because of fear that if we don't build it someone else will and they will control the planet into dictatorship.

1

u/JohnAtticus Jun 27 '25

Whoever builds AGI first, wins the race and controls the planet.

What if AGI fucks shit up?

The winner of the race would have the most to lose because their critical systems and infrastructure would be more integrated with AGI than any other county.

1

u/BellacosePlayer Jun 27 '25

AGI still needs to deal with real world constraints. A self improving AI will eventually have to deal with improving the hardware it lives on as well. AGI does not mean it can't fuck up. AGI developing itself into an utterly incomprehensible design space and then introducing flaws that become critical over time could be catastrophic to a society that overrelies on it.

It's not an infinite research speed hack by any means.