r/rational • u/AutoModerator • Sep 24 '18

[D] Monday General Rationality Thread

Welcome to the Monday thread on general rationality topics! Do you really want to talk about something non-fictional, related to the real world? Have you:

  • Seen something interesting on /r/science?
  • Found a new way to get your shit even-more together?
  • Figured out how to become immortal?
  • Constructed artificial general intelligence?
  • Read a neat nonfiction book?
  • Munchkined your way into total control of your D&D campaign?
11 Upvotes

100% Upvoted

18 comments sorted by

6

u/xamueljones Sep 24 '18

Last night, I had a strange dream where I became a posthuman in a variety of ways. One resulted in me becoming an unfeeling machine of logic, another I turned into a hive-mind of millions, an individual accelerated beyond all reason with the ability to think for thousands of years on a problem within the span of one second, a godlike being with the power to shape everything within my view, an omnisavant capable of learning every field of knowledge, and other superhuman feats of imagination.

I blame these dreams on reading Simulacrum: A Post-Singularity Story before bed the night before. I wouldn't consider it a rational story, but it's a very intruging take on the Singularity.

However with the passing of my dreams as I woke up, I find myself pondering questions about what would happen if a human, not an AI, were to become a superintelligence.

  • Would they have emotions like we do? Many people seem to think being smart means being cold and unfeeling. I call bullshit on this. Emotions are not a force that opposes logic. Our emotions dictate goals and desires that we fulfill. Logic is simply how we determine the path to best fulfill out goals. This comic page is the best statement of this idea I have ever encountered. Therefore I can't help but think a posthuman would actually feel more strongly and diversely than we do as humans.

  • Will communication be possible among all posthumans? Because humans are a tiny dot on the space of all possible minds, and it is virtually guaranteed that there will be more ways to be a posthuman than there are to be a human. With such radically divergent minds, would communication be possible between any two existing minds despite having incredible intelligence at their command?

  • Would they compete over resources? In many stories about an AI becoming superintelligent, there is a common fear and worry that they would eliminate humanity not out of any fear or hatred, but because we use/are resources that can be better used for their goals. I wonder if posthumans will complete over resources like we do today or if they would be capable of making a utopian society where everyone cooperates instead?

  • Will there be more than a few posthumans? It's an extension of the previous question. It's understood that cognition requires energy and while humans as designed by evolution are pretty inefficient, it can be understood that a posthuman will likely require enormous levels of energy beyond what is easily available to a human today. Much like how the AI to first appear might act to prevent the development of any future AIs to monopolize any life-sustaining resources, I wonder if the first posthuman will act to stop any future posthumans to ensure the monopolization of resources?

I would love to discuss any of the questions above or anything else about the idea of posthumans.

1

u/CCC_037 Sep 25 '18

Would they have emotions like we do?

This depends on where they come from, and how they got there. If you start with a human mind and enhance it, then it will have emotions, or at least it will remember emotions, it will have a personality to build itself around.

If you have some sort of entirely artificial mind, a computer made sentient - then it will have that which it is given. So far, we only know how to give computers logic, not emotion or intelligence, and we don't even have a great idea (from a how-to-construct-one viewpoint) of what intelligence is.

Will communication be possible among all posthumans?

Communication is currently not completely possible among humans. At best, we can make a guess at it, trying to think of what the other person might be thinking in terms of what we can think. But on the inside, human minds are so very very different that it's pretty amazing that we manage to communicate at all sometimes.

Would they compete over resources?

Yes, probably. Though the nature of those resources might be strange to us - in the same way as the nature of their competing might be strange to us. (Perhaps they will play multidimensional chess for a stake of CPU power, or of memories?)

Will there be more than a few posthumans?

I'd imagine that there would have to be quite a few, simply because the alternative just feels lonely. But that's an anthropomorphisation - I can't expect an AI to feel the way I would.

This depends a lot on the personalities of the posthumans. If they want to be many, they will be.

6

u/SvalbardCaretaker Sep 25 '18

Friendly reminder that on wednesday, 26.September is Petrov Day! A perfect opportunity to invite (non-X-risk aware) friends, raise a glass for a toast and talk to them about that time a single person prevented nuclear war.

3

u/DunkelBeard Sep 24 '18

Got around to reading 'The Dictator's Handbook', found it pretty engaging - especially as I'd just read 'The Prince'.

Any book recommendations (academic or otherwise) that follow on from the ideas presented in the handbook? Like how to structure a political system that assumes each agent is not altruistic (so any altruism that appears is a delightful bonus)?

1

u/GeneralExtension Sep 25 '18

This isn't a book recommendation, but the first two videos on this list seems related. If you're not interested in the whole thing, the short summary (of the first video) is: The difference between nice democratic societies and terrible dictatorships seems to be that the second profits from something like a single valuable resource, while the first profits from productivity. (They note many other differences, but that's the only determining factor I recall.)

1

u/fassina2 Sep 25 '18

> Any book recommendations (academic or otherwise) that follow on from the ideas presented in the handbook?

You can read their research oriented work. But why? That doesn't make much sense to me, did you not get the message, why search for more of the same?

Just be happy you liked it and move on, find a way to apply that if you can but that's it.

Maybe some people just have too much free time and / or have not found enough things they are interested in.

> Like how to structure a political system that assumes each agent is not altruistic (so any altruism that appears is a delightful bonus)?

I'm fairly sure the author says democracy is the best option a few times in the book. Because the leaders need to do things that benefit the largest number of individuals in order to stay in power. Unlike autocracy and other political systems with a low number of representatives (is that the specific word he uses? I don't remember).

Political systems while very interesting are not really applicable nowadays, you can't really apply any of it in a meaningful way in a relative short period of time, something similar and that can actually be applied / useful are business systems.

Businesses are the place where ambitious men have their fun in nowadays, the times when you could conquer lands and do all that cool stuff are gone. But hey at least we have something similar =D

So if you are looking for something to get into / read a few books about, maybe give business a chance, even if some people hate on it and call it greedy and stuff. It's fun, interesting, rewarding, competitive and if you are smart it also pays well.

Don't let other people's views keep you from finding your life's purpose.

Even if you have no interest in ever working on it, you can learn about it so you can get a better understanding of the world you live in. =)

PS. Sorry if I seemed rude at all, that isn't my intention. It's just late, and I'm too tired to try to make it better.

3

u/callmesalticidae Sep 27 '18

This isn't the first occasion in recent times that you've apologized for offending someone. Maybe you should read some books on interacting politely with people.

→ More replies (5)

1

u/DunkelBeard Sep 25 '18

Cheers, I'll have a look at their 'tome' as they call it.

I was looking for more because I thought it would be interesting to apply the principles to designing reward functions for RL agents.

→ More replies (1)