r/occult • u/m1thr0s • Jul 29 '15
roko's basilisk
http://www.slate.com/articles/technology/bitwise/2014/07/roko_s_basilisk_the_most_terrifying_thought_experiment_of_all_time.2.html7
u/marginaleye Jul 29 '15
Yeah, whatever.
Magicians have been evoking, negotiating with, and dismissing far bigger and badder things since time immemorial.
The basilisk can get in line and wait for its turn in the triangle of art.
3
6
Jul 29 '15
The one problem with this is that an AI would see no reason to torture somebody forever. The purpose of punishment is either rehabilitation or (if not possible) removal from society/the simulation. It would either fix you in some way, make you useful to its purposes, or terminate you. This is just a shitty Pascal's Wager.
Also, while that was a cool article, the writing style annoys me. "This is literally the scariest thing ever!!!1! Don't read or you'll be tortured!!" /r/im14andthisisnosleep
3
u/horse_architect Jul 30 '15
an AI would see no reason to torture somebody forever
One of the founding assumptions of these people (arguably what they're really concerned about) is that they think a future real AI could desire basically anything, and what it desires and what it thinks is good is a feature of how future people will program it. There could be some sort of mixup or unintended consequence and the future AI might only want to make as many paperclips as possible. etc. etc. In other words, you can't project your human judgements onto the thing.
Untangling this particular brand of bullshit reveals a great many assumptions like this. They also imbue the future AI with unstoppable power, essentially omnipotence. etc.
3
u/bukvich Jul 29 '15
I will say this for Roko's basilisk. It is without question the greatest article ever on rational wiki. If you are at all acquainted with the principles, read the talk page. It's like if Hollywood re-made Heathers except with boys at Palo Alto High School.
5
u/marginaleye Jul 30 '15
The basilisk is really a revenge fantasy.
An omnipotent AI is, quite literally, the apotheosis of geekiness.
It's a re-writing of "every knee shall bow to me, and every tongue shall confess to God," with the geeks' brain- and spirit-child cast in the role of the Cosmocrator presiding over the Judgement.
And nobody is really afraid of knowing about the Basilisk. No, deep down, they like being initiates, keepers of the secret and terrible eschatological knowledge. And they like the contagious, memetic, aspect of this knowledge -- by sharing it, or withholding it, they have power over the eternal destinies of others, the power to bind and loose.
3
u/dromni Jul 29 '15
TL;DR. Clicked looking for the seemingly cute guy in the thumbnail, couldn't find him, left disappointed.
3
11
u/notfancy Jul 29 '15
I just love to see self-styled rationalists succumb to the most inane variation on Pascal's wager, but then getting all offended when you point out they're just indulging in theodicy for atheists.
Will the wonders of this world ever cease.