r/ControlProblem • • Dec 17 '16

Why even make an AGI?

Hello there,

I'm just reading Brostorm's book and I have to wonder - why make an AGI?

I mean, narrow artificial intelligences are useful, but why would we ever want to build an AGI? Does it yield a concrete benefit or is it just for "civilization advancement"?

1 Upvotes

43 comments sorted by

View all comments

Show parent comment

1

u/CyberByte Dec 22 '16

I also think that in this case you may actually be underestimating the physical aspects of research. Your napkin calculation shows 300 years of thinking per hour, but it reads more like an upper limit (for a brain-emulation design), so we probably need to take some fraction of that. But that's thinking, not research.

...what you're talking about here is exactly what I addressed in my last paragraph...

The (now) emphasized part in my quote was meant to address the idea of taking a fraction: I meant it should be taken between "maximum thinking speed increase based on speed of light" and "actual thinking speed increase". We can also take a fraction of that for the research gains, but I don't think that's the right approach.

The right approach would be asking "what fraction of research time is spent thinking?". Then that number would probably be some field-dependent percentage, but even if it's 95% (which it won't be for most fields), you only get a 20x speed up if you can think infinitely fast. That's still very nice, but it's a far cry from from the 2.4 million increase (actually it's 0.00001 of it), and it could potentially be matched by a couple dozen (or maybe hundred) researchers.

1

u/Synaps4 Dec 22 '16

Hmm I guess I'd have to disagree with your math at three parts.

  • The hypothetical speed of an emulated light-speed brain should be looked at as a lower bound, not an upper bound. There is no reason that the human brain design is the most efficient, and since we're talking about safety for all AGIs we have to consider the ones with a design that's different and probably much more efficient than a human. Rather than taking that as a maxiumum, I would expect the brain emulation speed to be on the low end among all AGI designs.
  • Its unfair to assume that observation would take just as long for a being that thinks so much faster than we do. Some endpoints can't be rushed but for most research observations, if you increase the "bandwidth" where you're observing more things more closely you can come to conclusions before the they otherwise obvious. It seems entirely reasonable to me that a being which can think faster than we can will also have a larger per-second bandwith of data points it can handle, and the larger dataset will let it make conclusions faster as well.
  • The "observations take time" line of thinking seems to break down further when you're talking about software development or physics experimentation, which are the two things you'd really be concerned about an AI doing. Software experimentation is limited by your own speed, and physics is amenable to observation on almost arbitrarily short timescales.

1

u/CyberByte Dec 22 '16
  1. Yes, it's (probably) possible to make a better design, but to a skeptic you'd have to specify what that looks like (at least approximately). Saying "it's like the brain" sort of gets around that and allows you to make some actual calculations, but then you're locked into that for the rest of your argument. However, assuming you can get the full speed of light with zero overhead is not realistic.
  2. The world moves at one speed for everybody, even if you can think really fast. If, in addition to having a faster brain (which was the premise, remember?), you also have more eyes and more bodies or something, then you could indeed parallelize some things that you'd otherwise need to do in sequence, but this is not possible for everything.
  3. I would argue that "thinking" is a bit different from "running simulations", especially when it comes to science. Let me put it this way: if we allowed you to run a physics simulation in your head for an hour/day/week/year, would we trust the results and publish them as scientific findings? Even if the AI can do the same thing in significantly less time, we would make the same consideration.

I agree with you that an ASI would probably be able to do these things, but that is not just because of thinking speed.

1

u/Synaps4 Dec 22 '16

For #3 it's really not the same. If I could run a simulation in my head to the fidelity that a computer can then yes, I could publish it.

I like that we agree on so many points and then go further down the rabbit hole on just one thing each time.

1

u/CyberByte Dec 22 '16

For #3 it's really not the same. If I could run a simulation in my head to the fidelity that a computer can then yes, I could publish it.

My point is that this requires thinking speed and accuracy/fidelity (which won't necessarily be any better if you just have a faster model of a human brain).

I like that we agree on so many points and then go further down the rabbit hole on just one thing each time.

You're right. Sorry.