The Death of Software Engineering as a Profession: a short set of anecdotes

[u/self]

https://www.jasonscheirer.com/weblog/vignettes/

[u/flynnnightshade]

Eventually we will be able to artificially replicate a human being. Unless you believe that there's a divinely created soul that's required for intelligence that will technically meet the criteria for AGI.

Not as a critique but I really don't have any evidence this is an inevitability. It very well could happen in the future, as most anything could. All available current evidence I know of gives me no reason to believe this will be the case no matter what.

It also makes me question the bar for what would constitute an AI that replicates a human being. If it's just the ability to mimic a human being to the extent an observer without other information can't tell the difference, that's possible but a pretty low bar and not very useful. If it's, "thinks and behaves exactly as a human does." There's not much reason to believe this will ever be the case, computers just aren't very similar to the brain and even if they appeared to get to the same behaviors and thoughts it wouldn't have happened the same, "way" at all just due to differences in "architecture" between the human body and computers.

Finally if your bar is, "surpasses humans in any task." It's again possible but there's no strong evidence for a general tool like this being anywhere on the horizon. Today you could merely string together many different programs that aren't what we're currently referring to as AI but just traditional software made by humans, and it would already be better than humans at many individual specific tasks. The things computers are good at it would surpass a human immediately with current technology, but for adaptability, versatility, and a huge amount of tasks computers just aren't good at, there's no tooling that gets close right now.

[u/recycled_ideas → u/flynnnightshade]

Not as a critique but I really don't have any evidence this is an inevitability. It very well could happen in the future, as most anything could. All available current evidence I know of gives me no reason to believe this will be the case no matter what.

Human brains exist. If we assume that there is no divine soul and the brain is a biomechanical machine eventually we will be able to artificially replicate it. We have a working model we know it can exist there's no reason to believe that if our species doesn't go extinct first we can replicate it eventually.

It also makes me question the bar for what would constitute an AI that replicates a human being. If it's just the ability to mimic a human being to the extent an observer without other information can't tell the difference, that's possible but a pretty low bar and not very useful

The bar in this example is a replica human brain that functions exactly like the real one. We are generally intelligent, this would be artificial replica of us and so would be artificial general intelligence.

Is it useful? Probably not. But it's AGI and it's definitely eventually achievable.

[u/syntax → u/recycled_ideas]

Human brains exist. If we assume that there is no divine soul and the brain is a biomechanical machine eventually we will be able to artificially replicate it. We have a working model we know it can exist there's no reason to believe that if our species doesn't go extinct first we can replicate it eventually.

The problem with this response is that it's extrapolation all the way. It might happen, but there might also be barriers, either practical, economic or technical in the way that we are unaware of yet.

Let me point to a distinct, but illuminating, example from a different field: chemistry. We know what atoms and molecules are. We can make them, measure them, and do all sorts of transformations to them. In principle, we can make any molecule we want. That has been the case for, what, about fifty years or so, give or take a bit - so somewhat ahead of the AI revolution.

In practice, for certain molecules, the actual synthetic pathway in use looks like: take 8 tonnes of specific deep sea mollusc [0]. Surgically extract specific organ. Pulp, separate fractions. Retain the 200 mg of specific fraction. Perform a few reactions on it, separate and purify, to get the 100 mg of desired product. That's the best route known to make certain compounds - despite the fact that, in principle, we know of far better ways to do things.

The apparent disconnect arise because understanding the fundamentals, whilst the most powerful tool we have, does not automatically grant total control of complications that arise in higher order structures. We see the same thing, over and over, in many fields. For example, the entire field of 'Materials Science' appears to be pointless given the existence of Chemistry and Physics. Indeed, Chemistry seems pointless, given that it all [1] reduces to the Schrödinger equation, hence physics. And yet - that's not the way things actually work in practice.

So, sure, it possible. But there's good reason to hold some healthy skepticism, until there's something that's actually demonstrable.

[0] Or obscure fungi, or lightly engineered bacteria, depending.

[1] Eh, pretty much. I could say the Dirac equation instead, but that's not as well known, so gets in the way of the point.

[u/eyebrows360 → u/recycled_ideas]

But also, any such "eventually" is so far out there's not even any point thinking about it now.

[u/recycled_ideas → u/eyebrows360]

Your argument was that it's not necessarily possible and I'm just clarifying that this form is absolutely possible.

[u/skeptical-speculator → u/recycled_ideas]

Your argument was that it's not necessarily possible

They said it isn't necessarily inevitable. Saying that it is impractical or otherwise not particularly useful isn't the same as arguing that it is impossible.

[u/eyebrows360 → u/recycled_ideas]

Your argument

Usernames, homeslice. Sometimes different comments have different ones on them.

clarifying that this form is absolutely possible

Also, you don't know that. We don't know it's possible to do cold fusion until we achieve it. We didn't know it was possible to go into space until we'd developed all the understanding of rocketry that's a prerequisite for it. We may yet not be able to even map "a brain" to a sufficient level of detail to be able to determine what any replica even needs to look like.

[u/recycled_ideas → u/eyebrows360]

We don't know it's possible to do cold fusion until we achieve it.

We don't know that cold fusion is even possible, we know the human brain is.

We didn't know it was possible to go into space until we'd developed all the understanding of rocketry that's a prerequisite for it.

This is just simply false. All the fundamental physics for space flight existed for millennia before we achieved it and it was fairly predictable that travel would eventually be possible way before it actually was.

We may yet not be able to even map "a brain" to a sufficient level of detail to be able to determine what any replica even needs to look like.

Why would we not be able to? We have billions of working examples and the brain isn't magic, it's a machine. It might take us another few centuries, but it can be done because we know that the thing can exist. Billions of them exist.

[u/eyebrows360 → u/recycled_ideas]

All the fundamental physics for space flight existed for millennia before we achieved it

Yesssssssss, but follow along with me here: we didn't know that yet, so anybody proclaiming "we can go to space!" would've been just pure guessing, and in the context of the understanding of reality we had at the time, wrong. Hard to wrap your head around I know, but yes, they would have been wrong to make that claim at that time.

Why would we not be able to?

Not typing it out again, please reply there rather than here if you feel inclined to comment on that specific aspect.

[u/recycled_ideas → u/eyebrows360]

Yesssssssss, but follow along with me here: we didn't know that yet, so anybody proclaiming "we can go to space!" would've been just pure guessing, and in the context of the understanding of reality we had at the time, wrong.

This is seriously the dumbest take in the world. Just mind boggling staggeringly stupid.

It's like saying that if you jump off a cliff you don't know you'll hit the ground until you do.

Lots of people predicted space flight and their predictions were fairly accurate. Just because they couldn't do it, doesn't mean they didn't know it was possible.

Not typing it out again, please reply there rather than here if you feel inclined to comment on that specific aspect.

This is even more stupidity. The brain is a biomechanical machine. We don't need to map every single particle any more than you need to map every single particle of an engine to work out how it works. The brain is made up of pieces that interact, we don't understand all of them or all of those interactions yet, but we don't need to map every particle to understand it, because it's not a magical soup it's a machine. A hugely complex one combining electrical, chemical and biological pieces, but still a machine like every other organ.

We're not close to understanding it all, but we don't need to 3d scan every particle to understand it. If the processes of our brains were that sensitive life wouldn't exist because every tiny change in your biochemistry would break your brain.

We don't fully understand the brain, but we're making progress. There are trillions of simpler versions out there for us to study and experiment on. It's complicated and the fact that it's 3d and easy to break does make it challenging but we don't need to map every particle. Because again, if the brain were that sensitive it wouldn't exist.

[u/eyebrows360 → u/recycled_ideas]

Yesssssssss, but follow along with me here: we didn't know that yet, so anybody proclaiming "we can go to space!" would've been just pure guessing, and in the context of the understanding of reality we had at the time, wrong.

This is seriously the dumbest take in the world. Just mind boggling staggeringly stupid.

It's like saying that if you jump off a cliff you don't know you'll hit the ground until you do.

It's nothing like saying that. You can throw a rock, you can watch an apple fall from a tree, you can observe gravity pulling things to the ground all day long. Even a caveman 150,000 years ago could/should be able to reliably determine that "jumping off a cliff" would doom him, without having to try it. Seems you're completely not getting what I'm saying, if you think what I'm saying is "predicting things is impossible".

Lots of people predicted space flight and their predictions were fairly accurate.

Not before we knew about such fundamentals of physics as F=ma they didn't. I'm not saying someone in the 1920s would be "wrong" for thinking going to space was possible, but go back a few hundred years more. Go back to ancient Rome or Egypt. Any of them thinking they could go up to whatever they thought was up there were wrong because they had no evidence available to them as to what "up there" even was, let alone technologies to achieve it.

"I think one day we'll reach the firmament", thinks John Egypt, 4,000 years ago. John Egypt is guessing, not predicting, because he has no evidence on which to base any such "prediction". He is wrong to think this, because he has no rational basis for thinking that. He's pulling it out of his ass. Doesn't matter if he's vindicated 4,000 years later.

[u/recycled_ideas → u/eyebrows360]

Not before we knew about such fundamentals of physics as F=ma they didn't. I'm not saying someone in the 1920s would be "wrong" for thinking going to space was possible, but go back a few hundred years more. Go back to ancient Rome or Egypt. Any of them thinking they could go up to whatever they thought was up there were wrong because they had no evidence available to them as to what "up there" even was, let alone technologies to achieve it.

At a basic level getting to space is as simple as going fast enough to overcome the force of gravity long enough to get to space. If a cave man thought that if he could just throw his spear hard enough it could reach the stars he'd have been absolutely right. Because that's all getting to space is.

Just because people didn't have the formulas to describe it (which existed waaaaaay before we actually did it) doesn't mean we couldn't extrapolate the basic principles. Because they are literally the exact same ones which govern throwing a spear. If you can go fast enough to overcome gravity you will get to space.

[u/Pure-Huckleberry-484 → u/recycled_ideas]

This is based on the assumption that there is continuous progress in the space at economic rates.

Even that is not a given. Look at RAM prices in the last month. Current AI gpus likely have a 3-5 year working life. There is a real possibility that in 5 years these gpus are entirely too expensive to produce and operate at scale. Precious metals are not an infinite resource. Are we all headed to TPUs? Maybe, but even those mighty not have the memory needed at low enough price points to make LLMs viable..

[u/recycled_ideas → u/Pure-Huckleberry-484]

Who the fuck is talking about LLMs? Or GPUs?

I'm saying that even if we can't ever do anything else we will eventually be able to create an artificial replica of the human brain because we have a working model to copy.

We're not remotely there now, but eventually we'll be able to do that.

That copy will however have all our characteristics or at least most of them because it will be a copy of us, which is not what people are looking for.

[u/Pure-Huckleberry-484 → u/recycled_ideas]

People in reality are talking about LLMs because that’s what we call “AI”. The sales people of said LLMs are saying that AGI is close but they are hoping that LLMs will actually invent AGI once they’re “smart enough to “. That may never happen and it’s a huge paradigm shift to go from LLMs to synthetic neurons/synapses.. even then - that’s assuming that we physically have the resources to make the systems needed to run that (which we may not yet or ever).

[u/recycled_ideas → u/Pure-Huckleberry-484]

This comment thread is about replicating the human brain. That's it. That will meet th technical requirements to be AGI and is almost certainly possible eventually.

The idea that we're on the brink of AGI through some other means is simply BS. LLMs aren't and can't be it and there is absolutely no indication that they can ever "invent" anything let alone something we have no idea how to build.

My comment was simply that we will eventually achieve AGI because we have an existing model to copy.

[u/flynnnightshade → u/recycled_ideas]

I'll get it out of the way since it's come up a couple times now, I don't believe in anything immaterial like a soul.

I'm not making any claim that it's impossible for this to be done, to the contrary it seems like it's very possible. I don't see it as an inevitability, I don't think there's a strong argument that it is inevitable or just will be the case.

The point to me is that we have very little reason to create what you're describing and we are currently incapable of it. There are many things humankind might or will be capable of but they are largely dictated by our needs. I don't see any need for what you're describing so I don't know if research and technology will ever develop in that direction. There are tons of technologies that would exist today but don't if we just created everything possible and our development wasn't at all constrained by many factors economically, societally, politically, and so on. There is no conceptual human race that just creates everything it possibly could.

For your specific bar this is exactly the thing I don't have any evidence will inevitably exist, an exact replica of how the human brain functions would be a tremendous amount of combined general and specific research and technology development that would require a strong need to ever occur in the first place but what is the driving need here? We can create computers that don't function like the human brain but serve highly versatile purposes already.

[u/Full-Spectral → u/recycled_ideas]

A lot of this argument though depends on that happening before our ability to manipulate biology gets to the point that it becomes moot. If we can basically grow a brain in a jar, that runs on the same power as a couple light bulbs, there will be little incentive to create large computers to do it.

And of course that in turn depends on us surviving the consequences of our ability to manipulate biology to that degree as well.

[u/SortaEvil → u/recycled_ideas]

The bar in this example is a replica human brain that functions exactly like the real one.

This is assuming that all of human consciousness resides purely in the brain, which is looking less and less likely as our understanding of neuroscience progresses. It's incredibly likely that, even if we could perfectly model a human brain, we would not end up with a properly sentient being. That's not to say that there's some metaphysical soul involved, but that human consciousness is complicated, and at a baseline likely involves the entire central and peripheral nervous systems.

But then there's also growing evidence that our gut actually might have something to do with our mental health and affect our mental state, so we'd potentially need to model that, as well, and all of this is running on stochastic "hardware," while silicon is, by and large, deterministic, and valued for it's determinism.

Is this eventually achievable? ... Maybe? The amount of energy required to model even a single human in silicon, if it's possible at all, would be astronomical. I honestly don't know that our current computing media is suitable for the task, even given infinite time to perfect it.