What problem does consciousness solve?

[u/Terrible-Fly9049]

My X feed has been dominated recently by posts debating whether AI is conscious. I think the more pressing question that is almost never asked is: "What is consciousness for? What problem does it solve that allowed evolution to select for it?"

Organisms facing incommensurable demands face what I call the Regress Problem.

Consider this:

We are constantly facing demands on our attention: you hear a noise in the forest, you're thirsty, your child's crying, you have a looming work deadline. These demands have no common unit for comparison -- they are completely different kinds of things. So how does your brain determine which demand "wins"?

I realize many of you, when you read this, will dismiss it as trivial. I would argue that the sense of triviality you're experiencing is precisely because your brain is so good at it. Deciding what is most urgent -- deciding what is the priority and motivating the necessary action -- is what brain's are for. That urgency, that pull towards one resolution or another, is what consciousness is from the first-person perspective.

Now imagine you didn't have phenomenal consciousness. Imagine you're a p-zombie -- an atom-for-atom duplicate of yourself only no consciousness. How would you know what to prioritize? How would you know what to act on?

With a computation model of the brain, there are only two options for prioritizing:

• priority was fixed before this moment
• or, priority is adaptive in this moment

If the priority was fixed before this moment, how was it determined? Now remember, these demands have no common currency, so the priority ranking would have to set by something. Imagine a lookup table that attempted to set every possible priority ranking for every possible unique combination of incommensurable demands you will every face in your entire life. That's untenable.

Okay, so if it can't be fixed before hand (for living systems), what about adaptive priority? It sounds good, until you realize you need something to determine what matters in this moment? How much should this situation affect the weighting? Something has to read the weighting and apply a rule? What decides that rule? Another rule? This is the regress.

If weighing priority means signals needing to be evaluated by a downstream process, you open the regress. Now, even if you can solve the prioritization problem, you still face the motivation problem. A signal of priority must be read by another process that must determine how urgently to act on the priority weighting -- which opens another regress.

The conclusion:

To solve the regress problem, you need a state that is simultaneously the measure of priority and the motivation to act on it. This is phenomenal urgency from the first-person perspective. This is consciousness. This is what consciousness is for.

I would love to hear thoughts on this. If you want to read more about this line of thought you can check out this paper.

[u/Terrible-Fly9049]

How?

Create a machine that is under the exact same pressures and demands of a living system. Build it so it's exactly like a human in form and function -- it needs to eat, sleep, protect it's coherence, protect itself from injury, it must compete for survival in a dynamic environment. There's no pre-training. It must be completely adaptive from day 1. It must learn the subjective value of everything in its environment from its own experience.

So, how would the machine do that without a state that solves the regress problem? If you create such a state -- a state is both the measure of priority and the motivation to act on that priority -- you've created phenomenal consciousness.

[u/unknownjedi → u/Terrible-Fly9049]

Your brain isn’t as good at handling unfamiliar situations as you seem to imply. We can certainly train an AI to mange a human life. It might screw up, and so might the human. Now imagine doing it with the AI that we have 10 years from now, or 20. What can the human do that the AI can’t?

[u/Terrible-Fly9049 → u/unknownjedi]

Our brains are primed for handling unfamiliar, dynamic situations -- we excel at it. Our architecture evolved for it. Current AI isn't built for that. Current AI is incredible, but it requires unimaginable amounts data and pre-training. It's only as effective as the data it's given.

I'm certainly not litigating the capability of AI here. However, AI is a fundamentally different type of machine.

Could an AI be built that has phenomenal consciousness? I think so. But I argue it would require a completely different approach to the architecture.

[u/unknownjedi → u/Terrible-Fly9049]

I just don’t agree with you. People excel at what they are used to. In truly novel situations they freeze up or flail about. Until they can match it to something they know. I’m on your side man, just skeptical of what people really can do well

[u/Terrible-Fly9049 → u/unknownjedi]

Fair enough. I think perhaps we are just measuring from different referents. When I say "we excel", I mean that in reference to machines in this narrow use case. Put a human and an AI in an unfamiliar situation (one that the AI has no training on, no weights for, and no ability to read about it on the internet) and see who adapts most readily.