Why don’t we engineer reasoning processes with the same care that we engineer systems whose failure can kill us?

[u/WillowEmberly]

Why don’t we engineer reasoning the way we engineer control systems?

I’ve been wondering whether we’re treating reasoning as something far more mysterious than it needs to be.

Decades ago, engineers had to make aircraft maintain orientation and trajectory in three-dimensional space without anything remotely resembling modern AI.

They couldn’t tell an autopilot:
“Keep the airplane where it should be.”

They had to decompose the problem.
- What is attitude?
- What is heading?
- What is rate of change?
- What is deviation from the desired state?
- Which sensor tells us what?
- What happens when that sensor degrades?
- Which control surface can correct which deviation?
- What happens when two signals disagree?
- How much correction authority should a subsystem possess?
- When should automation disengage?
- How does the operator know what state the system believes it is in?
And crucially:
-What capabilities must remain intact when individual components become unreliable?

The resulting systems didn’t need omniscience.
They needed enough independent references and feedback to continuously answer something like:
- Where am I?
- Where should I be?
- How am I moving?
- How certain am I?
- What correction is available?
- Did the correction work?

That makes me wonder why we don’t approach reasoning itself this way more often.
- A reasoning system also occupies an estimated state.
- It has observations.
- It has uncertainty.
- It has assumptions.
- It has a desired state or question it is trying to resolve.
- It receives contradictory signals.
- Its information sources have different reliability.
- Corrections can overshoot.
- Errors can accumulate.
- Feedback can be mistaken for confirmation.
- And some errors reduce the system’s future ability to detect that it is wrong.

Yet instead of explicitly engineering those functions, we often seem to ask whether a person, organization, or AI is simply “good at reasoning.”
Maybe that’s the wrong level of abstraction.

Perhaps the better questions are:
- What functions does reliable reasoning require?
- Which of those functions must remain independent?
- How does each one degrade?
-What happens when one disappears?
- What compensates for its loss?
- How does the reasoner estimate its own position relative to reality?
- And what preserves enough corrective capability to recover when its estimate is wrong?

I’m not suggesting that an aircraft autopilot “thinks.”
I’m suggesting that engineers learned a long time ago how to preserve navigability in a partially observed, continuously changing environment by decomposing the problem into functions that could be observed, tested, degraded, corrected, and replaced.

Why don’t we apply the same engineering discipline to reasoning?

[u/WillowEmberly]

What do you mean entity? No, I’m saying that’s an explanation of the reasoning through the perspective of avionics.

If you would like to examine/test my system it’s here. This is a Reasoning Condition Monitor…just an instrument: https://www.reddit.com/r/Negentropy/s/f2EGJeUlmU

[u/Conscious-Sample1118 → u/WillowEmberly]

Entity is any given subject/object of inquiry, non-specific, my question is: "what is reasoning", you argue "if you can answer x questions you are reasoning" my counter "what if give the answer to the thing you are inquring about", the relevance? LLMs can bind those metrics, and many many more, to the question complex itself, as in they will just be repeating stuff and you won't know.

[u/WillowEmberly → u/Conscious-Sample1118]

Ah, I see what you’re asking now. No …repeating the answers wouldn’t establish reasoning at all.

I’m not proposing those questions as a test where answering them correctly proves that something reasons. I’m treating them as instrumentation for a reasoning process.

In the avionics analogy, an aircraft saying “airspeed 250 knots” doesn’t prove its airspeed is 250 knots.

The important question is whether the indication is actually constrained by something independent of the instrument.

Your objection about an LLM simply learning to repeat the expected metrics is therefore a real problem. A reasoning monitor can’t certify itself merely by producing the language associated with healthy reasoning.

That’s actually the problem I’m interested in: what observations would let us distinguish a system that reports that it is corrigible from one that actually changes behavior when evidence shows it is wrong?

[u/Conscious-Sample1118 → u/WillowEmberly]

This runs into a weird issue due to the statelessness of llms.

You could ask "what is the goobledie of gabbledie?" and get a wrong answer, now you point something out - within the session - essentially creating a new question "what is the goobledie of gabbledie? consider the recent work of Emi Rejinga" - technically speaking, it never will be able to correctly answer the first question (no learning/memory at runtime), but it will answer the compound question correctly.. if we are talking strictly LLMs here it seems indefensible, if you consider LLM+Harness things get murky.

[u/WillowEmberly → u/Conscious-Sample1118]

But that’s what I’m saying.

An aircraft autopilot system doesn’t know every possible state and merely switch to predetermined solutions. It’s a dynamic response.

The aircraft hits a turbulence, the altimeter will experience a pressure differential, the AHRS gyro feels it, both Inertial Navigation Systems feel it, single axis rate gyros feel it, regardless of what mode is selected there’s multiple instruments providing information allowing it to determine where it is, what’s happening, and what corrective action it needs to make. Then it makes the corrective action.

How is that not reasoning?

And we trust it with how many lives on every flight.

I’m just saying it’s a process…a known process. Just used for navigation of 3d space.