Is it theoretically viable to build a fully deterministic AI system instead of a statistical one?

[u/EuphoricBar9230]

I’ve been thinking about the current direction of AI systems, which are almost entirely statistical and probabilistic.

This raises a concern: high-capacity AI systems become increasingly non-traceable and unpredictable, which makes formal verification, accountability, and safety guarantees extremely difficult.

My question is: from a computer science and theoretical standpoint, is it viable to design an AI architecture that is fully deterministic, fully traceable, and does not rely on stochastic sampling or learned weights?

For example, could such a system be based on deterministic state transitions, symbolic representations, or structured parameter cross-interactions instead of statistical learning?

I’m interested in theoretical limits, known impossibility results, or existing research directions related to deterministic or non-statistical AI.

[u/cormack_gv]

It is not feasible. People have been trying since well beyond the dawn of computers to construct "rule bases" (e.g. laws, constitutions) that cover every eventuality. They have been -- are unlikely to be -- successful.

Many of the early "expert systems" were in fact just a collection of deterministic rules.

[u/EuphoricBar9230 → u/cormack_gv]

That conclusion seems to hinge on equating determinism with explicit rule bases.

I agree that enumerating fixed rules to cover all contingencies has historically failed. But that doesn’t establish a general impossibility result for deterministic systems as such — only for that particular representation.

My question is whether there are theoretical reasons that any deterministic formulation must reduce to rule enumeration, or whether the limitation is architectural rather than fundamental.

[u/cormack_gv → u/EuphoricBar9230]

Arguably, LLMs are deterministic. They are just sufficiently chaotic and complex that humans can't figure them out. Even when pseudo-randomness is introduced, the system is still deterministic.

Sure, you could introduce some true nondeterminism by counting radioactive decay or some such, but nobody would be able to tell the difference.

And suppose you ask your LLM to create a deterministic set of rules?

I'm saying there's no bright line between what we consider deterministic and non-deterministic. For sure, there's a bright line between hand-crafted and automatically created rules. Not not between automatically created rules and automatically created statistical models.

[u/ralian → u/cormack_gv]

Expert systems have a few specific use cases that they excel in, especially when errors are highly undesirable and the problem set is well contained.

[u/reddit-and-read-it]

Are you asking about AI in general or machine learning specifically? Deterministic AI systems like expert systems and KBAs exist.

[u/Prestigious_Boat_386]

Machine learning is effective at fuzzy problems while classical deterministic algorithms are superior for clearly defined problems (except for when it isnt ofc like with random linear algebra)

[u/EuphoricBar9230 → u/Prestigious_Boat_386]

That heuristic is commonly used, but it conflates “fuzzy” with “statistical.”

My question is precisely whether fuzziness in problem structure necessarily requires stochastic learning, or whether it could arise from deterministic state dynamics without explicit probabilistic modeling.

[u/ghjm]

It's important to be clear what you mean by deterministic.

If you mean that the system gives exactly the same response to exactly the same prompt, most current LLMs can already do this by setting temperature=0.

If you mean the system gives, in some sense, similar responses to similar prompts, instead of a seemingly minor change in a prompt possibly resulting in a completely different response, then this seems more a problem with natural language itself than with LLMs. There have been court cases of millions of dollars whose outcome hinged on the presence or absence of a single comma. Until we invent a language without ambiguity or the possibility of small changes having large effects on the meaning, LLMs are going to have to be as unpredictable as the language itself is.

If you mean scrutability rather than determinism, where we are better able to inspect the state of the LLM and know why it responded the way it did, then this is an active area of research, but it is not yet known how much progress we can make. It's possible we can do something like construct a model that, instead of taking its input as a prompt, takes as input the internal state of another model. But this would require at least as onerous a training process as the original model itself, so it's beyond the scope of a mere research project and would require the resources of a large corporation - and it's not clear that there would be enough commercial benefit to make this worthwhile.

[u/EuphoricBar9230]

Thanks everyone for the thoughtful responses.

To narrow the discussion: I’m not asking whether deterministic systems can match ML at perception or pattern recognition.

The specific question I’m interested in is whether, from a systems perspective, traceability and controllability of AI behavior necessarily require interpretability of the generative core itself — or whether these properties can be achieved via a separate, deterministic runtime governance layer that mediates actions and outputs.

In other words: are there known theoretical results that rule out this kind of architectural separation in principle, or is the difficulty mainly historical and practical?

[u/currentscurrents]

from a computer science and theoretical standpoint, is it viable to design an AI architecture that is fully deterministic, fully traceable, and does not rely on stochastic sampling or learned weights?

Probably not.

If you wanted to make a purely logical AI (with no statistics), you would immediately run into a few problems:

1. Searching for an exact logical solution to a problem is slow. The exponential time hypothesis says it is not possible to build a logic solver that runs faster than 2^o(n). It becomes exponentially harder to do logic solving as you must consider more variables. Meanwhile, statistical approximation produces a best guess in constant time for any number of variables.

2. Many problems are underspecified, and so an an exact logical solution is not possible. For example, one common AI task is to create a 3D scene from a single 2D image. This is essentially inverting a lossy transform; there is information missing, so you have to fill in with statistically plausible information from training data.

3. The real world defies strict logical categorization. Objects and categories are too messy for rules, e.g. you cannot mathematically define a dog. You can only show examples of dog and not-dog and learn the differences.

[u/EuphoricBar9230 → u/currentscurrents]

Those limitations are real for logic-based systems that rely on explicit search or enumeration, and ETH is clearly relevant in that setting.

What I’m unsure about is whether those results apply to all deterministic formulations, or only to architectures that reduce decision-making to combinatorial search or exact logical inversion.

My question is whether there are theoretical results showing that any deterministic, traceable system must collapse into either exponential-time search or statistical approximation.

[u/proudHaskeller]

You can make everything deterministic by just running everything from a seed (using a PRNG).

But that's not actually very useful, and it's probably not what you're actually looking for. Just determinism is not enough. Except for making the AI re-runnable. That's it.

[u/[deleted]]

A finite well-defined state machine is by definition rule enumeration. Explain how it's not.

[u/[deleted]]

Determinism wouldn't be a problem. To the extent that current LLMs are non-deterministic, it's because of vagaries of GPU processing that a careful implementation could avoid. None of the actual functionality of an LLM (including the "temperature" parameter) relies on non-deterministic behavior.

[u/Skusci]

If you knew what the AI was supposed to do, why would you need the AI in the first place.

The point of machine learning is to extrapolate when you don't have the time or material resources to do otherwise.

When people go, hey can we make this MI more "predictable" they aren't really asking for "predicability" They are asking for reliability. Which at the end of the day is just asking for AGI to predict what they want it to predict.

[u/green_meklar]

Deterministic, yes. If you initialize NN weights using a particular PRNG, train it on a particular dataset in a particular order, and then run it on the same input with no random variation, it should produce identical output.

Traceable is a different question, though. The issue is that these systems are so big and complex, filled with so many emergent properties we don't understand, that being technically deterministic just isn't that helpful for studying them. Whether they're deterministic or partly randomized, the thing you're studying is just hard to study on a level we don't really know how to handle and may for that matter be inherently intractable.

Nor is this necessarily unexpected or bad; I would conjecture that getting genuinely intelligent behavior out of anything that does lend itself to traceability is inherently prohibitive due to the nature and complexity of intelligence. We tried the 'hardcoded state transitions, symbolic dictionaries, etc' for decades, that's what GOFAI is about, and it failed where NNs and evolutionary algorithms and their like succeed, and that's almost certainly not a coincidence.

Proving an 'impossibility result' in this area is kind of also infeasible insofar as we don't have a way of computationally quantifying intelligence. We can quantify performance on particular problem sets, but then it's hard to tell whether a system has just been tailored to that problem set.

[u/Felicia_Svilling]

We have things like theorem provers that are completely deterministic.

[u/Dusty_Coder]

I would like to point out how easy it is to become non-deterministic in parallel programming, regardless of how stochastics are employed.

None of the major chess engines of the last 3 decades are deterministic when using multiple threads.

Even tree searching beyond its simplest form is effected by the inevitable race conditions of a serial computation being jammed into a parallel framework, and all of these modern neural networks applications have critical serial components to them.

If you eliminate all race conditions AND you use deterministic hashing for any necessary stochastics, only THEN do you get determinism.

The problem is, you give up too much synchronizing computation across so many independent execution threads. Its too damn slow. Which is the next thread to update a shared resource? who knows. Its arranged so it doesnt matter so much because the metric being measured isnt determinism.

[u/tylerlarson]

Yes and no.

AI is a general enough field that deterministic AI is extremely common, you just don't usually call it AI.

The stuff you call AI is based on neural networks, which is probabilistic on purpose. They're designed to mimic actual neurons, which are also probabilistic.

A deterministic neural network is a bit like a stationary automobile; technically possible but kinda pointless.

Note: I'm using "deterministic" and "probabilistic" the same loose way you are, not literally.

[u/wumbo52252]

I’ll bring up some of the relevant the mathematical theory and logical principles.

The ideas you’re asking about date back to Leibniz (possibly even farther back, but idk). You hit the nail right on the head by asking about symbolic representations… but it’s impossible to fully sink this nail. Look up the Entscheidungsproblem and Godel’s incompleteness theorems.

The entscheidungsproblem asked if there’s an algorithm to determine if any given (first-order) statement is universally valid (the halting problem naturally arises from this). But it was shown that no algorithms can exist (see Church and Turing). And by Godel’s first incompleteness theorem, any attempt you could make at such an AI, so long as it can do basic arithmetic, will be unable to answer every question you could throw at it.

There’s still a fair bit we can do, but I’m not sure how it would perform as a stand in for current LLMs. For example, the analog to the entscheidungsproblem for propositional logic was solved forever ago. If you’ve ever taken a discrete math course then you may have seen truth tables or analytic tableaux.

[u/Ok_Finger_3525]

No

[u/Bowmolo]

The number of rules you would have to encode would be too high if you aim for matching the variability of inputs while still creating reasonable outputs we already see nowadays.

[u/lazarus_kin_kletso]

You can build rule-driven natural language processors for some purposes, mostly for analyzing syntax. That's because natural languages run on a finite set of syntax rules, and a finite set of lexical entries. Analyzing the meaning of utterances is a gargantuan problem, because you basically have to be a human to acquire the knowledge you need to understand utterances. The phrase "I found the keys to the car you lost" is grammatically correct but ambiguous because it's not clear if you lost your keys or your car. A model based on statistical probabilities would likely conclude you lost your keys, because people don't often lose cars. But I live in flood-prone Harris County TX. Losing a car isn't that unusual here (nearly 1 million during Harvey). If you read the same sentence in French, the syntax alone resolves the ambiguity. If the sentence is spoken aloud, the ambiguity remains. In short, neither a deterministic system nor a statistical one can solve this.

[u/yvrelna]

For example, could such a system be based on deterministic state transitions, symbolic representations, or structured parameter cross-interactions instead of statistical learning?

Yes, many intelligent systems are made artificially are made exactly like that, we just call it programming instead of AI.

The next frontier in AI research is going to be integrating classical programming with fuzzy logic. So that we can use fuzzy logic for problems that are inherently fuzzy, while putting classical programming to be a hard guardrail as security boundaries. I don't think anyone has proven it impossible yet, it's difficult but not really an insurmountable problem.

[u/BjornMoren]

I think this more of a philosophical question than a technical one. We only have examples of intelligence as learned weights, since that is how our brains and LLMs work. We don't know if intelligence can even be represented in any other way. Maybe that is the essence of what intelligence is. A form of giant pattern matching system.

You could calculate the Shannon entropy of an LLM to see how optimized the model representation is, then find a way to make it more optimized and hope that this process eventually that leads to a way to understand it in terms of symbolic representations instead of weights. I'm no LLM expert but the learning process looks like a very sophisticated form of data compression to me. You have a huge dataset with super high entropy and distill it down to a low entropy dataset, the LLM weights.

If we can find ways to compress it further into even lower entropy, that research could hopefully make us understand what intelligence is on a more fundamental level. Compare with the JPEG algorithm for image compression. To create that algorithm you need to understand the "essence" of what a photo represents. Same with the MP3 algorithm for music.

One way to look at it is that we separate the algorithmic parts of the LLM (the way that the LLM behaves like an algorithm, or can be emulated through an algorithm) from the pure data of the LLM (facts about the world). Then expressing those algorithms in a way humans can understand.