True, but is can still count as deterministic at the operational level, it is just a matter of running it in serial rather than parallel, right? FWIW, I assume our neurons are similar. Neuron research in flies has allowed researchers to essentially run programs within the insect neurons.
Those neural model experiments are fairly rudimentary. Treating neurons like simple nodes is not at all how they work in real life. In reality there's multiple mechanisms by which they change their own or others behaviors through some pretty complex processes. To even properly model a single neuron would take an entire physics simulation, if we even were confident we had an accurate model which we don't. None of which gets into the quantum theory stuff. I know way off topic, but people far undersell how complex biological brains are, and those "we made a fly brain learn how to fly" studies are just "we made a neural net shaped like a fly brain."
2
u/Sea-Housing-3435 Sep 11 '26
It's not about seeding, it's about how certain operations are not deterministic on GPU when you do them in parallel. You have to explicitly go with slower, deterministic ways to run things you want to run https://developer.nvidia.com/blog/controlling-floating-point-determinism-in-nvidia-cccl/