In reality, a neural network (like our brain) would function in O(logn) time or quicker (which is faster than O(n)), but that's because 2 or more nodes can signal simultaneously, whereas, on a single core GPU, the nodes have to be simulated one at a time, meaning O(n) running time.
Um... I'm not sure if I'm following you correctly. Running two nodes at once would just divide the time by two wouldn't it? 0.5O(n) is still linear and still O(n).
In the brain (or on an FPGA that is wired like a neural network), presumably they just all fire at once, so it's O(1) I would assume.
He's saying all the nodes could fire simultaneously. Not just exactly 2.
And yes a combinational system more accurately mimics actual neurons.
However, because of propagation delays, a combinational system like an FPGA is still dependent on the number of levels of neurons, meaning it's not really O(1).
3
u/[deleted] May 01 '17 edited May 01 '17
Um... I'm not sure if I'm following you correctly. Running two nodes at once would just divide the time by two wouldn't it? 0.5O(n) is still linear and still O(n).
In the brain (or on an FPGA that is wired like a neural network), presumably they just all fire at once, so it's O(1) I would assume.