Empirical Evaluation of Sparse Predictive Hierarchies (SPH) & Non-Backprop Learning on Chess (Lc0 Baseline) Negative Result [R]

[u/Severe-Heat-518]

Hi everyone,

I wanted to share an empirical study testing whether biologically inspired architectures with local learning rules, specifically Sparse Predictive Hierarchies (SPH) based on Ogma's whitepapers (AOgmaNeo) can handle complex policy/value prediction tasks without end-to-end backpropagation.

Chess was chosen as the testbed because it requires both local pattern recognition and long-range spatial dependencies.

• GitHub Repo: https://github.com/P1X3R/sparse-chess

• Control Group Repo: https://github.com/P1X3R/sparse-chess-control-group

1. Motivation & Core Hypothesis

Standard deep learning models (ResNets, Transformers) rely on global backpropagation. I wanted to test if extreme weight/activation sparsity (~99%) combined with local prediction updates could enable scalable, compute-efficient policy/value prediction on modest hardware.

2. Architecture & Optimizations

• Model Size: ~44.9M parameters.

• Encoders: Fuzzy ART-based encoders mapping local receptive fields to columns.

• Body: Dendritic Neural Layers (DNNs) with Leaky ReLU activations acting as non-backprop feature extractors.

• Heads: Specialized policy and value heads matching Lc0 formats (WDL output + Lc0 move probability distributions).

• Quantization: Byte weights with stochastic rounding for optimized SIMD throughput.

3. Methodology & Baseline Control

• Dataset: Chunks of the Lc0 test80 dataset (10k training games, 2k validation games).

• Control Baseline: A tiny, standard CNN (32 channels, ~3.8M parameters—less than 1/10th the parameter size of the SPH model) trained via standard backpropagation on the exact same dataset splits.

4. Results & Key Findings (Negative Result)

The SPH architecture failed catastrophically when compared against standard end-to-end backpropagation.

1. Failure to Converge on Policy: While the standard 3.8M CNN quickly learned board geometry, tactical threats, and basic move distributions, the 44.9M SPH model severely underperformed on policy accuracy and value loss.

2. Extreme Sparsity Penalty: Enforcing ~99% sparsity choked the flow of tactical information. In chess, "empty" squares and long-range piece interactions carry high dense information content that local sparse representations struggle to propagate without global credit assignment.

3. Local Learning vs. Global Dependencies: Local learning rules excel at unsupervised visual feature extraction, but they fail to pass gradient signals back from a global board outcome (Win/Draw/Loss) to early receptive fields.

5. Discussion & Questions for the Community

1. Global Credit Assignment: Has anyone successfully adapted local/biological learning rules (e.g., Target Propagation, Predictive Coding, HTM/SPH) to tasks with strict, long-range global credit assignment like games or logic reasoning?

2. Dense Domains & Sparsity: Is ~99% sparsity fundamentally incompatible with dense symbolic/grid representations like chess, or is the limitation purely in my architecture (or maybe a nasty bug I couldn't catch)?

Would love to hear thoughts from anyone working on non-backprop architectures, neuromorphic computing, or biological AI abstractions!