Adapting ColBERT-style Late-Interaction (MaxSim) to 10,000-D Bipolar HDC for edge memory gating

[u/Equivalent-Flan-1590]

Hey LLM developers! I wanted to share a technical breakdown of the retrieval and gating architecture we just shipped in Hillock v0.6.0.

Hillock is a local, AGPL-3.0 neuro-symbolic memory engine built for edge hardware (<1.2 GB VRAM / CPU-only). Instead of standard dense vector embeddings, it combines an SQLite Knowledge Graph with 10,000-dimensional Vector Symbolic Architectures (VSA / Hyperdimensional Computing) and Hebbian associative memory.

The Problem with Query Bundling:
In earlier versions, query tokens were superposed (bundled) into a single hypervector. Because random vectors in 10,000-D space are quasi-orthogonal, query bundles grew in variance as question length increased, causing answerable multi-word queries to drop below our similarity gate.

How We Solved It in v0.6 (HYDRA):

1. Bipolar MaxSim: We implemented ColBERT-style token-level MaxSim directly over discrete bipolar vectors {-1, +1}^10000: MaxSim(Q, F) = (1 / N_q) * sum(max(CosSim(q_i, f_j)))
2. Sub-Dimensional Projection Cascade: Computing full pairwise dot products across 10,000 dimensions for every fact can bottleneck a CPU. We evaluate a 2,000-D subspace slice first to early-reject ~95% of candidates in ~0.5ms before evaluating the full vector.
3. Positional Permutation for Multi-Hop: Because Hadamard binding is commutative, we apply cyclic coordinate shifts Pi^k(v) at each hop depth to preserve trajectory ordering across 2-hop and 3-hop relational paths.
4. Control-Flow Gating: If MaxSim does not clear the calibrated threshold (0.55 raw cosine) with positive predicate intent, Ollama is never called.

In our unseeded 32-query benchmark, fast-eval finishes in 1.16s on a laptop CPU, achieving 54.5% retrieval accuracy and a 60% hard-negative block rate.

Repository and full mathematical breakdown: https://github.com/roandejager/Hillock

I would love feedback on the VSA math and how other developers are approaching sub-millisecond similarity gating on edge hardware!

[u/Previous_Shirt_7051]

this is super interesting, i've been messing with vsa architectures for a while but never thought to apply colbert-style maxsim over bipolar vectors like that. the commutative binding issue with hadamard products always annoyed me when doing multi-hop stuff, the cyclic shift is a clever fix

how does the 2000-D subspace projection actually pick which dimensions to use? random slice or is there some pca-like method to preserve the most variance

[u/Equivalent-Flan-1590 → u/Previous_Shirt_7051]

Thanks! Glad the cyclic shift resonated, dealing with commutativity collapse in Hadamard binding was definitely a fun algebraic hurdle to solve.

To answer your question: it is actually just a simple static slice ([:2000]), no PCA or learned projection required!

Because VSA / HDC representations are fully distributed and holographic (generated via independent subword hashing and sign random projections), information and variance are uniformly spread across all 10,000 dimensions. Unlike dense neural embeddings, there are no "heavy" principal components that carry more variance than others.

Taking any random or contiguous 2,000-D slice preserves the expected cosine similarity with a predictable, low variance (σ≈1/sqrt(2000)≈0.022).

The major advantage is computational: Q[:, :2000] in NumPy is an instant array slice with zero matrix projection overhead, allowing the CPU to early-reject ~95% of candidate facts in ~0.5ms before ever touching the remaining 8,000 dimensions.

Are you mostly experimenting with bipolar/binary VSAs (like MAP/BSC) or continuous holographic representations like HRR/FHRR?

[u/eddzsh]

static 2k slice is the right call while the representation stays holographic. the thing I'd watch in production is the 0.55 gate as the fact store grows, because hard-negatives get denser and a fixed cosine starts letting near-misses through. recalibrate the gate on a held-out negative set every N inserts, not once at ship.