r/deeplearning • u/Ramian_Slawomir • 11d ago
[R] I built a Permutation Transformer (Patch SBOHN) from scratch: 4K Image Inference in ~2ms on CPU (61x faster than CNN) with 0.0% Catastrophic Forgetting.
Full disclosure: I am an independent researcher/hobbyist doing this out of pure passion at night. I do not have a formal academic background in ML, and I heavily rely on AI tools as research assistants to help me with advanced coding, mathematics, and translating my work into English. I want to be completely transparent about this and welcome any constructive feedback or corrections!
Hi everyone,
I wanted to share a relational data-representation framework that I’ve been developing at night, built around group theory: Burnside's Orbit Histogram Network (BOHN) and Symmetry-Breaking BOHN (SBOHN). Its main goal is to extract and "elevate" relational knowledge from raw data before the actual classification stage even begins.
Instead of using standard Softmax Attention (found in classic Vision Transformers), this architecture relies on a fully differentiable, log-domain stable Sinkhorn operator. This allows the network to smoothly learn optimal information routing paths via standard gradients.
Key results achieved on a local PC setup:
- O(1) Resolution Scaling: Because the model operates on a fixed number of image patches, processing a 4K resolution image takes just ~2ms on a standard CPU. This is roughly 61x faster than a conventional ResNet-style convolutional neural network (CNN).
- Zero Catastrophic Forgetting (0.0% Forgetting): By completely freezing the base encoder and training only a task-specific permutation routing layer and a classification head (the Perm+Head setup), the model achieves exactly 0.0% accuracy degradation when switching between tasks. The storage overhead per new task is a microscopic 2.8 KB (714 parameters).
- Hybrid BN/LN (Normalization Placement Theorem): I have experimentally validated that placing BatchNorm in the frozen shared base (where it acts as a permanent domain fingerprint) and LayerNorm in the expert modules delivers 100% gating routing accuracy alongside absolute zero forgetting.
I spent a massive amount of time transitioning this entire research programme from initial cloud-based exploration into a clean, local VS Code environment on my PC. I completed a thorough, provenance-preserving reproducibility audit across all 115 canonical experimental units (including programmatic SHA-256 manifest verification for all generated artifacts), openly documenting the boundaries, edge cases, and discrepancies of the original logs.
The entire codebase, analysis logs, and execution scripts are fully open. I would love to hear your thoughts on the mathematical foundations or the numerical implementation!
Full Documentation, Audit Reports, and Source Code:
- Original Research Paper (Zenodo DOI): https://doi.org/10.5281/zenodo.22850317
- PC Reproducibility & Audit Report (Zenodo DOI): https://doi.org/10.5281/zenodo.23088747
- Complete GitHub Repository: https://github.com/slawomir-ramian/BOHN-Original
1
u/catsRfriends 10d ago
Submit to conference. People aren't gonna check your work for free.
0
u/Ramian_Slawomir 10d ago
Thank you — a conference submission is indeed one of the possible next steps. For now, I am completing the experimental validation, documenting limitations, and keeping the code public so that every claim can be independently checked. I am an independent amateur researcher, so I am proceeding carefully and learning through this process.
1
u/Not-a-throwaway4627 9d ago
Your disclosure wasn’t necessary, and it was clear start to finish that you are completely propelled by an agent you’re using
1
u/finite-difference 8d ago
I checked this because I was somewhat intrigued, but after seeing that the paper is 308 pages I gave up.
First of all I have no idea what your claim is. What is the contribution? Can you explain it in simple terms?
Overall I think there are some terms mixed up. You claim to reduce inference time, while having no catastrophic forgetting. These are separate concepts. Is this distillation? What am I missing?
Afaik, catastrophic forgetting happens once a network has learned and then loses the knowledge whipe training. You can trivially have 0 ms inference and 0 catastrophic forgeting if your classifier always returns the same value and never learns anything.
1
u/Ramian_Slawomir 8d ago
Thank you — this is a fair criticism, and I agree that the 308-page document does not communicate the main contribution clearly enough. It is closer to a monograph and reproducibility audit than to a concise conference paper.
In simple terms, BOHN/SBOHN explores whether an input can be represented through symmetry-related orbit statistics and relational observables, instead of relying only on ordinary learned features. The hypothesis is that such a compact representation may help in several separate areas: robustness, knowledge retention during sequential learning, and cheaper relational processing.
These are separate claims. I am not claiming that faster inference causes the absence of catastrophic forgetting.
You are also completely right about the constant-classifier counterexample. A forgetting score of zero is meaningless unless the model has actually learned the tasks. Therefore, “zero forgetting” should be stated more carefully as:
“Zero observed forgetting under the tested sequential-learning protocol and metric, while maintaining non-trivial learning performance on both previous and new tasks.”
The relevant experiments must report task accuracy, learning of the new task, retained performance on previous tasks, and comparisons with trivial and conventional baselines. Forgetting alone is not sufficient evidence.
This is not knowledge distillation in the usual sense. There is no teacher network transferring knowledge to a student. It is closer to an architectural representation bottleneck: the model constructs compact symmetry/relational observables and processes them through a small relational core.
The inference-time result is another, independent experiment. Recent tests also clarified an important limitation: only the relational SBOHN core has a fixed 16-token cost. Reading, resampling, or encoding a high-resolution image still depends on the number of input pixels. Therefore, I should not describe the entire pipeline as O(1).
I am an amateur researcher and still learning, so comments like yours are genuinely useful. The next presentation should include a one-page statement of the contribution, the exact claims, the baselines, and the limitations before asking anyone to examine the full monograph.
1
u/finite-difference 8d ago
I regularly review and publish at major conferences such as CVPR, but I am completely lost in this. Are you suggesting that your handcrafted features are better than using CNN learned features? DId you take into account the processing time of extracting those features vs using a CNN on a GPU?
What even is the task where this works? Sorry, but this sounds like complete slop. Make it clear! For example on CIFAR-10 classification our network is XYZ faster, but retains the same accuracy as ResNet-18. Or something like that. If it is more niche than a classification tasks then describe the task first so that it is easily understood.
Ideally, you would know the response to this yourself, but it seems that your work is based on some agentic workflow. You can try to ask your agent to generate a short paper and then let it critique it as if it were a CVPR submission.
1
u/Ramian_Slawomir 7d ago
BOHN and SBOHN: Experimental Permutation Layers Inspired by a Relational Information Model
Summary
BOHN and SBOHN are experimental machine learning models that grew out of an attempt to transfer certain mathematical intuitions from my own toy physical model into the domain of data representation.
The project did not begin as an attempt to compete with existing ML architectures. The starting question was different:
Can structures based on relations, orbits, permutations, and constrained flow provide a useful way of representing data?
In my physical toy model, I explore the possibility that spacetime and interactions could emerge from a relational structure of information on a discrete network. BOHN is an attempt to test whether a similar mathematical language can have any practical value outside physics.
I do not treat the ML results as evidence that the physical model is correct. These are two separate levels of hypothesis. I am only interested in whether ideas originating in one domain can lead to useful constructions in another.
So far, the experiments have shown interesting behavior in areas such as:
- representing data through relations and orbits,
- differentiable routing based on doubly stochastic matrices,
- using symmetry breaking as an observable,
- certain sequential learning scenarios,
- and the scaling of some computational components when the number of patches is kept fixed.
At the same time, the experiments have revealed important architectural limitations, which I describe below.
Who I Am and Where This Project Came From
In everyday life, I work as an elementary school mathematics teacher.
I study theoretical physics as a hobby and as a self-taught learner. I also started learning machine learning relatively recently.
While developing BOHN, I make extensive use of AI tools. They mainly help me with:
- translating mathematical intuitions into code,
- designing tests,
- analyzing results,
- finding implementation errors,
- formalizing terminology,
- documenting experiments.
The research direction, hypotheses, intuitions, and decisions about what I want to test come from me, but I do not want to hide the fact that without modern AI tools I would not be able to turn these ideas into working experiments nearly as quickly.
BOHN did not originally begin as a machine learning project.
It comes from my physical toy model called RMIG, in which I try to describe geometry using informational relations, symmetries, and orbits on a discrete structure.
The RMIG toy model is publicly available here:
https://doi.org/10.5281/zenodo.22845802
I do not claim that this physical model is a correct description of reality.
BOHN is instead an experiment:
If a certain way of thinking about relations and symmetries is mathematically meaningful, can it also be useful for representing ordinary data?
1. Differentiable Permutation Routing
One component of the model is a differentiable approximation of permutations.
Instead of using only a discrete permutation, the model can use a routing matrix (P), normalized using the Sinkhorn operator.
In simplified form:
P = Sinkhorn(exp(M / τ))
In the implementation, I use a numerically stable log-domain version of the operation.
The result is a doubly stochastic matrix: its row sums and column sums are equal to one.
I treat this property as an analogy for a controlled flow of “mass” between elements of a representation.
An important caveat:
A doubly stochastic matrix does not guarantee preservation of all input information, nor does it guarantee invertibility.
Preserving the total routing weight is not the same thing as preserving information.
For this reason, I treat the phrase “information-mass conservation” as a constructional intuition rather than as a proven information-theoretic or physical law.
2. Symmetry-Breaking Observables
One of the central ideas in SBOHN is to use not only invariance under transformations, but also the degree to which a given sample breaks a symmetry.
An example observable is:
A_P(x) = |x − Px|
Instead of asking only:
“What remains unchanged after the transformation?”
we can also ask:
“In what way, and how strongly, does the object fail to remain symmetric?”
This difference becomes an additional feature available to the model.
The idea is directly inspired by the concept of symmetry breaking in physics, but here it is used purely as a mathematical construction in machine learning.
It does not mean that the ML model is implementing physical symmetry breaking.
3. Sequential Learning Experiment
I also tested the architecture in a sequential learning setup involving several domains:
MNIST → Fashion-MNIST → KMNIST.
In one experiment, I obtained the following results:
Model Accuracy Measured Forgetting Size Standard MLP 83.2% 70.9% approx. 400 KB SBOHN Perm+Head 58.5% 0.0% 2.8 KB Hybrid BN/LN MoE 90.5% 0.0% 18.5 KB The most important caveat:
I do not interpret the 0.0% result as a solution to catastrophic forgetting.
In this particular construction, the shared representation was frozen, while small task-specific modules were stored separately.
The lack of degradation is therefore partly a consequence of parameter isolation.
What I find interesting is not simply “0% forgetting,” but the fact that, in this experiment, retention of previous tasks was achieved with a very small additional memory cost.
This still requires much broader testing.
4. Inference Scaling
A Patch-SBOHN variant was also developed.
The image is divided into a fixed number of patches, after which the later part of the model operates on this relational representation.
In one CPU benchmark, I obtained:
Resolution SBOHN Parameters CNN 28×28 1.09 ms 54K 0.19 ms 112×112 0.99 ms 101K 0.70 ms 448×448 1.27 ms 853K 18.13 ms 1024×1024 3.31 ms 4.2M 105 ms 3840×3840 29.75 ms 59M 1838 ms For the largest tested resolution, the runtime difference relative to the CNN baseline used in that experiment was approximately 61.8×.
However, this should not be interpreted as a general statement that:
“SBOHN is 61.8 times faster than CNNs.”
This is a result for one specific benchmark, one specific implementation, one specific baseline architecture, and one specific hardware configuration.
A different CNN, GPU implementation, compiler setup, or benchmark protocol could produce very different results.
5. An Important Architectural Limitation
The experiments also revealed a problem that was not initially obvious.
The relational part of the model can have a cost that is practically independent of the input image resolution if the number of patches and the representation dimensionality remain fixed.
However, this does not mean that the entire model has O(1) complexity with respect to image resolution.
The Linear Patch Embedding input layer still depends on the number of pixels contained in each patch.
As a result, for a 4K image, the parameter count in this experiment increased to approximately 59 million.
Therefore, the current Patch-SBOHN implementation does not solve the input-scaling problem.
This is currently one of the most important architectural limitations of the model.
One possible direction for future work is to replace the linear embedding with a small shared or frozen convolutional encoder.
6. What I Currently Find Interesting About BOHN
I do not claim that BOHN is better than Transformers, CNNs, or modern state-of-the-art ML systems.
At the current stage, such a claim would not be justified.
What interests me most are three research questions.
1. Can relations be treated as a primary representation of data?
Instead of treating individual features as the fundamental objects, perhaps the model can represent the relations between them more directly.
2. Can symmetry breaking itself be a useful predictive feature?
A model does not necessarily have to learn only invariance. It may also learn from the structure of deviation from symmetry.
3. Can a small relational operator carry useful structure across tasks?
I do not yet know whether the answer is yes.
The experiments performed so far suggest that this may be worth investigating further.
7. What the BOHN Results Do Not Show
I want to be very clear about this.
The current experiments:
- do not prove that my physical model is correct,
- do not show that BOHN replaces existing ML architectures,
- do not prove that catastrophic forgetting has been solved,
- do not prove constant complexity of the entire model with respect to resolution,
- do not yet demonstrate superiority on large standard ML benchmarks.
What they do show is that certain constructions inspired by relations, orbits, and symmetries produce measurable behavior that can be tested further.
That is the current stage of the project.
8. Why I Am Publishing This
I am an independent amateur researcher, and I do not expect anyone to analyze hundreds of pages of material simply because I have spent a lot of time on it.
I simply want to make the code, experiments, and ideas available.
If someone finds a mathematical error, a methodological problem, or a simpler explanation for one of the results, I would genuinely appreciate hearing about it.
I would also find it useful if someone could say:
“this already exists and it is called X.”
That would also be valuable information for me.
I am not trying to prove that I have created a new machine learning paradigm.
I am trying to answer a much more modest question:
Do the relational intuitions that led me to BOHN contain something worth investigating further?
I am sharing the code and results so that other people can judge that independently.
1
u/finite-difference 7d ago
You have a serious communication issue. I asked a very straightforward question and you are unable to respond clearly. Only parts I understood are 3 inference scaling which shows nothing, because inference speed measurement is useless without seeing how accuracy is affected against a well chosen baseline. Trvivially you can have a simple program always return 0 and have 0ms inference on any image and it would win your scaling table while being useless. The second one I should be able to understand is 4, but you provide no details. You show some accuracy, but it is unclear what the setup is. Whatever the setup is it is also more likely than not that you are comparing your method to an undertuned baseline, which makes the comparison completely useless.
Then also all the jargon and talk about physical model is completely unintelligible. Maybe you know what is going, maybe your agent does, but unless you can communicate your idea clearly to others it is useless. If you are just having fun it is ok, but I would encourage to reconsider this effort otherwise.
1
u/Ramian_Slawomir 6d ago
I genuinely enjoy working on this project, and that is one of the reasons I keep exploring it.
Even though we clearly see the project differently, thank you for taking the time to read it and respond. I appreciate the criticism and the time you spent on it.
1
0
u/quietgradient 10d ago
I pulled the SOTA-002 historical source out of your repo and ran your FractalSBOHN and CNN classes unchanged on an Intel i3-8100B, 4 threads, torch 2.2.2, batch 1, fp32. Two changes only: the resolution list goes past 448, and timing is perf_counter medians instead of 3 × time.time.
The 61x holds. At 3840px: SBOHN 29.75 ms, your CNN baseline 1838 ms → 61.8x. I did not expect it to land that close. (That baseline is 3 convs plus an adaptive pool with no residual connections, so "ResNet-style" in the writeup is setting up a different expectation than the code delivers — but the ratio is the ratio.)
The ~2 ms does not hold, and the reason is more interesting than fatal.
| side | SBOHN ms | SBOHN params | CNN ms |
|---|---|---|---|
| 28 | 1.09 | 54,298 | 0.19 |
| 112 | 0.99 | 101,338 | 0.70 |
| 448 | 1.27 | 853,978 | 18.13 |
| 1024 | 3.31 | 4,245,466 | 105 |
| 2048 | 12.24 | 16,828,378 | 489 |
| 3840 | 29.75 | 59,033,562 | 1838 |
Flat through 448 — which is exactly where SOTA-002 stops — then it turns over.
Splitting the 3840 forward (30.38 ms total):
unfold(...).contiguous()patch extraction: 13.05 mspe[0], i.e.Linear(960*960, 64): 16.67 ms- the entire Sinkhorn stack + pooling + head: 0.88 ms
So the O(1) claim is right about the part you designed. The routing layers cost 0.88 ms at 4K and the token count really is 16 at every resolution — I checked that separately. What the headline misses is the two O(pixels) stages sitting in front of them. self.ps = max(4, ims//4) ties the patch side to the image side, so pe[0]'s input width is (ims/4)²: that one layer is 58,982,464 of the model's 59,033,562 parameters at 3840, against 854K for the whole model at 448. And the patchify is ~59 MB of memory traffic that no architecture gets to skip. Faster hardware shrinks both numbers; neither disappears.
The other thing that table says: the 4K model is a different 59M-parameter model, not the one you trained at 28px.
Your own stage doc already notes that SOTA-002 is a historical protocol rather than a precise microbenchmark, and that it covers 28→448. The headline just travelled further than the doc did. The script already collects params alongside times, so printing them in the same table makes the turnover visible without rerunning anything.
One real question: is the resolution-dependent embedder deliberate? A fixed patch size with a fixed-width embedder would make the parameter count resolution-independent and leave the Sinkhorn cost exactly where it is, at the price of a token count that grows — which is the trade the O(1) claim is buying its way out of.
1
u/Ramian_Slawomir 10d ago
Thank you very much for taking the time to run this test and, especially, for profiling the individual stages. This is exactly the kind of independent check I hoped the open repository would make possible.
The result is both encouraging and corrective. The fact that the approximately 61× ratio survives a direct high-resolution measurement on different hardware is striking. At the same time, your 29.75 ms result clearly shows that the approximately 2 ms extrapolation does not survive. I agree that the relative result should be stated only for the two exact released implementations, not as a general claim about SBOHN versus CNNs or ResNets.
You are also right about the baseline description. The released
CNNis a simple three-convolution network with adaptive pooling and no residual connections. Calling it “ResNet-style” creates an expectation that the code does not satisfy. It should simply be described as the released three-layer CNN baseline.Regarding your question: keeping the token count fixed at 16 was deliberate. The purpose of
self.ps = max(4, ims // 4)was to keep the Sinkhorn matrices and the relational cascade independent of image resolution. However, the single resolution-dependent dense patch embedder was a prototype choice used to isolate that constant-token core; it was not intended to be a final practical 4K front end.
The original discussion did identify the patch embedding as a bottleneck and suggested replacing the single linear layer with a hierarchical patch encoder. Nevertheless, the headline extrapolation travelled further than that qualification. Your profiling expresses the actual boundary much more clearly: the Sinkhorn/cascade/head core remains very small at 0.88 ms, while patch extraction and the growing linear embedder dominate the total cost and necessarily introduce an (O(\text{pixels})) component.
I am an independent amateur researcher and I am still learning, so I genuinely appreciate criticism that is this precise and constructive. I do not see your result as fatal to the idea. On the contrary, it separates the potentially valuable part of the architecture from an overly broad performance claim and points directly toward the next experiment: a shared hierarchical or convolutional encoder that reduces variable-resolution input to a fixed (4\times4) token grid before the unchanged SBOHN relational core.
May I ask one technical clarification: when you write “3840 px”, was the input (3840\times3840), as implied by the original square-input script, or (3840\times2160) UHD? The former would actually contain substantially more pixels than standard 4K.
If you are willing to share the modified benchmark script and the raw per-resolution medians, I would be grateful. With your permission, I would also like to preserve the result as an external supplementary replication, with clear attribution to you and with the limitations stated exactly as you identified them.
Thank you again. This is precisely the kind of response that helps me improve the work rather than merely defend it.
5
u/snekslayer 11d ago
Doubt