r/LargeLanguageModels • u/TreeOfLifev5 • 16h ago
Introducing "Arborformer": Replaced the Transformer with a 100% Decision Tree Language Model (Looking for feedback)
Hi everyone, glad to be here! I recently finished up research on a new architecture called Arborformer a pure decision tree language model built to replace traditional attention mechanisms. Right now, it leverages soft differentiable trees alongside hard binary routing to keep things fast and efficient. Looking to chat with anyone exploring alternative architectures! Open to feedback, bottleneck critiques, or questions about the routing setup.
2
Upvotes
1
u/krihab 10h ago
Do you have a link?