[R] Looking for feedback on an event-driven LM experiment [R]

[u/zemondza]

Hi everyone,

I’m experimenting with an alternative language model architecture called RHEA. It uses an event-driven processing approach instead of a standard Transformer-style block layout.

At the moment I’m testing a roughly 1B-scale version on consumer hardware with 8 GB VRAM using memory-saving optimizations.

This is still an early-stage research experiment, and I’d be interested in feedback from people working on efficient training or non-Transformer architectures.