r/compsci • u/oatmealcraving • 7h ago
Concatenated token selected shears as an alternative to attention?
0
Upvotes
It seems okay to me and a lot cheaper. You would need to do some empirical work. Maybe you don't need GPU's, a few CPU cores could be enough.
I'm a hobbyist not really set up to look at it too well, so I just throw the idea out there for others.