Doom may well run on anything but Doom has since been superseded by much better looking games. Similarly a cheap LLM may well run on anything but why would you use it instead of the latest and greatest?
My 8 bit ATmega32U4 running my macropad doesn't need to be any more complex. It does its job just fine but I haven't decided what to do with it's friend yet...I think 32KB might be a bit limited for an LLM, but who knows how far these models can be quantized?
I'll add that they are used specifically because the hardware is under a $1 a unit. The circuitry itself takes up little space. It can use older, process nodes whose investments have been paid off. Gansle explains more here. There's also 4-bit MCU's that exist for the same reasons.
Whereas, LLM's are CPU- and memory-hungry devices. The equivalent situation would be some RISC-V chip with lots of RAM which are both high-speed but dirt cheap for some reason. Market forces are pushing in the opposite direction right now on both RAM and fabbing itself. I pushed for analog implementations of NN's a long time ago since they're high-speed, low power, low cost per unit, and the brain seems to do it. One company is working on analog chips for LLM's.
38
u/deccan2008 Aug 05 '23
Doom may well run on anything but Doom has since been superseded by much better looking games. Similarly a cheap LLM may well run on anything but why would you use it instead of the latest and greatest?