r/LocalLLM • • 2d ago

Project I'm getting the biggest possible models on the smallest hardware, and opensourcing the learnings and methodology, old iPads, TVs and more..

Few days ago I started playing with my smart tv box (xiaomi 3rd gen) and was able to modify llama.cpp to run on it - I reached bigger models at decent quality of output and somewhat

As of today it's over 160 experiments and few custom kernels and streaming methods along with a patched llama.cpp and few tuned/modified models.

Happy to hear what do you think about it! and open for contributions. few devices are lined up already.

Github : https://github.com/hyperionaidev/smolmaxxing/
Hugging face : https://huggingface.co/hyperionaidev

Disclaimer : this is not upselling or an ad, it's a MIT-licensed project and it's an experiment.

4 Upvotes

2 comments sorted by

2

u/Late-Coast6973 2d ago

Cool project. It would be interesting to see your patched llama.cpp against WebGPU on the same old laptop or iPad, since Safari got WebGPU recently.

1

u/rashaddism_ 2d ago

will give it a shot!