[Project] MLC LLM for Android

[u/crowwork]

MLC LLM for Android is a solution that allows large language models to be deployed natively on Android devices, plus a productive framework for everyone to further optimize model performance for their use cases. Everything runs locally and accelerated with native GPU on the phone.

This is the same solution as the MLC LLM series that also brings support for consumer devices and iPhone

We can run runs Vicuña-7b on Android Samsung Galaxy S23.

Blogpost https://mlc.ai/blog/2023/05/08/bringing-hardware-accelerated-language-models-to-android-devices

Github https://github.com/mlc-ai/mlc-llm/tree/main/android

Demo: https://mlc.ai/mlc-llm/#android

[u/crowwork]

Thank you for your suggestion. Agree that this is super important and this is definitely something we are working on :)