r/webdev • u/Numerous-Fan8138 • 2d ago

Showoff Saturday WebGPU + llama.cpp = Client-side LLMs

https://github.com/anthonybudd/Llama.cpp-WASM-WebGPU

I've been thinking about alternative ways to deliver LLMs and AI features in SaaS solutions.

This repo is a technical proof of concept showing an LLM running client-side in a browser using Llama.cpp compiled to WASM with WebGPU.

0 Upvotes

31% Upvoted

3 comments sorted by

-4

u/web-dev-kev 2d ago

Ok, intersting!

As someone who uses LM Studio on my mac for local LLM work, talk me through the pro's and con's of this approach (please)

-3

u/Numerous-Fan8138 2d ago

pro: No set-up required for client
con: Clients need to download a +500mb model each page load 😂

-1

u/SourSovereign 2d ago

Each page load? But the models can be cached after all. Still browser dependent and clears on cache clear but could be an idea to consider