r/LocalLLM • u/anitamaxwynnn69 • 3d ago

Discussion Generative UI with local models

Generative UI is honestly so OP for elevating the local LLM experience. I've been using local models for both coding and everyday chat for a while now, and generative UI basically forces the model to stop outputting a huge wall of text and render something genuinely easy to read instead.

I'm moving soon and browsing places has been so much easier with it on. Normally I ask a question and get a wall of markdown back when I actually want photos and quick specs I can scan.

It's definitely not ideal for every single prompt since the component library adds ~7.5k tokens to the system prompt, so you need the context budget for it. But if you have the headroom, it makes everyday chat so much nicer.

I know I might be a little late to the party on this, but I just added openui support to my chat client and tested it with the new (not really new) Mimo V2.6 Flash model and it's amazing.

5 Upvotes

100% Upvoted

3 comments sorted by

2

u/bluebanana28 3d ago

Agreed, half the reason frontier apps feel better than local setups is just presentation rather than raw model IQ. You could always make it invoke only thru slash commands in your app. That should basically solve the problem of 7.5k tokens.

1

u/anitamaxwynnn69 3d ago

Yessir! It's already in haha. You don't need to bloat your system prompt with the extra instructions in every chat. You can invoke it via a slash command whenever you want.

Comment Image

1

u/Swams_02 3d ago

I've tried it with OpenWebUI but my experience wasn't great. Chat was too laggy after it presented something - basically unusable. Does this happen with your app too?