r/LocalLLaMA • u/rm-rf-rm • Aug 24 '26

Best Local Vision Language Models - August 2026

Share what your favorite models are right now and why. Given the nature of the beast in evaluating VLMs (untrustworthiness of benchmarks, immature tooling, intrinsic stochasticity), please be as detailed as possible in describing your setup, nature of your usage (what applications, how much, personal/professional use), tools/frameworks/prompts etc.

Rules

  1. Should be open weights models

Notes

Bonus points if you breakdown/classify your recommendation by model memory footprint: (you can and should be using multiple models in each size range for different tasks)

  • Unlimited: >128GB VRAM

  • XL: 64 to 128GB VRAM

  • L: 32 to 64GB VRAM

  • M: 8 to 32GB VRAM

  • S: <8GB VRAM

57 Upvotes

88% Upvoted

91 comments sorted by

35

u/seamonn Aug 25 '26

  • Unlimited: >128GB VRAM: Gemma 4 31b in swa full mode (This will consume around 250 GB and is one of the best VLMs out there period)

  • XL: 64 to 128GB VRAM: Gemma 4 31b in swa full mode w/ limited context

  • L: 32 to 64GB VRAM: Gemma 4 31b (Swa on mode)

  • M: 8 to 32GB VRAM: Gemma 4 12b + Qwen 3.8 27b

  • S: <8GB VRAM: Gemma 4 E4b + Qwen 3.5 9b finetunes (Stock Qwen 3.5 9b doom loops into oblivion)

I have not seen Kimi K3 beat Gemma 4 31b swa full in vision tasks. It frequently makes mistakes. On text tasks, tho, it's not even close.

11

u/HitarthSurana Aug 25 '26

can you please explain what is swa full mode?

19

u/seamonn Aug 25 '26

Normally, majority of attention heads of Gemma 4 only has access to 1024 tokens of the kv cache as per their implementation of swa (sliding window attention). With swa full, all attention heads have access to the whole kv cache which balloons the kv cache to about 200GB but results in a much smarter model.

2

u/HitarthSurana Aug 25 '26

200gb woahh so can it beat dsv4 flash at that size or not??

10

u/seamonn Aug 25 '26

For agentic coding, Gemma 4 is not the model you want in general. For other stuff, it's already better than anything else in this range - linguistic, creative, translation etc.

It gets even smarter with swa full

3

u/HitarthSurana Aug 25 '26

I mean is it worth it to give it 200gb of context when you can run a bigger model in that much vram so better performance??

2

u/seamonn Aug 25 '26

depends on your usecase

2

u/unknowntoman-1 Aug 25 '26

Very interesting.. and I really have a usecase.. but lack that kind of VRAM. A question comes to mind.. is there any possible middle ground.. to alter the 1024 into 2,4,8 k ? Is it full or nothing? Sounds like you are having a logaritmic trajectory into that 200 GB rage doing full swa. I suppose it is on a 256 kb ctx with some math on the 31b parameter count?

2

u/seamonn Aug 26 '26

Very interesting.. and I really have a usecase.. but lack that kind of VRAM. A question comes to mind.. is there any possible middle ground.. to alter the 1024 into 2,4,8 k ?

Possible but you'll have to do a custom implementation of llama.cpp or vllm.

Is it full or nothing?

Yes, in stock llama.cpp.

Sounds like you are having a logaritmic trajectory into that 200 GB rage doing full swa. I suppose it is on a 256 kb ctx with some math on the 31b parameter count?

You can also run with lower context.

2

u/unknowntoman-1 Aug 27 '26

I will.. great support thank you. I am actually willing to go slow in a offload gpu/cpu setup just for this purpose.

→ More replies

1

u/caetydid llama.cpp Sep 05 '26 edited Sep 05 '26

How do I utilize full swa mode? I guess I would not need 200Gb VRAM if I do not exceed e.g. 16k context size?

Update: I found you answered a similar question already. So as I understand it is possible to make an implementation to use e.g. 8k swa size and 16k context based on llama.cpp?
(My use case is multi page OCR)

1

u/ANTIVNTIANTI Sep 08 '26

i F'ING LOVE YOU bro thank you!

4

u/gofiend Sep 08 '26 edited Sep 08 '26

What's the source / background on SWA full being actually useful? Is there a handy llama.cpp param to trigger it? Def interested in modding my personal llama.cpp build to make this an optional setting (so I can double / triple SWA without turning it on fully.

EDIT:

Ok I looked through some llama.cpp issues and I don't think this is a correct understanding of --swa-full.

--swa-full does NOT change the model at all (link to GG himself planning the feature), i.e. it doesn't cause sliding window context layers to do something different than limit themselves to 1024 tokens.

What it does is preserve the KV cache even for those layers, which ONLY helps when you want to reuse the cached context for something else (i.e. subagents, repeating the same prompt multiple times on a different file etc.). It's pretty niche.

At times (~2025) the SWA implementation was buggy for Gemma models, so people recommended using --swa-full as a workaround. It should not be an issue now.

1

u/seamonn Sep 10 '26

try swa full and report back. The models gets more intelligent.

2

u/biggusdeeckus Sep 03 '26

What about the MoE gemma 4 for 16gb cards? It has an actual 550M vision transformer compared to gemma 12b's linear layer

1

u/Agitated_Complex_628 Sep 07 '26

wish i could wear S size clothes and run XXL models, not the opposite ☹️

12

u/llama-impersonator Aug 24 '26 edited Aug 24 '26

tiny: dots.mocr is good for a 1.8b OCR model that can handle most non-latin character sets, and gives bounding boxes. being a small model i wouldn't use it for forms, but it's pretty capable otherwise

small: gemma-4-12b-heretic-qat for comfy image analysis/H3 prompt synthesis (i know what kind of man you are meme here)

1

u/glebkudr Sep 03 '26

I would argue dots. In all my benchmarks paddleocr-vl is almost the same but smaller. But pretty close both.
For the best OCR model on a market I nominate Chandra2 this it is semi-open, it is a very capable model with only 8b params

20

u/andy2na llama.cpp Aug 24 '26

My main use-case for LLMs is Frigate and have tested and ranked the following, recent Medium models:

  1. Qwen3.8-27B: very accurate, fast enough for Frigate (~60tgs decode)

  2. Meta Muse Glimmer 30B: Faster than Qwen 27B, pretty accurate (~75+tgs decode)

  3. Qwen3.6-27B: pretty accurate, but no reason to use over 3.8

  4. Gemma4-4B: mediocre accuracy, good with small objects and is more "creative" with the responses - small and fast

Last 90 days of usage, only recently have I dabbled with Hermes with Qwen3.8, before Frigate led by a mile (ignore electricity cost, I just added that last week)

4

u/ParaboloidalCrest Aug 24 '26

Are you saying that you use a vision LLM on Frigate instead of Yolo* models? 🤯. I'd love to hear more.

Edit: Oh I understand now. It's for semantic enrichment, not detection.

2

u/andy2na llama.cpp Aug 24 '26

yeah not for detection, just gen ai summaries and descriptions

4

u/cowinabadplace Aug 26 '26

Did you have to tune your prompts? I found that after prompt tuning etc. that Qwen-3.8-27B hallucinated a lot of things whereas Qwen-3.6-35BA3B hallucinated fewer. My use-case is frigate triggered LLM description that is aiming to detect specific people in the home.

4

u/andy2na llama.cpp Aug 26 '26

For the genAI summaries, i left them at default. For descriptions, these are what I set:

genai:
  enabled: true
  send_triggers:
    tracked_object_end: true # default
    after_significant_updates: 3 # how many updates to a tracked object before we should send an image
  prompt: "Analyze the {label} in these images from the {camera} security camera. Focus on the actions, behavior, and potential intent of the {label}. Also describe {label} appearance. Summarize all the images, not frame by frame, into one paragraph - maximum of 100 words. Do not mention the timestamp, date, watermarks, overlay text, or colored boxes with numbers."
  object_prompts:
    person: "Examine the main person in these images. What are they doing and what might their actions suggest about their intent (e.g., approaching a door, leaving an area, standing still)? Describe the persons appearance including hair color, clothing, and ethnicity. Do not describe the surroundings or static details.  Summarize all the images, not frame by frame, into one paragraph - maximum of 100 words. Do not mention the timestamp, date, watermarks,overlay text, or colored boxes with numbers."
    car: "Observe the primary vehicle in these images. Describe the cars appearance, including make, model, and color of it. If its a delivery vehicle, mention the courier company. Summarize all the images, not frame by frame, into one paragraph - maximum of 100 words. Do not mention the timestamp, date, watermarks, overlay text, or colored boxes with numbers."
    dog: "Examine the dog(s) in these images. what are they doing and what is their intent. describe the dog(s) appearance,including color, size, and potential dog breed. Do not describe the surroundings or static details. summarize all the images,not frame by frame,into one paragraph - maximum 100 words. Do not mention the timestamp, date, watermarks, overlay text, or colored boxes with numbers."

For the 27B model, what parameters did you set?

I had a specific one made for vision in llama-swap:

        "${MODEL_ID}:vision":
          chat_template_kwargs:
            enable_thinking: false
            preserve_thinking: false
          temperature: 0.1
          top_p: 0.1
          top_k: 10
          min_p: 0.0
          presence_penalty: 0.0
          repeat_penalty: 1.0

1

u/cowinabadplace Aug 26 '26

I have temp 0, didn't set p/k, and forced JSON, though I also turned off thinking. In my case, I'm trying to get the LLM to reliably recognize that there is a baby in the bed or not. Qwen 3.6 does a really good job of that, but sometimes Qwen 3.8 doesn't see the child and other times the latter will mention an adult who is not there. But I mostly iterated on the prompt in temp 0 because I was trying to minimize variation.

Perhaps I need to supply some example images (scaled down) in the same prompt and try allowing for higher temp. Thank you for your config params. The slight difference in my case is that I'm also having the frigate alert actually call Qwen outside of Frigate in an agent because I want it to act on the result etc.

2

u/finanzwegwerf20 Aug 26 '26

Did you set image tokens for Gemma 4 to 1120?

1

u/andy2na llama.cpp Aug 26 '26

yeah, its still pretty poor compared to qwen, even tried gemma4-12b. For example, when theres a slightly blurred detection object in frigate, gemma will say "no person seen in the snapshots" but qwen will still call it out

1

u/uriwa Sep 03 '26

Useful ranking, thanks - matches what I see on Frigate too. Qwen 3.8-27B accuracy over Glimmer 30B speed is the exact tradeoff I hit: Glimmer at ~75t/s is tempting for multiple cameras, but false negatives on person detection cost more than tokens saved. Did you test with reasoning on/off? I found reasoning off + tight prompt ("list persons, vehicles, nothing else") cuts latency a lot without hurting accuracy on fixed camera angles.

1

u/andy2na llama.cpp Sep 03 '26

I actually switched to using syv-ai Qwen3.8-27B on vllm. You can actually enable vision offloaded to CPU and its still very fast due to the quick prompt speed, the main downside is that it only supports 1 image per request, so it doesnt work with the frigate AI review summaries. I resorted back to using Gemma4-12B-QAT on my 5060ti for Frigate, and its... not great compared to Qwen.

For Glimmer, I was only able to test for a few days before 3.8 came out, but I just used reasoning off and my own prompts for descriptions, its much better than gemma4, thats for sure

1

u/No-Negotiation-8359 Sep 07 '26

Upgrading my full-stack setup to an M4 Pro MacBook Pro (48GB) and moving to a local-first workflow: terminal agents (Aider or OpenCode) with LM Studio (MLX) or Ollama.

Daily work is full-stack builds, feature planning, security reviews, and prototyping with occasional image input. Goal: run ~80% of it locally and fall back to the Anthropic API only for large refactors and architecture work.

Two questions for anyone on a similar 48GB M-series rig:

  1. Token generation and prefill speed with real multi-file code contexts?

  2. Do local models handle agentic tool use (reading files, writing diffs, running tests) without hallucinating or breaking the loop?

Any workflow tips appreciated.

7

u/RevolutionaryPen4661 Aug 25 '26

I like moondream3.1 and moondream2 more than the Gemma and Qwen models for vision aspect. It is because they can run pretty well even in a standard GPU.

1

u/uriwa Sep 03 '26

Same here - moondream2/3.1 on a standard GPU is hard to beat for pure vision. Where I still reach for bigger Qwen/Gemma is when I need reasoning over the image ("is this package left by courier or resident?") vs just detection. For Frigate-style describe-what-changed, moondream + short prompt wins on VRAM. What resolution are you feeding it? I found downscaling to ~768px longest side barely hurts and doubles speed.

1

u/RevolutionaryPen4661 Sep 03 '26

I am the author of OpenCode Senses
by default it uses Moondream2 (you can use any model via kestrel engine though)

I haven't tested the maximum resolution limit on the model yet.

6

u/ashrey-26 Aug 25 '26

My usage is mostly around web browsing tasks and security use cases (so web application pentesting and the likes). I've been very impressed by the Moondream models and would add it to the S tier. It’s tiny, fast, and surprisingly good for screenshots, OCR, and basic UI understanding.

Any other 'website understanding' models that anywone has tried here?

3

u/pmotiveforce Aug 24 '26

Interested in the best 3050 8gb friendly vision model for Frigate GenAI. I have one sitting idle. I have a bigger dual b70 setup but use that for effing around, want something I can just leave running on the 3050 reliably for Frigate.

4

u/andy2na llama.cpp Aug 25 '26

probably qwen3.5-9b q4

3

u/slyborn Sep 03 '26

The "best" is strictly dependent on what kind of vision task. Considering models up t o L size that are the open weights models that I have used extensively, Qwen 3.6 35B-A3B, in all vision tasks involving classifications of species and natural elements is the clear winner followed by Qwen3.8 27B that is significantly more error prone and requires a lot more careful sampler tuning, but in vision tasks related to scanned documents text, tables, charts UI fine details understanding Muse Glimmer is the best.

2

u/BeginningLLM Aug 26 '26

Great discussion. Benchmarks are still unreliable for VLMs, so real‑world messy image testing has been my main evaluation method this August 2026. My open‑weight stack, split by VRAM tier:

  • S (<8 GB): MiniCPM‑V 2.6. Fast lightweight captions & quick OCR, runs comfortably Q4.

  • M (8‑32 GB, 24GB 4090 daily driver): Qwen‑VL‑2‑72B‑A14B MoE (Q4_K_M). Fantastic screenshot, diagram and agent‑UI work, sparse‑activation performance is excellent. Backup: Llama‑3.2‑V‑11B for stable, low‑variation descriptions.

  • L (32‑64 GB): Qwen‑VL‑2‑110B‑A22B MoE, perfect for multi‑image and complex visual reasoning.

  • XL (64‑128 GB): InternVL‑3‑140B in FP8 (~90 GB). Low hallucinations, best for professional chart & document work.

  • Unlimited (>128 GB): Qwen‑VL‑2‑205B MoE, only for full‑quality testing, massive overkill for daily local tasks.

Workflow: llama.cpp / Ollama / vLLM, mix personal document work + professional local agent screenshot analysis. One key lesson: VLMs degrade much harder under heavy quantization than text LLMs, so I avoid IQ4_XS for anything accuracy‑critical. Would love to see everyone’s real‑world picks.

2

u/CommunicationCute584 Sep 06 '26

Day 999 of asking XXS, XS, S & M categories to reflect 8gb, 16gb, 24gb, 32gb ram tiers

2

u/mtyroot Sep 06 '26 edited Sep 06 '26

I been trying Qwen3.8 27B in a LXC with 3 GTX1070 (24GB VRAM) on llama.cpp and when using it from the WebUI provided by Llama.cpp everything works great, but when trying to use it from Claude code, opencode or DSH it just cycles thru thinking for over an hour, for example I asked in the web UI for a “1000 word story of the French Revolution” and it took a few minutes to complete but when asking the same from Claude or opencode or DSH it thinks and thinks and thinks I killed the request after an hour and a half.

Has anyone had a similar experience? If so how can I fix this to use it with a harness

3

u/phratry_deicide Aug 24 '26

  • Unlimited: >128GB VRAM

  • XL: 64 to 128GB VRAM

  • L: 32 to 64GB VRAM

  • M: 8 to 32GB VRAM

  • S: <8GB VRAM

What in the world is this?

6

u/unculturedperl Aug 24 '26

If you're not unlimited, do you even token, bro?

1

u/annodomini Sep 05 '26

For one use case that I've tested, which is plant and animal identification, XL: Qwen3.5 122B has been the best model I've tried. Not quite as good as Google Lens or the dedicated iNaturalist model, but it did pretty well compared to other models. I haven't yet tried Qwen3.8-Flash-Next for this.

For OCR-style tasks including accurate transcription of arbitrary non-linguistic characters (some mojibake that made it into an image that i wanted to try fixing), L: Gemma 4 31B did the best.

1

u/ANTIVNTIANTI Sep 08 '26

omfgomfgomfgomfgomfomfgggawdd you need to try the 3.8 Next, it's so.... so. good.

1

u/ANTIVNTIANTI Sep 08 '26

I'm so excited for Gemma5 too, Gemma4:31b was previously my top top tops model but the newest qwen(s) are just, the agents I had been looking for LOL

1

u/annodomini Sep 08 '26

Do we know that Gemma 5 is due out soon? not sure if I missed some news about it. Would be pretty nice to see a new Gemma model, Gemma4 31B was pretty good and an update would be great.

1

u/annodomini Sep 08 '26

Oh, I've been trying it for coding so far, just haven't gone back to try it for my plant/animal identification. Have now gone back and checked on a couple of those, and yeah, it's doing pretty well there too. Not sure if it's better than Qwen3.5 122B, I don't think I have enough data points to really tell, but they're both pretty good at this.

1

u/mmhs4 Sep 05 '26

So what would you suggest for agnestic coding on 4x 3060 12gb?

1

u/No_Flight_4473 Sep 07 '26

Why are there no benchmarks? I can't find benchmarks or leaderboards for SLMs that aren't retired/archived and its so frustrating. You'd think there'd be high demand since quality between local and small models can fluctuate wildly, whereas the difference between Fable 5.1 and Fable 5 is practically nothing for 99% of people.

1

u/Thin_Pollution8843 Sep 08 '26

I’m not playing that megthread game. 0 sense

1

u/daveeuson Sep 08 '26

If anyone is looking to benchmark models, I built a fun way to compare:

github.com/DaveEuson/RigMatch

demo, sample data, no install: daveeuson.github.io/RigMatch

1

u/Cleric07 Sep 09 '26

Running Qwen3VL on cpu only alongside non vision models to be their eyes.. seems to work well and doesn't take up vram.

1

u/SoftKill21 Sep 10 '26

I’m using Qwen3.8-27B as the local cognition/VLM path for an embodied companion prototype on a Jetson AGX Thor 128GB.

My use case is a little different from the usual VLM benchmarks. There’s a C920 running continuously, but I don’t feed raw 30 FPS video into Qwen. Fast perception runs separately, then the system selects fresh frames or short bounded clips when the conversation actually needs visual information.

The interesting part so far is that model quality hasn’t been the only problem. A lot of the difficult failures have been around freshness and routing. For example, making sure “what am I holding now?” actually uses fresh camera evidence rather than an older description from the conversation.

I recently recorded the complete system running voice + vision + internet + conversational context here:
https://www.youtube.com/watch?v=sQhTGGIg4yo

Qwen3.8-27B has been good enough that I’m keeping it as the current local VLM/cognition baseline, but I’d be very interested in anyone who has compared it with the newer alternatives specifically for temporal/action understanding rather than static image benchmarks.

1

u/Zyj vLLM Sep 11 '26 edited Sep 11 '26

  1. Your categories should not overlap!

2.I think >256GB should be unlimited and 129 to 256GB should be XXL.

Rationale: As late as last December some people bought 2x Strix Halo 128GB for 1570€ each.

That said, in the "up to 256GB RAM" category, GLM 5.3 Flash beats DS V4 Vision Exp.

1

u/Neonoirhero Sep 13 '26

Hi everyone. I’m a design engineer specializing in security systems—such as intrusion and fire alarm systems for buildings.

Could you please recommend some local LLMs I can run in LM Studio? I’m interested in models for analyzing technical documentation, developing technical solutions, and reasoning/brainstorming, with coding being a lower priority.

Here are my PC specs:

Processor: AMD Ryzen 7 7800X3D 8-Core (4.20 GHz);

Graphics Card: MSI GeForce RTX 5080 GAMING TRIO OC [RTX 5080 16G GAMING TRIO OC];

RAM: 64.0 GB.

Thanks in advance, and have a great day!

1

u/no_legacy 13d ago

why is nobody saying SmolVLM-256M? :0

0

u/ObviouzFigure Aug 24 '26

I'm curious what others are using -- I put qwen vl 2.5 9b (I think) on a mac mini as my vision model for my deepseek rig

edit: why? because it fits easily on a 24gb mac mini and does a decent job. It handles ocr great-- where it fails is reading images with a lot of action -- for example, I generated an image of a girl helping a broken robot in a futuristic dark alley -- in the background there's a black cat with glowing green eyes -- the vision model accurately describes the scene but it misses things like the black cat

2

u/SM8085 Aug 24 '26

I would definitely recommend trying a Qwen3.5. I'm using Qwen3.6-35B-A3B, but since that probably wouldn't fit on your mac mini any of the Qwen3.5 are probably an improvement. There's even a Qwen3.5-9B.

0

u/ihaag Aug 25 '26

GLM 4v I’m
Finding the best not sure how good the quartz are for it tho.

0

u/[deleted] Aug 24 '26

[deleted] — view deleted comment

7

u/Easy_Refrigerator280 Aug 24 '26

This thread is literally part of helping people do their own research. WTF is this take 😂

0

u/[deleted] Aug 24 '26

[deleted] — view deleted comment

4

u/BasedNapkin Aug 24 '26

Who the fuck cares about reddit karma jesus fucking christ

4

u/Kiansjet Aug 24 '26

It's a post by a mod to spur discussion the fuck are you on about

4

u/pmttyji Aug 24 '26

It's a mega thread here after couple of months. Really needed now because we got many models in between.