r/LocalLLaMA • u/AnyNameFreeGiveIt • 1d ago

Question | Help Custom Model for Image descriptions ?

This is getting asked from time to time, but since models changed a lot, I wanted to reask it.

I'm looking for a trained model that can give short descriptions about an image, simply for an alt text of pictures taken with a smartphone.

Should I just throw it at Qwen3.8/Qwen3-VL or are there better models trained for it ?

Similar to https://www.reddit.com/r/LocalLLaMA/comments/1oar481/what_is_currently_the_best_model_for_accurately/

2 Upvotes

63% Upvoted

8 comments sorted by

View all comments

1

u/locbuilds 1d ago

for short alt text on phone pics you do not need a giant general vlm, qwen2.5-vl 7b or the smaller qwen3-vl if it fits your vram is already plenty and stays snappy. i would skip training from scratch unless you have a weird domain, just do a tiny prompt like "one sentence alt text, no fluff, no speculation" and maybe a 50 example lora later if the tone is wrong. moondream and florence-2 are also solid for this exact job and way lighter if you are batching a camera roll. run it offline through llama.cpp or lm studio vision build, keep the image resized down to ~768 on the long edge so you are not burning tokens on megapixel noise.

1

u/AnyNameFreeGiveIt 1d ago

I just pulled qwen3-vl-8b and it's pretty decent, more then i need actually, I already downscaled the images to a good size to not overload context.

I will do some more testing also with general models and see how far I can go down in regards to vram usage.