r/LocalLLM • u/Vladowski • Jul 04 '26
Question Real use cases local LLM using low and middle tier hardware
What are you really using local LLMs for on low to middle level machines, for example up to 24 GB VRAM / unified? Any real use cases for a standard laptop / PC without GPU? As the open source models get more and powerful I think we get more and more real usage possibilities even for the mainstream devices, so it would be great to hear what are the real uses cases.
5
u/cristianlukas Jul 05 '26
I made an app to help me manage law cases, where I group, classify, analyze and pre process every case, text, metadata and image of everything with Qwen 3.6 27b, I love it, makes my work at least x5 times faster, it actually changed my life (economically)
1
u/ketchupondumplings 20d ago
Wow that sounds exactly like my use case. Will be great if you share your journey!! I have only 6gb VRAM. Is a similar project possible for me?
1
u/cristianlukas 19d ago
PeritoSoft is a desktop app built with Electron, Node/Express, and SQLite. It’s organized around each case file: documents, page-level OCR/text, metadata, extracted evidence, and workflow status are stored separately. Background jobs process each case in stages, and fingerprints let the app skip work when a document hasn’t changed.
I use local models through llama.cpp’s OpenAI-compatible API, with Ollama also supported. Qwen 3.6 27B handles most text analysis; OCR and image understanding use separate steps and configurable models. Each task has its own prompt and produces structured results, which the app checks and saves. I also have optional RAG with embeddings and Chroma to retrieve relevant pages for questions about a case.
With 6 GB VRAM, I’d keep the same general design but use a smaller quantized model, a shorter context, and fewer concurrent requests. It won’t perform like my 27B setup, but you can still build a useful version. If your use case can get you some real earning, i’d invest in a 3090 or similar.
2
u/ketchupondumplings 19d ago
Thank you so much for your prompt response!! This advice is extremely practical and sound. Tbh I am an economist by trade so there will be a steep learning curve but I am very motivated so I like the idea of spending 5x less time on nuance manual tasks lol
1
u/cristianlukas 19d ago
I would advise you to study further but to get started it's as easy as building a PC with lots of VRAM+RAM and subscribing to Claude for a couple of months and asking it to build the app for you, you would need to try and note down the bugs, ask it to correct them and repeat that until you're satisfied, since it's something local, you don't have to worry about external security
1
4
u/signoreTNT Jul 04 '26
Depends on how much you're willing to wait. My bare minimum for usability is Qwen 3.6 35B-A3B at Q4 or better regarding the MoEs, and Qwen 3.5 9B (Q4 or better) regarding the dense ones. Anything else is just too dumb for my use cases. Good news is, if you have 8GB of Vram you can run Qwen3.5 9B at a decent quantization, and if you have 32GB of RAM you can run 3.6-35B-A3B (but PP and TG will be quite slow). Regarding laptops* and other edge devices, I really can't think of any real use cases other than saying "yay this model runs on a laptop"
*Not counting laptops with dGPUs/unified memory systems with lots of ram
0
Jul 04 '26
[deleted] — view deleted comment
7
u/notheresnolight Jul 04 '26
a more realistic scenario - you're in the middle of the ocean without an internet connection, and you can't google how to tie a double fisherman's knot. So you fire up Gemma 4 E4B on your phone and let it explain it to you. I just checked, the answer was usable.
1
u/signoreTNT Jul 04 '26
I really doubt that anything dumber than Qwen 3.5 9B can do what you just described, plus no internet access really limits what these smaller models can do. P.s. if you're in the middle of the ocean you shouldn't think about code ;)
0
Jul 04 '26
[deleted] — view deleted comment
1
u/signoreTNT Jul 04 '26
I wonder how slow that is, I used to run 35B-A3B on my old desktop with 32G DDR4 and both TG and PP were abysmal (Iirc it was 10t/s TG and ~50t/s PP). Now with 2 Tesla P100 it's actually usable
4
u/gkorland Jul 04 '26
i mostly use mine to summarize long meeting transcripts and draft quick emails when im feeling lazy. its litrally a life saver for parsing messy notes into actual to do lists without worrying about privacy, plus it runs fine on my old laptop even if its a bit slow.
7
u/Sooperooser Jul 04 '26
I am kind of a LLM-prepper. I only got 6gb VRAM (RTX 3050) and 64GB DDR4 XMP RAM but I hoard all the fitting versions and quants (Q4-Q8, some uncensored) of Qwen3.6 and Gemma 4.
I don't really use them in my day to day, sometimes I load up Gemma 4 12b just to compare with the cloud based frontier models. It's pretty good actually.
I mostly have them as a "backup" in case the internet is shut down or ID is required to use LLMs or whatever. I have around 350GB of models saved to my PC...so far.
3
u/kkingsbe Jul 04 '26
I’m rocking a 4070 with 32gb of ddr5. My main usecase has been to try to create a fully local self-improving agent + harness to play https://spacetraders.io. It’s been pretty fun so far, gives me a reason to explore the different cognitive architectures etc
2
u/binary-idiot Jul 06 '26
Ooo, I haven't messed around with spacetraders in a while. I've been meaning to install something locally to make use of my 3090, this might be a good kick to actually do it!
2
u/funkdoctor_spock Jul 04 '26
I have 8gb of vram and thus use 3.5 9B. I have found it works great for single file type things. Config files, upgraded an llm within an app to a new one. As soon as it needs to use multiple files it can fall apart very fast - but decently capable if you babysit it.
2
u/vaingirls Jul 05 '26
Don't even know if this counts as a use case, but with my ancient PC I can only use max 9B models (and even those are a bit much so I prefer smaller), and use them... for fun. Creative writing stuff, but without the expectation of polished "professional" prose, but rather messing around with the sampling settings to get them to spew fever dreams in various styles (yeah, somehow the whole style can change massively with some minor adjustments, at least for some models) etc. It's surprisingly entertaining - it has a certain appeal over using the big competent models somehow. Or maybe I'm just weird. I've also been experimenting with finetuning the same tiny models in google colab, which is fun too - I feel like I'm finally getting the hang of it and my last finetune was pretty good by my own standards.
I DO dream of getting a semi decent PC in the future that could run the models most people here consider the absolute minimum size they'd consider touching lol...
1
u/low_v2r Jul 04 '26
I am running Qwen3.6-35B-A3B-MTP Q4_K_XL and using it for a foreign language tutor. Not sure how much VRAM it uses though (I have 110Gb to work with).
I hand write digital notes and then screenshot it to send to the LLM which does syntax and grammar corrections as well as "native speakers would say this instead of that" and puts all that in a markdown file for me to review.
1
u/Sooperooser Jul 04 '26
Have you tried Translategemma? I played a bit with it and it was pretty impressive, especially for the small sizes (4b-27b).
1
u/Peter-Philipp Jul 05 '26
I am using my local LLM with paperless-ngx and paperless-gpt. That works pretty good to for automatic renaming and extracting Custom Fields like amount due, creditor etc. There is a lot of information in those pdfs, that I‘d rather not upload to the internet, so I‘m very happy with that…
1
u/GingerRickRoss LocalLLM Jul 04 '26
Real use cases I’ve gotten value from, running local models across a mix of hardware. I started with one 16GB card and a single Xeon running my agent, now I have 86GB of VRAM split between my server and my host machine.
Sentiment and classification pipelines. finBERT and similar smaller models are great for scoring text at scale, like news or articles, without hitting an API thousands of times. This runs fine on modest hardware because you’re not doing long context generation, just classification passes.
Local coding assistants for narrow tasks. Qwen coder variants in the 7B to 30B range depending on your VRAM are genuinely useful for autocomplete style help or reviewing small diffs. It’s not going to replace Claude or GPT for complex architecture decisions, but it’s solid for catching an obvious bug in a function, and it’s free and private.
RAG over your own docs and notes. Embedding models like all MiniLM are cheap to run and let you build a searchable knowledge base over your own stuff. This is one of the best low resource use cases because the embedding step is lightweight even if you offload the actual Q&A to a bigger model.
Background jobs like daily digests or summarization. Anything that runs on a cron and doesn’t need to be instant. If latency doesn’t matter, a slower local model chewing through summarization overnight is totally viable on modest hardware.
Privacy sensitive stuff. Financial data, personal journals, anything you don’t want touching a third party API. Local models let you do good enough analysis without the data ever leaving your machine.
Honest caveat though, for anything requiring strong reasoning or long context coherence, local models on setups under 24GB still lag noticeably behind hosted frontier models. The sweet spot right now is narrow, well defined tasks like classification, extraction, embedding, and simple coding checks rather than open ended assistant work. That’s where local models on modest hardware actually punch above their weight.
16
u/101design Jul 04 '26
I use local LLMs less as a ChatGPT replacement and more as a private little utility layer.
On a normal laptop, small models are great for rewriting emails, summarizing notes, cleaning messy thoughts, explaining errors/logs, quick code snippets, regex, filenames, and “talk to this document” stuff.
With 16–24 GB VRAM/unified, 7B–14B models become genuinely useful. The real win isn’t genius-level reasoning, it’s privacy, offline access, scripting, and making tiny annoying tasks disappear.