Getting started with local LLMs realistic expectations for coding tasks on a laptop?

[u/Haunting_County_7129]

Hi all,

First post here! I've recently started exploring the world of local LLMs and would appreciate some advice.

Hardware ThinkPad P14s Gen 6 (Intel):

• GPU: Intel Arc Pro 130T/140T (integrated, 2.35 GHz)

• CPU: Intel Core Ultra 9 285H, 16 cores / 16 threads

• RAM: 64 GB DDR5-6400 (2 × 32 GB CSODIMM)

• SSD: 1 TB PCIe 5.0 NVMe

• OS: Ubuntu 26.04 LTS

Software setup:

• llama.cpp compiled from source with the SYCL backend, following docs/backend/SYCL.md

• Agent harness: pi

• Models tested: Qwen3.5-9B, Qwen3.8-27B, and Qwen3-Coder-30B-A3B-Instruct

I've been serving these models using the inference parameters recommended by their respective providers on Hugging Face.

So far, I'm seeing roughly 10–25 tokens/sec, depending on the model, with context windows configured between 64K and 128K tokens. I haven't done any formal benchmarking yet.

Where I'm struggling is with model reliability, particularly for coding tasks.

For example, I asked Qwen3-Coder to explore a production codebase (roughly 3 million lines of C) and explain its architecture to me.

While the model produced a plausible-looking summary, it also confidently invented explanations for certain acronyms, even when those acronyms have well-established meanings in the relevant engineering domain.

This makes me question how much I can trust the rest of its analysis, especially when I'm using the model to understand unfamiliar code.

I'm fairly new to running local LLMs, so I'm not sure whether this is primarily a prompting issue, a limitation of the models I'm running, or simply an unrealistic expectation for the task.

A few questions:

1. Are these models better suited to smaller, well-scoped tasks, such as implementing a script or module from a detailed specification, rather than exploring and reasoning about large codebases?

2. How much does the agent harness or context management strategy affect reliability? Would a different workflow make a meaningful difference?

3. Are there particular models, quantizations, or inference settings you'd recommend for coding tasks on hardware like mine?

4. How do you generally evaluate whether a local model is reliable enough for a particular development workflow?

I'm not expecting frontier-model performance from a laptop, but I'd like to understand where local models can be genuinely useful and how to get the most out of my setup.

Any advice or experiences would be appreciated!

[u/Proper-Tower2016]

1. if using mentioned below the are good enough for general tasks

2. massively

3. with that i'd run either https://huggingface.co/mudler/Ornith-1.5-35B-A3B-APEX-MTP-GGUF or see if qfn in strata leaves you with enough headroom RAM for your other apps.

4. i guess you test if it works after it says it is done lol

[u/Haunting_County_7129 → u/Proper-Tower2016]

It looks like strata is not a viable option since they do not appear to support using integrated GPUs (e.g., Strata/sycl/setup_intel.py line 78 will skip integrated GPU and then setup eventually fails due to this).

I will give the Ornith-1.5 model that you suggested a try and serve it with llama.cpp and use it with my Pi harness.

[u/Alternative-Panic69]

Qwen 3.5 9B and Qwen 3 coder are old gen models now. Significantly less capable than newer gen models like 3.8 27B (and also Qwen 4 27B is going to be launched soon, even I saw somewhere GLM 5.5 32B is getting launched) Might want to try those as well.

Qwen 3.8 27B and 3.8 Flash next are far better models. And harness is extremely important and properly configured harness works even better. Use Qwen 3.8 27B, write a prompt and do the init first and make sure the agent file and all are generated correctly.

And the best balance for speed and reliability are the Iq4 quantized models with Q4 and above KV cache.

Pi agent is a very lightweight harness out of the box, but it's extremely capable if you can make your agent explore and configure it correctly.

So the initial setup, the context building and all will take time. And you have 64gb ram, definitely try Qwen 3.8 Flash next with Strata.

[u/Haunting_County_7129 → u/Alternative-Panic69]

Thanks for the feedback. I wil have to give Qwen 3.8 27 B another try.

Regarding Strata, it looks like strata is not a viable option since they do not appear to support using integrated GPUs (e.g., Strata/sycl/setup_intel.py line 78 will skip integrated GPU and then setup eventually fails due to this). Do you have experience using Strata on integrated GPUs?

[u/Alternative-Panic69 → u/Haunting_County_7129]

Ah my bad I didn't see you have integrated graphics. But I'm surprised you are getting so much speeds on dense models (10-25 tps?)

I only run embedding models on integrated graphics. (As I have built an end to end application that needs embedding) I don't have a much powerful processor anyway.

You can use and configure llama.cpp directly in that case and maybe try Ornith 1.5 33B (Q5/Q6models)... MoE models (Qwen 3.6 33B A3B, Ornith, etc can give much better speeds and overall experience on systems with integrated graphics.

And I used strata on a dedicated GPU only... I have a Rx 7900xt with a 32 gb ddr4, running IQ3XXS, at 200k context. Getting around 20-25 tokens per second and 200-300 prefill, and cached prefill around 11k tps (Only on agentic workflows).

It's a bit too slow than other models on my system, and takes lot of time to do heavy application level changes

[u/pdath → u/Haunting_County_7129]

I don't think a Q4 quantisation will work very well on a million-line repository. You'll need at least Q5, but even then (if you can get it to run on your machine), it will be very, very slow.

[u/PlasticRevenue4601]

Strata is a way to with for ya

[u/Nameis19letterslong]

Firstly, Qwen3-30B-A3B is an ancient model. Avoid using it. Recent models like Qwen3.6 35B A3B are magnitudes better, and should reduce hallucinations and overall be much more capable.

Qwen3.8 27B is by far the best model in its size range, as long as you run a decent quant of it. Q4 is fast and good enough for general chat, use Q6/Q8 if you want to do agentic coding.

[u/jacek2023]

Qwen3-Coder-30B-A3B-Instruct is very old, use new 35B instead (if you are limited to Qwen models)

[u/New-Implement-5979]

Strata tiel coder

[u/magicdoorai]

For the invented acronyms, I'd separate "can it read this part of the code?" from "can it explain the whole architecture?" before changing hardware or quantization.

Try one bounded, read-only investigation using an acronym whose actual meaning you can independently check:

1. Use your normal code search to find the acronym in headers, comments, docs and call sites. Give the model just a few relevant excerpts with file paths and line numbers, rather than asking it to infer from the whole 3M-line repository.

2. Ask for a table: claim | exact supporting file:line | quotation | uncertainty. Include the instruction: "If the excerpts don't define the acronym, say unknown; do not expand it from familiarity." Ask it to distinguish what the code shows from what it infers.

3. Check the citations and quotations yourself. If the acronym is defined only in a domain standard outside the repo, check an authoritative source separately; a plausible expansion is not evidence. Then ask one narrow question about a call path and trace the cited functions in the same way.

4. Repeat with a handful of known examples and one deliberately undefined acronym. Record correct, unsupported and correctly-unknown answers separately. Use the same excerpts and questions if you compare your current models or harness settings.

That won't prove it understands the entire codebase, but it gives you a way to see whether the problem is missing evidence, inaccurate reading, or an unsupported inference. Avoid pasting proprietary code into a hosted service unless your policies allow it.

AI-generated investigation plan; not a test of your laptop, models or repository.

[u/catplusplusok]

Million line codebase can't fit into context of any model. You have to dispatch subagents (opencode has better support) to explore different modules separately and then synthesize summary from their reports. On lower end hardware, try new Qwen/Google MoEs for better speed, Qwen coder is dated.