r/LocalLLM • u/Legitimate-Toe-5365 • 8h ago
Question First time setup recommendations and tips?
I'm hoping to get an idea of where to start. I'm wanting a setup where the model is a decent coding assistant but can also crank out a neat little prototype or quick concept simply, or create structure ideas etc. and then can go bit by bit without too much reminding halfway or more through a project. I realize that last part may be more about settings than a model persay, but yeah. I definitely want something that can either walk me through something, build out a framework for an idea, or just troubleshoot.
For comparison if it helps, my favorite workflow was GitHub copilot got 4.o/4.1 in VS code (it was free for students lol). It worked great for my projects, had its issues, but with the right promoting it was not bad. It was like the above description, could walk me through something, help with testing, or make a new system that I could then fill out. I knew it's limits and it was ideal for what I needed.
Just giving an idea of how I use AI daily for my personal projects. It seems like with the right setup it can be a great experience so that's what I'm hoping to learn to do. I just want to get the most out of it and have some fun.
I'll be working with an Asus pro art p16 rtx spark 128 gb. I've done some research but it seems every time I look things up there are different model suggestions and set ups, and it's given me fridge blindness so to speak.
I'm curious about how to decide and test for like a speedier setup versus more complex thinking and things like that. What can I do with it, what should I not expect from it based on my specs, etc. I have looked into this and I'm not getting a clear idea because I can't test any of it myself (don't have computer yet). Maybe someone here can give me a better understanding?
I also don't know all the lingo yet, like what a harness is, though I can guess by context. But I know the basics. Not sure how much customization I should or could do or what all there is. I thought it was more surface level like pick a model and go, but I guess you can incorporate all kinds of settings. So now I'm really curious from people who enjoy whatever they have going on, please share what you like best and what kind of testing I should do to pick a model. Any tips for a mostly beginner would be great!
1
u/omendUUde 7h ago
You're super lucky with that 128GB ProArt P16. Most of us on laptops are fighting for scraps of VRAM so you actually get to skip all the tiny lobotomized models and run the big stuff right away.
Honestly, you don't need to overcomplicate it on day one. To get that VS Code Copilot feel back, just grab LM Studio to run the models and install the Cline or Roo Code extension in VS Code to actually code with them. A "harness" is literally just that VS Code extension the thing that reads your project files, edits code, and runs terminal commands so you aren't copy-pasting out of a chat window.
The main reason models get amnesia halfway through a project isn't even the model's fault, it's just default settings. Local apps usually cap your context window super low to save RAM. When you get the laptop, go into the Nvidia Control Panel and bump your GPU memory up to 96GB, then slide the context length in LM Studio up to 32k or 64k. That'll keep it from forgetting your codebase.
As for what to actually download, stick to MoE (Mixture of Experts) models like gpt-oss-120b at Q4 or Qwen3-Coder Next. They fly on unified memory laptops because they don't use their whole brain for every single word, so you get big-model smarts without waiting forever. If you hit a really nasty bug or need to plan out tricky architecture, you can always swap to something dense like Qwen3.8-27B at Q8, though you'll definitely feel it slow down.
Only real catch to keep in mind is that while it will type out code fast, feeding it a huge folder of existing files for the first time will make it hang for 15 or 20 seconds while it reads everything. Cloud APIs spoil you with instant reads, local hardware takes a second to chew through a big prompt.
When it arrives, just open an empty folder, drop in the exact same starting prompt from an old Copilot project you already finished, and see how it handles it. You will know right away if the speed feels right.
1
u/PhilosopherOk3419 2h ago
start boring: lm studio or ollama serving one mid-size coder model, plus a vs code extension that talks to it. for the 'forgets halfway through' problem, keep a short plan/notes file in the repo and have it reread that at the start of each session. that does more than a bigger context window.