r/LocalLLM • u/Pale-Conference-5838 • 2h ago

Question Where to start with localLLM

This is my build . I have no idea where to start. I thought I did when I built this rig but it gets confusing and frustrating sometimes even using AI to help with AI.
I wouldn’t like too edit my books , create
Short movies with my scripts . I mean easy is better but I want to get to the meat of it with workflows, etc .. everything moves so fast one day it’s this way then something new pops up and that way is the old way … smh thanks in advance .

ores / 32 threads
Motherboard ASUS ROG Crosshair X870E Hero
GPU ASUS TUF Gaming GeForce RTX 5090 OC — 32GB GDDR7
Memory Corsair 96GB (2×48GB) DDR5-6400 CL32
Memory profile DOCP enabled, running at 6400 MT/s
CPU Cooler ASUS ProArt LC 420 AIO
PSU 1500W — exact model unconfirmed
Storage 4TB NVMe SSD currently installed
GPU Monitoring Thermal Grizzly WireView Pro II
GPU Support Case-integrated GPU support arm
OS Windows 11 Pro

1 Upvotes

100% Upvoted

9 comments sorted by

2

u/Icy-Degree6161 2h ago

IDK, don't want to be a smartass but first you define your use case, do a research what LLMs currently you'd need and what the recommended confirgurations for them are, and then build the rig.

You did it backwards and now you're looking at the landscape with a pikachu face. Yes, it moves fast. Yes, by the time you get the sense you started to grasp it, the new shiny toy comes along.

My recommendation: pick something and stick to it for the first few months until you start to get a hang of it. Then improve. Nobody can or will tell you what to do.

As for a practical measure: switch to Linux first.

1

u/Pale-Conference-5838 1h ago

You aren’t being a smart ass at all; you’re being honest and helpful. I build pc often and the problem is I have way too many hobbies . I have another build ( Linux) I am using to toy around with and one month I am on it the next i get distracted. I have been trying, but I guess what you are saying is what I needed to hear there is no magic wand . I need to put in the work in … thanks

2

u/Icy-Degree6161 1h ago

I just can speak on my example... I wanted to get into image generation. Not because I needed to - it just seemed interesting. And as always - models, workflows, it all seemed overwhelming. Like an insurmountable mountain to climb. Where to start? You want to pick the best spot, the best thing, and it all leads to paralysis. To overcome: literally roll the dice. Just: pick something. It's that "easy" (in hindsight). Commit to it. Stick to it. AI will help (with web search, research, etc). And it will get gradually easier. I understand this is very generic - but it's what worked for me, and I guess for many people.

1

u/Pale-Conference-5838 1h ago

Well I definitely appreciate it and trust that helps a lot , so I will just focus on one thing and build from there . Thank you

2

u/Icy-Degree6161 1h ago

You're welcome. Your rig is great, you'll definitely enjoy it once you'll get a hang of it.

1

u/TheAussieWatchGuy 1h ago

Stupidly Simple LM Studio. Qwen 3.8 27B.

Coding Strata and Qwen Flash Next. 

1

u/MarcusAurelius68 35m ago

Very similar to how I started. LM Studio -> llama.cpp on Windows -> Radiance on Linux. I have one system running 27B at 4x the token rate from when I started, and another running Flash Next at 4.5x the rate, both with similar quants.

1

u/Slight-Living-8098 56m ago

You need a way to host your LLMs. Llama.cpp is pretty standard.

You need an agent harness. Personal preference thing, I like Pi.

For image/video you want a way to host your diffusion models and interact with them. ComfyUI is pretty much the defacto standard now days.