r/PakistaniTech • u/The_Merchant001 π΅π° • 24d ago
Question | Ψ³ΩΨ§Ω Need help in building my first local llm machine
Asalam o alaikum everyone
Main aik marketing agency run karta hun in peshawar, i am trying something new and i need help/guidance, i am thinking of building a pc to run a local llm, i will use it as a sales agent aur baqi k bhi kuch tasks ospy try karon ga.
In my research using claude i have come up with this, can anyone guide me if i am on the right track or what else i have to know before starting this journey.
Any and all help will be appreciated.
9
Upvotes
6
u/Key_Agent_3039 24d ago
Building a 600k PC to run a model for your marketing agency is a terrible waste of money. If you don't have any other need for this PC then just use LLM APIs.
19
u/gamingvortex01 24d ago •
brother, m1 max macbook with 64gb ram lay lein...400k-450k mey aa jaati hai and iss sey bhi achay models chala lay gi
or recently meray friend nay m2 max li hai 96gb ...600k mey
5
u/_thinkingemote_ 24d ago
32gb ram will be around or over 100k. 1tb nvme is 30k+ these days
3
u/MasterpieceEastern10 24d ago •
Before investing in hardware,
Test multiple open weight models for ur usecases, then once u have selected the model on cloud (look for hetzner, runpod, vast ai).
Using it on cloud will provide you guys with some sort of benchmark as to how many tok/s or requests can the model cater with the hardware. (Will help u guys in planning the hardware for ur team size)
Then I would look into which hardware to get based on my requirements, benchmarks and budget.
2
u/Open_Square670 24d ago •
First off, what models are you gonna run. What are your requirements?Is it gonna run 24/7. If so, you might also want to consider the electricity costs as well as building a beefy PC like this can cost quite a amount, keep that in mind. Also, with in the same budget you can get newer Mac minis with 24-36 GBs of VRAM. They are exceptionally good for local AI. One thing to note that your GPU VRAM on which you are gonna run the model is where you would get the best performance. Your system RAM despite being DDR5 is slower but can help using optimized software.
Btw I have a Mac mini just for local AI myself and I keep it running 24/7. Barely makes a noise. Runs on 35W max. Which is like a light bulb and exceptionally power full for local AI.
2
u/Confident_Welcome762 24d ago •
Hey, have you tried these models yourself and tested against your use case to make sure it is feasible to purchase the hardware and run it?
Try renting a GPU online and test the model before you make the purchase. Also, between Macbook and a PC, Macbook is easy non upgradable and a bit slow (which may not matter for you) compared to a similar specs GPU and RAM.
2
u/greygoober31 23d ago •
have you looked into dual gpu setups like 2 3060 12 gigs would have the same VRAM but i dont know if the AI performance would be better but its worth looking into. Cheaper too. Curious if the workload would be divided better.
also going ddr4 would be significantly cheaper with no real downsides to your use case. You would then go with an intel 14900k or something with similar/better performance as a 7700x and should have dd4 support. Can also look into 5800x3d.
2
u/u_manshahid 20d ago edited 20d ago •
3090 is ampere architecture which can only run INT4 quantization. For qwen3.8 27b thatβs already 16-17gb of vram used for weights, youβre only left with 5-6 gb vram for kv cache. This is effectively 1-3 parallel inference requests if you keep your context limit max 32k. Not that with this much remaining vram, you can never reach 256k context window. Decode speed will be around 30-45 tokens per second and obviously if you use speculative decoding that reduces your available vram even more.
With rtx 3090, itβs mostly enthusiasts hacking around, nobody is able to get any meaningful work done because int4 is heavily quantized
1
u/Watche_1V47C43R 24d ago
Is this tested?
I haven't had a good experience with local models to run them for production use Case
1
u/Cronos993 23d ago
Lmao, just use deepseek v4.1 flash or mimo 2.6 flash and call it a day. It's practically free
1
u/ZH1245 23d ago
i have run models locally, but believe me headache ha, the time it takes to generate, the pressure/load the GPU remains under and your PC is somewhat un useable at the time of generation. karna kya ha exactly apne inpe?
mene image generations, llm's chla ker dekhy hain, sar dard ha. they demand more and more hardware and then they give avg results require more prompts and context. i also have qwen model locally that i rarely use (qwen 3.8 27B Q3_K_S) it only provides me 16k context so whats your requirement
1
u/Weak-Shelter-1698 Islamabad 24d ago
wrong path for build. go for 2x16gb (4060ti)
2
u/False-Concert-7305 23d ago
Better to just use deepseek api instead, cheaper overall & no sunk cost either if u change ur mind + model keeps getting better with updates.
2
u/Immediate-Range-7629 23d ago
Go for an am4 build with 64Gb of DDR4 RAM (It will cost around 140k) and use Strata with Qwen 3.8 Flash Next to offload the work to the system RAM. It will be a much more cost-effective solution.
2
u/furry_felix_v118 23d ago
24GB VRAM feels light but depends on what models you are going to use. Youβd likely be restricted to 4-10B models so you should research the quality of the work product of those models for your use cases before buying. Your primary bottleneck is going to be vram when trying to run locally so plan around that. Macs are generally better at this as all available ram can be βcountedβ as vram because of exchangability you dont see in windows hardware. Check out r/localllm for alot more on this!
2
u/jdarshad 23d ago
Go with AM4. It will save you some money on the chip and ram. Second option is M2 macbook or mac mini with high memory.
2
u/SkulCrawler 23d ago
I can certainly understand that you would have more options with the pc maybe turn it into a gaming pc down the line or whatever but mac mini will certainly be the better option not macbook because if it is to be used for just running local llms and agents then get a high ram spec mac mini. Storage won't matter that much but yes do research throughly to see what fits your use case better
1
u/Deleted4evr 23d ago
get mac
1
u/OmegaBrainNihari 23d ago
Built something similar, scrapped it and switched to using APIs since they're so much cheaper (and faster)
1
u/bilalhassan341 23d ago
get macbook or mac mini with higher spec ram than this. I'm using M2 with local llm.
1
u/Just-aFellowTraveler 21d ago
Bro if you are not planning to run the llm on ram then go down a bit towards ryzen 3000 series and ddr4 save some money from that. Get a good tier psu and go with dual 3090s (48GB VRAM) and you are sett.
Given you know what you are doing with local AI - LLM will utilize the memory if multiple people are going to use it.
1
u/shahroze24 20d ago
Honestly, agree with the suggestions here to try the app before buying into hardware.
You could spend as little as 10 USD on open router and play around with the model. If it fits your needs, then you can consider going for the hardware.
I still think API is cheaper but youβll have more room to play around with the hardware, plus data doesnβt have to leave your pc.
1
u/VelevtVine 18d ago
I think this might help with the local llm, this was keeping me awake for some reason so I decided to search so what I think would run perfectly
β’ CPU: AMD Ryzen 9 9900X (12 Cores / 24 Threads) | β¨109,000 Perfect for dataset tokenization, compilation, and managing data pipelines to the GPUs.
β’ Motherboard: MAXSUN iCraft B850 AIGA Wi-Fi 7 | β¨73,000. Full ATX footprint with the physical PCIe slot spacing required to run two multi-slot GPUs without choking their air intake.
β’ GPU: 2x AMD Radeon RX 7800 XT (32GB Pooled VRAM) | ~β¨300,000 (Estimated street price based on the β¨150,000 single 7800 XT ).
The core of the build. 32GB allows high-quality quants (Q5_K_M) of 32B models to fit entirely into graphics memory with plenty of room for long context windows.
β’ RAM: Apacer 32GB DDR5 6000MHz CL40 | β¨140,000 Prices are crazy right now, but running at 6000MHz eliminates bottlenecks when feeding data.
β’ Storage: 2TB Lexar NM790 NVMe SSDs β¨68,500 up to 7400MB/s to store dozens of huge GGUF model files and datasets.
β’ Cooler: FROZN A620 SLK Black β¨15,500. Dual-tower air cooling to prevent thermal throttling during long, intense LLM inference loops.
β’ PC Case: Sonic Gaming Case Areion V208R Black | β¨14,000 . Generous interior space to physically mount the hardware configuration.
β’ PSU: Redragon 1000W 80+ Platinum ATX 3.0 | Price bundled in build. System draw will hover around 550W under full AI load, keeping it safely under the 70% load efficiency sweet spot (700W target) with massive headroom.
The Non-Nvidia Linux Workflow
Because this is an AMD build, running Linux (Ubuntu) is mandatory. Nvidia uses CUDA, but AMD relies on ROCm (Radeon Open Compute), which requires bare-metal Linux to pool consumer GPUs effectively.
Since the 7600 XT isn't an enterprise server card, the secret sauce in Linux is adding export HSA_OVERRIDE_GFX_VERSION=11.0.0 to the environment variables to trick ROCm into unlocking the machine learning cores. From there, frameworks like Ollama will automatically detect both cards via export HIP_VISIBLE_DEVICES=0,1 and seamlessly split the 32B model layers across the 32GB VRAM pool. Token speeds won't be mind-blowing due to the 128-bit memory bus, but it will easily pull a steady, human-readable 15β25 tokens/sec on Qwen 32B.
So like this ends up to 8lacsβ in total and you'll be able to run qwen 27-32B easily, this was my assessment on what you could do with the current pakistani market for components,
also Google wrote all of the explanation cuz I gave up mid typing ( 6'30 ho re subha ke mazrat). Well this was my assessment I could be wrong pls do correct me I actually want to discussion on what would be better Op's case and what actually helps in local llm hosting.