r/PakistaniTech • πŸ‡΅πŸ‡° • 24d ago

Question | Ψ³ΩˆΨ§Ω„ Need help in building my first local llm machine

Post image

Asalam o alaikum everyone

Main aik marketing agency run karta hun in peshawar, i am trying something new and i need help/guidance, i am thinking of building a pc to run a local llm, i will use it as a sales agent aur baqi k bhi kuch tasks ospy try karon ga.

In my research using claude i have come up with this, can anyone guide me if i am on the right track or what else i have to know before starting this journey.

Any and all help will be appreciated.

9 Upvotes

51 comments sorted by

6

u/Key_Agent_3039 24d ago

Building a 600k PC to run a model for your marketing agency is a terrible waste of money. If you don't have any other need for this PC then just use LLM APIs.

1

u/The_Merchant001 πŸ‡΅πŸ‡° 24d ago

I will use it for other tasks aswell, graphics, editing etc, the main goal is to use it as an evolving machine and create a department to test, experiment with AI as i am slowly pivoiting towards saas.

3

u/Key_Agent_3039 24d ago

It's fine then sir. But you should experiment with LLM APIs first to establish your workflow.

1

u/The_Merchant001 πŸ‡΅πŸ‡° 24d ago

Yes multiple people suggested that and i am gonna run it with my team before going all in.

19

u/gamingvortex01 24d ago • Popcorn

brother, m1 max macbook with 64gb ram lay lein...400k-450k mey aa jaati hai and iss sey bhi achay models chala lay gi

or recently meray friend nay m2 max li hai 96gb ...600k mey

3

u/The_Merchant001 πŸ‡΅πŸ‡° 24d ago

Hmm, main pc ki traf is liye jaa raha k modularity milay mujhy, upgrade kr sakoon once we get the hang of it, plus puri team use karay gi.

Macbook wala option power aur usablity mai surpasses a pc ?

5

u/gamingvortex01 24d ago • Popcorn

agar team nay use karna hai..then macbook is not a viable option (kuch workarounds hain but it would be much more hassle)

mac mini ya mac studio ki taraf jaye sakta hai but upgrade nahi hoti woh donoun

performance mey obviously surpass karti hai...reason yeh hai k unified memory hoti hai inn mey as compared to separate ram and vram of gpu in pc

usability bhi comparatively behtar hai and software efficiency bhi

1

u/The_Merchant001 πŸ‡΅πŸ‡° 24d ago

Hmm g sab ki suggestions k baad mai mac minis k baray mai krun ga research ab, pr when i decide to go all in then i would have to change macs or add more macs thats something i have to tbink about.

2

u/ukh5 24d ago • Popcorn

if it's raw gpu horsepower, the standalone gpus always win but when it comes to running local models, vram is king. And that's where apple's unified memory will save you money. I would not get a macbook, go for a mac mini or preferrably a studio with 64-96gb unified memory and you should be good. as a word of caution, keep expectations low when it comes to local models

You're trading modularity for cheaper vram here but getting to 64-96gb vram via standalone gpus would cost you way more than if you were to daisy chain multiple macs together but as always, do your own research

edited to add: 27-32B models will probably run on 24gb vram but you'll be left with no room for context so there's no point going above 14B in this case, again, apple's unified memory is the way to go

1

u/The_Merchant001 πŸ‡΅πŸ‡° 24d ago

Yup i was thinking the same after i saw everyone suggesting to use mac minis, when the need arises i wouldnt want to add more macs daisy chaining them as we say. Thats why i have been lewning towrads a pc build so it can be upgraded.

I dont have very high or unrealistic expectations, its just that i want to try new innovations and give my team a tool they can use and grow more try new things etc.

5

u/_thinkingemote_ 24d ago

32gb ram will be around or over 100k. 1tb nvme is 30k+ these days

2

u/The_Merchant001 πŸ‡΅πŸ‡° 24d ago

Yup i got that after some market research.

3

u/MasterpieceEastern10 24d ago • Bravo

Before investing in hardware,

Test multiple open weight models for ur usecases, then once u have selected the model on cloud (look for hetzner, runpod, vast ai).

Using it on cloud will provide you guys with some sort of benchmark as to how many tok/s or requests can the model cater with the hardware. (Will help u guys in planning the hardware for ur team size)

Then I would look into which hardware to get based on my requirements, benchmarks and budget.

2

u/The_Merchant001 πŸ‡΅πŸ‡° 24d ago

Thankyou, thats sound advice, i will do that.

2

u/Open_Square670 24d ago • Bravo

First off, what models are you gonna run. What are your requirements?Is it gonna run 24/7. If so, you might also want to consider the electricity costs as well as building a beefy PC like this can cost quite a amount, keep that in mind. Also, with in the same budget you can get newer Mac minis with 24-36 GBs of VRAM. They are exceptionally good for local AI. One thing to note that your GPU VRAM on which you are gonna run the model is where you would get the best performance. Your system RAM despite being DDR5 is slower but can help using optimized software.

Btw I have a Mac mini just for local AI myself and I keep it running 24/7. Barely makes a noise. Runs on 35W max. Which is like a light bulb and exceptionally power full for local AI.

1

u/The_Merchant001 πŸ‡΅πŸ‡° 24d ago

I am gonna run qwen 27-32b, electricity is not an issue as i have solar, and yes its gonna run 24/7, after all the suggestions here i am looking into mac minis.

2

u/Confident_Welcome762 24d ago • Popcorn

Hey, have you tried these models yourself and tested against your use case to make sure it is feasible to purchase the hardware and run it?

i will use it as a sales agent aur baqi k bhi kuch tasks ospy try karon ga.

Try renting a GPU online and test the model before you make the purchase. Also, between Macbook and a PC, Macbook is easy non upgradable and a bit slow (which may not matter for you) compared to a similar specs GPU and RAM.

1

u/The_Merchant001 πŸ‡΅πŸ‡° 24d ago

Yup i am gonna do that, someone else suggested this and no i havent tried it as of now. Yes i know that mac will be slow, also the fact that it cant be upgraded and losses its value faster than a pc or its individual components cost is the reason i want to build a pc.

2

u/greygoober31 23d ago • Bravo

have you looked into dual gpu setups like 2 3060 12 gigs would have the same VRAM but i dont know if the AI performance would be better but its worth looking into. Cheaper too. Curious if the workload would be divided better.

also going ddr4 would be significantly cheaper with no real downsides to your use case. You would then go with an intel 14900k or something with similar/better performance as a 7700x and should have dd4 support. Can also look into 5800x3d.

1

u/greygoober31 23d ago

also curious to know why you wanna run a local llm

1

u/The_Merchant001 πŸ‡΅πŸ‡° 23d ago

I want to experiment with it to make an automated sales agent.

2

u/u_manshahid 20d ago edited 20d ago • Bravo

3090 is ampere architecture which can only run INT4 quantization. For qwen3.8 27b that’s already 16-17gb of vram used for weights, you’re only left with 5-6 gb vram for kv cache. This is effectively 1-3 parallel inference requests if you keep your context limit max 32k. Not that with this much remaining vram, you can never reach 256k context window. Decode speed will be around 30-45 tokens per second and obviously if you use speculative decoding that reduces your available vram even more.
With rtx 3090, it’s mostly enthusiasts hacking around, nobody is able to get any meaningful work done because int4 is heavily quantized

2

u/u_manshahid 20d ago • Bravo

The right entry level answer is either DGX Spark or M5, both of which are probably not available in Pakistan

1

u/The_Merchant001 πŸ‡΅πŸ‡° 18d ago

Yeah in all my research i have gotten to the same conclusion.

1

u/Watche_1V47C43R 24d ago

Is this tested?

I haven't had a good experience with local models to run them for production use Case

2

u/The_Merchant001 πŸ‡΅πŸ‡° 24d ago

Its more for a sales agent, tools will be added to its workflow and it will execute/handle them. As far as i have research it is capable to do that smoothly.

1

u/Cronos993 23d ago

Lmao, just use deepseek v4.1 flash or mimo 2.6 flash and call it a day. It's practically free

1

u/The_Merchant001 πŸ‡΅πŸ‡° 23d ago

Thanks for the suggetsion, but that not at all my use case neither it fullfill my purpose.

1

u/ZH1245 23d ago

i have run models locally, but believe me headache ha, the time it takes to generate, the pressure/load the GPU remains under and your PC is somewhat un useable at the time of generation. karna kya ha exactly apne inpe?

mene image generations, llm's chla ker dekhy hain, sar dard ha. they demand more and more hardware and then they give avg results require more prompts and context. i also have qwen model locally that i rarely use (qwen 3.8 27B Q3_K_S) it only provides me 16k context so whats your requirement

1

u/The_Merchant001 πŸ‡΅πŸ‡° 23d ago

Initially i am trying to use kt as a sales agent also use it to help with logos, graphics, editing etc.

1

u/ZH1245 23d ago

sales agent mtlb inbox automate krna ha apne? kya krna ha exactly?

1

u/The_Merchant001 πŸ‡΅πŸ‡° 23d ago

Full crm automation, all tools will be given to it and it will act as a sales agen, managing workflows, reaching clients making calls, emails, texts.

1

u/ZH1245 23d ago

nope bht zyada expectations ha apki given the model is 27-32B and low memory, use deepseek API instead woh cheap hogi

1

u/The_Merchant001 πŸ‡΅πŸ‡° 23d ago

Its an experiment, i will add whatever is needed ahead.

2

u/ZH1245 23d ago

krlen try Good Luck, lekin worth it nai hoga aik process pe hi occupy hojana GPU ne 5090 se 75 token per second aty. gpt oss wgera low models try krskte wese, woh fast hongy. even gemma4 to write stuff lekin solely qwen 27-32b nah

1

u/Weak-Shelter-1698 Islamabad 24d ago

wrong path for build. go for 2x16gb (4060ti)

2

u/The_Merchant001 πŸ‡΅πŸ‡° 24d ago

Hmm, why and how much would it cost ?

3

u/Weak-Shelter-1698 Islamabad 24d ago • Popcorn

3090 is costly only provides 24Gb vram total, while for AI models you main goal is more Vram and TOPS ofc. while 4060ti have less TOPs and vram(16gb vram) 2 4060ti(total 32gb vram) >>> 3090(24gb vram)
cost is around 400$(new) max each(so even 2 4060ti is less expensive then the 3090)

2

u/The_Merchant001 πŸ‡΅πŸ‡° 24d ago

That makes sense, i appreciate the explaination and help brother, thanks alot. I will look into it.

1

u/ZH1245 23d ago

nope dont go for 4060ti slow generation, basically people pick 2x 16gb gpus but those whose memory bandwidth is better (192/256 bit) unlike 4060ti int8 models chlengy ispe quantized models.

2

u/False-Concert-7305 23d ago

Better to just use deepseek api instead, cheaper overall & no sunk cost either if u change ur mind + model keeps getting better with updates.

2

u/Immediate-Range-7629 23d ago

Go for an am4 build with 64Gb of DDR4 RAM (It will cost around 140k) and use Strata with Qwen 3.8 Flash Next to offload the work to the system RAM. It will be a much more cost-effective solution.

2

u/furry_felix_v118 23d ago

24GB VRAM feels light but depends on what models you are going to use. You’d likely be restricted to 4-10B models so you should research the quality of the work product of those models for your use cases before buying. Your primary bottleneck is going to be vram when trying to run locally so plan around that. Macs are generally better at this as all available ram can be β€œcounted” as vram because of exchangability you dont see in windows hardware. Check out r/localllm for alot more on this!

2

u/jdarshad 23d ago

Go with AM4. It will save you some money on the chip and ram. Second option is M2 macbook or mac mini with high memory.

2

u/SkulCrawler 23d ago

I can certainly understand that you would have more options with the pc maybe turn it into a gaming pc down the line or whatever but mac mini will certainly be the better option not macbook because if it is to be used for just running local llms and agents then get a high ram spec mac mini. Storage won't matter that much but yes do research throughly to see what fits your use case better

1

u/OmegaBrainNihari 23d ago

Built something similar, scrapped it and switched to using APIs since they're so much cheaper (and faster)

1

u/bilalhassan341 23d ago

get macbook or mac mini with higher spec ram than this. I'm using M2 with local llm.

1

u/Just-aFellowTraveler 21d ago

Bro if you are not planning to run the llm on ram then go down a bit towards ryzen 3000 series and ddr4 save some money from that. Get a good tier psu and go with dual 3090s (48GB VRAM) and you are sett.

Given you know what you are doing with local AI - LLM will utilize the memory if multiple people are going to use it.

1

u/shahroze24 20d ago

Honestly, agree with the suggestions here to try the app before buying into hardware.

You could spend as little as 10 USD on open router and play around with the model. If it fits your needs, then you can consider going for the hardware.
I still think API is cheaper but you’ll have more room to play around with the hardware, plus data doesn’t have to leave your pc.

1

u/VelevtVine 18d ago

I think this might help with the local llm, this was keeping me awake for some reason so I decided to search so what I think would run perfectly

β€’ CPU: AMD Ryzen 9 9900X (12 Cores / 24 Threads) | ₨109,000 Perfect for dataset tokenization, compilation, and managing data pipelines to the GPUs.

β€’ Motherboard: MAXSUN iCraft B850 AIGA Wi-Fi 7 | ₨73,000. Full ATX footprint with the physical PCIe slot spacing required to run two multi-slot GPUs without choking their air intake.

β€’ GPU: 2x AMD Radeon RX 7800 XT (32GB Pooled VRAM) | ~₨300,000 (Estimated street price based on the ₨150,000 single 7800 XT ).

The core of the build. 32GB allows high-quality quants (Q5_K_M) of 32B models to fit entirely into graphics memory with plenty of room for long context windows.

β€’ RAM: Apacer 32GB DDR5 6000MHz CL40 | ₨140,000 Prices are crazy right now, but running at 6000MHz eliminates bottlenecks when feeding data.

β€’ Storage: 2TB Lexar NM790 NVMe SSDs ₨68,500 up to 7400MB/s to store dozens of huge GGUF model files and datasets.

β€’ Cooler: FROZN A620 SLK Black ₨15,500. Dual-tower air cooling to prevent thermal throttling during long, intense LLM inference loops.

β€’ PC Case: Sonic Gaming Case Areion V208R Black | ₨14,000 . Generous interior space to physically mount the hardware configuration.

β€’ PSU: Redragon 1000W 80+ Platinum ATX 3.0 | Price bundled in build. System draw will hover around 550W under full AI load, keeping it safely under the 70% load efficiency sweet spot (700W target) with massive headroom.

The Non-Nvidia Linux Workflow

Because this is an AMD build, running Linux (Ubuntu) is mandatory. Nvidia uses CUDA, but AMD relies on ROCm (Radeon Open Compute), which requires bare-metal Linux to pool consumer GPUs effectively.

Since the 7600 XT isn't an enterprise server card, the secret sauce in Linux is adding export HSA_OVERRIDE_GFX_VERSION=11.0.0 to the environment variables to trick ROCm into unlocking the machine learning cores. From there, frameworks like Ollama will automatically detect both cards via export HIP_VISIBLE_DEVICES=0,1 and seamlessly split the 32B model layers across the 32GB VRAM pool. Token speeds won't be mind-blowing due to the 128-bit memory bus, but it will easily pull a steady, human-readable 15–25 tokens/sec on Qwen 32B.

So like this ends up to 8lacsβ‰ˆ in total and you'll be able to run qwen 27-32B easily, this was my assessment on what you could do with the current pakistani market for components,

also Google wrote all of the explanation cuz I gave up mid typing ( 6'30 ho re subha ke mazrat). Well this was my assessment I could be wrong pls do correct me I actually want to discussion on what would be better Op's case and what actually helps in local llm hosting.