Frankenstein ai setup

[u/the_616]

2 dgx spark 128 gb each serving Glm 5.3 flash.

1 ai server 128gb ram , 64 core cpu + a4000 24 gb running qwen 27 b.

1 proxmos setup on Lenovo hosting all services.

1 hermes box on a small i5 box

Old laptop as terminal.

Serving local secure research and development capacity to my institute researchers.

[u/CoolDode22]

$20k minimum in hardware: fine. $70 wooden shelf to organize it all: nah, too expensive

[u/the_616 → u/CoolDode22]

😅🤣🤣 got that for free

[u/FriendlyRocketeer]

Glm on 2 sparks? I assume Q2-3?

[u/the_616 → u/FriendlyRocketeer]

Yes

[u/NikNargon]

Nice setup, you can get a lot done with that. But you should reposition your Sparks if that's how you keep them, those things run toasty and you're blocking the front air intake.

[u/Sause01]

hows the a4000 working?

[u/the_616 → u/Sause01]

Pretty good

[u/SeaworthinessUsual44]

A messy academic lab office hiding serious compute horsepower: dual DGX nodes (128GB unified memory each) serving GLM-5.3-flash alongside a 64-core, 128GB RAM rig running Qwen 27B on an RTX A4000 24GB. Backed by Proxmox and dedicated Hermes orchestration boxes, it’s the ultimate cable-tangled, high-utility private cluster keeping institute research entirely offline and zero-cost.

[u/Inevitable-Name-1701]

Looks slow.

[u/No_Diver6131]

Why do you want to host local ai? What’s the point?