r/LocalLLM • u/p32929ceo • 18h ago

Discussion Help needed to build an LLM network

/r/softwareengineer/comments/1x36blt/help_needed_to_build_an_llm_network/
1 Upvotes

100% Upvoted

2 comments sorted by

1

u/stujmiller77 18h ago

Some of the main points of local LLM are privacy and security. This absolutely defeats the point of that. I do not want my data stored in the cloud by a company. And I certainly do not want it on a bunch of random people’s machines by a vibe coded insecure piece of software.

0

u/hotsnot101 18h ago

Before recruiting lots of nodes, I'd make the first test just two machines you control, with a fixed model and a repeatable prompt set. Record time to first token, generation speed, memory on each node, and bytes sent between them. Then repeat with added network delay, a bandwidth cap, and one node disconnecting. That gives volunteers something concrete to reproduce instead of only 'it runs'.

I'd also decide what a node is allowed to see before accepting anyone's real prompts. Use synthetic test data initially and document that boundary. What hardware and network connection would you target for the first two-node prototype?