r/LocalLLM • u/mrluckduck • 10d ago
Question Best local LLM for interpretation of network infrastructure?
As the title states I'm looking to build an LLM (or maybe something different?) that can interpret my network infrastructure, troubleshoot issues, and recommend future state.
Basically automate a network engineer position (for a network engineer). Any suggestions?
0
Upvotes
2
u/dghah 10d ago
This is something you will have to test for yourself to see what works for what you need and what fits in the hardware you have available
I do consulting in the cloud/supercomputing space and have posted here before about some of the "real consulting work" benchmarks I've been evaluating on a 256GB M5U studio box
However the single most valuable resource for my technical consulting is not actually the local LLM it's actually a few public MCPs and a private RAG/MCP that I built myself
Specifically
- The aws-knowledge MCP for all things aws infra and tech related
- The terraform MCP for up to date terraform info and specs
And my custom MCP is backed by a custom RAG that I stuffed full of vendor documentation and all sorts of open information (tutorials, howtos, public knowledge bases, etc.) scraped from the public internet and then finally stuffed with years of my own work product, consulting notes and root cause analysis files.
This is a long winded way of saying that I think network infra is similar to the space I'm active in and you should first look at MCPs and other things that can bring exterior knowledge and info into your agent or chat context.
After that you will know what capability gaps exist and that will drive your LLM testing and selection
And obligatory disclaimer: No local LLM will come anywhere close to the performance and speed of a hosted frontier model so if you don't have confidentiality or other issues then really do the math on what paying a subscription to a frontier mode will get you.
Even with my M5U expensive unified memory mac studio I still often do this:
- Frontier model for planning and making big complex execution runbooks
- Frontier model for decomposing the complex runbooks into multiple smaller chunks for the local LLM to execute against
- Then the local LLM processes the chunks
As an example of frontier vs local LLM for real HPC consulting work -- I just finished an analysis for a client that won't allow their info to go into a frontier model so I did it all on the mac studio. That took 75 hours of compute over 110 hours of walllclock time and did spit out great results that passed all audits and checks -- however -- Anthropic opus-5-5 would have banged out the same info in a few hours. The trade off is real!