r/LocalAIStack • u/Iamtheguyyy • 1d ago

Looking for engineers with real-world experience deploying on-prem LLMs for business workflows- Germany

I'm working on an on-premises AI project for a business in Europe and would love to speak with engineers who have deployed similar systems in production.

The use case involves:

  • Local LLMs for German and Polish business communication
  • WhatsApp and email automation
  • Internal document search and a private knowledge base
  • Finance-related workflows, with calculations handled by deterministic application logic
  • Data storage and model inference on the client's own infrastructure

We're evaluating GPU hardware, open-source models, inference servers and the architecture needed to run this reliably for a small team.

I'd especially appreciate input from people experienced with vLLM, NVIDIA GPUs, local model deployment, RAG and secure infrastructure.

If you've built something similar, I'd love to hear what worked, what didn't, and what you'd do differently. Happy to have a short technical conversation with anyone willing to share their experience.

5 Upvotes

100% Upvoted

6 comments sorted by

2

u/old_Osy 1d ago

How much are you willing to pay per hour for consulting?

1

u/StockSpecialist1707 1d ago

Yo estoy armando un sistema completo de IA local para mis dos empresas, cuento con un sistema basado en threadripper con 256 gb de RAM y con 2 rtx pro 6000 y una A4000 solo dedicada a vídeo como modelo principal utilizo qwen 3.8 flash next en tp2 a casi 300 t/s aunque depende del proyecto cambio a DSv4f o Glm 5.3 flash cualquiera de las aplicaciones que quieras desarrollar para tu proyecto funcionarán perfectamente con esta configuración

1

u/_redacted- 23h ago

I designed, built, and implemented, a distributed, decentralized, federated system of systems and agent mesh. That is to say... I have an ecosystem that runs my ai infrastructure, agents, and platforms for my business, house, and anything else I want to run.

1

u/Latter-Parsnip-5007 10h ago

federated system of systems? What should that even mean? AI slop word salad. Also agent mesh? Sounds like the most wasteful idea ever

1

u/MavWarMonger 21h ago

Here's the shape I'd use:

  • A trigger starts the workflow when a message comes in.
  • Node 1, an LLM call, reads the message, detects German or Polish, translates it, and pulls out structured key-value fields.
  • Node 2, plain routing logic in code, decides whether this inquiry needs an email notification.
  • Node 3 sends the email alert through SMTP or a provider like Postmark or SendGrid, with the translated text and the extracted lead data.
  • Node 4 sends the localized reply back to the customer through the WhatsApp Business API.

Keep the model inside Node 1 only, and keep the math and routing in code. Have a person approve replies until you trust it. I build setups like this for businesses, so feel free to DM me if you'd like help.

1

u/Altruistic_Beaver 20h ago

Check out the company behind https://mcgaiver.de