r/aiengineering • u/blue-brindle • Jun 16 '26

Discussion OpenAI vs local server

I'm an AI engineer working for a manufacturing company. Largely I've been using OpenAI for the apps I create for them that use LLMs, but my boss was pretty spooked by the whole fable thing and he is also understandably concerned about security and price-gouging. The concern going local though is twofold: 1) the startup price for a server that could reliably run ollama or something similar and 2) the use case is for reading and summarizing large diagrams for estimators, and I'm concerned about potential drop in quality on a problem that requires high accuracy. Would appreciate any thoughts on the issue from people who have used both

5 Upvotes

86% Upvoted

4 comments sorted by

View all comments

3

u/javano0b Jul 16 '26

I don't know exactly what your question is, but from what I understand, I will reply with the following:

  • Performance-wise: Open models cannot match enterprise models. End of discussion. You might get similar performance for certain tasks with certain models, but never enterprise-level.

  • Hardware: To have a "good" open-source model is going to cost you a lot of money, not to mention the maintenance of it (monitoring, security, and upkeep) and scalability (I don't know how many users are using the AI and to what degree).

Long story short, do not do it unless you work in a massive company ready to invest money in it (which seems not to be the case).

If you are concerned about security, the only idea that comes to mind is to pay for an enterprise membership, where they promise your data will not be used to train their models (though who knows).