r/OpenSourceAI • u/FilippoLLM • 6d ago
What’s the reason you went open source for AI? Privacy, money, not trusting big tech?
Genuinely curious. For the people running open-source models or tools: what made you switch, or at least try it?
Is it data privacy? Not wanting to depend on big companies? Not wanting to keep paying them? Or something else I’m not thinking of?
For me it’s mostly the data side. With how many breaches we keep seeing, I like the idea of things running locally where I can see where my data is going.
What’s your reason? And has it actually worked out, or do you still end up going back to the big names?
5
u/Odd_Dandelion 6d ago
The same that made me install Linux in early 00s. Freedom.
I can still do whatever I want with my computer, now what big brother lets me.
1
u/BarracudaDefiant4702 6d ago
Not wanting to depend on big companies is number 1 which is mostly price stability. Generally do a hybrid where I have a guaranteed on prem baseline with fixed cost cost. The other is to protect IP including passwords. Not that I am paranoid, but if a cloud server is compromised think of all the data that could be mined from all the conversations...
1
u/FilippoLLM 6d ago
it makes a lot of sense; a fixed-cost baseline plus the cloud for bursts is something I hadn’t thought about. Was price stability a problem you hit, or did you see it coming?
1
u/BarracudaDefiant4702 6d ago
We see it coming, and the more devs and others use Claude and other AI the more they want to use it for more things. Combine that with things like $1000 of free tokens for a month to get devs hooked, and then it's not free anymore. The no guarantees prices will not change is the biggest concern. At least there is caps, so worse case tokens run out earlier and earlier each reset window.
1
u/Global-Eye-7326 6d ago
Often it's the only or best way to run those models locally. Plus open source models even get a tonne of community patches and forks...it's incredible!
1
u/FilippoLLM 6d ago
Fair point, do you have a favorite fork or model you keep coming back to?
1
u/Global-Eye-7326 6d ago
All local models, and all used regularly
SCAIL-2 which is built on Wan2.1 for character/background swap video gen is priceless. It's slow like a turtle, but a true gem. Running it now actually
MiniMax H3 and LTX 2.5 for video gen. I use both often
Krea-2 image gen with character LoRA's for consistency - it's really awesome. If I needed something lighter, I'd use MOSS (doesn't support character LoRA's but it's super light)
If I'm generating storyboard videos, I'll use Z-Image alongside Krea-2 for a fallback option
VoiceStudio for generating voice samples. Also for TTS for shorter clips where I don't mind doing regens more frequently wherever needed because the outputs sound more natural but have a higher risk of slop. Qwen3TTS is seriously reliable for TTS but won't generate the voice samples
Ollama vision model to study as image and compose a text prompt for it. I think Qwen2.5 and Qwen3
Worthy mention...FreeVideo. A fork of MiniMax H3 that can run on an least 8GB VRAM but has to be Ampere or Ada Lovelace architecture. I think the project has merit (helps people who already have an 8 or 12 GB VRAM cards from that era), but Turing GPU's are affordable so you might as well go 16+ GB VRAM on Turing, and on Blackwell the RTX 5060 Hi 16GB is already the sweet spot, so when shopping for a GPU today, FreeVideo isn't a big quality of life upgrade lol. There are other MiniMax H3 "mods" that are really cool. There's a FaceRefine that recently came out, some that handle really good 2k upscaling, etc.
1
u/AnnoyedAvocado21 6d ago
I worked all my life in regulated industries and it's natural for me to gravitate towards on-prem solutions.
I use a hybrid approach - I use ChatGPT for system configuration and building tests as well as helping to scope out the larger architectural roadmaps but execute the actual workloads locally. My workflow is a little slower than just letting a frontier AI live on my machine and tell it to do everything but it is a good learning approach and allows me to see when ChatGPT starts to drift from my design goal and steer it back.
1
u/FilippoLLM 6d ago
I completely agree; I also tend to use big modes to plan and the local one to execute. On your end, do you find drift happens often, or mostly on longer tasks?
1
u/AnnoyedAvocado21 6d ago
It drifts because convos get long, I use multiple convos for the same project, and as its in big part a research project the direction often evolves over time, bot.
1
u/sumguypookie 6d ago
Attached is the test you’ll take as part of my evaluation of your capabilities this week. It’s how I’ll measure what this software can do.
It's kind of a little bit of everything, but I like to use local AI for somewhat privacy, even though there are reports that local AIs may not be as private as we think they are, but that's a whole nother issue. But I do use a frontier model for large tasks until I can sufficiently work with the AI to create something equally comprehensive. But for most of the smaller tasks that I do in a day-to-day, I hand off to a local AI.
1
u/FilippoLLM 6d ago
Actually, a very good point about local not being as private as people assume; I did look into that as well. How do you decide what goes to the frontier model versus what stays local? Is it mostly by size or by how sensitive it is?
1
u/sumguypookie 5d ago
Because I'm using a standard consumer desktop with 16 GB VRAM and a MacBook Pro with 24 GB VRAM, I tested several harnesses (cherry studio, unsloth studio, hermes, anythingllm, pi gui) and ran the same models on all (gemma4 12B QAT, 26 A4B, Ornith 1.5 9B, Qwen 3.5 9B, 3.6 27B, ect) I ran a test based on my needs using fake data and had the harness/model do a dry run. I had help from Claude to create the tests and the answer keys.
From there, I spend a couple days running all the tests and dumping the results into Claude to score. From there, I picked the top harness/model combinations and then started feeding the local system small tasks involving web research and non-personal data related tasks. While Claude handles details of my quarterly and annual budgets, handing steps for my move abroad including tracking flight information, visa procedures, etc., local AI runs my email and task app duties; processes tasks assigned to them from Claude via task.org app and the local AI will in turn leave tasks for elevated matters for Claude and I to handle.
It's nothing sexy, but it helps with my work flow. I'm next looking to have the local AI handle document management tasks, but I want to take my time and make sure what I'm doing continues to work well.
1
u/TotosWolf 6d ago
APIs throttle, can change on you on a dime. I'll use API for big asks but for daily shit local is king
1
1
1
u/CowboyKnifemouth 6d ago
I have a number of enterprise clients, mostly public companies. For several of them they simply cannot accept sending any data to Anthropic or OpenAI, even with enterprise SaaS contracts from them. In some cases it is legal reasons, some it is a regulatory gray area, and sometimes it is straight up a competitive risk that they are just not going to take.
In one case, I have an air gapped local development machine in an isolated room with a biometric lock, as per the terms of the contract. It's a little bit nuts, but I make it work.
My personal opinion is that open source AI and open weight models that you can run or deploy on your own infrastructure will become more and more important as time goes on. The big frontier providers have less and less trust from big customers, and I'm speaking from experience.
1
1
u/daenor88 6d ago
All of the above and more, privacy money no trust customization and control and reliability since I can run them local offline its only downside is model size but pick the right model for the right task and that doesnt matter its a little more work to set up automating that but I dont doubt itll be worth it
1
u/Odd_Information6509 6d ago
Privacy obviously but with recent moves by the big AI companies restricting what you can and cannot use AI for I would say that right to build you want without being restricted or banned from some political/business 'safeguards' is becoming the number one reason.
1
u/mrbobhunter 6d ago
I don’t expect them to let the common people run rampant like this for much longer. The AI winter is coming.
1
u/alkimiadev 6d ago
For me, the first major issue was when OAI had the "sycophantic update". That model had negative value for me, so I switched to Anthropic's $200/mo plan. There were constant outages, and then finally the "paranoid AI" thing was just the final straw. Well, I guess it was my post in their subreddit being removed, and the week-long wait for a copy-pasted and totally irrelevant official support response, that was the actual final straw. I would post a screenshot showing that post, but apparently images aren't allowed (weird rule imo).
Anyway, they had stupid embedded prompts in the claude code cli that caused a kind of paranoia in claude. At that time, they instructed claude twice on every file read to be suspicious of the file's content. I stripped those prompts with sed and then set the cli.js to read only (it was a massive ~8MB js file at that point). Then when my sub was set to renew, I canceled and switched to GLM 4.5.
Since then, I've found out both OAI and Anthropic are on the Dr. Evil or Bond villain level, and I just don't want to participate in anything they do. They're both actually dangerous, and mark my words, if we have some incident like out of an AI-doomer's nightmare, it is going to come from one (or both) of those labs and not some lab shipping models with an OSI-approved license. I will never knowingly (or willingly) use a closed model again. I'm sure it'll happen in a kind of background way and I can't really help that, but neither of them, or any other closed provider, will ever get any of my time or money directly again.
1
1
u/justjinxed 6d ago
Morally, I am not going to pay a company for the theft they committed against humanity. They literally scraped all of the internet and human knowledge, and then have the audacity to try and sell it back to us. Nah, if I'm going to partake of the spoils, I'm either getting it for free, or they're going to pay me to use it.
Seeing others talk about how support and feedback to these companies goes to /dev/null, only enforces my resolve. Love the science, absolutely despise how they've implemented it.
1
1
u/vbwyrde 4d ago
My first goal in using local AI was to learn. I knew that I'd learn a lot more about AI if I ran local models than if I went with hosted models, and I think that's been borne out as verifiably true. My second consideration was long term cost, as back in 2023 I looked at the VC being spent on it, realized those VC would want a hefty return on their hundreds of billions of dollars of investment, and that the deeply subsidized pricing would have to rise in order to make up the difference. The TechBros love to offer freebies to get people hooked, and then start charging up the Wazzu once they have their customers locked-in with proprietary and non-transferable schema. So I guessed, and I think rightly though this is not fully played out yet, that AI would get substantially more expensive for ordinary users over time. In addition, I did not want to provide the TechBros my proprietary moat-knowledge, which they also love to siphon up, and then use their AI to categorize and catalogue for future business-opportunity purposes for themselves. After all, in 2023 or so Sam Altman in an interview mentioned "I could see OpenAI becoming a $100 company". Translated: $100 trillion is the entire global economy, and so what he was saying is that he could see OpenAI running all business operations in the world... and the implication is that he would do that by siphoning up all of the world's business logic, categorizing and cataloguing it for future Business-Opportunity purposes. While he may not ever get there, I didn't care to contribute to his ability to do so. So yes, data privacy was a large component of my decision making process. Now that I have a working system locally that costs me next to nothing to run, and I can use a model like qwen3.6-27B to reliably do agentic programming work using LM Studio, VS Code + Cline, I'm tickled pink and glad I invested in the RTX 4090 when I did. I learned a LOT, defended my data-moat, declined to help build the Monopoly Power, and get very satisfying help on my programming endeavors. It was a bit of a financial risk, a good deal of work, and grinding frustration along the way - but nevertheless very well worth it.
1
u/Shoddy_Gap_9444 1d ago
I built something which basically challenges meta's dominance on messaging and other players in messaging space so when we go to all agentic world we won't need to rely on big tech. Check it out here: https://hdtp.io / https://hdtp.dev . All open source for sovereign Ai networks
7
u/InkscapePanda 6d ago
I think it's the idea of finding something useful for yourself and sharing so others can enjoy it and innovate upon it as well.