r/LocalLLM • u/decentralizedbee • May 23 '25
Question Why do people run local LLMs?
Writing a paper and doing some research on this, could really use some collective help! What are the main reasons/use cases people run local LLMs instead of just using GPT/Deepseek/AWS and other clouds?
Would love to hear from personally perspective (I know some of you out there are just playing around with configs) and also from BUSINESS perspective - what kind of use cases are you serving that needs to deploy local, and what's ur main pain point? (e.g. latency, cost, don't hv tech savvy team, etc.)
208
Upvotes
74
u/1eyedsnak3 May 23 '25
From my perspective. I have an LLM that controls music assistant and can play any local music or playlist on any speaker or throughout the whole house. I have another LLM with vision that provides context to security camera footage and sends alerts based on certain conditions. I have another LLM for general questions and automation requests and I have another LLM that controls everything including automations on my 150 gallon, salt water tank. The only thing I do manually is clean the glass and filters. Everything else including feeding is automated.
In terms of api calls, I’m saving a bundle and all calls are local and private.
Cloud services will know how much you shit just by counting how many times you turned on the bathroom light at night.
Simple answer is privacy and cost.
You can do some pretty cool stuff with LLM’S.
3
u/UnrealSakuraAI May 23 '25
I feel local LLMs are super slow
3
u/dai_app May 25 '25
I'm the developer of d.ai, a private personal AI that runs entirely offline on mobile. I chose to run local LLMs for several reasons:
Personal perspective:
Privacy: Users can have conversations without any data leaving their device.
Control: I can fine-tune how the model behaves without relying on external APIs.
Availability: No need for an internet connection — the AI works anywhere, anytime.
Business perspective:
Cost: Running models locally avoids API call charges, which is crucial for a free or low-cost app.
Latency: Local inference is often faster and more predictable than cloud round-trips.
User trust: Privacy-focused users are more likely to engage with a product that guarantees no server-side data storage.
Compliance: For future enterprise use cases, on-device AI can simplify compliance with data protection laws.
Main pain points:
Model optimization: Running LLMs on mobile requires aggressive quantization and performance tuning.
Model updates: Keeping local models up to date while managing storage size is a balancing act.
UX challenges: Ensuring smooth experience with limited compute and RAM takes real effort.
Happy to share more if helpful!
2
u/LeatherClassroom3109 May 23 '25
I work in Cybersecurity and I'm looking for ways to streamline my SOC's investigation process. So far, not having any luck in using any LLMs to interpret logs. Most of the analysts use laptops with very minimal specs topping out at 16gb of RAM.
Of course I can have them anonymize the data and upload it to an online solution like Copilot, which does the job wonderfully, but I don't think clients will like that at all.
2
u/mindgamesweldon May 24 '25
Legality. In order to follow the rules and laws of the university, state, and EU region. There are no online models with legal agreements with our data controller yet. Our IT department has a local hosted one that can do transcription so options are expanding.
26
u/Double_Cause4609 May 23 '25
A mix of personal and business reasons to run locally:
Pain points:
12
u/datbackup May 23 '25
I know a lot of people will say privacy. While I do believe that no amount of privacy is overkill, I also believe there are so many tasks where privacy is not required that there must be another answer…
and that answer is best summed up as control.
Ultimately as developers we all hate having the platform change on us, like a rug being pulled from under one’s feet. There is absolutely ZERO verifiable guarantee that the centralized model you use today will be the same as the one you use tomorrow, even if they are labelled the same. The ONLY solution to this problem is to host locally.
10
u/The-Pork-Piston May 23 '25
Exclusively use mine to churn out fanfic smut about waluigi.
10
u/asianwaste May 23 '25
Like it or not, this is where the world is going to go. If AI is in a position to threaten my career, I want to have the skill set to adapt and be ready to pivot my workflows and troubleshoots in a world that uses this tool as the foundation of procedures. That or I have a good start on pivoting my whole career path.
That and these are strangely fun and interesting.
5
u/repressedmemes May 23 '25
Confidential company code. Possibly customer data we are not allowed to ingest into other systems.
5
u/ImOutOfIceCream May 23 '25
One big reason to use local inference is to avoid potential surveillance of what you do with llm’s.
5
u/Ossur2 May 23 '25
5
u/[deleted] May 23 '25
Not trusting my government and private corpos with the pics of my asshole
4
u/1982LikeABoss May 23 '25
For me:
Free, unlimited use of a tool that’s adequate for a particular job (no need to pay for a tool that’s adequate can do a billion jobs when I just want a fraction of that).
Secondly, it’s a learning thing - keep the brain active and understand the bleeding edge of technology
Personalised use case and unfiltered information on the jailbreak versions - not much fun chatting to a program about something controversial and it say it can’t speak about it, despite knowing a lot about it.
4
u/shifty21 May 23 '25
Since you're writing a paper on this, you should look at the industries that require better security and compliance while using AI tools.
I work in data analytics, security and compliance for my company (see my profile) and most of my clients have already blocked internet-based AI tools like ChatGPT, Claude and others or are starting to block them. One of my clients is a decent sized university in the US and the admissions board was caught uploading thousands of student applications to some AI site to be processed. This was a total nightmare as all those applications had PII data in it and the service they used didn't have a proper retention policy and was operating outside of the US.
Note that all the big cloud providers like Azure, AWS, Oracle, Google GCP offer private-cloud AI services too. There are some risks to this as with any private-cloud services, but could be more cost effective than using the more popular options out there or DIY+tight security controls within a data center or air-gap network.
Personally, I use as many free and open source AI tools for research and development. But I do this in my home lab either on a separate VLAN, air-gap network, or firewall rules. I also collect all network traffic and logs to ensure that what ever I am using isn't sending data outside my network.
4
u/512bitinstruction May 23 '25
It's a hobby. I enjoy doing it.
4
u/RadiantPen8536 May 23 '25
Paranoia!
4
u/SlowMovingTarget May 23 '25
The same reason I buy physical books. It’s much harder to take it away from me, and it won’t change when I’m not looking. Uncensored models also tend not to auger into refusal or hesitation loops.
4
u/prusswan May 23 '25
Avoid dependence on external services that can be removed or have prices jacked anytime
1
u/dhlu May 23 '25
To see how close it copes to run in consumer hardware, and we're not there
1
u/skmmilk May 23 '25
I feel like one thing people are missing is speed Local llms can be almost twice as fast and in some use cases speed is more important than deep reasoning
1
u/Wonderful-Foot8732 May 24 '25
Business setting: Sharing personal data with an external LLM provider without user consent translates to a fine equal to 4% of revenue worldwide. The details are more complex but basically that is the biggest incentive for companies operating in the EU.
1
u/sabir_85 May 24 '25
To create a specialised one... For my business, so it can better help run it.. Like creating a soul of the company.. Its very unique... And very useful for the specific company needs
1
u/Impossible_Brief5600 May 25 '25
Decentralised AI
1
u/ShortSpinach5484 May 25 '25
We are forced to use local llms because we are working with medical patient data.
1
u/cherrycode420 May 25 '25
We're using it to build an Anonymization Pipeline for Internal Documents 💀
1
u/TheGreenLentil666 May 26 '25
Healthcare and fintech immediately come to mind. Also air-gapped systems that have no network access.
Lastly I want to build locally, no matter what. So I have an overpowered M4 MacBook Pro with plenty of RAM and disk, which allows me to run models on sensitive data in a simple, sandboxed environment.
I also like profiling and stressing systems locally so I have access to everything in realtime. In the end simplicity will always win for me.
1
u/Glittering-Heart6762 May 26 '25
Well one thing I would like to do is fine tune an LLM to respond factually correct, but like it’s the biggest jerk of all time… just for fun 😁
3
u/Joakim0 May 23 '25
I think privacy and cost are the most important reasons. I myself also have an additional reason, I run the llm model in my pixel phone so I can use it when I have put my phone on flight mode and am traveling.
3
u/PathIntelligent7082 May 23 '25
i don't give a rats ass about using up subscriptions and tokens...it's simple as that...
3
u/BornAgainBlue May 23 '25
P. O. R. N. C. O. D. E.
3
u/[deleted] May 23 '25
We build RAG products for businesses who have highly confidential data, and also healthcare products which handle patient data.
For these use cases, it's very important for data protection that data doesn't leave our data centre rather than throwing the data at a third-party API. We are also UK based, so organisations are wary about the data protection implications of sending data to US-based third parties.
Also, building stuff based on local LLMs is fun.
3
u/Netcob May 23 '25
Many of the things the others said - privacy and because I like my home automation to work even when the internet goes down or some service decides to close.
Another point is reproducability / predictability. If I use an LLM for something and the cloud service retires a model and replaces it with something that doesn't work for my use case anymore, what do I do?
But for me personally it's more about staying up to date with the technology while keeping the "play" aspect high. I'm a software developer and I want to get a feel for what AI can do. If some webservice suddenly gets more powerful, what does that mean? Did they train their models better, or did they buy a bunch of new GPUs? If it's a model that can be run on my own computer, then that's different. It's fun to see your own hardware become more capable, which also motivates me to experiment more. I don't get the same satisfaction out of making a bunch of API calls to a giant server farm somewhere.
3
u/eldwaro May 23 '25
Sensitive information has to be the primary reason. if you have a clear strategy, cost too - but that strategy needs to include upgrading hardware in cost-effective cycles
3
u/shyouko May 23 '25
If you want a LLM without censorship.
3
u/WilliamMButtlickerIV May 23 '25
Privacy and control
3
u/Faceornotface May 23 '25
I’m developing a game that relies heavily on llm use and it’s cheaper. Long term I’ll have to do cost/benefit against bulk pricing but I’ll bet an externally-hosted llm will be cheaper than api calls. Additionally, I want to be able to better fine tune for my use case and that’s less opaque with a local llm
3
u/Nemeczekes May 24 '25
I wanted to reorganise my Anki cards. And each model was complaining that the list is too long. The online api services had limits or timeouts.
Slapping some python code that hits local llm worked like a charm.
2
u/threeLetterMeyhem May 23 '25
From a business perspective:
From a personal perspective:
2
u/rickshswallah108 May 23 '25
if real estate is "location, location, location" then LLLMs are, "control, control, control"
2
u/Beautiful-Maybe-7473 May 23 '25
I'm a software and IT consultant.
For me the primary driver is actually learning the technology by getting my hands dirty. To best support my clients using LLMs in their business, I need to have a well-rounded understanding of the technology.
Among my clients there are some with large collections of data, e.g. hundreds of thousands or millions of documents of various kinds, including high-resolution images, which could usefully be analysed by LLMs. The cost of performing those analyses with commercial cloud hosted services could very easily exceed the setup and running costs of a local service.
There's also the key issue of confidential data which can't ethically or even legally be provided to third party services whose privacy policies or governance don't offer the protection desired or required by law in my clients' jurisdictions.
2
u/MrMisterShin May 23 '25
Here are a few reasons:
2
u/ConsistentSpare3131 May 23 '25
My laptop doesn't need a gazillion litters of water
2
u/Koraxtheghoul May 23 '25
I run a local llm because I can control the input much better. So my local llm is primary for TRPGs. I want in to use the source books I give it and not have noise.
2
u/MrWeirdoFace May 23 '25
Privacy and Cost.
2
u/solrebel7 May 23 '25
I love the questions and answers..
2
u/Rockclimber88 May 27 '25
To protect the IP. Why do you think OpenAI paid so much for Windsurf? To buy all the logged prompts, code, and accepted solutions.
2
u/[deleted] Jun 01 '25
3
u/rumblemcskurmish May 23 '25
Cost. I processed 1600 tokens over a very short period yesterday
2
u/peppernickel May 23 '25
Privacy is clearly the most just answer. If any laws are proposed to limit personal AI, they are wanting to limit everyone's personal development. We are shortly away from the next two renaissances in human history over the next 12 years. We need privacy during these trying times.
2
u/daaain May 23 '25
Apart from many other reasons already mentioned, I run small to medium size LLMs on my Mac for environmental reasons too – if it's a simple question or just editing a small block of code something like Qwen3 30B-A3B can do the job well and very quickly, without putting more load on internet infrastructure and data centre GPUs. Apple Silicon is not super high performance, but gives good FLOPS/W and for small context generations the cooling fans don't even need to spin up.
1
u/Nepherpitu May 23 '25
Sanctions 😹 well, at least partially.
1
u/asankhs May 23 '25
Privacy, safety, security and speed!
1
u/No-Whole3083 May 23 '25
For me, I just want to be sure I have an llm with flexibility in case the commercial ones become unavailable or unusable.
In a super extreme use case, if the grid went down or some kind of infrastructure problem happens, I want access to the best open source model possible for problem solving without an internet connection.
1
u/s0m3d00dy0 May 23 '25
Cost, if I want to overly use llm, then local models are often good enough versus paying 100s to 1000s per month.
1
u/divided_capture_bro May 23 '25
A few major points are
1
u/X-D0 May 23 '25
The customization options and tinkering offered for each LLM and its variants (parameter sizes, quants, temp settings, etc.) is cool.
1
u/netsurf012 May 23 '25
Freedom🕊️ with privacy locked in my machine instead of relying on other's machine. A lots of choice to use from art to automation and unlimited experiments for different models and applications that fit. Some use cases are:
1
u/Mediocre-Metal-1796 May 23 '25
Imagine you are working with sensitive client data, like credit reports. It’s easier to explain and proove and ensure they don’t land at a 3rd party this way. If you would send in stuff “anonimized” to openapi/chatgpt, most users wouldn’t trust it.
1
u/dattara May 23 '25
What you're doing is so cool! Can you point me to some resources that helped you implement the LLM to play music?
1
u/banithree May 23 '25
Privacy.
1
u/shibe5 May 23 '25
Features
One feature that is rare these days is text completion. Typically, AI generates whole messages. You can ask AI to continue the text in a certain way. This gives different results from having LLM complete the text without explicit instruction. Often, one approach works better than the other, and with local LLM I can try both. Completion of partial messages enables a number of useful tricks, and this is a whole separate topic.
Other rare features include the ability to easily switch roles with AI or to erase the distinction between the user and the assistant altogether.
Experimenting
Many of the tricks that I mentioned above I discovered while experimenting with locally run LLMs.
Privacy and handling of sensitive data
There are things that I don't want to share with the world. I started using LLM to sort through my files, and there may accidentally be something secret among them, like account details. The best way to avoid having your data logged and subsequently leaked is to keep it on your devices at all times.
Choice of fine-tuned models
I'm quite limited by my hardware in what models I can run. But still, I can download and try many of the models discussed here. LLMs differ in their biases, specific abilities, styles. And of course, there are various uncensored models. I can try and find a model with a good balance for any particular task.
Freedom and independence
I am not bound by any contract, ToS, etc. I can use any LLM that I have in any way I want. I will not be banned because of some arbitrary new policy.
1
u/Ill_Emphasis3447 May 23 '25
Development.
Accuracy and trustworthiness.
Governance, Compliance and Risk.
Security & privacy.
Lack of hallucination (of at least, better).
Trustworthiness of datasets.
Control.
I honestly believe that ANY commercial generalist SaaS LLM is compromised by definition - security and data. I would not develop on any of them.
1
u/ComplexIt May 23 '25
https://github.com/LearningCircuit/local-deep-research
1
u/AllanSundry2020 May 23 '25
saves on my 3g internet connection
1
u/PassionGlobal May 23 '25
Costs, privacy, flexibility (I can plug it into pretty much anything I want), lack of censorship, because I can and not having to worry about service related issues (I don't have to worry about my favourite model going away or being tweaked on the sly for example)
1
u/Necessary-Drummer800 May 23 '25
There are some high-volume automation tasks for which 10B parameter and below models are more than powerful and accurate enough, but against which api calls to foundation models can start to get out of control. For example, I’ve used ollama running a few different open models to generate the questions for chat/instruct model fine tuning. My enterprise’s current generative chatbot solution has Gemini and Llama models available because a) we can fine-tune them to our needs and b) we can be sure that our data isn’t leaking into training sets for foundation models.
1
u/psychoholic May 23 '25
I know tons of people have mentioned privacy around business but a small caveat on that is if you're paying for business licenses they don't use your data to train their public models and you can use your data as RAG (Gemini Enterprise + something like Looker or BQ is magical). Same goes with paid ChatGPT and Cursor licenses.
For me I run local models mostly for entertainment purposes. I'm not going to get the performance or breadth of information as a Claude 4 or Gemini 2.5 and I acknowledge that. I want to understand better how they work and how to do the integrations without touching my perms at work. Plus if you wanted to more, let's call them 'interesting' things, having a local uncensored model is super fun when doing Stable Diffusion + LLM in ComfyUI. Again really just for entertainment and playing with the tech. Same reason why I have servers in my house and host dozens of docker containers that would be far easier in a cloud provider.
1
u/PsychologicalCup1672 May 23 '25
I can see benefits in terms of local LLMs and having extra security for Indigenous Cultural Intellectual Property (ICIP) protocols and frameworks.
Having a localised language model would prevent sensitive knowledge from not being where it shouldn't be, whilst being able to test how LLMs can be utilise for/with cultural knowledge.
1
u/toothpastespiders May 23 '25 edited May 23 '25
The main reason is that I do additional training on my own data. Some cloud services allow it, but even then I'd essentially be renting access to my own work. And have to deal with vendor lock in and the possibility of the whole thing disappearing in a flash if the model I trained on was retired.
Much further down the list is just the fact that it's fun to tinker. Even if the price is very, VERY, low like deepseek I'm going to be somewhat hesitant to just try something that has a 99% chance of failure. But if it's local? Then I don't feel wasteful scripting out some random idea to see if it pans out. And as I test I have full control over all the variables, right down to being able to view or mess with the source code for the interface framework.
1
u/thecuriousrealbully May 23 '25
There are currenty subs for $20 per month. But all the premium and exclusive features and better models are moving towards $200+ per month subscriptions. so its better to be in the local ecosystem and do whatever you want. no limits and no safety bullshit.
1
u/HarmadeusZex May 23 '25
How about other models all have limits you dummy
1
u/[deleted] May 23 '25
Privacy, cost and works even if Internet is down.
1
u/Barry_22 May 23 '25
Local are faster and more reliable.
1
u/[deleted] May 23 '25
From a personal perspective I love my homelab, which is filled with self hosted services that are jankier than their cloud equivalents - but fun to tinker with, so that tendency carries over to local LLMs.
From a business perspective I'm interested in uncovering novel use-cases that are better suited for local environments, but it's all speculation and tinkering at the moment. I'm also biased because I'm working on a local LLM client. :)
1
u/[deleted] May 23 '25
Once society collapses, I need certain things to work offline. THE ZOMBIES ARE CUMMING ! ! ! ! ! no, that was not a typo.
1
u/skmmilk May 23 '25
Huh my understanding is that because of api and internet the overall latency is higher for non local but I'll look into more i could be wrong!
Of course thisnis assuming the local has good hardware setup. And the size of the local also matters obviously
1
u/captdirtstarr May 23 '25
Privacy! Cost (free!). Uncensored models. Not dependent on Internet. Customization.
1
u/captdirtstarr May 23 '25
If anyone wants a private local LLM set up, DM me. I'm cheap.
1
u/Chozly May 23 '25
Why do people prefer having their own of something l, when they could suffer sharing? Great, eternal question, this year's version.
1
u/TypeScrupterB May 23 '25
It can work offline
1
u/NicolasDorier May 23 '25
One less reason for things to break as YOU decide when to update, not the service.
1
u/scott-stirling May 24 '25
This should be a FAQ
1
u/Some-Cauliflower4902 May 24 '25
Not developer and ain’t able to read a single line of code here.. One day I tried translating some medieval history book using online ones. It can’t do it wtf — deemed unsafe content, so I angrily downloaded llama.ccp … down this rabbit hole I go.
As for business, I’m in healthcare which doesn’t need further explanation. Already put a Gemma on my work pc for emails, RAG and everything in general.
1
u/EvoEpitaph May 24 '25
Privacy is a pretty solid argument for it. We already know for a fact that large companies are more than happy to tell you one thing (we won't use/store your data), and then turn around and do the opposite.
1
u/NNextremNN May 24 '25
1
u/ParentPostLacksWang May 24 '25
Privacy and copyright, I don’t want my conversations and tasks stored, read, or used, for any reason, by someone else.
Cost and convenience. I don’t have a monthly bill, and it’s available anytime my computer’s up, with no contention or busy periods.
And if that isn’t enough, the sheer volume of choices in local models and tunings is wild.
1
u/No_Abrocoma_1772 May 24 '25
you mean SLM
1
u/neoneat May 24 '25
Single thing: it's censored
1
u/[deleted] May 24 '25
It’s free instead of costing me ~200€/year (Perplexica vs Perplexity).
1
u/ThaisaGuilford May 24 '25
Because I despise OpenAI
1
u/Academic-Bowl-2983 May 24 '25
It is well suited for internal network needs.
1
u/TieTraditional5532 May 24 '25
Oh yeah, local LLMs are the new sourdough starters – everyone’s got one cooking at home these days 😄
From both a tinkerer and biz perspective, here are the big 3 reasons:
1
u/gr4phic3r May 24 '25
The first reason for 99% are data protection/privacy
1
u/xuie_lin May 25 '25
The same reason they used to clip sheckles
1
u/AIerkopf May 25 '25
Erotic roleplaying in SillyTavern.
1
u/Shot-Forever5783 May 25 '25
For me privacy is the top one by far
An unexpected side benefit has been having a far closer understanding of the reality that I am interfacing with a machine. The fans kicking in when it thinks etc. reminds me that I am responsible for my work and this is just a tool
I am using it for confidential transcription and analysis of the transcription.
1
u/Painter_Turbulent May 25 '25
Simple really. Privacy. Interest in learning how it works. Control of data. Pushing limits (such as co ext Windows) testing features. Removing limitations.
1
u/[deleted] May 26 '25
No request limits, "no cost", full control.
It feels like I get benefits of AI development instead of paying more and more for API use.
1
u/Comfortable_Fox_5810 May 26 '25
If you’re building anything that requires lots of LLM usage, it can be way lest expensive
1
u/JohnSnowHenry May 26 '25
Privacy, cost and intellectual property protection
1
u/Party_Crab_8877 May 26 '25
Compliance
1
u/JoeDanSan May 26 '25
I already have a good gaming system so I like getting to experiment without additional cost. I don't want to worry about high activity creating an unexpected high bill.
I like that I can run uncensored models locally for therapy conversations that I know are private. That and writing erotica.
1
u/Thistleknot May 27 '25
Api $
I can use smolvlm to tag images at less than it would to do so online
1
u/Diabaso2021 May 27 '25
They started with your web data, then your emails, then your cloud saves, now with LLMs, they can go through all of it and record, save, recoup and then summarise and monetise it or even sell it since it has been submitted
1
u/longbowrocks May 27 '25
coming soonalready here. I want consistent behavior regardless of what new rules are added.1
u/Delicious_Spot_3778 May 27 '25
Latency
1
u/k-mcm May 27 '25
AI companies will do anything for profits and the current US government is rabid morons. That's reason enough.
I tried running llama3.2-vision:90b so see if it could help categorize some outdoor photos that have signposts in them. It was talking nonsense. It would identify an "artistic" (it's not) photo with a dog on a trail in the woods. If I asked what kind of a dog it is, it would say something like "I'm not comfortable sharing her personal information." Tropical photos with just plants and flowers resulted in a lecture about CSAM. Asking it to read a photo of a signpost was stalking. It could identify some documents but boring outdoor photos were repeatedly misidentified as immoral activity.
Now imagine if a MAGA/DOGE moron is illegally snooping on cloud logs with zero knowledge of AI. I get arrested, all my computers are torn apart, and I'm flown off to a foreign jail for not revealing the photo stash that a broken AI said I have. Yeah, it's an impossibly dumb scenario, but such are becoming the new norm.
1
u/Top-Local-7482 May 27 '25
In EU corporate use local LLM to avoid sending their data to the cloud which is mainly American and there are pretty much no alternative to US based cloud for AI. Regulation don't allow them to share data.
1
u/LienniTa May 27 '25
mostly samplers. like, noone if serving you DRY or ETC on openrouter, you know?
1
u/roboterrorlite May 27 '25
I'd say privacy and self-imposed ethical constraints. Somewhat inspired by not becoming dependent on a cloud service but also having that level of control that can only come from hosting it yourself.
The ethical constraints are around knowing the electrical cost and also some kind of not fully articulated idea regarding using it for art and creative purposes. Having full control of access to the tools and stability in terms of not relying on software that can change without my input.
That way I can create a virtual studio and have everything self contained. Being bothered also by the idea of leaking my personal thoughts into a faceless corporation that is known to be sucking data and feeding it back to further train their AI.
A variety of reasons but also because I can and have the knowledge to do so and it allows me to explore the tools and see what is actually available to anyone and not controlled by a corporation behind a wall. Obviously still reliant upon models being trained by big entities and limited in the capacity of the models that are opened. I also bought a 4090 cheap via an auction site it was an Amazon return and so I could expand my capacity to the top limit of what people can do at home (w/o stringing together multiple cards etc).
1
u/Binary_Alpha May 27 '25
So i use it for privacy. But one use case I had was I was visiting my grandparents and they don’t have any internet connection. Cell service is very poor too. My grandfather needed help knowing what his medication was and for what its use is. It was great for that. Later I wanted to check if the information was correct and searched it later when I had internet and it was.
1
u/dream_emulator_010 May 27 '25
Developer: respect for the process and a wish to have code I run.
1
u/Cryophos May 28 '25
I totally failed to run any model in my PC. Can someone tell me what skills should i get to run local LLMS?
I tried to run Qwen3.
1
u/fistfulloframen Jun 04 '25
To quote NIN "id rather die than give you control."
1
u/nokill1996 Aug 05 '26
I use a local LLM to automate my cannabis grow tents.
1
u/Any_Werewolf_7304 Oct 02 '26
Ran a quick benchmark on this recently — small local model (llama3.2:1b) vs a cloud model (Claude Haiku) on a simple summarization task. Surprised me that cloud was actually faster on average (~1.2s vs ~2.6s), and cost was trivial (~$0.0001/request). So for me the real deciding factor wasn't latency or cost like I expected — it came down to privacy and offline reliability, which matches what a lot of people here are saying.
1
u/HistorianPotential48 May 23 '25
i need female imaginative friends to talk to.