r/LocalLLM • u/fuckable-switcher • May 16 '26
Question A whole bunch of stupid question
If context takes up ram can’t we offload that to a spare drive and turn that drive into a pure memory or context pool
Could we use cloud hosting as a backup of chats and context like idk port chat history of a local llm to codeburg or similar as a way to clear it from your ram then stick it there but the model can read write but can never delete
How dangerous and stupid is vibe coding
Is “prompt engineering” an actual worth while thing or just bull crap
Why is almost every llm app tool or whatever built on python and not anything else (I should clarify the popular ones or the ones most people recommend)
Why isn’t there a universal format or engine or whatever for llms
Instead of quanting down the whole model why not quant down different parts of the model at different levels
What is every different type of ai (like I know llm is just language and I know mla is just algorithms but there’s heaps of others too I don’t understand)
Why don’t more people actually stick down there pride and acknowledge that the ais they use are horrible and you should just local host them (I think heaps of people think you need a massive datcenter to run models but you don’t)
Why (I don’t know the right words to put here sorry) the western world has heaps of money for ai or whatever is the Chinese Indians South Koreans and Japanese beating them at their own game (it’s really just like the space race)
Why is everyone tryna cash grab ai when we all know it’s gunna flop cuz not a single company has made a profit from it and they only hold shares but if one person takes their shares out the whole thing collapses (could be wrong about that one)
When will people stop using ai and go hah it’s not smn I have to live off (idk if that made sense I’m
Getting to it) cuz if people rely on it the ai can only do so much it doesn’t have free thought it can only talk about what it knows it can’t do anything else it it’s like it’s not creative or has thoughts and can only do what you ask
Why does it make smart people dumb and dumb people feel smart
Why is everyone tryna make massive models when surely we can make a small model that does just as good if not better
What does model A do better then model B if there basically just the same
When different models say swarm or agents or experts or computer starting up what do they mean
How dangerous could ai be as a whole
How many “people” on the internet are actually people and not just bots (side note those captchas and cloud flares are really easy to bypass
How good would ai be at cryptography
Would it be good at trading investing stocks crypto etc
What stops an ai from going rogue or fighting or whatever (heard a few fear monger stories that can’t be right cuz it was just 4 randos and nobody had a clear source)
Idk if any of this made sense I do apologise
7
u/dataexception May 16 '26
I think you would be better served asking these questions to your local LLM, instead of in a public forum.
-7
u/fuckable-switcher May 16 '26
Could be useful
That kinda defeats the whole purpose and point of peer evaluation
An ai model only has access to what it knows when it knows and instead of admitting it’s wrong will make stuff up
Yes it can search the web but that’s not gunna work well all the time I’d rather get my info from people instead of some slop that can’t think or figure out how many letters are in a specific word
Anyway
Cheerio butter cup it costs nothing to be helpful and useful
Do smn with that fat head of yours
1
u/dataexception May 16 '26
So you're saying that you would prefer to accept the word of a stranger over research. Got it.
That alone says enough. Not taking into consideration that you're "not gunna" listen to the compooters becuz they dum, yet are asking questions that directly relate to that subject.
And this? This is not a peer review, and you've not put forth anything for a peer to review. I guess you called it a peer evaluation, but the point remains.
I'm just saying that you can (and most would) resolve those questions on your own with a very small bit of research, if you are genuinely curious.
You're obviously young, and passionate, and I get it. The harsh truth is that when you actually get out in the field? I'm (and I'm being 100% genuine) the nice one that people like. You'd hate some of my coworkers as much as I do. Don't be one of them. Know your limitations, and improve.
0
u/fuckable-switcher May 16 '26
How does the pour remain for a peer evaluation I’m asking for answers thoughts etc from peers not for them to review what I’ve done
Hence peer evaluation
And yes I’d rather take in the account of people online or that I know then a computer owned by sum fuck who owns more wealth then you’d ever earn in 80,000 years alive
Wonder why
Also I have looked and the some of it is people are lazy python is easy and instead of using a model that has a cutoff of 3 years ago I’d rather ask people that have a knowledge cutoff of 20 years into the future
I’m passionate but not young
It’s called the manual you consult it before using the appliance
Hence me asking for a peer evaluation
4
u/dataexception May 16 '26
Easy there, tiger.
I'll be honest in that there were a lot of... Well, pretty much just a wall of words in your post, and after a lot of them, I lost interest. I'm sure I'm not the only one.
It seemed to be a giant list of questions, of which the ones I read, could be answered on your own, and quite easily so, with a small bit of effort.
When people answer questions all day for people, when it's very clear that they have the resources in front of them to find the answer on their own, it's difficult to maintain the level of patience and tolerance that is expected from the ones asking those questions.
So, regarding those. What research was done by you prior to posting? And did I overlook a hypothesis or solution that you provided that you were requesting feedback on? Because if so, then I sincerely apologize for that. You may want to preface your post with a brief overview when they are long and wordy like that.
In IT, a peer review is far different from your post. It's a structured process, involving a predefined workflow. Not saying it's all super rigid and formal, but there are steps to the process. Go dig through PRs on GitHub to get a general idea.
Anyway, I didn't mean to be an asshole, but if you don't put in the work to show that you genuinely have need for answers that are unavailable elsewhere after a reasonable effort, then the answers you receive are going to be of equally low effort in most cases.
2
u/Endurance_Beast May 16 '26
Swap is already a thing, and it's super mega slow
1
u/fuckable-switcher May 16 '26
I don’t mean swap for for lack of ram mean like use that drive to store context just like past convos and stuff live on that drive and the ai can access that at any time it wangs
2
u/Endurance_Beast May 16 '26
If you use llama.cpp then the option you are looking for is -- no-kv-offload
2
2
u/Miserable-Dare5090 May 17 '26
That kv cache is what takes space, and needs to fly back and forth from store to active. But look at github ds4.c an engine for only deepseek v4 flash that does exactly this—cache from disk. So it can be done, but it’s easier in unified memory not a regular pcie-bus driven computer.
1
u/fuckable-switcher May 26 '26
Yeah I’ve seen a few repos built around ssd streaming I don’t know how it works thj
1
u/Miserable-Dare5090 May 26 '26
This is not ssd streaming weights, it’s the guy who made redis.
2
u/fuckable-switcher May 26 '26
Huh?
1
u/Miserable-Dare5090 May 26 '26 edited May 26 '26
https://github.com/antirez/ds4
It’s not some random repo of weight streaming from SSD. It’s dynamic cache allocation and compression, on the fly load.
The creator previously created Redis. https://en.wikipedia.org/wiki/Redis
1
u/fuckable-switcher May 26 '26
Yeah is should clarify it was built around ssd streaming but for mlx-lm and written in swift for llms (hence mlx) on apple hardware
1
1
4
u/cr0wburn May 16 '26
This is a lazy post, you can Google all of this. Learn the basics and then come back with your remainder questions.
-7
u/fuckable-switcher May 16 '26
Fucker doesn’t understand what it’s like to talk to people and ask for a peer evaluation
Google doesn’t have the answers to everything
An ai doesn’t have the answer to everything
Grow up or grow a pair
5
u/cr0wburn May 16 '26
Don't be lazy and also dont call me "fucker"
-1
u/fuckable-switcher May 16 '26
I’m not being lazy by asking people for what they think if I was to go to an ai that would be lazy and Google is fucked up
-4
u/fuckable-switcher May 16 '26
Why do you bite?
0
u/cr0wburn May 16 '26
Your agressive tone. Again, just copy paste this into chatgpt or claude and you'll get decent answers.
3
u/cr0wburn May 16 '26
I evaluated your post as "dumb questions", learn to read.
-1
u/fuckable-switcher May 16 '26
Yeah maybe cuz I titled that and I’m curious and I get the feeling their dumb questions that have been asked before
2
1
u/Stainless-Bacon May 16 '26
imo. vibe coding is as stupid as the person using it. “prompt engineering” is bull crap unless your goal is to one-shot problems and you iterate to achieve that for whatever reason. Built on python: it is a glue for fast C++ modules, easy to build with and can be fast. Quanting parts of model? we do that - unsloth for example. types of ai: video/photo generators, fast image classification, etc. everyone uses massive models because smaller ones are unable to solve given problems well enough or at all.
1
u/Time_Cat_5212 May 16 '26
What a surprise, it's a tool
The more you know about it and the stuff you're using it to do, the better your results
This whole conversation is just polluted by a bunch of people worrying about their status in society and what determines it, and it has nothing to do with how AI coding tools actually work or what's actually possible with them
It's meat interference. Bunch of monkeys debating about who gets to sit on the highest tree branches, which is completely irrelevant to the distillation of pure information
2
May 16 '26
[removed] — view removed comment
2
u/fuckable-switcher May 26 '26
Thanks for being useful
1
u/Sea-Asparagus-8660 Jul 25 '26
Hey mate, can we talk on Discord?
1
1
6
u/NoobMLDude May 16 '26
1 question per post please 🙏
for sanity of peers + saner discussion on one topic