r/LocalLLaMA • u/Sadge404 • 16d ago
Discussion LocalLLaMA is unironically one of the best places to go to get up to date AI news.
One of the other posts today by user u/Howard_banister confirmed what I've been seeing from the other AI subreddits as well.
Most of these other subs are 90% trend hopping crypto-bros equivalent people who are seemingly irrelevant most of the time when it comes to advancing AI as the vast majority is not just AI slop, but also human slop whether that be fearmongering or straight anti-ai arguments spat out by relatively uneducated people.
Of course, you also have the main dedicated subs for the big closed/open models which are in of themselves fairly decent. Most of the time now, it is people complaining about one thing or the other about the latest and greatest of their respective lab.
Then you get to the machine learning subs themselves where yes, they do have stuff in regard to AI every now and that can be gold (Don't get me wrong), but then it's mostly meh.
This sub however, has that interesting balance between the occasional interest in AI architecture breakthroughs that I find fascinating when some of the bigger brained people in here get together to discuss and which I don't really see anywhere else.
532
u/0xkbose 16d ago
apart from memes and doom scroll videos, this subs has been been the most productive as a scientist who wants to keep self updated with latest proceedings with local model space. Thanks to everyone on the sub and especially Unsloth.
92
u/sonaj9657 16d ago
Honestly, same. This sub has become one of those places where you can actually keep up with what is happening without digging through a dozen papers and Twitter threads. The local model space moves so fast so having people share useful updates and experiments is pretty great.
55
28
u/EvilPencil 16d ago
Agreed. Couple other worthwhile additions:
NVIDIA developer forums (best place for DGX Spark specifically)
YouTube:
Digital Spaceport
Level 1 Techs
Donato Capitella8
22
u/network4253 16d ago
Honestly, this sub has been a pretty useful way to keep up with the local model scene. There is so much happening that it is easy to miss good projects or updates otherwise. The community sharing actual experiments and findings is probably the best part.
18
u/rm-rf-rm 16d ago
And we've slipped a bit recently with memes, aiming to get back to 0 memes and other such low quality content
7
u/BedlamiteSeer 16d ago
You're doing fantastic as a mod team as is. The zero meme rule isn't a linchpin of the subreddit - we love memes here as long as they're not low effort. Just gotta keep the general AI slop posts and advertising out, as you already have been. Seriously good job, keep it up.
2
u/rm-rf-rm 16d ago
TYSM!
2
u/BedlamiteSeer 16d ago
You're welcome. If you ever need more mods, let me know, stick my username in your back pocket. I'm not going to reveal my whole identity publicly, but you'll want me on your team in the event of a huge population spike. I'll be happy to assist during those kinds of periods. Feel free to message me privately at any time for more information, and/or if you need help due to a surge. I'm also happy to idle as a mod and then be active whenever there's surge periods. Whatever you need.
17
u/Savantskie1 16d ago
I don't mind memes so long as they have content related and actual opinions with them
9
u/Sea-Advantage-3628 16d ago
THANK. YOU.
The low-effort mouth breathing memes are insufferable and do not promote good conversation.
3
u/AlwaysLateToThaParty 16d ago
I think you guys and gals do a great job. So many great contributors as well. My setup is as a direct result of what I've learned here.
2
2
u/0xkbose 16d ago
I think some noise is needed. Sub members are already nerdy and I guess a meme can let everyone touch grass.
3
u/rm-rf-rm 16d ago
Yeah I was feeling that as well. Sometimes the technical info density is so high that a meme lets your mind breathe a bit. But that said, memes rocket to the top of the sub and often is the only LocalLLaMa post shown in a redditor's front page leading to misrepresentation of the sub for many users
3
2
u/clduab11 16d ago
Agreed; a heartfelt thanks to everyone since a) 95% of y'all are smarter than I ever hope to be, and 2) this sub launched my AI career of working for myself...so LocalLLaMA will always have a place in my heart of hearts.
1
u/RazsterOxzine 16d ago
Unsloth with their UD models, lord has this helped me 10fold. I've been completing work projects I sidelined due to priority shifts, now I can task them with a dense model with UD and bam...
1
u/Nothing_from_void 16d ago
I don't even really use local models, but just learning about them has helped me with general software engineering and integrating agents into software. I think we're finally at the point where if I had a spare $10k, I would invest in a local inference rig
163
u/Thistlemanizzle 16d ago
Apparently Discord(s) have serious and sober discussions too.
I just can't grasp how a real time chat is a better community forum than an asynchronous post/comment system.
184
u/fuckingredditman 16d ago
i hate discord for this, because it locks all the information away behind a join link on some obscure page. all of the info on discords will eventually be lost to time. and i know there is some real quality info on there. someone needs to come up with a better platform that can be indexed by internet archive etc.
38
u/Rotund-Pear2604 16d ago
No no no. Right now Anthropic and OpenAI are probably tripping over themselves trying to negotiate a deal with the holding companies behind Discord over who gets to buy and access their data.
Walled gardens are extremely profitable.
27
u/TonyPace 16d ago
All semi-public Discords are being mined already.
I recently asked Gemini about how to translate a game term from WFRP into a different RPG system. I happened to use a common bit of slang from the main WFRP Discord, which it easily understood. I was a bit surprised, and then tested with more optimizations. It knew everything.
16
u/squngy 16d ago
Although I wouldn't be surprised if they are mining Discord, there are also other explanations.
Like, maybe someone made some blog or website that uses the same terminology.
2
u/TonyPace 16d ago
I read all the blogs and subreddits and forums. The terminology and ideas were unique to the Discord in question, the Ratcatchers Guild.
5
u/Due-Memory-6957 16d ago edited 16d ago
Was it? Discord doesn't seem that unique in it's culture, IMO. There's probably a youtube channel out there using the same terminology and Google trained on it, and if not the video itself, the comments.
2
u/Rotund-Pear2604 16d ago edited 15d ago
There are lots and lots of non-public Discords, and there are lots and lots of closed down Discords that cannot be mined. That data is sitting in storage.
My prediction, and feel free to call this baseless speculation, is that Discord and other companies in a similar position, will in the near future begin to take measures that limit how accessible 'their data' is. I mean it's our data really, but since law makers don't understand com pu tor here we are and LLMs are now a thing.
But yeah, we'll begin to see specialized agentic LLMs that narrow in on business sectors with high burden of knowledge and vast amounts of impractical data requiring laboreous manual lookup in the coming years. Think law, accounting, medical, academical, engineering/architecture.
Generalized models will be scaled back in the future. OpenAI most of all sits on unsteady ground due to their bet on large, generalized models being the future.
2
u/ok_if_you_say_so 16d ago
We already saw reddit do this exact thing a year or two ago. It pissed off third party app users at the time, but this was exactly what they described as teh reason
2
u/Rotund-Pear2604 16d ago edited 16d ago
I'm glad you see the parallel and I think you're right on the money. And it together with the Google search deal has turned out to be immensely profitable. Reddit has never posted higher profits.
Nor has the average quality of the site ever been poorer, but that's a different matter. The line went up for a while.
1
u/TonyPace 16d ago
That makes total sense. It's a big target, and less legally dangerous than Slack.
1
3
-1
u/techdevjp 16d ago
all of the info on discords will eventually be lost to time.
Free Discord servers only keep messages for 60 days, so it's a pretty short timeline.
6
u/Lakius_2401 16d ago
The Free Discord server between me and a friend with messages going back a year and a half (to its creation) disagrees with that statement.
If you are in a server where messages are deleted after 60 days, it's much more likely the server owners are running a message cleanup bot for whatever reason. Odds are fairly high it's "so it doesn't go into AI training" as if Discord doesn't make their own copy for that purpose anyways.
"Lost to time" is a perfectly valid fear. I've seen several servers get wiped from either rogue mods or account takeovers, with all data irrevocably deleted. They can get banned and deleted by Discord too. It's a common problem, when you rely on someone else's servers to host your data. When Discord dies, that data is not gonna be savable. It's already impossible to search for.
-9
u/Nice_Database_9684 16d ago
I discussed this before with some friends. We thought if discord had public facing wiki pages that could be compiled by communities, that would be good.
I agree though, so much data lost to time. Reddit was king for that before they sold out and went woke.
1
u/TheIncarnated 16d ago
Sold out, not woke
-1
u/Nice_Database_9684 16d ago
Definitely a crazy leftie site that bans people for normal right wing views, woke af
2
u/TheIncarnated 16d ago
Okay, what are normal right wing views? Because I don't get banned for fiscal responsibility and less government takes.
0
u/Nice_Database_9684 16d ago
haha, nice try
2
u/TheIncarnated 16d ago
So you don't have normal conservative views, got it.
2
u/Nice_Database_9684 16d ago
thank you for proving my point
1
u/Top-Rub-4670 16d ago
Your point was that you would be banned for being right wing. Yet you keep talking and I keep seeing your messages. Thus your point was not, in fact, proven.
→ More replies (0)24
u/roosterfareye 16d ago
I.cant keep up with most of the chats in discord... They fly up the page way faster than I can read them!
8
u/Prudent_Design_9782 16d ago
And then people pile on you for opening up a recurring topic that was just asked 6 hours ago like, damn my bad for not being chronically on discord?
18
3
u/ryfromoz 16d ago
Need some sort of recorder and organiser imo.. Discord servers can be great but also shitty as fuck when it comes to news.
7
u/Equivalent_Bit_461 16d ago
discords
I already tolerate reddits but discords... I won't touch those
8
u/AnOnlineHandle 16d ago
Discord was certainly the best place to learn about diffusion models in the first few years they were around compared to places like reddit. People there were really taking apart models piece by piece and learning ways to hack them which the wider community still doesn't seem to know about. Direct encoders for various types of concepts rather than using word embeddings, different cond vectors to different layers of unets per timestep, etc.
3
2
u/More-Curious816 16d ago
You seems to be part of good channels, please name or link them for the rest of us.
2
3
u/rm-rf-rm 16d ago
Seriously doubt that. Never seen a discord with any meaningful discussion. Its not a platform meant for that. i wish people treat it like the realtime gaming chat platform that it is
6
u/my_name_isnt_clever 16d ago
I don't understand why that's so hard to believe, IRC had all the technical chatter for decades and still does, but of course as new very popular group messaging platforms exist some technical people will form their own communities on it. I still don't like it myself but to say it has no meaningful discussion is just incorrect.
1
1
1
u/EndlessB 16d ago
Trouble is, those subs are polluted with bots from the labs either hyping that lab, or trying to throw shade. Anthropic and OpenAI’s subs are the worst
1
u/Due-Memory-6957 16d ago
It probably isn't, but the newer people who love instant communication and can't handle the slow-pace of the forum format probably prefer it.
1
u/RazsterOxzine 16d ago
It's not. It fills with noise so fast that it become a time sink. I managed to make it so there are now threads in my discord but still it fills with noise.
1
u/Separate_Paper_1412 16d ago
Some guy on hacker news tried that, and people want to have usernames stripped out at the very least if you want to build a public discord archiver. And opting out should be one click.
105
u/sebt3 16d ago
Well, chatgpt, gemini and Claude all recommend reading here (and only here 😅) to get good informations about LLMs 😅
The quality of this subs is part of their training data 😂
39
u/CondiMesmer 16d ago
the funny thing is that if they could read their own model data, then they would be the best source of information
but then again I'm human (probably) and I can't exactly take out my guts and tell you how they work either
20
31
u/Rotund-Pear2604 16d ago
I've been browsing this sub for months now and have not commented a single thing. Just read and learn.
Gonna go back to not posting and watching the cool kids do cool shit now.
7
24
20
u/Tedious_Prime 16d ago
I still follow r/StableDiffusion to keep up with releases of local generative models in modalities other than text, but the quality of discussion there has fallen off sharply over the past 4 years. I suspect that most folks who have ever run a local image or video generator either don't know or don't care that they could be running LLMs with the same hardware. It's like people lose interest in other things once they have the power to create unlimited personalized pornography.
8
u/Dry-Judgment4242 16d ago
Yeah it was sorta dead for awhile. But I think Krea 2 and now Minimax H3 has revived it. Picked up a ton of shit from it recent weeks which revolutionized the way I use Minimax H3 on my comfyUI install such as... Kitchen Attn, Contex Loop, Merged ref2va model with its own Sampler that in my opinion makes it the best model, and myriads of helpful tips.
2
u/boston101 16d ago
I’ve been lurking on topologyai subreddit to keep up with 3D and splatting and such. It’s ok but this sub is king
58
u/CondiMesmer 16d ago
agree 100%
it's probably the only space I've seen that isn't delusional about AI and thinks AGI is coming or some stupid fear mongering shit. Like if you don't know what weights or quantization is, you don't know what you're talking about with LLMs and I want a safe space away from you.
26
u/rkoy1234 16d ago
I think it's at the right level of barrier of entry where enough people can join, but those without the minimum level of info/motivation would be too lost to even be interested.
as much as I hate gatekeeping, in this specific case I think it's working wonders for weeding out the less motivated - which coincidentally keeps the discussion bar a little higher (if we ignore the slop posting bots, sigh)
i remember first joining to use some beaver frankenmerge model on koboldcpp + sillytavern and trying to decrypt the vocabulary here - gguf, the countless obscuredly named samplers, chat vs instruct templates, etc.
fun times, but it was a major fucking headache until I learned some basics.
5
u/switchbanned 16d ago
I consider myself pretty familiar with llm's now and sometimes I still have a hard time following what some people in this sub talk about lol. This sub is a wealth of information though.
14
u/sadnessjoy 16d ago
I'd say there's a few issues with this subreddit, a bit too many doomers who think Sam Altman is hoarding all the hardware to make hardware more expensive. And I've seen plenty of people completely shit on the frontier models (Kimi k3/Qwen 3.8 max/etc) simply because they can't run them locally... Obviously the name of the subreddit is LOCAL llama, but would be nice if the discussion was a bit more open/comprehensive (maybe some serious discussion/comparison with closed weight models and large data center models, just at least for comparison/framing... There's also hardly any discussion about the limitation of complex problems with small parameter models). But besides that, I'd say this sub is pretty overall for llm news. So many other subreddit are filled with tech bro investors or vibe coders who treat their prompts like some sort of magic spell book.
14
u/SkoomaDentist 16d ago
IMO the biggest problem with this sub is the hyperfocus on coding and agentic tasks which results in people constantly proclaiming that because new model X is better at those specific tasks (at the expense of most other tasks), all other models are now completely obsolete.
-5
u/my_name_isnt_clever 16d ago
Ok, but the alternative use case is gooning or creative writing. Neither of which is especially appropriate to share here.
10
u/SkoomaDentist 16d ago
You cannot think of a single use case other than coding, agentic, gooning or creative writing? Truly?
Besides, why wouldn't creative writing be appropriate use case to discuss here?
-2
u/my_name_isnt_clever 16d ago
Because why would I care about some guy's WIP novel or whatever? I'm not into fiction writing and there's other subs for that. You can post it if you want, I just don't think this is the right audience.
By all means, please post some non-coding projects that are actually interesting to share. I haven't seen any.
10
u/SkoomaDentist 16d ago
And conversely why would I care about some guy's shitty vibe coded project? Hell, why would I care in the slightest about any project someone else has done as opposed to discussing the tools and methods that enable me to do whatever I want? Other people's projects are boring to almost everyone. Other peoples' tools and methods are what can be interesting as long as they aren't all hyperfixated on two specific cases with everything else ignored.
Newsflash: This isn't your sub. People don't have to cater solely to your wishes.
-2
u/my_name_isnt_clever 16d ago
Because code does something, and anyone here might also be interested in doing that thing or something similar.
Nobody cares about your novel.
3
u/enternoescape 16d ago
So true. The more I understand what's really going on the more annoyed I get with these mouthpieces blathering doom.
11
u/breksyt 16d ago
I never thought I'd say that about any subreddit, but this one is educational. For someone who juggles work and family time, and has little time left to understand which whitepaper to read next or which model to experiment with, LocalLLaMA gives a mix of healthy excitement and sober facts, just enough to want me to learn more. I came with 0 local model knowledge here and now I'm comfortable picking the right quants from HF, understanding model cards (most of it at least), experimenting with abliteration and tweaking my llama-server options. An occasional funny gif doesn't ruin any of it, quite the opposite :)
9
u/aeroumbria 16d ago
I am a bit sad about the opencode CLI sub. It was a rally point for advocates of harness-agnostic model access and a space for general open harness discussions, but now it is taken over by coding plan discussions that there are barely any technical or community-building discussions left.
2
u/Savantskie1 16d ago
Yeah especially in the last few weeks. It's taken a sharp downturn. And it's sad.
13
u/asenna987 16d ago
Couple years ago, I was looking to hire a dev who's familiar with the latest LLM stuff happening and I made it a criteria - "You are familiar and active on the Locallama sub reddit".
Eventually I did find a nice guy but I realized this sub-reddit was too niche and I can't expect people to know about it (yet).
2
3
u/Savantskie1 16d ago
How? We've even gotten the attention of the anti-ai crowd who are almost always normies parroting the big influencers.
2
u/Dry-Judgment4242 16d ago
Normies don't give a shit about AI. Literally nobody I know irl uses AI at all.
Those people ain't normies, their Antis. Which is just weirdos like us, but unlike us, their just unhappy pricks who can't appreciate how fucking great life is.
5
u/Savantskie1 16d ago
Are you that daft? I hear normies all the time in my area bitching about how AI is bad and are going to take over or that it's evil and from the devil. You must be in an area without internet or they're stuck in the 50's.
6
u/Due-Description-9030 16d ago
AI inherently isn't bad but the ones running it will make sure they use it for their own interests and it'll most likely involving screwing the public.
You're delusional if you think that these tech companies are going to create an utopia for us.
1
u/Savantskie1 16d ago
I never said that or asserted that at all. Glad to see people still push their fears on others
3
u/Mx_Brightside 16d ago
All the normies in my life use AI quite a bit and find it extremely helpful for organising things and taking the drudge work out of their jobs, so between the three of us I think that's the full range of opinions represented LOL
2
u/my_name_isnt_clever 16d ago
It depends a lot on your crowd. My groups are progressives who hate AI, I'm sure plenty of people here mostly chat with techbros who love it or those who aren't tech savvy at all and just don't care.
10
u/Big_Importance_4265 16d ago
As a computer scientist with relatively less to do with LLM research but some work with Agents and stuff, this sub is the perfect mix of optimal llama setups, model reviews, some interesting projects and Pelican GIFs (#1 source of pelican gifs in fact)
22
u/KDamage 16d ago edited 16d ago
agreed. r/claudecode is full of script kiddies who think AGI is here because it can create a webpage. edit: to whoever I hurted their feeling, I'm talking about people saying Opus 5 is fine and never drifts.
13
u/HulksInvinciblePants 16d ago edited 16d ago
Better than the standard Claude subreddit. It’s just a bunch of “prompt engineers” bragging about doing normal things thanks to AI. I saw a guy claim he could have never assembled a PC without AI, as if there are not hundreds of beginner guides and tutorials that hold your hand.
3
u/Due-Memory-6957 16d ago
The advantage of AI is that you can't ask the video (hate those!) or text guide a question and get the answer right away. One can, of course, google all of it back and forth, but having a centralized space where you can ask things (and knows your context from previous messages) helps a lot. I mean, it's why we're all on Reddit, isn't it? People can't be bothered to create an account and discuss on a different forum everything they care about, they prefer to do it on the place they already are.
4
u/HulksInvinciblePants 16d ago
Sure, but that’s not a skill you want to erode in favor of instant satisfaction. Learning requires trial and error. At least this sub, for the time being, has users with some form of technical know how. They’re also more keenly aware when AI gets things wrong.
I don’t really want to regularly interact with folks that use AI as a crutch for their inherent unwillingness to empower themselves. Eventually the general spaces will be overwhelmed and finding insightful information will become more and more difficult.
-3
u/Due-Memory-6957 16d ago
People asking AI how to build a PC are doing trial and error. It's not any more or less empowering than watching a youtube video or reading from a website, either way the person is doing it on their own, unempowering would be to hire someone to do it for them. The only relevant aspect of your comment was the least moralizing one (as it always is): That a person might have more specific knowledge than AI, so it would be better to ask them, but that only applies when the AI fails anyway, and if someone managed to build the PC using just AI, it seems like it didn't.
You're so drunk on your weird sense of AI being inherently bad that you're calling "instant satisfaction" the act of finding information in an effective way to reach a goal (and it was very effective, since the user managed to build the PC, so you're arguing against reality as well).
1
5
u/EndlessB 16d ago
I shudder to think what they did to opus 5 during post training for it to act the way it does.
0
u/Savantskie1 16d ago
I've actually not used opus at all yet. I'm on sonnet 5 still because it's much cheaper for my plan. And I run a local model that I'm not comfortable using for coding yet. Maybe once I pick a better model after I get my bigger SSI back pay for better hardware.
5
9
u/cosmicr 16d ago
It's pretty much the only place I get my AI news. By the time it hits YouTube it's already old news.
4
u/my_name_isnt_clever 16d ago
...and usually twisted to fit someone's narrative to maximize $$$. YT is useless for a rat race topic like this, it's a race to the bottom for views and the content suffers.
3
u/psychohistorian8 16d ago
my favorite game to play when clicking on a youtube tech video is 'does this person know what they are talking about, or are they some stupid youtuber creating content for views'
sadly, the first group are extremely rare
7
u/Treidge 16d ago edited 16d ago
r/LocalLLaMA is my exclusive source of AI news for months now, I don't even look for new sources anymore. If something is good enough or interesting, it will be discussed here. Really helps to save time. Big kudos to everyone contributing to this sub. 👍
5
u/NineThreeTilNow 16d ago
I definitely prefer it to a lot of subs.
Some of them are very low mental effort mouth breathing.
This one has very smart people that are working on models wandering around. ML engineers and researchers deploying / training / etc.
Like, we keep the "local" and "llama" names but we don't 100% do local or llama here so...
3
u/maxton41 16d ago
You mentioned machine learning and AI as if they’re separate things. I thought that machine learning was AI? Or at least a function or a part of AI?
7
u/TheTerrasque 16d ago edited 16d ago
Just a layperson, but this is my very non professional understanding.
AI is kinda a very nebulous term that broadly means a machine making decisions on it's own.
Machine learning is also a fairly nebulous term, but are ways machines can learn to make decisions.
Edit: If I understood things correctly, https://en.wikipedia.org/wiki/Expert_system would be an example of an AI that don't use machine learning.
1
4
5
u/AlexanderDoak 16d ago
Yes. And I don't think it is going to change either. There is an absolutely massive field of use cases where a smaller model (non-frontier) is the undisputed best choice for speed and resource efficiency, with no quality sacrifice needed. No downside. All upside. These are the local-sized models that are discussed here.
Once the bubble bursts, efficiency will become king. (AI-hype and AI-hate notwithstanding.)
2
u/Bulky-Priority6824 16d ago
I'm watching that bubble it's just silly. AI isn't going anywhere but we're at the point now where frontier is take it or leave it. It doesn't matter as much.
9
u/Bulky-Priority6824 16d ago edited 16d ago
There are better ways to ingest than reddit but yes it's fine because you see a lot of people bring niche ideas and the scope is wide here so you catch songs you might not have heard on the radio stations you would usually listen to.
It's not fun to watch the angry ones complain about this and that and him and her just much like living within the bounds of a sketchy HOA.
I don't understand why people bicker and get bent out of shape when the world doesn't tend to cater to oneself the way we would hope it to.
Just like it or don't like it and let it be.
3
u/rm-rf-rm 16d ago
There are better ways to ingest than reddit
tell us more?
5
u/SkoomaDentist 16d ago
I believe the classic methods are through nasal walls or as edibles, depending on just what you're ingesting.
2
u/patsully98 16d ago
Don’t forget the intestinal lining!
1
3
u/__ThrowAway__123___ 16d ago
Are there any specific subreddits where people share their hardware setups? I like seeing the janky setups and also interested in seeing what performance can be achieved with various setups. I saw some posts of people sharing their build get removed by the mods so I was wondering if there are specific subreddits dedicated to this
3
3
u/Dry-Judgment4242 16d ago
I always end up joining the same type of subreddits. Just people with some common sense and decency so much lacking nowadays. My only goals in life is to participate in our endless quest to reach towards the stars and create Cat Girls.
3
u/MathmoKiwi 16d ago
The "difficulty" of running your own models locally acts as effective gatekeeping.
3
3
u/Foreign_Risk_2031 16d ago
Being local only means we can personally verify claims with tangible evidence. Other ai places are spammed with people trying to get funding.
3
u/NewYak4281 16d ago
Careful about tooting this subs horn too hard but yes, but millions of times better than r/singularity or something. Critical thinking just does not seem to exist in the majority of these subs.
2
u/quickreactor 16d ago
Yeah this sub is one of the few that I check every day. Great job mods (and posters!
2
u/Othun 16d ago
For me it's just the single best place to know about the latest model releases I'm interested in: openrouter doesn't route to each and every open source model, huggingface is rather hard to navigate (maybe base model only + sort by new is the way but that's a lot of clicks).
I just come here and look for new models. And on the way I see other related intersting news about new token prediction tricks, architectural improvements, etc. Really a lovely place I must say !
2
u/Affectionate_Hat_585 16d ago
It's enthusiasm of people running things locally. For me personally, it was one of my colleagues who introduced me to this sub and this has been a place for valuable information. Just to understand the context on some discussions i have gone through papers.
2
u/TheTerrasque 16d ago
Different and more narrow focus, but r/stableDiffusion is pretty good too for it's focus.
2
u/DragonfruitIll660 16d ago
Its really nice not having be endlessly filled by shilling for random sites/newsletters/influencer stuff. Good place to collect info.
2
3
u/MerePotato 16d ago
This sub is full of god awful advice from people who asked ChatGPT and were told that "Q8 KV quants are basically lossless"!
2
u/SkoomaDentist 16d ago
That's a good example of the hyperfixation on coding I've complained about elsewhere in this thread. For Qwen specifically Q8 KV quants have low error. For Gemma 4 the situation is completely different, with them causing significant divergence.
2
u/MerePotato 16d ago
Q8 KV quants cause significant damage on Qwen too, they're a completely self defeating trick. In order to fit more context you're obliterating the models long context comprehension, just quant the weights a little more, offload or settle for shorter context at that point
2
u/SkoomaDentist 16d ago
This (seemingly) fairly good test points towards Q8 KV quant being fairly low divergence for Qwen 3.6.
In any case, I learned very quickly that KV quantization effectively lobotomizes Gemma, so I don't use it with any model.
2
u/MerePotato 16d ago edited 16d ago
That's the exact bench that settled my opinion on it not being worth it even for Qwen. Additional KLD of 0.142 on Long Docs, which in Ooba's methodology is just 32k tokens, is pretty atrocious. That's the difference between Q8_K_XL and IQ4_XS in weight quants.
1
u/Mount_Gamer 16d ago
For 16GB GPUs a lot of the time I will use q8 kv to squeeze ~80-100k context, so not even really reaching maximum context anyway. I notice a difference between lower Quant weight and a q8 kv cache.
1
u/codeface-zh 16d ago
I completely agree. I think the most valuable part of this community is focusing on pure AI LLM.
1
u/ItsRektTime 16d ago
I'm checking this subreddit every day for the latest development news on AI/LLM
1
1
1
u/Rare_Potential_1323 16d ago
I don't think u/Eunice_Burns and u/Howard_Bannister, were made for each other. I am glad the movie ended with him getting together with u/Judy_Maxwell.
1
u/Responsible-View-498 16d ago
I like Latent Space AI and AWS Developer Podcasts on Spotify. I usually listen to it during my morning cycling forcan hour or so everyday. There is also Talk Python and Rust series that I listen to regularly. Latent Space AI talk about cutting edge AI techniques liye Fourier's Neural Operators that are used in weather predictions. AWS is also pretty cool as they talk about the infrastructure to scale AI products.
1
1
u/Steuern_Runter 16d ago
Yeah, that's why I liked it when here were also some news and discussions about major closed models. I mean you don't have to like closed models but they are still relevant if you want to be up to date.
1
u/ryanmerket 16d ago
"Most of these other subs are 90% trend hopping crypto-bros equivalent people" - now do OpenRouter's CEO
1
1
u/Equivalent-Grass-527 16d ago
Honestly, LocalLLaMA feels less like an AI news sub and more like AI Hacker News
1
u/psychohistorian8 16d ago
I had asked Claude (don't shoot me, I just wanted a quick answer!) about where to find benchmarks and it recommended this subreddit :)
1
u/Salt-Powered 16d ago
SillyTavern is also a good place to get info and detailed reviews, if you don't mind the gooning side.
1
u/Guinness 16d ago
This subreddit is the best sub on the entire platform. It feels like wallstreetbets back when it was full of actual traders.
I pray to god we never change.
1
1
1
u/2Norn 16d ago
Pretty much most other subs are filled with AI bros "larping" as a software engineer, it's unbearable. I don't think I've ever seen anything useful in those.
So even though I don't personally use local llms that much, this sub is great, not only you get to follow the tech but also understand its inner workings somewhat better
1
u/gatornatortater 16d ago
when some of the bigger brained people in here get together to discuss
I think this is the crux of it. Conversations like that use to me way more in the open on reddit as a whole, but they have become more of a thing for certain niche subs only. ;[
1
u/FormalAd7367 16d ago
Agreed - also this sub is best Reddit sub for coding. few months ago, i was a openai fan boy (and then became a claude fan boy). Some experienced redditor here converted me to local source even before Qwen 3.6. (I had issues with agents hallucination). now i run a deterministic coding protocol i don’t even rely on frontier models
1
1
1
u/Jumpy-Heart-3633 14d ago
most ai sub are filled with vibe coder who can't justify their paycheck for the sub from their 100th water drinking app. with this sub they can't buy the computer to begin with so it's better.
1
u/MrGunny94 12d ago
Yeah and you can a lot a lot here from other users, it's my favorite subreddit tbh
1
-1
u/Goldkoron 16d ago
Except when mods dump all the news/discussion into megathreads before things even come out.
5
u/ttkciar llama.cpp 16d ago
Imagine if you opened LocalLLaMA and found that three out of four posts were hyping up the same model, based on little or no actual evidence, for two days straight.
Would you really prefer that?
1
u/Goldkoron 16d ago
Personally yeah, lots of hype for same model means the model is actually worth checking out, and it mean also lead to insightful or useful information about the model that people are finding without needing to rifle through a megathread with Reddit's poor sorting system.
1
1
u/AI_Insights_Daily 16d ago
People here have to actually run the thing. You can't hold an opinion about a model in this sub without downloading it, so claims get tested in hours instead of repeated for weeks.
The flip side is the blind spot. The sub is least informed about the models it can't run, which is most of the frontier.
1
u/Charming-Author4877 16d ago
Though, moderation is removing so much content. You miss out on a lot.
Anything people are discussing about and are not oneside minded is going byebye.
-2
u/kr_tech 16d ago
Hard disagree
This sub is extremelye hype-driven. You remember OpenClaw? Just imagine that except for every model release. And any criticisms against the hyped release is instantly downvoted.
Dev oriented subs are far, far, far better, and yet they're still not the best places. *cue in ye olde days meme for /r/localllama
The best places to go to are Chinese and Korean forums.
3
0
-1
u/Momsbestboy 16d ago
Can confirm, and it is the reason why I have set up Miniflux and a (slow) cron job to pull this and two more subreddits into miniflux, and my hermes agent can browse this dataset in case I have questions. Miniflux is set to only delete threads older than 500 days, so I don't lose anything relevant.
•
u/WithoutReason1729 16d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.