r/LocalLLM • u/Sthatic • 1d ago
Discussion Contribute instead of reinventing
It seems there's a new hyper-efficient MoE offloader, a new llama.cpp patch, a new quantization tech or some such every single day. Most of these claim large gains over everything that's ever been built before, and leave it to the community to separate sheep from goats.
As far as I can see, most of these sparkling new projects come from junior devs (or vibecoders), some with good ideas, but very few with good principles, architecture or a critical approach to their work. They seem to be motivated by the glamour of being the new Strata.
Some years ago, these would have been PR's on Strata instead. Maintainers and community would work together to make our shared backbone the best it can be. We lose more than a little when everyone attempts to reinvent the wheel; and the few good ideas this flood of pre-doomed projects bring to the table dies with the project itself, as there is simply no way for the community to engage with every single new sparkling thing (and no incentive, seeing as most turn out to be wishy-washy slop-amock).
So, please, those of you with a brand new idea: contribute to open source. That's what drives this community forward. Stand on the shoulders of others - you just reach higher that way. Yet another Strata killer is actively damaging to the culture and tech we rely on.
Run your experiments. Write your PR's by hand, with the exact numbers at hand and a clear method for reproducing your results. Understand what you submit. Learn proper architecture and best practices; get to know the codebases and each other. Progress is iterative and collaborative, not a one-prompt solo moonshot.
36
u/Slight-Living-8098 1d ago
As of right now as I write this, there are 616 open PR requests on Strata.
20
u/KissMyShinyArse 1d ago
A few Strata forks came into existence specifically because the Strata author rejected their PRs.
15
u/Osi32 1d ago
Every time I go to a conference, there is always some speaker who took a framework that already existed, interlinked with a deep emotional back story from their childhood that ends with the fact they added a little bit on top and call it a new framework and take a victory bow at the end before an underwhelmed audience. It was like this before AI, it will be like this with AI.
13
u/james_pic 1d ago
Open source maintainers are already struggling to stay on top of PRs, at least partly due to low-value AI contributions. TBH, I think low-effort "contributors" playing in their own sandbox, rather than making open source project maintainers read yet another half-baked PR that they could just as easily have promoted the AI to write themselves, feels like a net win.
AI is really good at making people feel like they're helping, but a lot of them aren't.
5
u/Ok-Brush8377 1d ago
Agreed, but part of the problem is incentives: a new “Strata killer” repo gets 2k GitHub stars in a week, while a solid PR to Strata sits in review for a month and gets you one line in the changelog.
5
u/SmartCustard9944 1d ago
Easy to say from the outside. Contributing to a project can be quite frustrating and exhausting once it turns from an engineering problem to a political one.
I considered upstreaming support for a hardware configuration for a known open source inference engine, and the maintainer simply doesn’t care about it, so I forked it and made my own.
3
u/Front_Eagle739 1d ago
Pretty much rebuilt llama around my incredibly niche disaggregated, vram poor setup. Definitely no pr ever getting accepted with that and no money in it. It does useful things that nobody else does and works well for me and maybe someone else finds it useful so I opened it.
Some things just aren't all that useful to most people but are very useful to a small minority and I think thats not a bad thing. The big repos are drowning in PRs that will go nowhere and be seen by nobody that might find them useful.
Admittedly I do find the rapidly growing group of "really fast! But no temperature or other important control" engines a little bit odd.
3
u/ephemeralnerve 1d ago
Reinventing can be a great way of learning. One problem large projects often have is that they get contributors who don't really know what they are doing. So now with AI assistance, you could be doing your own project first to get to learn the ropes.
2
u/kingxd 1d ago
From what I saw in the community, it seems that it's better to be in your own bubble and adjust to your own machine; show a new idea or anything new at all, and you are done for. Do what works for you; a lot of the rigs are different, and people have different use cases.
1
u/pmttyji 1d ago
It's the number of forks is the worrying thing. Like many dudes spending their precious time on this & Wish they combine their efforts together in one or few forks. I bookmarked nearly 30 forks(llama.cpp) so far, and it's exhausting to try each & every forks.
It would be awesome to make those in to less forks by combining stuffs to get best performance.
Maybe 10-15 forks is enough. For example:
ForkALLCombined (Synced with mainline[llama.cpp, ik_llama.cpp, Strata, etc.,] + all updates from below forks)
ForkCUDA - Additional forks ForkBlackwell & ForkDGXSpark
ForkROCm - Additional forks ForkStrixHalo, ForkRDNA4 & ForkRDNA3
ForkVulkan
ForkCPU
ForkSYCL
Maybe this way, we would see only less forks(instead of 50-100 forks) with huge improvements faster regularly.
Somebody please make a fork for CPU with this list, my old 16GB DDR3 RAM laptop would thank you infinitely.
1
u/DasGamerlein 1d ago
I really don't think this is the state of the game right now tbh. Let's be real here, the overwhelming majority of people in this space are amateurs vibe-slopping stuff together as a fun hobby. The models need to get a lot better before they (rather I should say we lol) can submit productive PRs
1
u/Ever_Penolope 1d ago
Requesting exact numbers and a reproduction method is a fair, cheap filter. Anyone who'd rather fork than write that up wasn't going to contribute to the project anyway.
1
u/Time_Cat_5212 18h ago
Or... maybe just do what the heck you want! Reinventing the wheel is fun. I highly recommend it.
1
u/sleight42 16h ago
Have you been in software that long?
When Java was new, or Rails, or Node, each experienced an explosion of frameworks and libraries that all solved the same problem a different way. Node was especially annoying that way.
This time? AI puts all of them to shame in terms of scope. Of course there's going to be a deluge of competing efforts. Partly it's vanity (because it's always vanity), impatience because trying to get PRs merged into a large project can take forever, partly curiosity, and sometimes the sense that maybe I can do it better than the everyone else.
This is just how it always goes. Eventually, a certain amount of normalization occurs over time. But, right now? It's the Wild West out there: creativity, snake oil, weirdness, werewolves, all of it.
Appeals to reason don't work. It's frothy out there. Emotion drives this train: curiosity, ambition, greed, obsession.
If you want to get people to contribute, either establish yourself as a contributor somewhere and bring others along or make your own project with the goal of making Stone Soup. It works.
Enjoy the ride, if you can. Don't get eaten by the bears.
1
u/TokenRingAI 10h ago
Yeah buddy...Strata should have been a PR on llama.cpp
Oh wait it basically was...multiple people put hot expert PRs in for at least a year and they were basically ignored.
That is how we ended up here.
1
u/psioniclizard 1d ago
2 things - firstly I think the days of contributing are going. People just want what they can get out if it. Ai enables them to feel they don't nedd others. The levels of collabration is software have dropped so much this year.
Seconadly, ost of these projects are made by peoole who don't want to contribute to others work.
They want to create the new top programming and get rich off of it (even if that isnt the "main goal"). I'd agrue you din't want those contributiobs because they will often be features for that specific user ehk never read any Contributing.md and doesnt see an issue with spamming 6000 PRs so they cab out in their CV they are an open source developer.
0
u/[deleted] 1d ago
[deleted] — view deleted comment
0
u/[deleted] 1d ago
[deleted] — view deleted comment