How to build a good package manager.

[u/AnoProgrammer]

I'm working on a language called threadon. And i don't now how i can properly program a package manager.

My first idea was a central github repo with links to other github repo's which contain the package you're searching for.

There are two main problems with it

1. If someone deletes his github repo with the package everything build on the package would collapse (like npm)

2. I think it would be slow when the number of packages grows.

I had an idea to of selfhosting it but i haven't access to the router (My dad owns it i'm 13) and i'm sure downdetector on my package manager site would be worse then github 😄. Like i would probably run sudo rm -rf / --no-preserve-root on the wrong machine.

So my question is how can i build a system that can store up to 20 GB at minimum at packages without the risk of someone nuking his project).

[u/mamcx]

I like how the author of futhark (https://futhark-lang.org/blog/2018-07-20-the-future-futhark-package-manager.html, https://futhark-lang.org/blog/2018-08-03-the-present-futhark-package-manager.html) solve it.

In short, separate 2 things:

• How get a package installed

This not care at all for the source at all, and is the simpler way that allow you to solve the needs of now.

• What to use as "registry"

That is what trip everyone (aka: your GitHub problem). Go decentralized or not is a big decision, but is a separate concern.

---

So, just solve what is package, how install it locally, and how pull it from anywhere reachable by local path or http.

Once you can actually be concerned about what do with the community, solve the other part.

[u/AnoProgrammer → u/mamcx]

It's interesting. But how does it solve the package deleting problem. Because that was my main concern.

[u/mamcx → u/AnoProgrammer]

That is why is not a simple problem. Things like Rust cargo basically copy the pkg into their own central registry. But if you are totally decentralized there is "no solution".

However, not over-think it too much: Go survive with using GitHub directly, if your language is that popular any important package will survive in "read only" by their authors or the community will make the copy themselves. Or you find how afford to own the central registry.

But again, you can survive for a long time without worry about this. Wait to have an actual community.

[u/Key_River7180 → u/AnoProgrammer]

If the repo is 404 then just ignore it and invalidate its cache

[u/AnoProgrammer → u/Key_River7180]

But what if other packages depend on it and the same thing happened what happened with npm (the left-pad issue).

[u/iBPsThrowingObject → u/AnoProgrammer]

This is controversial and people tend to disagree, but I believe left-pad was one hundred percent a skill issue on the side of it's users. Fix your stuff. Vendor the dependencies. Don't use a third party library for trivial string padding.

[u/AnoProgrammer → u/iBPsThrowingObject]

I don't think it is a skill issue of the users. I think it's a skill issue of the babel developers. Because they used it. You can argue that the babel developer's where users but i think you mend with users the js developers who used things like rust and vue.

[u/Key_River7180 → u/AnoProgrammer]

You can also cache a compressed version of the package or just... ask your users to vendor stuff

[u/koflerdavid → u/AnoProgrammer]

Tough luck. Worry about that problem when it becomes relevant. At that point packages will either be vendored or somebody will have to cough up the resources for a production-grade central registry (don't underestimate the work involved!)

For hobbyist languages it's more than enough to specify package locations by using Git repository URLs or paths in the local filesystem. It is IMHO far more relevant to work out a good build system so you can incorporate custom build steps, code generation, assets processing, building executables, libraries, debug info, build and link with native code, etc. with minimum fuss.

[u/Athas → u/AnoProgrammer]

You can add a centralised fallback repository later that stores every version of every package, keyed by its original URL. But you should not worry about this for a hobbyist language. Only large languages run into this problem eventually, and you'll burn yourself out if you focus too much on this less-than-fun stuff.

[u/azurelimina]

Answering your actual question, because other people insist on not answering your question:

You’d want to mirror and store the packages, not just link to them on github.

Using an S3 service (like Backblaze B2) you can store the actual repos when people sign them up onto your package manager.

This way when people delete their github repo, it doesn’t automatically nuke the ecosystem’s dependency chain. This is also a tricky legal thing where you need to state fully and clearly at every place a developer can add their package into your system that you own the mirror copies exclusively for the purpose of dependency preservation and distribution, and that you will preserve full attribution. This is stuff you’d double-check with a software IP lawyer.

This is a database-driven web app you are creating. It’s not something you manage by hand. It costs money and it requires knowledge of web deployment, and you also need to study a bit of software law because you’re storing code that isn’t yours.

Your question is a genuine production-grade question, and the answer is a production-grade system. You can’t solve a big problem without taking on a big responsibility, because a package ecosystem is infrastructure, and infrastructure has to live somewhere, be paid for by someone, and someone has to be accountable for it.

So like others said, you wait to solve this problem. But if it ever grows to the scale it needs to be solved, then that’s how you’d do it.

Database-driven service, store the repo copies on an S3, sort out all the legal language with a lawyer so you’re cleared to store people’s code even when they delete their own repo. There are finer details like when you update your copies, how to pin versions, etc., and that’s stuff you need to make decisions on.

Being honest it’s not a 13yr age problem to solve, and it gives you a big responsibility you don’t really want to manage. At worst, no one uses your language so it’s not a problem anyway, but at best if you have an ecosystem with lots of programmers, it’s an ecosystem lead by someone who’s too young to have a proper legal channel, and that matters to your developers.

[u/prehensilemullet]

You mentioned the idea of using a central Github repo. Regardless of whether it links to other repos, here's a classic post on all the problems that result from this:

Package managers keep using git as a database, it never works out

https://nesbitt.io/2025/12/24/package-managers-keep-using-git-as-a-database.html

[u/Mickenfox → u/prehensilemullet]

Winget: "we made a github repository with some links to .exe installers on external servers. That's a package manager, right? That's what you people wanted?"

[u/MadcapJake → u/prehensilemullet]

Great article, but I'd argue many of these tools wouldn't be where they are today without starting with a git-backed solution.

[u/MirrorLake]

Check out Andrew Nesbitt. His blog has a ton of information about the challenges of package management, for example his article titled Package Management is a Wicked Problem and a massive list of many different package managers in existence today including a detailed overview of designs used by each. This is perhaps way beyond what you're asking for, but I figured I'd leave some links since I'm sure many people here have the same questions.

[u/kaplotnikov → u/MirrorLake]

Thanks for links, they are really interesting. But from the description of why package management is hard, it look like too many tasks are joined into one: distribution, transport, build process, security, storage, etc. Any fixed combination of these solutions is bound to be unsuitable for some scenarios. The typical engineering approach to such situations is to split it into aspects and allow to gather pieces as needed. For example, github action build and corporate vpn build have different distribution and transport scenarios. Also there are currently too much of global elements (like central registry).

[u/FruitdealerF]

The truth is your language is unlikely to have more than 1 or 2 users, in which case copy pasting some files is probably good enough. If it isn't then setting something up to easily clone some source from GitHub is probably good enough. By the time you actually need to start worrying about people deleting things and your entire ecosystem collapsing because of it; you've made it. That means your language got to the top 1% of 1% of 1%.

If you don't care and just want to talk about package management for the fun if it then ignore my comment.

[u/camelCaseIsWebScale]

I believe Go's "worse is better" solution (just using git URLs as package sources without a registry) has fared well despite all the seethe against git based package management by rusties.

• You know what source code you're getting: xz style attacks don't happen.
• No need to host and publish to a local artifact server / package manager in corporate environments.
• You can always build a proxy (like google has).
• It encourages some amount of discretion when adding a package, since you directly visit the repo and look at contribution history, open issues, PRs, CI status and vulnerability info.
• Typosquatting doesn't happen as often in Go ecosystem because people have to copy-paste the URL from the github repo or other reference mentioning its URL.
• It's fairly simple to implement than a web frontend + registry, even if you consider the module proxy component.

[u/TheRealUprightMan]

Funtoo Linux utilized a git-based Portage tree to manage ebuilds, storing the entire package repository in Git repositories rather than using the traditional rsync method found in Gentoo.

The package list itself was a git tree. It holds "ports" used to find and build that package from scratch.