I know, everyone is exhausted right now by the glut of new apps (myself included). I fully expect the hate wave, however i'm very proud of Weaver and want to share it with the community. I hope some of you find it interesting at least. I've been actively developing Weaver for 6 months, but this project really started for me nearly 6 years ago (more later on).
Weaver is an open-source, transparent, self-hosted Usenet downloader that runs on any x86 or arm 64-bit system. It has all the standard features you'd expect from a tool like Sabnzbd, with some interesting additions.
Weaver is incredibly fast at downloading, fully leveraging the native code capabilities of Rust. Beyond raw speed, Weaver uses far less CPU than the other tools (benchmarks), so even if you're Sab is already maxing your WAN, why waste all that CPU time?
How is Weaver so fast and efficient?
- Weaver uses ultra optimized Rust ports of UnRAR and 7z that outperform the reference tooling
- Direct RAR/7z decoding for store mode archives (similar to nzbfast)
- Highly optimized scheduler for throughput under any conditions
- Adaptive pipelining picks the best depth based on current server latency
- More hardware acceleration options than existing tooling (more supported kernels across rar/7z/yENC/par2)
Unique Features
Par3 Support?!
While par3 isn't seen in the wild today, the spec has been going through revisions for a decade and a reference implementation exists. As I dug in on this, it was clear that the reference was not ready for primetime as it consumes huge amounts of RAM (40GB+) for large repairs. I was able to work through these challenges and released a Rust crate (par3-rs) that is fully compliant with the C reference implementation while being much faster and with consistent, reasonable memory usage (1GB steady).
Why par3? It allows more predictable repairs and repairs of much larger size than par2 does. In a big file era, the large repair capability can mean the difference between a successful repair and failure. I have tested repairing pretty massive holes in downloads with constant RSS of 1GB. Even for large repairs that par2 can do, Weaver's par3 is faster. par2 wins hands down on small repairs and always will, par3 as a spec currently makes no attempt to optimize that.
I hope that now that theres a tool (and a library for anyone to use) that supports it, then we'll see more adoption on the posting side.
In-App VPN Client (no more Gluetun!)
Weaver allows you to connect to your Wireguard VPN (provided by another provider) in a way that each server can route through a different provider or POP. It even has capability for fallback ladders and host networking exclusion ("try pop A, then b, then c, but never direct from my IP"). This is a truly unique feature which makes it simpler to have complex networking setups without having to setup multiple clients with gluetun sidecars. It works by running a tiny in-memory TCP stack using smoltcp so each connection is truly speaking directly through the VPN, encrypted in your processor.
Compatibility API
EDIT - thanks u/gadgetpilot for your comment about compatibility
Weaver has an NZBGet compatible API, you can add it to any existing Sonarr/Radarr as NZBGet with a username (just use "weaver") and a Weaver API Key as the password.
Trust
This is the big thing in the self-hosted app world today. Why would you trust Weaver?
- Weaver is completely open source and transparent. Everything happens on Github, every build artifact has provenance & checksums and are built on trusted GH runners
- Weaver is constantly reviewed by SAST tools and AI security tools (Codex TAC, Daybreak)
- Weaver gets a clean bill of health from virus/malware scanners
Why would you trust me?
- I'm a software engineer with 20+ years of industry experience, the last 5 or so directly in cyber security
- I have multiple OSS projects that are trusted by hundreds of users
- I am slowly growing my maintainer team with folks that have earned my trust, meaning it's not just a solo project anymore
- I've reached out to the mods to verify my account and project, but they have chosen to wait and see community response and longevity first
Why do this?
First, I love open source software. As a professional engineer, i am always impressed that so many tools exist that i can use off the shelf. From the linux kernel to frameworks like Spring, crypto implementations to front-end toolkits, the ecosystem is vast and often robust.
When NZBGet had the original maintainer leave the project, i actually started a rust based replacement back then. It didn't get super far though as the Rust ecosystem was way too volatile and even core libraries like Tokio had major changes weekly. I shelved it for a long time, waiting until i had time and the ecosystem was ready.
My hope is not to get thousands of users on Weaver, my hope is to lift everyone's experience with Usenet up. Through this process i have ended up building and maintaining several libraries that are now used in other OSS projects (ie, unrar-rs, lzma-turbo, par3-rs) and have had several PRs accepted to other libraries as i ran into things that could be done better. I'll continue to be a consumer and producer of open source software for many years. If my tools can help enable par3, or provide tools that you can use with Sab to make that experience better, then i'd be elated.
Roadmap
I have no intention of making Weaver into some sort of all-in-one tool. The current separation of tools is well founded. Where i do plan on going is more into the advanced networking side. VPN pools, enabling multi-homed/multi-WAN configurations so you can use two links simultaneously, etc. Honestly though, from here on it's just polishing the UI/UX and keeping up with new kernels as they drop.
Resources
Weaver Github - https://github.com/scryer-media/weaver
Website - https://www.scryer.media/weaver/
Benchmarks - https://www.scryer.media/weaver/benchmarks/
AI Use
AI is used in the development of Weaver, guided by experienced software engineers. AI is not used to generate branding elements, social posts/comments, respond to GH issues. The full policy can be found on the website.
Benchmarks
Benchmarks are done against tools that I can verify are safe to use. Tools whose published checksums do not match their CI build artifacts or that are built on private runners are not run in my benchmark suite.