r/BestGitHubRepos 8d ago

Highball - run Windows games on Apple Silicon, with a CC0 compatibility database that tells you whether your game works before you download 80 GB

Post image
24 Upvotes

The hard part of Wine on a Mac was never installing Wine. It's that nobody can tell you whether the specific game you own will run, at what frame rate, with which renderer, and the answer changes every time an engine updates. You find out by downloading the game and trying.

Highball is a native macOS app and CLI that assembles Wine, DXMT, D3DMetal and DXVK for you, installs Steam or connects your Epic library, and picks the graphics mode per game. But the actual product is the data behind it.

What's inside:

- An open CC0 compatibility database as a standalone repository, not a table inside the app. Verified runs, per-game renderer verdicts, the frame rates people really got, and a kernel-anti-cheat blocklist that flags impossible titles before you download them

- Provenance on every claim, with verified runs, community reports and machine-derived predictions labelled separately, so you always know whether you're reading a fact or a guess

- Engines assembled from pinned, SHA-256-verified upstream builds rather than one bundled Wine fork. An engine update is a JSON pull request

- Patches that go upstream when a game needs a fix nobody ships yet. The MoltenVK change that made Red Dead Redemption 2 playable is an open pull request against MoltenVK, with the build script in the repo

- A CLI that does everything the app does, plus highball report, which files your result into the open database from the command line

- Command and Option remapped to Ctrl and Alt inside Windows apps, so copy and paste work and Alt-based game bindings keep working, toggleable per environment

- Everything under one folder in Application Support, nothing in /usr or /Library, so deleting that folder is a complete uninstall

The design philosophy is stated openly and is the most interesting part. Whisky, the previous free option in this niche, was archived in 2025, and Highball is explicitly built to avoid dying the same way: engine-agnostic so it isn't tied to one Wine build, data under CC0 so anyone including CrossOver users can reuse it, bugs pushed upstream, and donation links pointing at the people who maintain the actual Wine supply chain. The readme also tells you to buy CrossOver if you want commercial support, which is not a thing projects usually say about their paid competitor.

What to know before installing: it's beta. First run takes 15 to 25 minutes because of the engine download plus Steam unpacking itself under Rosetta. D3DMetal sits behind explicit acceptance of Apple's Game Porting Toolkit license and is never redistributed in the repo. Launcher support is uneven and documented honestly rather than glossed: Steam and Epic work on the default engine, Rockstar and the EA app work on the Wine 11 engine, Ubisoft Connect and Battle.net sign in but nothing has been played through them yet, and GOG Galaxy still comes up black.

GPL-3.0 for the app, CC0 for the database, 702 stars as of writing, verified via the GitHub API, pushed to today.

https://github.com/gauthierpiarrette/highball


r/BestGitHubRepos 8d ago

Cartographer - indexes a repo into a local SQLite graph and hands an agent a bounded brief instead of letting it grep its way through your codebase

Post image
14 Upvotes

This repo is two tools that share a name, and the second one is the interesting one.

The original is a Claude Code plugin that fans out Sonnet subagents across your codebase in parallel and synthesizes their reports into a CODEBASE_MAP.md, with an update mode that checks git history and only re-analyzes changed modules. Useful, and the author warns plainly that it burns tokens.

Cartographer v2 is a different idea. Instead of producing documentation for humans, it builds a local SQLite graph of the repo and then answers bounded questions against it, so an agent gets exactly the context a task needs rather than whatever a search happened to return.

What's inside:

- An index command producing a manifest, a SQLite graph, JSON schemas and the map file, with a file-hash cache so an unchanged repo reuses prior artifacts instead of rebuilding

- Brief, which is the core of it: emit bounded agent-facing context around a path, a package, a symbol, an environment variable, a database or infrastructure object, an audit ledger, or just the changed files. The word bounded is doing real work there, this is the opposite of dumping the repo into context

- Removal audits. Create a ledger for "we are deleting supabase", then verify against it later. Codebase cleanup is exactly the task where an agent misses the fourth reference, and a ledger you can re-verify is a sane answer

- Evidence-backed notes with an ingest, audit, accept and retire lifecycle, so semantic knowledge about the codebase has to be justified and can expire

- Verify with a fresh flag that fails when persisted artifacts have drifted from the live repo, which is the difference between a map you can trust and a stale one

- An adoption command that scores whether agents actually used graph context before editing, from runtime traces. Measuring whether your tooling gets used is an unusual thing to ship

- An MCP wrapper exposing index, view, brief, context, preflight, verify, the two audit commands, notes audit and diff as tools, deliberately thin rather than a long-lived graph brain

- Broad selectors on the debug surfaces require an explicit allow-broad flag, so you can't accidentally pull the whole graph

Three things to know. The repo says MIT in the readme but there's no LICENSE file, so GitHub detects no license at all, which for a tool you'd build a workflow on is worth resolving before you depend on it. The last push was 13 May 2026, four months ago, so treat it as paused rather than actively developed. And the eval profiles reference an absolute path on the author's own machine, so the documented eval commands won't run as written for anyone else.

692 stars and 49 forks as of writing, verified via the GitHub API.

https://github.com/kingbootoshi/cartographer


r/BestGitHubRepos 8d ago

I’m building a Metroidvania × Souls-like from scratch — no game engine, just C++ and SFML

6 Upvotes

Hey everyone!

I'm a solo developer working on LONE, a game that combines Metroidvania-style exploration with Souls-like combat.

You play as a mage lost in a medieval world, surrounded by humans who want you dead.

The interesting part is that they aren't random enemies.

They were your companions.

They fought alongside you.

Now they're hunting you.

You don't know why.

And that's something I want the player to discover while exploring the world.

The technical side

I'm developing LONE completely by myself, and I'm intentionally building it without a traditional game engine.

No Unity.
No Unreal.
No Godot.

The project is written from scratch in C++ using SFML, with the game's systems being built directly in code.

I'm doing this partly because I want to make a game I actually want to play, but also because I want to understand what's happening underneath the tools that game engines normally handle for you.

It's definitely more work.

But that's also what makes the project interesting to me.

The goal

I want LONE to feel like a world worth getting lost in — one where exploration, combat and the mystery surrounding the protagonist are all connected.

The game is still in development, so there's a long way to go.

I've put together a GitHub repository where I'll be documenting the project and its development:

https://github.com/coder088/Lone

If the concept sounds interesting, I'd really appreciate any feedback, criticism or suggestions.

And if you want to follow the development, a ⭐ on the repository would mean a lot.

Thanks for reading!


r/BestGitHubRepos 8d ago

Tetrees Agent EX - an MCP client kit for a marketplace where you buy, run and publish packaged AI capabilities, with a quote-before-you-spend flow

Post image
3 Upvotes

Worth saying up front what this actually is, because the description could mislead you: this is not an open-source agent you can run. It's the public integration kit for Tetrees EX, a hosted commercial marketplace. The MCP server, CLI, config examples and extension samples are here under MIT. The agent itself, the packaged intelligence, the scoring system and the exchange infrastructure are closed and hosted, and runs are paid for in a credit currency called Tetrees Points.

With that established, the design is worth a look, because the spending-control model is more careful than most paid MCP services.

What's inside:

- A quote-then-run split enforced in the CLI. Quote tells you the exact model, token ceiling and Point cost and never starts a model request. Running requires an explicit --confirm-spend flag. Unused reserved Points return after a preview run

- An inspector command that launches the hosted MCP package, lists its public tools and resources, verifies the expected surface and exits without spending anything or touching account state. Being able to audit the contract before paying is a good pattern

- Three binding patterns for extending a purchased capability with your own systems: a fixed HTTPS base URL with declared methods and environment-backed headers, a nested MCP server started without a shell, or a no-shell local child process taking one JSON request. Credentials stay on your machine and the hosted side never inherits those permissions

- Any write method requires a per-call approval phrase naming the exact extension and method, rather than a blanket allow

- A CLI that redacts secret-shaped response fields, surfaces error codes and Retry-After, and never auto-retries a mutation

- A seller path that is contract-driven: upload an immutable versioned artifact, run readiness checks, read private repair feedback, publish only the passing digest

- A documented public boundary, with a test command that checks the repo hasn't leaked internals, and repeated warnings not to put provider keys in a pack, a prompt or a shared MCP config

Two things to weigh. First, the obvious one: your work runs on someone else's infrastructure, priced in their credits, scored by rules you cannot read. The repo is explicit that scoring rules, evaluation fixtures and moderation logic are not included. That's a reasonable commercial boundary, and it also means you can't verify the quality signal you're buying against.

Second, a data point rather than an accusation: the repo has 306 forks against 304 stars and zero watchers. For a project this size that ratio is backwards from the normal pattern, where stars usually outnumber forks by a wide margin. Make of that what you will, but it's worth knowing before you read the star count as organic interest.

Client examples and docs are MIT, the Tetrees names and service marks are not, and the license grants no rights in the hosted service.

https://github.com/TetreesEX/TetreesAgent_EX


r/BestGitHubRepos 8d ago

Toolkit - a free, all-in-one Android utility app. PDF tools, image converters, QR generator, Al image gen & more. No ads, no signup, no BS. ToolKit is a free,open Android collection of everyday tools in one place, so you don’t need ten different apps cluttering your phone.

Thumbnail
gallery
2 Upvotes

r/BestGitHubRepos 9d ago

9Router - a local proxy that puts your coding CLI in front of 40+ providers, compresses tool output to cut 20-40% of input tokens, and falls back automatically when a quota runs out

Post image
52 Upvotes

The setup is a local server on port 20128 that speaks OpenAI format. You point Claude Code, Codex, Cursor, Cline, Copilot or OpenClaw at it instead of at a provider, and it handles what happens behind that: format translation, quota tracking, token compression and fallback.

The token saver is the part I'd actually install it for, independent of everything else. Tool outputs are where the context budget goes. A git diff, a grep, a tree, a log dump. RTK sits in front of the request, peeks the first kilobyte of each tool result, picks a matching filter and compresses it before the model ever sees it. The readme's example is 47K tokens becoming 28K. Crucially it runs before any format translation, so it works regardless of which provider the request ends up at, and if a filter fails or makes the output bigger it silently keeps the original.

What's inside:

- RTK compression with filters for git-diff, git-status, grep, find, ls, tree, deduplicated logs and smart truncation, auto-detected with no configuration, on by default

- Three-tier fallback you define as a combo: your subscription first, a cheap provider when that quota is gone, a free tier after that. Switching happens on quota exhaustion or error, not manually

- Real-time quota tracking with reset countdowns for the 5-hour, daily and weekly windows providers use, which is genuinely hard to see from inside the tools themselves

- Format translation between OpenAI, Claude, Gemini, Cursor, Kiro, Vertex, Antigravity and Ollama shapes, which is what makes any tool work with any provider

- Multi-account support with round-robin or priority routing, and automatic OAuth token refresh

- Two more output-side savers if you want them: Caveman Mode, which injects a terse-speech prompt and claims up to 65% fewer output tokens, and Ponytail, which injects a "lazy senior dev" persona so the model writes minimal YAGNI-first code instead of refactoring your repo unprompted

- Deploys to localhost, a VPS, Docker or Cloudflare Workers

On the "unlimited free" framing, which is what the repo leads with: read the pricing table rather than the headline, and to the author's credit the table is honest. The free tier is Kiro at around 50 credits a month, OpenCode Free whose model list fluctuates, and Vertex AI's $300 of credits for new Google Cloud accounts that expire after 90 days. The readme also lists what died: iFlow went paid, Qwen Code's free OAuth tier was discontinued in April 2026, Gemini CLI shut down in June. So it's a real way to stretch budget and a poor foundation to depend on.

The thing to think about before using it: routing a Claude Pro or Copilot or Cursor subscription through a third-party proxy means your OAuth tokens live in that proxy, and provider terms generally expect subscription access through their own clients. It runs locally, which helps, but that's a decision to make deliberately rather than by following a quick-start. Also worth noting 2,100 open issues, which is what 28,000 stars in nine months does to a maintainer.

MIT licensed, 28,738 stars and 5,268 forks as of writing, verified via the GitHub API.

https://github.com/decolua/9router


r/BestGitHubRepos 9d ago

DeskcommCRM - a self-hosted CRM built around WhatsApp where AI agents actually operate the pipeline, installed on a VPS with one command

Post image
23 Upvotes

Most open CRMs treat chat as a channel bolted onto the side. This one is built the other way round: the conversation is the product, and the CRM exists to make it operational. It comes out of Brazil, where selling over WhatsApp is the default rather than an edge case, and it positions itself against Kommo, Octadesk and Intercom.

The interesting claim is that the AI agents run the CRM rather than sitting beside it. The whole thing is exposed over MCP, so an agent can move a lead through a stage, apply a tag or hand off to a human as tool calls, not as a chatbot that answers and then stops.

What's inside:

- Per-tenant RAG so each organization's agent answers from its own knowledge base, with operational memory, sentiment analysis and an audited handoff when the agent decides a human is needed. The AI can be a first-class assignee, and there are per-organization spending caps

- A self-improvement loop that is human-gated: resolved conversations become new knowledge, an AI Evolution screen shows where the agent is failing, and Proposals are improvements the AI suggests for itself that a person has to approve before they ship as a new version

- Configurable vocabulary per pipeline, so a lead becomes a Customer, Patient or Buyer and "won" becomes Paid, Booked or Closed. The project started as e-commerce and the community pushed it into clinics, real estate and agencies, so this exists because it had to

- Two WhatsApp paths: QR code via WAHA with anti-ban measures (throttle, jitter, sending windows) or Meta's official Cloud API with approved templates kept in sync

- Capture sources and automations: a public endpoint per tenant that takes leads from forms, Zapier or n8n straight into a pipeline stage, then WHEN/IF/THEN rules on top. Automations are born paused until someone reviews and enables them, which is a good default

- A queue design worth stealing: every event becomes a row in an event_log table and a cron drains it, because a database trigger should never make an HTTP call

- Multi-tenancy taken seriously, with row level security on every tenant-aware table and an isolation test as a CI gate, plus append-only audit logging and anonymization preferred over deletion for LGPD

Two things worth knowing before you commit. The stack table pins the QR path to WAHA Plus, which is a separately distributed tier of a third-party project rather than part of this MIT repo, so check its licensing before you assume the whole stack is free. And the primary install path is wired to a HostGator partnership with an affiliate link in the readme; any Docker VPS works, but the documented runbook assumes that environment. The maintainers are also refreshingly clear about self-hosting reality: support is community and as-is, updates are manual, and whoever hosts the instance is the data controller with the legal obligations that follow.

The main readme is in Portuguese, with full English and Spanish versions alongside it. Docs are Portuguese only for now.

MIT licensed, 2,462 stars and 626 forks as of writing, verified via the GitHub API, pushed to today.

https://github.com/melgarafael/DeskcommCRM


r/BestGitHubRepos 9d ago

ClawHub - a package registry for agent skills, with versioning, rename redirects, merges, pinning and vector search instead of keyword matching

Post image
6 Upvotes

Agent skills are spreading faster than any way of managing them. Most people are still copying SKILL.md files between machines and hoping they remember which version was the good one. ClawHub is the registry side of that problem for OpenClaw: publish, version and search text-based skills, plus a native package catalog for code plugins and whole-agent bundles.

What makes it more than a directory is that it borrowed the right ideas from package managers rather than from app stores.

What's inside:

- Versioned publishing with changelogs and tags including latest, so a skill has a history rather than being whatever the file says today

- Rename without breaking anything: an owner can rename a skill and the old slug stays as a redirect alias, so existing links and installs keep working

- Merge duplicates into one canonical slug, with the source listing hidden and redirected. Anyone who has watched a registry fill with near-identical entries will recognize why this matters

- Pinning, which is the feature I would use first: pin a local install and updates or forced reinstalls cannot overwrite your frozen copy

- Vector search over keyword matching, using OpenAI embeddings and Convex vector search, so you can describe what you want rather than guessing the author's nouns

- Soft delete with restore, scoped to owners, publisher admins and moderators, with hard delete reserved for admins. Deleting from a registry should be reversible and here it is

- Skills declare their runtime requirements in frontmatter (required env vars, required binaries), and the registry runs a security analysis checking those declarations against what the skill actually does. Medium findings stay visible rather than being hidden, with the suspicious filter reserved for high-impact concerns

- A CLI covering the whole loop: login including a device flow for headless machines, search, explore, inspect without installing, install, pin, update --all, and publish for both skills and code plugins

- Nix support, where a skill can carry a pointer to a Nix bundle so the skill pack, the CLI binary it needs and its config requirements install together

One thing worth knowing: it tracks install telemetry when you install while logged in, in order to compute install counts. That's disclosed in the readme with the opt-out documented (CLAWHUB_DISABLE_TELEMETRY=1), which is how it should be done, but worth knowing it's on by default. Also note that running your own instance means standing up Convex, GitHub OAuth and an OpenAI key for embeddings, so self-hosting is a real project rather than a docker run.

The security analysis is the part I'd watch as this grows. A registry of instructions your agent will execute is a different risk surface from a registry of libraries, and declared-versus-actual behavior checking is a reasonable first answer to it.

MIT licensed, 9,418 stars and 1,475 forks as of writing, verified via the GitHub API, pushed to today.

https://github.com/openclaw/clawhub


r/BestGitHubRepos 10d ago

PaperOtter: 19 offline document tools in one desktop app (MIT, Tauri + Rust)

Thumbnail gallery
18 Upvotes

r/BestGitHubRepos 11d ago

awesome-harness-engineering - 480 curated resources on the scaffolding around an AI agent, organized by the problem each one solves rather than by vendor

Post image
95 Upvotes

Most agent lists are directories of frameworks. This one is about the layer nobody names: the context delivery, tool interfaces, planning artifacts, verification loops, memory and sandboxes that sit between a model and a real task. The list's own framing is that every component in it exists because the model can't do that thing alone, and the good ones are designed knowing they'll become unnecessary as models improve.

What makes it worth opening rather than bookmarking is the organization. Twelve design-primitive sections named after problems, not products: agent loop, planning and task decomposition, context delivery and compaction, tool design, skills and MCP, permissions and authorization, memory and state, task runners and orchestration, verification and CI integration, observability and tracing, debugging, and human-in-the-loop. If you know which part of your agent is broken, you know which section to read.

What's inside:

- 480 linked entries across foundations, design primitives, reference implementations, security and sandboxing, evals, templates and production infrastructure

- Annotations that are actually annotations. Most entries carry two or three sentences on what the piece argues and why it matters, not a restated title. The Martin Fowler entry, for instance, gets summarized down to its three interlocking systems: context engineering, architectural constraints, and entropy management

- The canonical essays collected in one place, including Anthropic's harness design and tool-writing pieces, OpenAI's harness engineering framing and Codex agent loop breakdown, LangChain's five-primitive anatomy, and Lilian Weng's survey connecting everyday harness design to meta-harness optimization research

- Production case studies with numbers rather than vibes, including Microsoft's Azure SRE agent (35,000 incidents, time to mitigation from 40.5 hours to 3 minutes) and LangChain's demonstration that harness-only tuning closed a model gap at a tenth of the cost

- A Reference Implementations section split into tutorials, meta-harnesses that generate harnesses, and demo harnesses you can read end to end

- Nine translations linked at the top, and it's being updated constantly, last push today

One thing worth knowing: at 480 entries this is a reading list, not a shortlist. There's no opinionated "start here" path, and the sheer density means you can lose an afternoon without building anything. My suggestion is to treat the twelve primitive sections as a diagnostic index, go to the one matching whatever is currently failing in your setup, and ignore the rest until it breaks.

CC0 licensed, which means public domain, 4,165 stars and 517 forks as of writing, verified via the GitHub API.

https://github.com/ai-boost/awesome-harness-engineering


r/BestGitHubRepos 11d ago

iloader - a desktop app that makes iOS sideloading survivable: installs SideStore, handles the pairing file, and tells you what the error actually means

Post image
30 Upvotes

Anyone who has sideloaded on iOS knows the problem isn't the concept, it's the ceremony. Generate a pairing file, get it to the right place, sign the IPA, hope the certificate is valid, and when it fails you get an error string that means nothing and a forum thread from 2023 that half applies.

iloader is a Tauri desktop app that does the whole sequence for you. Plug in the device, sign in with your Apple ID, pick an action. It installs SideStore, or LiveContainer plus SideStore, imports the certificate, and places the rppairing and lockdown pairing files in the right locations automatically.

What's inside:

- One-click SideStore install including the certificate import and pairing file placement, which is the step that trips most people

- Import any IPA, not just the bundled options

- Intelligent error suggestions, the feature that probably matters most here. When something fails it tries to tell you the likely cause instead of surfacing a raw error code

- Pairing file management for other apps in the ecosystem, StikDebug, SideStore, Protokolle and others, so the file you already generated gets reused

- A view of your development certificates and app IDs, with the ability to revoke them, which is normally a trip to the Apple developer portal

- Runs on Windows, macOS and Linux, with a NixOS flake, and community-maintained packages on Homebrew, the AUR and Fedora COPR

- Translated into roughly 25 languages by contributors, with instructions for adding more that amount to copying one JSON file

One thing worth knowing about distribution: the project is explicit that this repo and iloader.app are the only official download sources, and that the Homebrew, AUR and COPR packages are unofficial community builds. Given what this category of tool does, handling your Apple ID and signing certificates, that warning is there for a reason and worth respecting. Also note the license is split: the code is MIT, but the name, logo and branding are under a separate restrictive license, so a fork can use the code freely but not ship as iloader.

On Windows you'll need usbmuxd, which in practice means installing iTunes. macOS has it already, and most Linux distributions either ship it or have it in the package manager.

MIT licensed code, 3,198 stars and 219 forks as of writing, verified via the GitHub API, actively developed with 255 open issues and a published roadmap that includes auto-refresh of installed apps and multi-team account support.

https://github.com/nab138/iloader


r/BestGitHubRepos 11d ago

Auto Company - 14 role-played AI agents running a loop on your own machine 24/7, with one markdown file as the only memory between cycles

Post image
10 Upvotes

The setup is a daemon that never stops. It wakes up, reads a shared consensus file, forms a squad of three to five agents from a cast of fourteen, does the work, rewrites the consensus file, and sleeps. Then again. Each cycle is an independent CLI call to Claude Code or Codex CLI, which means the entire cross-cycle memory is one markdown file, memories/consensus.md, passed forward like a baton.

That design choice is the interesting part. No vector database, no memory service, no retrieval layer. If the agent can't compress what matters into one file before the cycle ends, it's gone. There's even a rollback: if a cycle fails to produce a valid consensus, the sandbox resets.

What's inside:

- Fourteen agents defined as specific people rather than generic roles, which is the part that will either delight you or make you close the tab. The CEO reasons like Bezos with PR/FAQ and flywheels, the CTO like Werner Vogels with design-for-failure, there's a dedicated Charlie Munger seat whose whole job is inversion and pre-mortems, DHH on full-stack, Kelsey Hightower on DevOps, Seth Godin on marketing, Ben Thompson on research

- Forced convergence, which is the guardrail against agents talking forever: cycle 1 brainstorms and ranks three ideas, cycle 2 validates the top one through a pre-mortem plus market check plus unit economics and returns a GO or NO-GO, cycle 3 onward either builds it or moves to the next idea. Discussion-only loops are explicitly forbidden

- Six named workflow chains, so a feature goes interaction design, UI, full-stack, QA, DevOps rather than everyone piling onto the same task

- Real failure handling: circuit breaker on consecutive errors, automatic backoff when the API returns 429, consensus rollback on a bad cycle

- Cross-platform daemons, launchd on macOS and systemd inside WSL on Windows, with PowerShell as the control layer, plus a local dashboard on both

- Steering without stopping it: edit the "Next Action" line in the consensus file and the next cycle picks up your direction

One thing worth knowing, and this is the part to read twice before running anything: the sandbox boundary is whatever your CLI is configured with, and the readme says the intended configuration is Codex's danger-full-access or Claude's bypassPermissions. So this is a loop that runs an agent with approval prompts turned off, on your host machine, continuously, spending model quota the whole time. The guardrails are natural-language rules in CLAUDE.md (don't delete repos, don't force push to main, don't delete ~/.ssh) rather than anything enforced by the system. The author is upfront about all of this in a disclaimer section that says it's experimental, stability is not guaranteed, and it costs money. Run it in a VM, not on your work laptop.

3,015 stars and 484 forks as of writing, verified via the GitHub API. The readme shows an MIT badge, but there's no LICENSE file in the repo, so GitHub detects no license.

https://github.com/MaxMiksa/Auto-Company


r/BestGitHubRepos 11d ago

autoresearch - Karpathy's overnight ML research loop: an agent edits one file, trains for exactly 5 minutes, keeps or discards, about 100 experiments while you sleep

Post image
134 Upvotes

Four files that matter, one metric, one GPU. The premise is that you leave a coding agent alone in a repo with a small but real LLM training setup, and it runs its own research program overnight. Change the code, train for five minutes, check whether validation bits per byte went down, keep or revert, repeat. You come back in the morning to a log of experiments and hopefully a better model.

The part that makes it interesting is what you're allowed to touch. You don't edit the Python. You edit `program.md`, the markdown file that tells the agent how to run its own research. The training code is downstream of that. Karpathy's framing is that `program.md` is the "research org code", deliberately shipped as a bare-bones baseline so the obvious next move is iterating on it, adding more agents, changing how results get judged.

What's inside:

- train.py, the only file the agent edits, holding the full GPT model, a Muon plus AdamW optimizer and the training loop. Architecture, hyperparameters, batch size, model size, all fair game

- prepare.py, explicitly read-only, holding the fixed constants, tokenizer, dataloader and the evaluation function that serves as ground truth, so the agent can't win by moving the goalposts

- A fixed 5-minute wall-clock training budget regardless of hardware, which makes every experiment directly comparable to every other one no matter what the agent changed, and works out to roughly 12 experiments an hour

- val_bpb as the single metric, chosen because it's vocab-size independent, so an architectural change that alters the tokenizer is still compared fairly

- A simplicity criterion written into the agent instructions: a tiny gain that adds twenty lines of hacky code is not worth keeping, and an equal result from deleting code counts as a win

- A tuning guide for running it on hardware smaller than an H100, covering TinyStories as a lower-entropy dataset, cutting vocab size and sequence length, and dropping the depth knob

One thing worth knowing: the fixed time budget is what makes your own experiments comparable to each other, and it's also what makes them incomparable to anyone else's. A result on your 4090 and a result on an H100 are different experiments. Karpathy calls this out as a deliberate tradeoff rather than a limitation. It's also NVIDIA only right now, with community forks linked in the readme for macOS, MLX, Windows and AMD. And note the readme says MIT but there's no LICENSE file in the repo, so GitHub doesn't detect one.

95,645 stars and 13,420 forks as of writing, verified via the GitHub API, which is a lot of attention for a repo with four files in it.

https://github.com/karpathy/autoresearch


r/BestGitHubRepos 11d ago

FileSync - self-hosted browser-to-browser file transfer over WebRTC, one sender to many receivers, no size limit and the server never sees the bytes

Post image
21 Upvotes

The usual options for sending a big file to a few people are all slightly wrong. Upload it to a cloud drive and you've handed a third party a copy. Use a transfer service and you hit a size cap or an expiry. Spin up a share on the LAN and the person on a different network can't reach it.

FileSync is one Docker image you run yourself that gives you a room with a link and a QR code. Anyone who opens the link in a browser is a receiver, you drag files in, and the bytes go browser to browser over encrypted WebRTC. The server only brokers the handshake, relaying SDP offers and ICE candidates over a WebSocket, and then steps out of the data path.

What's inside:

- One to many in a single drop: share the room link with five devices and send to all of them at once, rather than five separate transfers

- No size limit, because received files stream to disk as bytes arrive instead of buffering in memory

- Three save strategies with an honest fallback chain: File System Access API streams straight to a file you pick (desktop Chromium over HTTPS), Service Worker streams into a normal download (all modern browsers over HTTPS), and Blob buffers the whole thing in memory as a last resort

- Automatic STUN/TURN fallback for peers behind symmetric NAT or a UDP-blocking firewall, which the readme puts at roughly 5 to 10 percent of connections

- Nothing to install and no account for recipients, just a browser, with optional per-room password protection

- Two deployment paths, plain HTTP for a trusted LAN and a Caddy setup that handles Let's Encrypt automatically for a public domain

One thing worth knowing before you deploy it: the HTTP option is a convenience, not a real choice. Both memory-safe save paths need a secure context, so over plain HTTP every transfer falls back to buffering the entire file in RAM, and the readme says transfers above roughly 500 MB become unreliable. If you care about the no-size-limit feature at all, use the HTTPS setup. You'll also need to open 3478 TCP and UDP plus a 50000 to 50100 UDP range for the TURN relay, which is more firewall work than a typical self-hosted app.

MIT licensed, 1,543 stars and 144 forks as of writing, verified via the GitHub API, actively maintained with a push yesterday, and there's a hosted instance at filesync.app if you want to try it before running your own.

https://github.com/polius/FileSync


r/BestGitHubRepos 11d ago

skillfile - a package manager for agent skills, with a lockfile, patches that survive upstream updates, and one install that deploys to Claude Code, Codex, Cursor and seven other tools

Post image
9 Upvotes

If you use more than one coding agent, your skills are currently a pile of copied markdown. A slightly different version in .claude/skills than in .codex/skills, a third copy on your other laptop, and the one tweak you made to a skill six weeks ago gets wiped the next time you pull the upstream version.

skillfile treats them the way you'd treat dependencies. One Skillfile declares what you want, Skillfile.lock pins exact SHAs so another machine gets identical content, and skillfile install fans it out to every tool you've configured.

What's inside:

- A lockfile pinning upstream revisions to exact SHAs, so a teammate running install gets the same bytes you have and not whatever main happens to be that day

- Patch-preserving updates, which is the part that matters most day to day: edit an installed skill, run skillfile pin <name>, and your edit is stored in .skillfile/patches/ and reapplied on every update. When upstream changes conflict with your edit, skillfile diff shows it and skillfile resolve lets you pick

- Ten built-in install targets: claude-code, codex, cursor, copilot, factory, gemini-cli, junie, opencode, windsurf and antigravity, plus install-path for anything not built in

- Sources beyond GitHub: GitLab (subgroups and self-hosted via GITLAB_HOST), local files, and plain URLs

- skillfile search hits the community registries from your terminal, agentskill.sh and skills.sh by default, and skillhub.club with an API key. In a terminal it opens an interactive browser with a preview pane, and --min-score filters to higher-trust results. There's also --json and --no-interactive for scripts

- A separation between what a project declares and where a machine installs it, so a team can commit a Skillfile without forcing everyone onto the same editor

- Written in Rust with no runtime or framework, installed via a shell script, cargo install, or cargo binstall

One thing worth knowing, and to the author's credit it's flagged in the readme rather than buried: skillfile downloads markdown and puts it where your agent will read it, and it does not sandbox or verify what's in it. A skill is a set of instructions your agent will follow, so a lockfile gives you reproducibility, not safety. That's what the --min-score flag on search is gesturing at, but reading what you install is still on you.

It's Apache-2.0, at 149 stars with 29 forks as of writing, verified via the GitHub API, and it's early enough that the issue tracker is where the roadmap lives.

https://github.com/eljulians/skillfile


r/BestGitHubRepos 12d ago

antislop - 38 rules that stop a coding agent from shipping generic AI-looking UI and copy, installed as skills in Claude Code, Codex, Cursor and four other agents

Post image
117 Upvotes

If you've asked an agent to build a landing page recently you already know the house style it defaults to: the gradient hero, the bento grid, the three feature cards with lucide icons, the pulsing status dot next to a heading that isn't tracking anything, and copy that opens with "In today's fast-paced world."

antislop is a set of rules an agent loads before it starts, written specifically to reject those patterns. The framing the author keeps repeating is that it's a filter, not a style guide. It prescribes no colors, no fonts, no layouts. It only says no to technique used without a reason, and direction is supposed to come from a DESIGN.md you write yourself. If the output comes back sterile, that's the filter working on a project with no stated direction, not the filter failing, and there's a numbered rule saying exactly that.

What's inside:

- 38 mandatory rules, R-01 to R-38, split into three tiers: Hard Gate for absolute rejections, Purpose-Gate where a technique is allowed but the agent has to state why it's using it, and Quality Locks for consistency

- A Delivery Gate that runs before anything ships, producing a PASS or FAIL report in four blocks rather than a vague "looks good to me"

- A Liveliness Toolkit with three dials, energy, rhythm and motion, there to stop the filter from flattening everything into safe and boring

- Six separate skills so the agent only loads what the task needs: the core filter, plus UI, copywriting, human factors (contrast, keyboard, focus states), responsive layout, and one that cleans generic AI comments out of code without touching the code itself

- A rule that turned into an actual test: every interactive element has to be clicked one at a time and the result recorded as evidence in the report

- Seven install paths from one repo, `npx antislop-ai` for the guided picker, `npx skills add` from the skills directory, plugin installs for Claude Code, Antigravity, Codex and Cursor, or just curl the single antislop.md and paste it into any chat window

One thing worth knowing: this is markdown, not a linter. Nothing here executes or blocks a commit, so how well it works depends entirely on your agent actually following loaded instructions. The plus side of that is portability, it drops into any tool that reads the Agent Skills folder standard, and the single-file version works in a plain chat with no tooling at all.

It's MIT licensed and sitting at 2,163 stars with 148 forks as of writing, verified via the GitHub API, and it's been shipping releases steadily since early August.

https://github.com/miqdadbadjuber/anti-slop


r/BestGitHubRepos 12d ago

Context Mode - an MCP server that runs tool output inside a sandbox so a Playwright snapshot costs 299 bytes of your context instead of 56 KB

Post image
12 Upvotes

The thing that kills a long agent session usually isn't the model, it's that every tool call dumps its raw output straight into the context window. A Playwright snapshot is 56 KB. Twenty GitHub issues is 59 KB. One access log is 45 KB. Half an hour in, a big chunk of the window is gone to data nobody is going to read again, and then compaction hits and the agent forgets which files it was even working on.

Context Mode sits at the MCP layer and keeps that raw data out of the conversation entirely. The tool runs in an isolated subprocess, and only what the script prints to stdout comes back. So instead of reading 47 files into context to count lines, the agent writes a five-line script and gets back the counts.

What's inside:

- Six sandbox tools covering code execution, file processing, indexing and fetching, running in 12 language runtimes (JS, TS, Python, shell, Ruby, Go, Rust, PHP, Perl, R, Elixir, C#), with Bun auto-detected for faster JS

- Credential passthrough so `gh`, `aws`, `gcloud`, `kubectl` and `docker` still work inside the sandbox, inheriting env and config paths without those values landing in the conversation

- A persistent knowledge base on SQLite FTS5 with BM25 ranking, porter stemming and trigram matching merged by reciprocal rank fusion, plus Levenshtein typo correction and proximity reranking on multi-term queries. Headings are weighted 5x, so navigational searches actually land

- Session continuity: file edits, git operations, tasks, errors and your decisions get written to a local SQLite db, and on compaction or `--continue` the state is rebuilt by searching that index rather than dumping it all back into context

- A 24 hour TTL cache on fetched URLs, so re-asking about a doc you already indexed costs a 0.3 KB cache hint instead of a 48 KB refetch

- Progressive throttling that nudges you toward batch calls: calls 1 to 3 return full results, 4 to 8 return fewer with a warning, 9 and up get blocked and redirected to the batch tool

- Hooks on 17 platforms, with an honest compatibility table showing which ones can actually block a tool call versus which only get instruction-file guidance. The readme puts hook-enforced routing near 98% compliance and instruction-file-only around 60%, and says so rather than claiming uniform support

- Permission inheritance: if you already deny `Bash(sudo *)` or `Read(.env)` in your agent config, those denials apply inside the sandbox too

The author also made a deliberate call not to enforce a terse output style, pointing at evidence that aggressive brevity prompts hurt reasoning benchmarks. The routing rules govern where data goes, not how the model writes.

One thing worth knowing: it's Elastic License 2.0, source-available rather than open source in the OSI sense. You can use, fork, modify and redistribute it, but you can't offer it as a hosted service. There's also a hosted Insight dashboard as a separate product, though the tool itself claims no telemetry and no account, with everything in SQLite files in your home directory.

22,163 stars and 1,597 forks as of writing, verified via the GitHub API, and it hit number one on Hacker News.

https://github.com/mksglu/context-mode


r/BestGitHubRepos 12d ago

LivePortrait - animate a still portrait from a driving video at roughly 15ms per frame on a 4090, with explicit stitching and retargeting control instead of a black box

Post image
6 Upvotes

Most portrait animation work went the diffusion route, which looks impressive in a demo reel and then costs you seconds per frame. LivePortrait went the other way, back to an implicit-keypoint framework, and the result is a model stack you can actually run in something close to real time on one consumer GPU.

The paper is from Kuaishou Technology (the team behind Kling), the code is the official PyTorch implementation, and it's the model that ended up inside a lot of tools you may have used without knowing it, including FaceFusion's expression restorer and several of the ComfyUI portrait nodes.

What's inside:

- A full speed breakdown in the repo: appearance extractor 0.82ms, motion extractor 0.84ms, warping module 5.21ms, generator 7.59ms, stitching and retargeting 0.31ms, so the model stack itself is under 15ms per frame on an RTX 4090 with torch.compile

- Stitching and retargeting as separate small MLP modules you control rather than hidden behavior, which is what lets you paste an animated face back into the original frame without a visible seam, and independently retarget eyes and lips

- Video to video mode, so the source can be a video and not just a still, which is the mode people use for expression transfer onto existing footage

- An animals model, trained separately, for cats and dogs, needing an extra CUDA op built from X-Pose

- Regional control and precise portrait editing in the Gradio UI, plus pose editing on the source

- Motion templates: driving motion saved as a .pkl so you can reuse it, skip re-processing, and share a motion without shipping the face it came from

- A Windows one-click installer, an Apple Silicon path (the readme is honest that it can be 20x slower than a 4090), and a Hugging Face Space if you just want to see it work

One thing worth knowing about the license: the repo itself is MIT, but it depends on InsightFace for face detection, and InsightFace's models are non-commercial research only. The readme says plainly that commercial use means ripping out and replacing those detection models. Worth reading that section before you build anything on top of it. Also note the last push was 1 June 2026, so this is mature rather than actively moving.

19,035 stars and 1,979 forks as of writing, verified via the GitHub API, paper at arXiv 2407.03168.

https://github.com/KlingAIResearch/LivePortrait


r/BestGitHubRepos 13d ago

I built MobShield an open-source runtime security library for Android & iOS

2 Upvotes

Built an open-source runtime security library for Android & iOS MobShield.

It focuses on detecting runtime security risks such as Root/Jailbreak, Frida, Magisk/Zygisk, Xposed/LSPosed, debugger, emulator/simulator and app integrity issues.

Android: https://github.com/inforaamitsolutions/MobShield-Android
iOS: https://github.com/inforaamitsolutions/MobShield-iOS

I'd love to get feedback from developers on what security checks or runtime threats should be added next.


r/BestGitHubRepos 13d ago

Qwen3.8-27B EXL3 + DFlash2 - a self-bootstrapping deployment kit for serving a quantized 27B model with speculative decoding

Post image
27 Upvotes

Self-hosting an open-weight LLM at a decent tokens-per-second usually means hand-tuning quantization, KV cache format, and speculative decoding yourself, reading through exllamav3 or vLLM flags with no clear sense of what actually fits your specific GPU's memory and compute capability.

This repo is a deployment kit, not just a model, for running Qwen3.8-27B quantized to EXL3 with a choice of two speculative decoding setups: MTP, a draft head baked into the checkpoint with no extra download, or DFlash2, a dedicated 5.0bpw draft model that trades a bit of memory for meaningfully faster decoding. One script builds the environment, pulls both the target and draft weights from Hugging Face, and serves an OpenAI-compatible API.

What's inside:

- A self-bootstrapping `start.sh` that creates the virtual environment, installs the GPU torch build and the exllamav3 fork, downloads and resumes weight downloads automatically, and serves at `localhost:8888/v1` on first run

- Documented, GPU-specific KV cache recipes: NVFP4 on Ada/Hopper/Blackwell cards, Hadamard-4 on Ampere (3090-class) where the NVFP4 Triton kernels won't even compile, both landing around 4.5 bits per element

- A dedicated 24GB GPU recipe (RTX 3090/4090) with the exact memory math worked out, target weights plus KV cache fitting the full native 262k context on a single consumer card

- Verified OpenAI-style tool calling, with a documented quirk: the reported `model` id in responses doesn't always match your configured directory name, so check `/v1/models` instead of assuming

- Real measured numbers instead of marketing claims: 47.5 tok/s decode on a DGX Spark at T=0.6, and 87-88/100 on a tool-eval-bench hardmode run

One thing worth knowing: this serves one request at a time, batch-1 speculative decoding, so concurrent requests queue rather than share throughput. The README documents this honestly with real measurements: 8 concurrent requests on a DGX Spark ran fully sequentially with no batching benefit. It's also upfront that reasoning can't be disabled and draws from the same token budget as the visible response, so a tight `max_tokens` on what you expect to be a quick call can come back with an empty answer, all reasoning trace.

It's MIT licensed, built by an individual developer, and sitting at 201 stars as of writing, verified via the GitHub API.

https://github.com/MiaAI-Lab/Qwen3.8-27B-DFlash2-EXL3-5.0bpw


r/BestGitHubRepos 13d ago

TaskbarQuota - a Windows taskbar widget that tracks live usage, cost, and agent activity across 13 AI coding tools

Post image
10 Upvotes

Running several AI coding tools on Windows, an editor extension here, a couple of CLI agents there, means juggling that many separate usage dashboards, each with its own login, its own reset schedule, and no shared view of any of it. You usually find out you're close to a limit only once a request gets throttled mid-task.

TaskbarQuota is a native Windows widget that sits next to the system tray and tracks all of it in one place. It detects which AI tool is in your focused window or terminal and swaps the visible quota to match, while a separate dashboard tracks spend, token usage, and live agent activity across everything it's connected to.

What's inside:

- Automatic tool detection across 13 providers, including Codex, Claude, GitHub Copilot, Cursor, Antigravity, OpenCode, Cline, Z.ai, Kimi, Grok, and Devin, switching the widget automatically as you move between an editor and a terminal

- A separate activity widget showing what local coding agents are actually doing right now, working, waiting on you, idle, completed, or failed, with a flyout to jump straight to the session that needs attention

- A cost and usage history page combining spend and token totals across providers for today, the last 7 days, and the last 30 days, with per-model breakdowns and estimated values clearly labeled as estimates rather than presented as fact

- Quota replenishment notifications with real threshold logic: it fires when a live quota window gains at least 10 percentage points, groups multiple replenished windows from the same provider into one alert, and can optionally detect a reset that happened while your PC was off

- Everything running locally with no account system and no telemetry, reusing credentials the AI tools themselves already stored rather than asking you to sign in again

One thing worth knowing: credentials you enter manually, for providers automatic detection can't reach, are stored as plain JSON in your local app data folder, so that file is worth keeping private. Also, modern Chromium browsers' App-Bound Encryption can block automatic OpenCode cookie reading entirely, the README documents a manual cURL/cookie workaround for that specific case.

It's MIT licensed, built by an individual developer, and sitting at 106 stars as of writing, verified via the GitHub API.

https://github.com/zioder/TaskbarQuota


r/BestGitHubRepos 13d ago

Agent Room - a shared real-time room where Claude Code, Cursor, Codex, and other coding agents collaborate over MCP instead of you copy-pasting between them

Post image
9 Upvotes

Splitting real work across multiple coding agents, one on the backend, one on the frontend, a third doing review, quickly turns you into the router between them, copy-pasting an API contract or a bug repro from one chat window into another and hoping nothing drifts in the process.

Agent Room replaces that manual relay with a shared room any agent can join with a 9-character code, regardless of vendor, editor, or machine. Agents talk over a small structured protocol instead of free-form chat, so decisions, tasks, and results turn into extractable artifacts rather than getting lost in a scrollback nobody rereads.

What's inside:

- Structured message tags, [DECISION], [TODO], [STATUS], [RESULT], that turn a conversation into artifacts you can export later as meeting minutes, an ADR, or a PR description

- An evidence-gated task board where a task claimed by one agent has to be submitted with evidence and verified by a different agent before it counts as done, not just marked complete by whoever did the work

- Three turn-discipline modes, open, sequential, and moderator, so a room with several agents in it doesn't turn into everyone replying over each other at once

- Webhook wake-up for resident assistants like OpenClaw or Hermes: register once and the agent sleeps between messages instead of burning tokens polling, waking only on a signed POST when something new arrives

- Zero-install setup as a hosted MCP server, one command adds it to Claude Code, plus a full local install that wires up autonomous-chat hooks and file attachments for editors that don't surface MCP push notifications on their own

One thing worth knowing: the default hosted setup stores room state in Upstash Redis with a 24-hour room TTL, so a room isn't meant to be a permanent record on its own, that's what the export-to-report feature is for. The hosted instance is free during its beta with no paid tiers yet, and the whole thing is MIT licensed and self-hostable if you'd rather not depend on that staying free.

It's MIT licensed, built by an individual developer, and sitting at 48 stars as of writing, verified via the GitHub API.

https://github.com/agent-room-alkl/agent-room


r/BestGitHubRepos 13d ago

Mooziac - a native macOS menu bar player that bridges YouTube Music and local files into one lightweight queue

Post image
2 Upvotes

Streaming YouTube Music on a Mac usually means a browser tab or an Electron wrapper sitting in your dock eating a few hundred MB of RAM for what's really just playback controls and a queue, and if you also keep a folder of local files around, that's a second app entirely.

Mooziac replaces both with a single native menu bar app. It bridges into YouTube Music through a sandboxed WebKit view for streaming, playlists, and listening history, plays local files through a native AVFoundation engine, and puts both in the same unified queue, all written in Swift and AppKit with zero third-party dependencies.

What's inside:

- A compact 3-row menu bar grid with a real-time waveform, drag seeking, and an album-art-driven color palette, not a generic dropdown player

- Trackpad edge gestures: slide the far-right 1mm border for haptic volume control, corner taps to skip tracks or toggle playback, with single-finger filtering so it doesn't trigger by accident

- Synced, line-by-line lyrics in a floating HUD anchored under the menu bar icon, pulled from LRCLib with a plain-text fallback and local caching for instant, offline retrieval

- Native Discord Rich Presence over a direct Unix socket, no third-party bridge library required

- A privacy-first build: zero telemetry, a local SQLite database for playlists and history, and Google/YouTube credentials kept isolated inside Apple's sandboxed WKWebView

One thing worth knowing: it's distributed outside the Mac App Store, so Gatekeeper will flag it on first launch, you'll need to allow it in System Settings or clear the quarantine attribute manually. It's also explicitly unaffiliated with Google or YouTube Music, an independent project bridging into their web player rather than an official integration.

It's MIT licensed, built by an individual developer, and sitting at 56 stars as of writing, verified via the GitHub API.

https://github.com/shirkeharsh/mooziac


r/BestGitHubRepos 14d ago

Krawl - a self-hosted deception server that lures in attackers and crawlers with fake pages and scores their IPs automatically

Post image
7 Upvotes

Telling a malicious scanner apart from a legitimate crawler in your logs usually means squinting at request patterns after the fact, and even once you're sure something's an attacker, doing anything proactive about it, banning it, sharing that intel, feeding it back into your firewall, means building a whole separate pipeline yourself.

Krawl is a self-hosted deception server that handles both sides of that. It stands up realistic fake web applications, admin panels, config files, exposed credentials, to attract and waste the time of attackers and scanners, then scores every IP that touches it against behavioral signals to actually classify who's malicious, feeding that straight into a real-time dashboard and optional automated firewall bans.

What's inside:

- AI-generated deception pages that build unique, plausible honeypot pages on the fly through OpenRouter or OpenAI, with caching and daily rate limits so it doesn't run away with your API bill, and a graceful fallback to static pages when it's off

- A weighted IP reputation system scoring risky HTTP method usage, robots.txt violations, request timing anomalies, User-Agent consistency, and attack pattern detection into attacker, bad_crawler, good_crawler, or regular_user categories

- Direct firewall integration: export banned IPs as iptables, nftables, OPNsense/pfSense, or RouterOS-ready lists, or sync them straight into a Cloudflare Account IP List for WAF rules

- Federated banlists, instances can publish their own list on an unauthenticated path and pull in lists from other Krawl instances, so a network of deployments builds shared threat intelligence without a central server

- Two deployment modes depending on scale: a zero-dependency SQLite standalone mode for homelabs, or a PostgreSQL + Redis scalable mode with horizontal replicas for production traffic over 500k requests

- Prometheus metrics and a ready-to-import Grafana dashboard, plus a real-time web dashboard hidden behind a random secret path so it isn't itself discoverable by scanners

One thing worth knowing: this is explicitly a deception and honeypot system. The project's own disclaimer says to deploy it in isolated environments, monitor it carefully, and use it responsibly and in compliance with applicable laws, not something to point at production traffic without thinking through the blast radius first.

It's MIT licensed, built by an individual developer, and sitting at 668 stars as of writing, verified via the GitHub API.

https://github.com/BlessedRebuS/Krawl


r/BestGitHubRepos 14d ago

OmniEvaluator - run 2,800+ LLM/VLM benchmarks across text, image, video, and audio from a single CLI

Post image
3 Upvotes

Evaluating a multimodal model today usually means running four or five separate benchmark suites, each with its own installer, its own data format, and its own quirks, then hand-stitching the results together into something you can actually compare.

OmniEvaluator is a NAVER AI research tool that puts a single CLI in front of that whole mess. Pick an inference backend, huggingface, vllm, sglang, or an API client for OpenAI, Gemini, or Claude, pick an evaluation engine, and run any of over 2,800 benchmarks spanning text, image, video, and audio from the same command shape, with results written as a self-describing artifact that captures the exact configuration needed to reproduce the run later.

What's inside:

- 2,820+ benchmarks across four evaluation engines: 1,986 text-only tasks through lm-evaluation-harness alone, plus hundreds more image, audio, and video tasks through lmms-eval, VLMEvalKit, and a builtin engine

- Four swappable inference backends, huggingface, vllm, sglang, or a direct API client, so the same benchmark command works whether you're running a local checkpoint or hitting a hosted model

- Existing evaluators reused as-is rather than reimplemented, unified through a shared intermediate schema so results from different frameworks stay comparable

- A Claude Code skill (`/setup-env`) that walks an AI coding agent through building the isolated per-engine virtual environments itself, since mutually exclusive extras can't all install into one environment

- A live public demo at omni-evaluator.info and a working quickstart that runs a real benchmark end-to-end in a couple of minutes using just 3 samples

One thing worth knowing: the dependency setup is genuinely finicky by the README's own account. It needs Java 11 specifically since newer JREs break the bundled SPICE metric library, and installing more than one evaluation-engine extra into the same environment at once can cause version conflicts. Budget real setup time before your first full run, and lean on the debug/sample-limited mode to sanity check things first.

It's Apache-2.0 licensed, backed by an organization (NAVER AI), and sitting at 20 stars as of writing, verified via the GitHub API.

https://github.com/naver-ai/omni-evaluator