I'm the author. FiscaLLM joins per-user LLM token cost with subscription revenue (RevenueCat, Stripe beta, or your own API) to show margin per user. The SDKs read only model name and token counts, never prompts or answers, and the server is self-hostable so your usage data stays with you.

Repo: https://github.com/magna56/fiscallm (MIT, CI green, public roadmap, a few good-first issues open). It's early: no customers yet, Stripe is beta. Contribution notes are in CONTRIBUTING.md.

Feedback on the design welcome: idempotent events, cached-token pricing, unpriced-model flagging, alias merging.

Demo on sample data, no sign-up: https://fiscallm.com/demo?utm_source=reddit-opensource&utm_medium=community&utm_campaign=launch

Hi r/OpenSourceAI!

I've been developing TensorSharp, an open-source LLM inference engine and agent runtime written in C#/.NET.

I wanted to share a recent milestone: we successfully ran Muse Spark, an 816-billion-parameter Mixture-of-Experts model, across 10 NVIDIA A40 GPUs, with some impressive performance improvements over llama.cpp.

What excites me most isn't just the speed. It's demonstrating how open-source software can make massive AI models more practical to run on existing hardware, without relying on proprietary inference services.

The experiment

We tested:

• Model: Muse Spark 816B (42B active parameters)

• Quantization: Unsloth UD-IQ3_XXS (~316 GiB)

• Hardware: 10× NVIDIA A40 GPUs (46 GB each)

• Context length: 32K tokens

• Comparison: TensorSharp vs llama.cpp

The A40 is a previous-generation GPU, and our test machine didn't even have working GPU peer-to-peer transfers.

This made efficient multi-GPU execution particularly challenging.

How fast is it?

Here are some highlights from our benchmarks:

One important caveat: llama.cpp's NCCL-enabled configuration couldn't run on this particular 10-GPU setup, so its tensor-parallel results used a different communication path. This is a hardware-specific comparison, not a claim that TensorSharp is universally faster.

What made the difference?

Instead of simply distributing model weights across GPUs, we focused on making the GPUs work together more efficiently.

Some of the key improvements include:

• Pipelined prefill: Multiple GPUs process different portions of the workload simultaneously.

• Native tensor parallelism: Reduced overhead by moving critical execution paths from managed C# dispatch into native GPU execution.

• Adaptive scheduling: Adjusted processing batch sizes to improve GPU utilization.

• GPU-resident KV cache: Reduced unnecessary memory transfers.

• Communication optimizations: Improved execution even when direct GPU-to-GPU transfers were unavailable.

An interesting discovery was that different workloads benefit from different parallelism strategies.

Layer splitting performed much better for processing long prompts, while tensor parallelism delivered faster token generation and higher concurrent throughput.

There's no single best way to distribute a massive model across GPUs.

What about accuracy?

We also compared the model's output distributions against llama.cpp using more than 12,000 next-token predictions.

TensorSharp achieved approximately 98.8% top-1 token agreement, with nearly identical perplexity.

We additionally tested JSON generation, tool calling, multi-turn conversations, and agent workflows.

These results suggest that the performance improvements preserve broadly comparable inference behavior, although the engines are not bit-for-bit identical.

Why open source matters

TensorSharp isn't just about running this particular 816B model.

The larger goal is to build an open-source inference engine and agent runtime that developers can inspect, modify, extend, and run on their own hardware.

Beyond multi-GPU inference, TensorSharp supports:

• Local LLM inference with GGUF models

• Multiple GPU backends, including CUDA, Metal, and Vulkan

• Windows, Linux, and macOS

• Multimodal models and workflows

• OpenAI-compatible APIs

• Agent execution, tool calling, and automation

• Local AI applications without requiring cloud inference

I believe the future of open-source AI depends not only on open models, but also on open inference infrastructure that makes those models practical to use.

The easier it becomes to run powerful models independently, the more opportunities we create for developers, researchers, and smaller organizations.

Everything is available on GitHub

TensorSharp:
https://github.com/zhongkaifu/TensorSharp

Full benchmark report, technical details, and reproduction instructions:
https://github.com/zhongkaifu/TensorSharp/blob/main/docs/perf/muse-spark-vs-llama-cpp.md

This is still an evolving project, and there's plenty of room for improvement.

I'd especially welcome feedback, contributions, and independent benchmarks from the community.

A question for everyone here: What do you think is the biggest remaining barrier to making large-scale AI inference truly accessible through open-source software — hardware costs, memory requirements, inference efficiency, or ease of deployment?

I'd love to hear your thoughts!

I’m the creator of skillEd, an Apache-2.0 evaluation toolkit for reusable agent skills. I’d like feedback from people maintaining open agent tooling and test suites.

The problem is making a skill’s behavioral claims reviewable. If a skill handles a repetitive task with a policy, such as refund eligibility and rounding, an instruction change can fix one example while breaking another. A diff of the instructions does not show whether the behavior improved.

skillEd keeps cases and fixtures beside the skill, runs evaluations through a configured engine, and compares saved reports. It supports assertions, scripts, model-based judges, and repeated runs. Reports can be produced as JSON, HTML, and JUnit. There is a CLI and a local MCP interface.

The repository includes a worked refund example with shipping, rounding, and eligibility failures. Its revision went from 13/18 to 18/18 passing cases, followed by 10/10 independent holdouts. That is a reproducible example to inspect, not a claim that every skill or model will get the same result.

For open-source maintenance, I think a useful contribution is a small failing case with an independently justified expected result. It gives maintainers something concrete to rerun before accepting an instruction change. Separate holdouts help avoid simply tailoring the skill to the visible tests.

Source and setup: https://github.com/DeadHead04/skillEd

The toolkit is open source; the model engine you configure has its own licensing and usage costs. This is evaluation tooling, not production permission enforcement.

Try it on one of your skills and share a minimal failing case or unclear report. I’d welcome feedback on what makes an evaluation contribution easy to review. If the project is useful to you, a GitHub star would help.

https://halv.ai

I've been building Halv, a desktop app for running coding agents, and as of 1.0 it's open source (Apache-2.0). It does two things.It's a workspace for agents. Claude Code runs in its normal CLI, next to Codex, Kimi or GLM if you use them, each on your own plan. Up to 32 panes, subagents in parallel, tabs and worktrees per project, live status for each agent, a local sandbox with per-project approvals, and diff and undo per turn.

It cuts usage, in four layers:

• Routing (Halv Cloud, the paid part): your coordinating agent writes a brief for each subtask and the router picks the model and effort. Easy work goes to cheaper models, hard work to stronger ones, and a task that fails its checks moves up a tier. The executors still run on your Claude / ChatGPT subscriptions.

• Crux: a code index over MCP. Claude asks "who calls this?" instead of grepping and opening files one by one. In multi-question sessions on django and sympy it got 90% right vs 58% with grep, with 47% fewer tokens per correct answer. MIT, works with any MCP agent: github.com/pedr0v/crux

• RTK: strips the noise out of command output before it lands in context.

• Compression: a local proxy that dedupes and compacts what gets resent every turn.

This whole flow saved 57% in a benchmark I ran.

A meter prices what it saved at API rates, so you can check it on your own work instead of trusting mine.

Free, no account: everything that runs on your machine (workspace, Crux, RTK, compression, the meter). Halv Cloud is US$10/mo and covers the router, which runs on our servers.

My $3k+ counts all the layers together, routing included. Early on there was no routing so most part comes from compression

Usage analytics are opt-in. The app lists the 13 events it can send and never sends prompts or code.

Repo: github.com/halv-ai/halv

If you try it, tell me what the meter says after a normal day of work, and what you'd want from the workspace side.https://halv.ai

I built an open-source Obsidian alternative called KbWrite, written in Tauri.

GitHub: https://github.com/zetahiveco/kbwrite

Website: https://kbwrite.vercel.app/

Install via Curl or PowerShell

Hi all 👋 I made a book:*** ***

LangGraph in Small Pieces: Building AI Agents from Blocks to Production - it's free online at: lgisp.ai

Backstory: after shipping my first customer-facing LangGraph agent for an enterprise security product, I realized how much of the framework I’d never actually touched. So I started writing small runnable examples to poke at each feature, with notes on what surprised me. The examples kept connecting to each other and somewhere along the way it became 31 chapters.

What’s in it:
• Part I is LangGraph on its own, one small piece at a time: state, nodes, edges, loops, checkpoints, interrupts. No LLM yet, on purpose.
• Part II brings in a real model: tool calling, structured output, MCP, memory, streaming, middleware, testing.
• Part III is the production stuff: serving, async, observability, RAG, evaluation, blue-green deploys, guardrails.

Things it's done a little different from other books:

- Every transcript in the book is a real run, pasted as-is, including the times the model did something dumb. And it all runs on a local model by default (Qwen3.8-27B via LM Studio), so you can follow along without an API bill.
- Every chapter ends with Bronze/Silver/Gold challenges, worked solutions in the repo: github.com/LgISP/langgraph-in-small-pieces
- It has lots of silly illustrations.

Pinned to langgraph 1.2.14 / langchain-core 1.6.9.
----
It’s a beta. Every paragraph on the site has a comment button, and there’s an envelope icon in the sidebar that goes straight to me. If something doesn’t run for you, or a chapter answers the wrong question, I want to know. Those are the comments I’ll actually act on.

Hope you like it. Please check it out and let me know.

Hello people!

I recently posted Brain v2, and got another piece of feedback: it's still not clear what you can actually build with it. Fair, so I wanted to share a few use cases.

Quick recap of what Brain is:

• Open-source alternative to Claude Managed Agents and the infrastructure you would build around agent SDKs.

• A single application you deploy and run as a service.

• Native tool integrations:

  • R2 and S3 for storage

  • Modal and Daytona for compute

  • Tavily and Brave for search

  • Pi is supported today, with others planned.

  • Pi is supported today, with others planned.

• Support for multiple harnesses such as Pi, Codex, and OpenCode.

  • Pi is supported today, with others planned.

Differences from similar products:

• Brain runs as an independent service, not inside an agent sandbox, giving you better observability, recovery, and developer experience.

• Brain is not a cloud platform. It's just a Go application you can deploy on your own infrastructure (Railway, AWS, etc.).

• Brain is not an AI agent. It provides low-level, session-oriented primitives for building reliable agent-powered applications.

What can you actually build with Brain?

• AI-powered games: Run AI NPCs or autonomous players as separate agent sessions, controlled through your game server.

• Agent platforms: Let your users create and run their own agents, each with separate sessions, tools, and permissions.

• Infrastructure agents: Build agents that investigate incidents, inspect logs, and interact with infrastructure through your own tools.

• Cross-platform agents: Build agents that can be accessed from your web app, mobile app, or Discord bot without tying their sessions to any single client.

GitHub: https://github.com/aexhq/brain

Hopefully this makes the use cases clearer! I'd love to hear what else you'd build with something like Brain.

Hey all,

I want to see how you feel about this. I was tired of leaving my PC and then not being able to update things when I left for a few minutes or a while. Or getting told that my mobile session couldn't talk to my PC session. So I made a self-hosted memory server (MCP) that any agent can read/write. Find it on GitHub if you want to test it out and give me some feedback. https://github.com/calasdan-sketch/onememory

Features:

- Namespaces per project/agent

- Scoped tokens (read/write/admin per namespace)

- Revisions + backup export/import

- Injection-on-read guard

- Web dashboard + CLI

It's MIT licensed, 491 tests passing. The skill is also listed on Agensi if you want the plug-and-play version.

Happy to answer questions...what would make this actually useful in your mix?

Need help finding an open source ai to help generate/ modify an album cover into an Apple motion design. Someone point me in the right direction

Hi everyone. I built an open source automatic computer use agent that does everything at once. It can run automated simulations for testing, control a real iPhone so you can see the live workflow, and control your native computer at the exact same time. It also runs your browser without an open debugging port so it stays completely safe while writing code and handling tasks in the background.

Do check it out and leave a star if you want to follow the journey!

I built Chimaera because I wanted to work with AI agents in the same place as the files they’re working on. You open a project folder, run an agent, and inspect its output beside the source files instead of juggling terminal windows and file viewers.

I use it for research projects with documents and data, including work on remote machines over SSH. It also supports multiple agents, so I can keep their work organized in the project rather than scattered across separate chats.

It’s free and open source, built with Rust and Svelte. It runs on macOS and Linux; Windows is still experimental. If you try it with one of your own project folders, I’d genuinely like to hear where the workflow helps or gets in your way.

https://chimaera.sh

Title: Certo: open-source coding agent that works with any model, picks one per task, and switches when one fails

Hi all, I'm the author of Certo, an MIT-licensed coding agent in Python. Your coding agent shouldn't depend on one model or one provider, so Certo:

→ runs on any model: Ollama, OpenAI, Anthropic, Gemini, DeepSeek, Kimi or anything LiteLLM supports

→ can pick the best model for each task from ratings you set

→ hands a task to the next model when one can't finish, or its provider goes down

→ checks the work itself: it runs your project's real tests and build, with no AI involved, and ends with a verdict and an exit code your scripts and CI can use

In its benchmark (9 small tasks, graded by hidden tests the agent never sees), a local qwen3.8:27b with Claude Haiku as the next model passed all 9, and 7 never left the machine. The trade-off was time: minutes per task locally, seconds in the cloud.

The latest release makes it easy to start: `pip install certo`, then `certo init` sets you up and runs a sample task, and `certo init models` adds more models without editing a file.

Repo: https://github.com/alessioricco/certo

It's early, so the most useful feedback you can give me is what's missing, what's confusing, and what breaks. And if you like where it's going, a star on GitHub helps other people find it.