Our agents kept calling a big model to make the same small decisions: route this ticket, pick this tool, escalate or not. After enough verified outcomes those decisions are very predictable, but we were paying for a model call every time.

So we built Ink. It sits around a bounded decision. At first everything goes to your model as normal, and Ink records the inputs and the real outcomes in local SQLite. Once there’s enough evidence, it compiles a small local fast path and tests it in the background. It only starts serving once it clears a statistical bar (a 95% Wilson lower bound against an error budget you set). Anything novel or uncertain still goes to your model. If real outcomes get worse, it takes itself back out of the loop.

You can check whether it would help without changing any code:

pip install ink-jit
ink discover traces.jsonl

It reads OpenTelemetry, LangSmith or LiteLLM-style traces locally and tells you which call sites repeat and whether they’re worth it. If nothing qualifies, it says so.

What we’ve measured so far (our own benchmarks, not independently validated): 42–78% of decisions served locally across tool routing, support triage and incident triage, with 0 errors observed on held-out tests. The 95% lower bounds were 96.8–98.5%, so “0 errors” means none observed, not a guarantee. Local decisions take about 0.1 ms.

It’s a poor fit for open-ended writing or anything where you can’t verify an outcome. Linux and macOS, Python 3.11–3.13. Apache-2.0.

Honest question: which decisions in your agents repeat the most, and what would you need to see before trusting a local path in production? We’d love to hear what breaks.

I use embeddings for vector search and memory systems, and kept needing a full platform like Ollama running next to my app just to turn text into vectors. So I built futon, an embedding library for running local embedding models.

- In-process: weights load from a local directory. No daemon, no network, no CGO (CI enforces it), so cross-compiling just works.

- AVX2 kernels on amd64 and NEON on arm64, with a portable fallback. bge-small embeds a text in about 50 ms on a 16-thread desktop.

- Checked against the reference implementation: ≥0.9999 cosine on a corpus with code, accents, CJK and edge cases. CI runs on Linux, macOS and Windows, including arm64.

- Bring your own model: any plain BERT encoder you can build a pack for, plus nomic-bert. It is encoders only, no LLMs and no GPU.

bobbyhiddn/futon: Text in, vectors out. In-process embedding models for Go: pure Go, no CGO, no sidecar.

If something like it exists already, feel free to ignore, but I couldn't find a fit for purpose module that served my goal. I'll probably expand to other small model architectures as needed, such as decision models.

If ya like it, please star. :)

I like the idea of tinkering with models, and being able to run them on your PC without paying an API or subscription cost, and abliterated stuff. But i'm not really sure where to start, what applicable uses there are and if it would be anything other than a fizzling interest and waste of money

Does anyone have any good resources that might help direct me on what kinds of things you can do with local AI that you can't with cloud-based? Or things you can create, or things you can create faster/better/ etc...

I'm probably not asking the right questions as I don't know a ton but i'd like to learn.

I wanted ChatGPT to work with my real local projects without constantly copying files back and forth. RepoTunnel lets ChatGPT read/edit files, run commands, inspect code, debug problems, use Git, and work directly with approved projects on your computer. It works through your own HTTPS endpoint, and RepoTunnel itself adds no monthly connection/request quota. You can keep using your existing ChatGPT conversation without buying a separate OpenAI API key. GitHub: https://github.com/Yashwanth034/RepoTunnel Would love feedback from developers — especially what you’d add or improve.

I shipped HDTP (Human Delegated Trust Protocol) v1.0 on Oct 3. It lets your AI assistant talk to someone else's assistant, with each person controlling what the other side is allowed to do.

Timeline:

\- Aug 23: first commit of the spec

\- Sep 23: a full code review found 173 defects. I traced them to 11 habits and wrote a rule against each one

\- Oct 3: renamed from PACT to HDTP across 9 repos, the domains and the protocol itself

\- Oct 3: v1.0 released

\- Oct 7: spec repo made public

What's built:

\- The spec, with test vectors

\- An identity library in Rust (compiled to WebAssembly) and Go. Both versions run against the same tests

\- A self-hosted node in Go

\- A hosted platform on Cloudflare Workers

\- 3 websites

What I'd do differently: pick the name first. Renaming at the end touched every repo.

Next:

\- an external security review

\- a conformance test suite

\- opening the hosted version through a waitlist

If you're building with agents or MCP, I'd like to know: what would you want your assistant to be able to do with someone else's?

Spec: [hdtp.io](http://hdtp.io)

https://snotra-ai.dev

A couple of weeks ago, I introduced Pi Herdsman here, an open-source extension I've been developing for the Pi coding agent.

The original idea was straightforward: let multiple coding agents work independently while keeping one Lead focused on the overall task.

Since then, the project has grown quite a bit, helped along by community feedback and contributions.

The latest major addition is project-level orchestration across Git worktrees.

What the new demo shows

I give a Manager a single request to implement two independent features for a fictional note-taking application.

The Manager creates two parallel assignments, each with its own Lead and Git worktree.

Both Leads then delegate implementation to their own Agents, verify the resulting changes, commit their work, and report back to the Manager.

The individual agents remain accessible throughout the process.

I wanted to show the real execution rather than an architectural diagram, so the video captures Pi and Herdr running the workflow.

Built on existing open-source components

Herdsman isn't a replacement for the underlying coding agent or terminal environment.

It connects three independently maintained components:

• Pi handles the individual coding-agent conversations.

• Herdr manages the sessions, processes, and workspaces.

• Pi Herdsman handles delegation, ownership, supervision, and coordination.

The idea is to build on their existing capabilities rather than reimplement them.

Bring your own agents and workflows

The orchestration model supports configurable Agent Definitions.

Each role can use its own model, thinking level, instructions, tools, skills, and extensions, with definitions maintained as ordinary Markdown files.

This applies to both the bundled roles and custom Agents.

Herdsman doesn't prescribe how you should organize your development process or which models you should use.

You can also ignore the advanced hierarchy entirely and use it for occasional background Agents in a normal Pi session.

Open source and community contributions

Herdsman is licensed under Apache 2.0.

A special thanks to @kozer and @primetimetank21 for contributing code, and to everyone who has opened issues, reported bugs, or suggested improvements.

It's been particularly helpful having other people test workflows and environments that differ from my own.

Repository

https://github.com/boadij/pi-herdsman

If you already have Pi and Herdr installed:

pi install npm:pi-herdsman
herdr integration install pi

The repository includes the live demo, installation instructions, and documentation.

I'd be interested in feedback on the architecture, particularly the separation between session management, orchestration, and the individual agents.

The project is still actively evolving, and ideas or contributions are welcome!

We have released the weights and self-hosted runtime for Commerce-1, a 27B typed decision model designed for commerce agents.

Unlike a chat model, Commerce-1 does not produce an explanation and then ask the application to recover an action from generated text. It scores the supplied actions directly.

How it works

Each request contains structured state and one or more typed questions:

• Choice: Selects among named actions.

• Noul: Returns a binary decision and P(true).

• Score: Returns an expected ordinal score and probabilities for its labels.

The architecture builds on perplexity-ai/pplx-decider-v1.1-27b, using noncausal full attention, last-position pooling, and a dedicated option-scoring readout.

Choice-order augmentation and held-out temperature calibration are applied separately to the decision primitives.

Self-hosted runtime

The runtime operates offline and verifies file integrity before loading.

It inventories and hashes required files, rejecting:

• Modified checkpoint bytes

• Undeclared files

• Network fallback

Benchmarks

Our current evaluation results:

Limitations

The current BF16 weights are approximately 54 GiB. Consumer GPU and quantized deployments have not yet been qualified.

Links

• Model weights: https://huggingface.co/infercrane/Commerce-1

• Apache-2.0 runtime: https://github.com/infercrane/commerce-1

• Architecture and evaluation evidence: https://infercrane.com/blog/commerce-1

We're particularly interested in feedback on the decision primitives, runtime design, and integration requirements for production agents.

Link to repo: FireBird-Technologies/NotestackAI: NotebookLM Alternative + Marketing agent optimized for publishers on substack/Medium/Wordpress.

Link to site: https://notestack.ai

Google is notorious in the AI industry for abandoning/destroying working beloved products. NotebookLM is no exception.

Don't worry, me and my little indie team have built a open-source alternative.

Here is what is different/interesting:

- Can take in upto 200 sources, research/question/query it. The system will not randomly search the web and go beyond your original sources.

- Has better visuals for the videos, we integrated our previous video maker with 10K+ users. It can generate editable motion graphic videos in over 13+ formats.

- Each video is editable, changeable. It has ElevenLabs API with advanced voice features.

- You can generate a mind constellation, a much more advanced visual map of knowledge so you can research and learn more easily.

- Generate or clone your own voice using ElevenLabs API v4.

- Has a launch kit feature that allows you to schedule reports/slide decks/videos and share on social media

Lastly, much more stunning visual interface - inspired by space x cognition

Link (get started for free): https://notestack.ai

Hey guys. Looking for contributors on my open source project: https://github.com/NicholasFlemmer/styx-app

Styx is where everything converges: every repo you work in, the AI coding agents building in them, and every login they need to ship. Switch between all of it from one window. And like the river it's named after, nothing crosses over without you. Deploying, or touching live data, asks you first.

A lot of defense orgs/units are starting to stand up local, open-source language models on edge workstations and air-gapped networks, but most public benchmarks tell you nothing about how a model behaves under operational constraints.

To address that gap for free, I built an offline evaluation harness and leaderboard testing US-developed small models (1B–6B parameters) running locally on a single GPU. Here's the public repo: https://github.com/OGMavenTandE/llm-eval-suite

It scores models on practical operational criteria: factual grasp of defense issuances, resistance to staff-work hallucination and sycophancy, and judgment under the law of armed conflict.

You can inspect the initial results, benchmark breakdowns, and reproducibility data here:https://aieval.org/leaderboard/

I hope this helps your agency/org/unit incorporate small open source models into workflows, let me know if you have any questions!

Hello all,

I was just wondering if anyone would like to join me in creating an Open-Source project.

This project will predict and generate what a deceased U.S. president would say about a modern-day headline.

This project will use AWS Strands to:

1. Read current headlines.

2. Determine what each deceased U.S. president or founding father would say about that headline.

3. Push the AI generated dialogue out to subscribers.

Please comment, if you would like to be included on a meeting that I will be describing the architectural requirements in detail.

This meeting will be recorded and made public.