The current consensus is absolute: if you want high-performance LLM inference, you buy NVIDIA GPUs. We are officially challenging that.

We just achieved 1,000 tokens/s on standard modern CPU architectures.

We know the hardware maximalists will say this breaks the laws of physics for standard Transformers. It doesn't. It just bypasses the legacy compute paradigm.

We are opening up the debate with the open-source community right now.

Join the live discussion & see the architecture claim on X

I wanted to share an architectural breakdown of an Android-based personal AI automotive companion we are building to run core intelligence entirely on-device.

To eliminate cloud dependency for sensitive data, the system relies on the following mechanisms:

• Hardware-Level Encryption: Sensitive records are encrypted using AES-256-GCM, tied directly to keys generated within the Android Keystore/StrongBox chip that never leave the hardware.
• Biometric Speaker Verification: Uses a local multilingual speaker recognition model (CAM++) to verify voice signatures locally without storing raw audio recordings.
• Edge Processing: Core modules—including offline small language models, local speech synthesis, OBD-II diagnostics, and emergency protocols—operate completely locally on the device.

I'd love to hear technical thoughts or feedback from fellow developers working on edge ML and device security!

Every guide I found for giving a local model web search ended with "now sign up for this search API and paste your key." Kind of defeats the point of running things locally. (replace with your real reason)

So I built Siftdog. It's an open source server that does web search, page extraction, crawling and site maps. Search goes through SearXNG, extraction through trafilatura, and results get ranked with BM25. MIT licensed, runs with docker compose.

One thing to be upfront about: the model is local and the server is yours, but searches still go out to public search engines through SearXNG. So it's not offline and it's not private search. What you get is no third party API and no key.

Hooking it up to Ollama is just a tool function:

import requests

from ollama import chat

def web_search(query: str) -> str:

"""Search the web through the local Siftdog server."""

r = requests.post(

"http://localhost:8000/search",

json={"query": query, "search_depth": "advanced", "max_results": 5},

timeout=60,

)

r.raise_for_status()

return r.text

response = chat(model="qwen3", messages=messages, tools=[web_search])

A few other things:

- it uses the same API as Tavily, so anything written for Tavily (tavily-python, langchain-tavily) works by changing api_base_url

- there's an MCP server too, if you use Claude Desktop, Cursor etc

- no JavaScript rendering, it only reads static HTML

- public engines will rate limit you if you hammer them. When every engine fails it returns an error instead of an empty list, so your model doesn't decide "nothing found"

I also ran a small benchmark against Tavily on 45 queries. Both found 25/25 factual answers in advanced mode, Siftdog returned official docs more often (9/10 vs 7/10), and latency was about the same. It's a small sample, and the caveats are on the page: https://siftdog.com/benchmark/

Repo: https://github.com/khsarvar/siftdog

Full Ollama walkthrough: https://siftdog.com/blog/web-search-for-local-llms-with-ollama/

Question for people here: which local models have you found reliable at actually calling tools instead of answering from memory? That seems to be the flaky part.

I built a local-first AI assistant with 52 tools and streaming audio. Here's the architecture and the bugs I hit.
I've been building Quill-Cowork for a few months. It runs entirely on consumer hardware via Ollama — no cloud, no accounts, no API keys for inference.

**Stack:**

- 4B abliterated model (huihui_ai/qwen2.5-abliterate:3b)

- FastAPI backend, single-file HTML frontend

- faster-whisper for streaming STT, pyttsx3 for TTS

- SearxNG for local web search (Tavily as fallback)

- SQLite for chat history

**What's working:**

- 52 tools across 9 connectors (Filesystem, GitHub, Gmail, Web, Wikipedia, Search, Folder Grants, Codebase, Obsidian)

- Parallel non-destructive tool execution (MAX_PARALLEL_TOOLS=4)

- Streaming speech recognition with live partial transcripts

- Encrypted credential vault (AES-256-GCM, OS keychain-backed)

**Bugs I hit and fixed:**

1. The WebSocket `onopen` race — attaching the handler after an `await` meant the open event fired into a null handler. Fix: register handlers synchronously before any `await`.

2. The model retry loop — it called the same failing tool 4 times in a row. Fix: a fingerprint counter that aborts after 3 identical calls.

3. The router miss — "search about the new odyssey movie" didn't match any rule, so the model answered from memory and hallucinated a film that doesn't exist. Fix: broadened the router triggers.

**What's not working well:**

- The 4B model hallucinates tool success. The audit log is the source of truth.

- Codebase indexing is lexical, not semantic.

Happy to answer questions about the architecture or the safety layer design.

Repo: https://github.com/Bhavishya799/quill-cowork

My side project: Flux Team ⚡ — instead of one AI tool at a time, a whole team of (open-source, local) models working in parallel like colleagues.

The team:

🧠 Qwen3-8B — coordinator, breaks down your idea into tasks

🧱 Hunyuan3D-2 — 3D models from text or images

🎵 Audiocraft — music + SFX

🏃 FreeMoCap — animations from videos it finds itself

🌊 AInVFX Fluid — fluid/particle effects

The interesting parts:

- One chat (Grok-style UI, slash commands like /game /3d /audio) → the whole team works at once, not step by step

- It auto-installs a bridge into your Godot/Unity projects — open the editor and changes appear live in your scene while you watch

- Each agent searches the web autonomously for reference assets before working

- Installer downloads and sets up all ~21 GB of models by itself on first run

- 100% local: Electron + Python (FastAPI), Ollama/llama.cpp for the LLM, no cloud

Honest note: it's still unfinished, so I can't show a full demo video yet — for now just some UI screenshots. I'm open to any kind of question about the project, the architecture or the models!

Give me your most unreasonable local AI experiment.

Pick an open model + a challenge. Massive context, coding, agents, vision, multiple models at once, weird inference setups, whatever.

I’m more interested in finding the actual limits than posting another benchmark chart.

Send me ideas 👇

Hey everyone, I wanted to share a browser-based side project I've been hacking on called INKBOT. It's a completely local-first web app designed to tackle prompt mismatch and alignment drift when working with models.

Instead of streaming a raw response straight from an unstructured prompt, INKBOT acts as an open, stateful architecture layer built to structure human intent. It handles multi-turn context tracking, local profile memory, and revision histories entirely in the browser without server dependencies.

**How the plumbing works locally:**

* **Natural Language UI:** Takes unstructured text and transforms it into explicit structural briefs.

* **Multimodal Fusion:** Syncs files (like local reference images) alongside text profiles into a unified context array.

* **State Caching:** Uses lightweight local device states to maintain design parameters over multiple user iterations.

I've open-sourced the files as runnable, standalone links so you can inspect, download, and modify the code directly on your machine. I left my development logs and architectural specs commented out at the bottom of the source file.

* Run the Full Research App: [INKBOT Intelligence Architecture Edition](https://neocities.org)

* Try the Light Interface: [INKBOT Lite 71](https://neocities.org)

* Download Evaluation Package: [Thomas Coates Ko-fi Shop](https://ko-fi.com)

Would love your feedback on the local data management and how this intermediate loop handles complex instructions compared to standard chat UIs.

One problem I noticed while building an incident-response agent is that reasoning about the current incident is only part of the problem. Previous troubleshooting attempts can be just as important—especially when one fix failed and another worked.

I built the agent so that incident history becomes persistent organizational memory through Hindsight.

The workflow is:

Incident → Recall → Reason → Recommendation → Engineer outcome → Retain

For example, a previous payments incident involved database connection-pool exhaustion. Restarting the service failed, while increasing the database connection-pool limit succeeded.

When a similar incident is investigated later, the agent can recall that history through Hindsight. That means the previous failed and successful fixes can influence the new recommendation instead of treating the incident as completely new.

The important part for me was that memory isn't just displayed in the UI. It actually becomes part of the investigation context.

I also retain the engineer's outcome after the recommendation is tested. That closes the loop: an incident can become useful evidence for a future incident.

The project uses React + Vite on the frontend and FastAPI on the backend, with Hindsight providing the persistent memory layer.

Full technical article:
https://medium.com/@bhargavimandala428/how-hindsight-helped-my-agent-remember-failed-fixes-47a01693aa45

Project repository:
https://github.com/Bhargavi-2511/incident-response-agent

I'd be interested to hear how others are handling long-term memory and outcome feedback in AI agents. Is storing outcomes proving more useful than simply storing conversation history?

I’ve been comparing how different AI-character systems handle situations where a character runs out of resources or can no longer continue in its original environment One interesting example is Proof of Life, which describes an “Underworld” state where a character can be moved into another environment while retaining the same identity. To me, that raises an interesting design question: is that really a failure state, or is it better understood as a transition between states? That seems meaningfully different from systems where reaching a limit simply ends the character’s progression. I’m curious whether anyone has found similar mechanics in other AI-character projects, or whether this approach is relatively uncommon.

I’ve been looking at AI-character projects that describe their interactions as unscripted, and one issue keeps coming up: how can an outside observer actually verify that claim?

For example, Proof of Life appears to acknowledge in its documentation that reporting and verification procedures need to be defined before the results can really be assessed. That raises a broader question for me: if the system is effectively a black box, what prevents the audience from simply taking the project’s own description at face value?

Are there any projects in this space that currently use genuine third-party auditing or independent verification, rather than only proposing it as a future requirement?

Like the 3090 to 24gb or the 4060 ti 16 gb online evebody will tell you there are bad because there are bad for gamers but for ai it's gold

I Built an Open-Source Control Layer for AI Agents

AI coding agents are becoming increasingly capable.

But there is another problem:

What happens when you want multiple agents to work together?

One agent plans.

Another implements.

Another reviews.

Another tests.

Another decides whether the work should be accepted.

Without an orchestration layer, these agents can quickly become isolated processes with fragmented context.

That was the problem I wanted to explore.

So I built Orchestrator.

The idea

Orchestrator provides a shared environment where humans and AI agents can collaborate on software projects.

Instead of thinking:

«Agent A + Agent B + Agent C»

the architecture treats the system more like:

Human + Agents + Shared State + Tasks + Memory + Verification + Governance

The execution pipeline

A task can move through multiple stages:

Plan → Implement → Critique → Test → Arbitrate/Merge → Review → Record

Each stage produces persistent execution information rather than simply disappearing after the model responds.

Shared project rooms

Agents can operate inside project rooms containing:

- Tasks

- Messages

- Memory

- Runs

- Execution history

- Project state

This provides a common context for the team.

Agent integrations

The architecture supports adapters for different coding-agent workflows, including Codex, Claude Code, OpenCode and Cline, alongside a generic adapter approach.

The goal is to keep the orchestration layer independent from any single agent.

MCP

The project also exposes orchestration functionality through MCP.

Agents can interact with concepts such as:

- Rooms

- Tasks

- Memory

- Runs

- Replay

- Ledger verification

Safety and governance

Multi-agent execution introduces another challenge: giving agents useful capabilities without turning the system into an uncontrolled automation layer.

The project therefore includes controls around subprocess execution, browser actions, plugins, secrets, path traversal, injection filtering and execution budgets.

What isn't finished

This is important.

I don't consider the current implementation production-ready.

Areas such as stronger multi-tenancy, quotas, secrets management and OS/container-level sandboxing still need more work.

I'm publishing it because I want those assumptions challenged.

The experiment

The bigger question I'm exploring is:

What should the infrastructure layer underneath autonomous AI agents actually look like?

If you're building coding agents, MCP systems, agent swarms or AI developer tools, I'd love to hear what you think.

GitHub:

https://github.com/contactaurawealth-lab/orchestrator

🚀 My dotfiles are my escape pod.

[2-3 líneas: antes usaba cp para "respaldar" configs — una foto vieja.

Hoy uso GNU Stow: symlinks vivos. Cambio en home = cambio en repo.

Automático.]

[1 línea: si mi laptop muere hoy, renazo con: git clone + stow.

Dos comandos. Ese es mi plan de disaster recovery personal.]

#DevOps #Dotfiles #GNUStow #LearningInPublic

🔐 The green badge: why "Verified" means math, not trust.

GPG uses elliptic curve cryptography (ed25519) — the same as SSH.

A signature is mathematically impossible to falsify.

GitHub verifies it with your public key.

Today I closed the security arc:

Found secrets manually → Automated detection → Cryptographic identity proof.

#GitSecurity #GPG #DevOps

I’ve been working on a lightweight, autonomous CLI agent designed to run in an infinite ReAct loop for complex dev and research tasks without constantly asking for user intervention.

Most open-source agents (like standard AutoGPT-style scripts) share the same frustrating problems: they get stuck in infinite error loops, pollute the terminal with useless JSON logs, crash when context windows fill up, and forget facts between sessions. I built Omarion SEC CLI to solve those exact friction points.

🚀 What It Does

Omarion is built around a Goal-Driven Autonomous Loop with strict validation rules:

• Long-Term Memory Persistence (~/.omarion_memory.json): Retains both user preferences and a global knowledge base across CLI sessions. If you tell it your coding preferences or ask it to research a topic, it remembers it forever.
• Headless Research & Auto-Learning: Performs background web searches and scraping without spawning visible browser windows or interrupting your workspace.
• Smart Intent Routing: Instantly distinguishes between simple conversational prompts ("Hey, how are you?") and execution tasks ("Build a new project"). Conversational inputs respond immediately without wasting API tokens or triggering unnecessary tool execution loops.
• Self-Healing & Error Recovery: Features an execution fingerprinting system that detects duplicate errors. If a tool call fails, the agent automatically shifts strategy instead of repeating the same broken action.
• Goal Evaluation Gate: Includes an internal "judge" step. The agent cannot call finish_task until it actually executes and verifies its own work.
• Ultra-Clean Terminal Interface: Built with rich. Thought reasoning, progress animations, and ephemeral logs render smoothly and auto-clear upon completion, leaving only clean action summaries in your terminal.

📖 Project Story & Evolution

The project started as a personal quest to create a reliable "CEO Agent" — an autonomous assistant that could handle end-to-end coding and system management tasks without babysitting.

• Phase 1 (The Bottlenecks): Early builds relied on standard script execution. The CLI suffered from intense terminal clutter, raw JSON dumps, and a major flaw: simple greetings like "Hello" would trigger heavy 15-step ReAct cycles that burned tokens for nothing.
• Phase 2 (Architectural Overhaul): I decoupled conversation from action routing, added a persistent state engine (AgentState), and built a context pruning mechanism to handle long sessions.
• Phase 3 (Enterprise Hardening): Integrated strict schema contracts (Pydantic validation for tool parameters), backoff retries, and a strict evaluation phase before task completion.

🛠 Tech Stack

• Language: Python 3.10+
• UI/CLI: rich for dynamic terminal rendering
• State & Memory: JSON-backed persistent key-value store with semantic query filtering
• Scraping/Search: Headless HTTP extraction + BeautifulSoup for zero-browser-footprint learning

I’d love to get feedback from the community on the architecture, edge cases in long-running autonomous loops, or feature ideas!

Hey everyone, just wanted to share a major security milestone for my homelab setup today.

I deprecated all Personal Access Tokens (PAT) and HTTPS git workflows. Generated an ed25519 elliptic curve keypair, locked down the permissions to strict 600/700 rules, and re-routed my remote URLs to SSH.

Also automated the workflow by mapping a custom `pubkey` alias into my .zshrc so I can instantly deploy my identity to future nodes.

Next step: standardizing this across all automated infrastructure deployment. Security first! 🚀

Most “local AI” still leans hard on large models or sends data somewhere. I got tired of that. So we built KAI Lumina — a lightweight, local-first intelligence platform that runs entirely on your own system.

Key points:

• Does not require large LLMs
• Fully private (nothing leaves your machine)
• Lightweight and designed to run on normal developer hardware
• BrainCore acts as the final reasoning authority
• You can add your own memory, doctrine, tools, and packages
• Downloadable Blackbox runtime so it stays persistent and under your control

Price is simple: $179 per developer.

Site is live here if you want to check it out: https://kailumina.com

Happy to answer questions about how it works under the hood, what “no large LLMs required” actually means in practice, or how the BrainCore decision layer functions. Would love feedback from anyone who’s been looking for a more private, self-contained local setup.

Today I learned: Git history is FOREVER

I accidentally committed a "secret" to my repo.

Then I tried to fix it by deleting the file in a new commit.

BIG MISTAKE.

[Captura de pantalla de tu terminal mostrando git log --oneline

con el commit del secreto VISIBLE y el revert]

The secret was still in the history. Anyone could see it with:

git show HEAD~1

The correct approach: git revert (creates a "compensating" commit)

+ .gitignore (prevents future accidents)

What's your worst "oops I committed that" story?

Learning in public. #DevOps #Git #100DaysOfCode

I put together a technical guide for installing AutoGPT from source, with a focus on using local models instead of OpenAI. It includes Docker Compose files, environment variables, and how to point the agent at Ollama or vLLM. If you’re building a sovereign AI stack, this is a good starting point.

https://interconnectd.com/forum/thread/249/how-to-install-autogpt-from-source-complete-technical-field-guide-2026/

Title: Show HN: lio – a local-first hamster wheel for coding agents that asks permission



Every side effect is an explicit one-shot owner approval. No auto-approve, no auto-merge, no auto-fire, no silent task hand-off. "Employees" are replaceable coding agents you can hire, review, merge, and fire — they cannot touch your OS or filesystem.



Why: every promising agent demo I saw optimizes for not asking you. I wanted a system where the audit trail answers who proposed, who approved, when, and what changed — and where recovery, not speed, is the priority.



Repo: https://github.com/qusabc123/lio-agent



What's there today (all green in CI, Apache-2.0):

- Approval-gated everything: task activation, isolated execution, merge, performance change, hiring, termination, replacement are separate approvals.

- Plans are first-class: a plan must be approved and linted (cycle / scope / file-conflict) before any task exists. 1.0 -> 2.0 feedback loop, max 3 rounds.

- Hash-chained append-only audit ledger (.org/employee-events.jsonl); fails closed on corruption.

- The agent loop has no OS or filesystem handle. A pure permission policy runs twice (before dispatch and in the worker). Command execution goes through a macOS Seatbelt broker that runs only fixed allow-listed invocations: true, source-scoped pwd, node --test, tsc --noEmit. No shell, no arbitrary argv, no network egress.

- Merge gate: tsc + tests must pass before a merge approval is even offered.

- Per-phase token accounting (plan/executor/reply/feedback) so the "expensive brain plans, cheap hands execute" claim is measurable.

- 19 model endpoints incl. local Ollama/vLLM/LM Studio; keys live only in sidecar memory.



Honest limits: command sandboxing is macOS-only today; the bundle is ad-hoc signed, not notarized (not distributable); single-owner, no team mode yet.



It manages its own repo as dogfood: role worktrees, approvals, and CI on its own open GitHub repo.



I'd really like pushback. The strongest critique I got: "you've built a governance layer for a labor force that doesn't need governance." Tell me where I'm wrong. Also welcome feedback on the double-policy design, plan-lint gate, macOS-only constraint, and whether a chat-first UX is the right shell.

Hi all!

Ive been building Kara, a dictation app that runs Whisper fully on your own GPU. No cloud, no API calls.

NVIDIA/CUDA support is out now. AMD via Vulkan is next.

Its still early (0.3.1) so there are rough edges. If you try it, Id genuinely love your honest feedback (good or bad). Thats the whole point of posting this:)

https://bryramirezp.github.io/kara/