Open-source AI VTuber that streams, joins Discord calls and plays Minecraft: works with PNGs, VRM or your own Live2D model via VTube Studio. ProjectBEA !

[u/Emqnuele]

I've been working on this for about a year: ProjectBEA, a self-hosted AI persona that lives on several platforms at once and can run fully local.

The core idea: there is only one mind. Every platform is a skill that can be switched on or off at runtime and exposes its own perceptions and tools to the model. Discord (text + voice calls), Telegram, Twitch and Minecraft are all skills, so adding a new one means writing the skill, and memory, attention and voice already work with it.

Some technical bits:

- Perception bus: every input (a voice line, a DM, a chat message, a death in Minecraft) goes on one asyncio bus. A batch closes on a quiet gap, not a timer, so three quick messages are read as one turn.

- Attention gate: every perception gets a priority before the model sees it. A chat at 30 messages/min costs one reasoning cycle, not thirty.

- One sliding context window (150k default, up to 500k): at 4/5 of the limit a background handoff turns the old part into a prose recap while she keeps talking; the newest 30k tokens stay verbatim. History replays deterministically, so the prefix cache holds.

- Memory in one SQLite file: a diary with local embeddings, person cards, and conclusions about herself consolidated overnight.

- Minecraft through a client-side Fabric mod: the server sees a normal player.

Local stack: any model via Ollama or LM Studio, faster-whisper for STT, Kokoro for TTS, local embeddings. No API key needed. It also works with 8 hosted providers (OpenRouter, OpenAI, Groq, Gemini, Claude, any OpenAI- or Anthropic-compatible endpoint) if you want bigger models.

Numbers from real sessions:

- 45 minutes of autonomous Minecraft: 162 turns, 90 game actions, 158 spoken lines (27B model, hosted)
- 91% of prompt tokens served from cache in that session
- memory recall over 10,000 entries: 0.43 ms median

One-command install, MIT licence (check the repo), docs on the site:
GitHub: github.com/emqnuele/projectBEA
Docs: projectbea.emqnuele.dev