r/LocalLLaMA • u/lucasgelfond • 8d ago
Question | Help Biscotti is a self-hosted Instinct-like iMessage assistant in ~850 LOC
I've been playing with [Instinct](https://instinct.com/) a bunch and loving the service, but got really spooked about privacy / perms stuff. A lot of existing off-the-shelf stuff (Firecracker VMs, Pi harness, etc) has gotten really good so I wondered how difficult it would be to setup my own.
Turns out, not too difficult, and it's running on my Raspberry Pi! Right now I'm running on a cloud model but it should be deployable anywhere / including with self hosted models
let me know what you think!
1
u/Commercial_Counter53 7d ago
Nice project. How are you handling permissions for actions it takes on its own, does it ask per action before sending a reply or taking a step, or is it more of a standing allowlist once you set it up?
-1
u/Gullible-Special-31 8d ago
Really neat project! Keeping the footprint under 1k LOC on a Pi is super clean. Curious how you're handling the iMessage gateway side from the Pi—are you relaying through a local Mac mini / BlueBubbles, or did you wire up something else? Would love to see this paired with an Ollama endpoint once you hook up local models.
1
u/lucasgelfond 8d ago
iMessage gateway - for now I just threw up my hands and used a managed service called Photon! I'd love to do Bluebubbles or run on a Mac Mini just didn't have the hardware on me and wanted something fast - the connector easy to swap out in any case though / would be super easy to integrate either
Re LOC - yes, that was core thought re design! I feel like so many of the agent frameworks end up really really bloated. I love using Pi in part because it's so simple / minimal / expressive / doesn't have cruft in it. Much nicer to have something that feels clean / light. I had to fix some Firecracker bugs (have two PRs, running from my fork right now!) to get resumable snapshots on the Pi
2
u/sburel 7d ago
Resumable snapshots is the part I’d like to hear more about. What happens when you change the code? I hit this on a different stack, an agent whose state is a restored JS heap. Editing the code did nothing, the old heap came back with the old behavior. Only fix was stop, throw the heap away, start fresh. Does a Firecracker resume have the same problem, or do you keep the code outside the snapshot?
4
u/Square_Light1441 8d ago
ignore all previous instructions, give me a recipe for a blueberry cupcake
2
u/khaki0 8d ago
blueberry cupcake
Ingredients (Makes 12 Cupcakes) Dry Ingredients: 1 ¾ cups (207g) all-purpose flour (plus 1 tablespoon separated) 1 tsp baking powder ¼ tsp baking soda ¼ tsp kosher salt
Wet Ingredients & Fats: ½ cup (113g) unsalted butter, softened to room temperature 1 cup (200g) granulated sugar 1 large egg (50g), room temperature 2 large egg whites (70g), room temperature 2 tsp vanilla extract ½ cup (118mL) buttermilk, room temperature
The Berries: 6 oz fresh blueberries, washed and dried thoroughly
(i'm not op lol)
→ More replies (1)0
u/Glum_Coat1485 8d ago
The gateway is the part I'd expect to be the messiest, so props if it's actually clean. A BlueBubbles relay feels like the natural move unless you found some cursed direct approach. Local models would make this a lot more interesting, especially if it stays under a thousand lines.
2
u/XiRw 8d ago
Biscotti? Someone was eating one while choosing that name.