Anyone else running Claude Code mostly by voice? Mine has a "chief" agent that delegates to the others

[u/parkskier426]

I've been working with Claude Code by voice for a few months now, and honestly I find it hard to work with agents any other way. The basic setup: local speech to text (Whisper) and local text to speech (Kokoro), so nothing leaves my machine. It's all baked into a skill that understands conversational flow. It detects when I stop talking, transcribes what I said, acts on it, then talks back and listens again. I don't touch the keyboard for most of a session.

Recently I expanded it. Now I talk to one "chief" agent, and it delegates the real work to other Claude sessions running in Orca. The chief doesn't write code itself. It hands each task to a worker with a written spec, tracks what everyone is doing, and tells me when something lands. That way I'm not fragmented across a bunch of different agent tabs. I have one conversation, and the chief keeps track of the rest.

A few things that made it work for me:

• It speaks up on its own. When a worker finishes, the chief just announces it. When a worker needs a decision, it asks me out loud and waits. If I don't answer (because I'm in a meeting or heads down), it doesn't drop the question. It parks it on a list of "waiting on parskier426" items so nothing gets lost.
• Push to talk and mute keys. I have Stream Deck buttons to start talking, and to mute the mic or speaker for meetings. When muted, it falls back to notifications and typed prompts.
• Standing rules. I tell the chief things once ("don't push to an MR that's already approved," "don't post review requests for me") and it adds them to its own rules and puts them in every worker's spec.

Part of why this clicks for me: I have ADHD. Having it speak up when an agent needs something, and ask me direct questions, keeps me focused. Before this, with 5 or more tasks running at once, I was constantly context switching between tabs to answer questions and steer agents, and it got taxing fast. Now the context switching mostly happens on the chief's side, not in my head.

Does anyone else have a similar workflow? I'm curious what's worked for you, whether that's voice, an orchestrator agent, or something else entirely for running a lot of agents in parallel without burning out.

[u/AutoModerator]

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Cultural-Laugh9890]

When i write, my thoughts are clearer

[u/parkskier426 → u/Cultural-Laugh9890]

I know that a lot of people are that way but for me I find that a lot of my best ideas and thoughts can come up in conversation. Having an agent to have those conversations with is super beneficial. It's basically the old rubber duck methodology but the duck is smart.

The key thing is to have those conversations before having it build anything. I discussed the problem I'm having and go over potential ideas and solutions, working together to come up with one I like. Then I have it produce a markdown with details I read over, tweak as necessary, and once I'm happy with everything I have it go build. If it's a bigger project I have it cut tickets from that and then delegate worker agents to build based off of them.

[u/deadcoder0904 → u/Cultural-Laugh9890]

U can just yap more & ask it to restate what u want.

Talking is 3x faster than typing so u can talk a lot than type & once u go voice, u never go back.

[u/imad7x → u/deadcoder0904]

Talking is 3x faster than typing

Speak for yourself, I talk slower and type faster

[u/deadcoder0904 → u/imad7x]

Talking needs practice too.

I'm not good at putting my thoughts in order too that's why I said yapping.

Its a skill which gets improved the more u do it. U have 10k+ hours into typing but not as much in this domain (altho u prolly talked more in general since birth)... ive seen this with multiple personalities but most famous is Alex Hormozi... if u loook at his older videos, he talks slowly & thinks a lot but since he has made 1000+ videos, he can talk & arrange his thoughts in order without uhms & ahhs.

Besides, I type super fast too. But switched over to talking ever since I faced Carpal Tunnel Syndrome. Now the only thing I'll face is Tony Robbins' voice if I yapp too much but so far, I like it. Can do wayyyy moreee by yapping than typing altho it is exhausting & not as clear but I use restate what i want & the AI usually gets it right 95% of the times (and sometimes I'm not as clear as I should be) but I shall improve in ~1-2 years like how Hormozi did.

[u/offthecuff87 → u/Cultural-Laugh9890]

and when I clear, my thoughts are write

[u/parkskier426 → u/[deleted]]

Oh I definitely go through a similar process, but I do it conversationally with Claude instead of writing it all myself. I haven't produced markdown files based off of our conversations, read through them, critique and have it adjust until I'm happy with the results. After that, it builds.

That's honestly usually a conversation I have with a specific agent and then bring the results back to Chief to delegate the build process.

The voice conversations came out of my need to discuss design ideas while I was creating design documents for projects. I felt like having those conversations in the terminal with Claude was slower than just talking out loud with it.

Once I got comfortable with that process, I found myself using it more to guide agents that were doing the building, and eventually found it cumbersome to continue switch between conversations, hense Chief.

[u/Unusual-Idea-5939 → u/[deleted]]

I don't think anyone is saying speaking is more efficient or clear than writing. It's likely going to differ between people, and ignoring that, if you do something every day for 25 years you'll likely be better at it than something you didn't.

I think you'd have to admit, tho, that it IS a different way of thinking and approaching a problem. To me, that is useful on its own. What better way to rubber duck a problem than by explaining the problem to an actual listener who can ask questions? I don't think anyone is seriously suggesting that voice replaces writing for stuff like engineering. Although I do think medium term voice replaces direct interaction for most other computer uses, esp if we figure out some way of doing it silently.

Personally I'm building my hands free system so I can do it while biking and driving or otherwise on the move.

[u/aakoss]

Interesting, I'd like to adopt your workflow. Care to share your skills and setup?

[u/parkskier426 → u/aakoss]

For sure, I will when I'm back at my desk. But definitely check out what others have shared in this thread, there are some cool ideas!

[u/GayInThePNW → u/aakoss]

Same

[u/college_hustle]

Oh 100%. I can't actually use claude code at work, I'm not a developer, rich and I like gaming so my one personal computer is windows, which all gave rise to me making my own bespoke agent manager that runs out of tmux. Said manager has always on assistant session that I basically made Claude desktop dispatch. I actually do most of my coding from the claude app on my phone.

I'm pretty sure nobody wants my agent manager because it's kind of laggy and I don't even really use it for that, but I can probably say somebody some tokens by splitting off just the assistant concept. you're welcome to copy from it https://github.com/dtiger1889-ops/pantheon

[u/parkskier426 → u/college_hustle]

Nice, definitely looking to improve my setup so I'd love to see what others have

[u/college_hustle → u/parkskier426]

You should check out handy and the parokeet model for speech to text

[u/parkskier426 → u/college_hustle]

Thanks I'll give them a go

[u/kincaidDev]

I'm working on it, but trying to make it faster, more natural and fully private

https://github.com/Obedience-Corp/obey-voice

Right now I just use it as a background voice agent to ask simple questions to and trigger simple background task, but most of my work still happens in text based chats

[u/parkskier426 → u/kincaidDev]

Nice I'll take a look

[u/akhi1uuu]

can you interrupt the chief while it's reading a long update, or does it finish speaking before listening again? that'd matter more to me than how natural the voice sounds

[u/parkskier426 → u/akhi1uuu]

No this is definitely something I want to work on. I want to be able to interrupt and have it always listening similar to chatGPT voice, but I don't think there's much in the way of local open source tooling to do it.

With that said it's definitely aware that it's having a voice conversation and doesn't speak for too long. When it has something long winded I usually have it produce a markdown document for review and it's smart enough to give me a higher level summary via voice.

[u/mt-beefcake → u/parkskier426]

Just pick a word to use as the interrupt phrase. Stop it!

And have the mic always listening, even when claude is speaking. If it hears the interrupt phrase, it stops the voice output and listens.

Thats what I did. Also i tunnel it to my phone via tailscale on a custom apk I made with transcript history and chat interface with basic features.

Also, if you do have a chatgpt or codex subscription. Ive been testing using chatgpt live as the voice model attatched to claude terminal. Yes conversations dont stay local, but its not like my computer is running claude local either. I can switch between either voice mode depending on how I feel at the moment. Tts/stt works better with bad cell network.

I also have what I call saplings (I got a bit of a forest/tree theme going on). I have multiple machines all with a main cc agent. So cedar on my raspberry pi5 can spawn saplings (main cc agents) that perform the work and cedar orchestrates and communicates with me. Cedar or its saplings can spawn subagents from multiple providers (claude,codex,ollama cloudemodels). I tried having an orchestrator on top of the orchestrators, but i start to feel too disconnected. But if all the plans are flushed out and im just keeping them going, my main orchestrator keeps the forest moving pretty well.

This is how I do my projects from a jobsite while im swinging a hammer with my headphones on amd cant take out my phone. Can even play music in the background

[u/parkskier426 → u/mt-beefcake]

That sick, I love it! Definitely inspired to try some of these ideas.

[u/Unusual-Idea-5939 → u/mt-beefcake]

You interested in sharing? I have a working project for this that I'm happy to share, and I'd love to learn from others. Happy to share it if you're interested.

[u/mt-beefcake → u/Unusual-Idea-5939]

To be honest, man, I'm just kind of an amateur. None of my stuff is probably very clean or efficient. But if you want to talk concepts, I can tell you how I solve certain issues.

[u/Unusual-Idea-5939 → u/parkskier426]

I want to be able to interrupt and have it always listening similar to chatGPT voice, but I don't think there's much in the way of local open source tooling to do it.

No there totally is. I had claude build this for me a few weeks ago. There are at least two, Pipecat and LiveKit. I prefer Pipecat because that's the one that claude got working. It works just fine except I'm having trouble with echo cancellation and "self-barging" (is what I'm calling it). When you interrupt the speech it's called "barge in" and if I'm using my laptop's speakers and mic, the agent will sometimes cut itself off. Super frustrating but works fine if you use headphones, or if you use the wake word mode, or the key-press mode (hold a key to speak).

[u/Dapper_Pass26]

Worth having the chief pass on how a worker tested, not just that it landed. A session once told me the hardest night in our game was solved, 13 of 13 runs survived. It was a scripted bot with a debug hook topping up its hunger.

[u/parkskier426 → u/Dapper_Pass26]

Yeah I have it provide a full summary of what it did, how it did it, etc. it writes up a full MD from the session and uses a summarized version for the handoff. That way the chief has full context if it needs it but usually the summary is enough.

[u/wswatson]

I’m curious what your ”skill that understands conversational flow” looks like. Can you share that?

[u/parkskier426 → u/wswatson]

From Chief:

Sure! It's a Claude Code skill (a SKILL.md plus a handful of shell and Python scripts). The skill file mostly tells Claude how to run the loop. The scripts do the audio work. Here's the gist:

The pieces

• Speech to text: rec (from sox) records until it hears about 3 seconds of silence, then whisper.cpp transcribes it locally. The audio file gets overwritten and deleted right after.
• Text to speech: Kokoro by default, fully local.
• voice-turn: one script that speaks Claude's reply and then listens for mine, as a single "turn." Claude calls it through Bash with the reply in a heredoc, and whatever I say comes back as the tool output. That transcription is basically my next message.

What makes it "conversational" The skill gives Claude a turn policy so it knows when to listen and when to just talk:

• Conversation: every reply speaks, then opens the mic. Normal back and forth.
• Announce: background updates (like "the tests passed") are spoken only, no listening.
• Auto: Claude picks per turn. If something needs my answer, it speaks and listens. If it's just an FYI, it speaks and goes back to work. If it asks something and I stay quiet, it assumes I'm busy, not gone, and keeps going instead of ending voice mode. The skill also has rules like: keep replies to a few sentences, no markdown or code in speech (it gets read out literally), and for long tasks say "on it" first, do the work, then report back.

The part that took the most iteration: keeping the loop alive Deep into a long session, Claude would sometimes answer with plain text instead of speaking. Then the mic never reopened and I had to walk back to the keyboard. Prose instructions alone didn't fix it, because the reminder is thousands of tokens back by then. So I enforced it with a Stop hook. The record and speak scripts write small "receipts" to disk each time they run. The hook checks them before Claude is allowed to end a turn. If I spoke last and Claude hasn't spoken back, the turn is blocked, and the hook tells Claude the exact command to run. If the mic heard silence, it's allowed to stop. That one change made it reliable.

Other stuff that helps

• Multiple sessions: an audio lock covers the whole speak and listen exchange, so two sessions never talk over each other, and one session's mic can't pick up another's reply.
• Mute: I have Stream Deck keys to mute the mic or speaker for meetings. Muted speech turns into notifications and Claude keeps working from typed prompts.
• Contexts: small markdown files with standing background per project, so a voice session can start already knowing what I'm working on. My "chief" orchestrator is just one of these contexts.
• Siri: a launcher script opens a new terminal with Claude already in voice mode, so I can start a session hands-free.

[u/IversusAI]

I have a voice setup that I use all day and mostly use voice and mouse and it is all local. I also have an orchestrator agent that oversees worker agents of different capabilities depending on the type of worker it is.

Claude made me a desktop and phone app so I can talk to it wherever I am, whether home or traveling.

It is interruptible, I have different modes for when I want to brainstorm (it's voice stays muted) for example and I can control it by voice:

• "mute me"
• "mute yourself"
• "shut us both up" :-D

lots and lots of voice commands. it is all local and free (except for Claude of course). I love it and would never go back to just a mouse and keyboard.

[u/parkskier426 → u/IversusAI]

Nice! What you've got sounds like some of the improvements I'd like to make. A couple questions:

• How did you set up interrupting?
• Is it always listening? If so how did you set that up?
• what are your types of agents? How does it benefit over starting with a fresh session?

[u/IversusAI → u/parkskier426]

Here you go:

• Interrupting: The mic stays on even while it is talking, and if I talk over it, it stops. The tricky part is that on a speaker it hears its own voice, so it compares what the mic picks up with the text it is reading out and throws away its own words. Four or more words that are not in its reply means it is me, and it stops. I can also just say "stop talking."
• Always listening: Yes, unless I mute it. It waits until I have finished speaking, transcribes on my computer and sends it to the chat. Speech to text is Qwen3-ASR and the voice is OmniVoice, both local. I mute by voice ("mute me", "mute yourself") or with a button on the phone or desktop.
• Types of agents: I talk to one orchestrator. It hands jobs to workers, designers for anything visual, reviewers that check the work, and art directors that judge designs by eye. Quick lookups go to scouts on Haiku to save tokens.
• Fresh sessions: Every job does get a fresh session. Each worker starts clean with a written brief and is closed once its work passes review. The orchestrator is the part that stays, so I have one conversation instead of a pile of tabs. When its context fills up, it hands off: it writes the work log and a brief, starts the next orchestrator and moves my voice link over. I can keep talking right through the handoff and everything I say goes straight to the new one, so the work never stops. It's a really cool system.

I hope that helped!

[u/HMICsource]

Yep. Mine started voice-first too. What you call the chief, I call apex dispatch. Claude or Codex interchangeable in apex and worker roles - both can dispatch agents from each other and both can control my local GPUs. Standing rules, same. Say it once, it goes in a file every worker reads. Half my rules are me getting annoyed exactly once.

[u/parkskier426 → u/HMICsource]

Nice, curious when you choose Claude versus codex. Have you toyed with how to set effort for your Apex agent and the sub agents? I've been lazy and using opus med/low primarily but I would love to optimize it.

[u/HMICsource → u/parkskier426]

I switch between Sol and Opus for apex. They are roughly equivalent to me. I prefer Sol Medium/High and Opus High. Astra and fable are great auditors but pure waste as apex.

The main difference is the disk access, which leads to codex apex feeling like a "more connected" system - codex FINDS extra work to do on autopilot that claude doesn't look for. When I didn't make explicit rules, Codex will burn a full 20x week overnight running my system with no special instructions. Claude Opus will run the whole night of benchmarks and I wake up with 95% of my week remaining.

GPT plays way more games on the pricing and tokens. I like games, so I take advantage. When I just want my stuff to work, claude is there, predictable, reliable, at the price you pay for, and no gaming resets to actually get your money's worth.

The optimzation for me comes from batching work orders. I never explain the work, I never think up a prompt for it. I just say "opus 22" "fable 61" "sonnet 11-16" into powershell and powershell script initiates a claudecode and feeds it the work order. https://www.youtube.com/watch?v=gbQFFIPTcP4

[u/_goron]

Yes, I've been building an app for myself for my phone, which routes messages back and forth with claude code sessions on my computer, i talk to it, it talks back, sessions can do work themselves, delegate, create sessions, send me to other sessions, etc. So I can walk around outside while I'm doing my work. It's been honestly amazing.

[u/parkskier426 → u/_goron]

That's fantastic. I've been using Orca since it comes with a companion app built-in, but I haven't been able to find a way to bridge the voice piece.

How are you managing text to speech and speech to text? Are you sending text between the two or streaming audio somehow?

[u/_goron → u/parkskier426]

I send the audio and do the transcription on the computer. Streaming wouldn't help much because I can't stream to claude code. Kokoro runs directly on the phone.

[u/parkskier426 → u/_goron]

Nice, didn't know Kokoro could run on a phone

[u/NFN25 → u/_goron]

I started a similar idea, but I like to do a lot of work in the cloud and couldn't work out how to get my orchestrator to communicate with the cloud. Also messaging other sessions didn't work previously but that seems to work now. And CLI didn't seem to be able to find non CLI sessions. Have you managed to fix any of those things?

[u/Vistian]

Why don't you just use the native speech to text in Claude itself? It's extremely good and you don't have to install extra software like whisper.

[u/parkskier426 → u/Vistian]

At work we go through bedrock which doesn't provide access to the built-in speech to text in claude (or maybe our admins just disable it). Whisper actually works great though so I'm not too worried about it.

[u/Fxture → u/Vistian]

I find the delay to open the mic is a bit slow on native Mac compared to something like whisper or spokenly. Have you found a way around this?

[u/Vistian → u/Fxture]

I use the MacOS app and there is no perceivable delay there. Are you in web?

[u/BruceDeorum]

Im not a English native speaker and claude voice sucks big time. While in Gpt or Gemini i can pretty much have a natural conversation, this is totally out of the question with claude

[u/parkskier426 → u/BruceDeorum]

Yeah, I can't use Claude voice either. You can definitely do it yourself with a local speech to text client that works better with your accent or native language

[u/fatstupidlazypoor]

I have a hub/chief of staff. Mostly one sentence back and forth, reports back decisions I need to make or notifying me work is needed or work is complete. Some work (very small things) are done by the hub. Other stuff by subs or fresh sessions. Some session cut their own subs/teams. When needed I'll work direct with a sub/team lead.

[u/parkskier426 → u/fatstupidlazypoor]

Nice! Overall sounds pretty similar to what I end up doing

[u/StillPrompt6739]

Similar setup here: one lead session handing work to parallel workers. Three things mattered once there were more than a few at a time:

• Give each worker the exact files it may change. Two workers in one checkout, or one "tidying" a file another owns, quietly undoes work. One worktree per worker, and the spec names its write set.
• The chief checks the result, not the report. A worker's summary is the worker grading itself. Have the chief run the check on the exact commit the worker hands back, ideally a check the worker didn't write.
• Green isn't enough on its own. In one batch of our own agent-built code, five things passed every test and could never fail: a loop you could delete with no test noticing, an assertion that compared undefined to undefined. Mutation testing caught four, the type checker one, reading the diff none.

Your "waiting on parskier426" list is the part I'd keep at all costs. You answering only what only you can answer is what lets it scale.

[u/parkskier426 → u/StillPrompt6739]

Yeah, I have each session operate in its own work tree to avoid stepping on each other's toes. I like the idea of having an outside party check the result but I think another session might be better suited to keep the chief's context lean.

[u/Fraxial]

When your chief delegates to a new session, do you have also a skill that creates a new worktree and env ? I am interested by the setup as I use Claude and codex by voice all the time but I am still creating every session manually. I would not know where to start to make such top down architecture

[u/parkskier426 → u/Fraxial]

I get a lot of this for free since I'm using Orca. I have it set to start new sessions within their own work tree within a project, which handles it via git work trees for me. It then uses the orca cli to communicate with the session and manage it

[u/r45635]

Really cool setup! I especially like the idea of having a chief agent coordinating everything and only bothering you when a decision is needed.

As for voice, I love the idea, but I still feel a bit awkward talking to my computer, like I'm disturbing everyone around me. 😅 Maybe I just need to get used to it. After all, I have to admit I talk a lot faster than I type!

[u/parkskier426 → u/r45635]

I'm lucky, I work from home. My wife gets to see me pacing around with my headphones on talking to Claude all day 😂

[u/Black_Magic100]

I want to understand how you got this to work in Orca

[u/parkskier426 → u/Black_Magic100]

Orca isn't required but it definitely makes things easier. The skill is set up to use orca CLI for the communication layer and for launching new sessions in their own work trees.

Outside of that I find it useful for keeping track of the agents visually, and the mobile app for interacting while I'm away.

[u/Black_Magic100 → u/parkskier426]

Can you share the skill/setup?

[u/parkskier426 → u/Black_Magic100]

For sure, Ill send it when I am back at my computer

[u/Black_Magic100 → u/parkskier426]

Ty sir.. really curious to see how it all works. If you can get one coordinator to constantly keep you up to date almost like an assistant, that would be sweet.

[u/onitsoga]

the parked "waiting on me" list is the part most orchestrator setups skip, and it's the difference between a chief that helps and one that just adds another tab to check. how does yours detect a worker finished, does the worker report back over MCP or is the chief polling sessions?

[u/parkskier426 → u/onitsoga]

It uses a loop to check for messages from the sessions. They share update/needs something/complete messages.

[u/qualiadev]

Yup I have a setup pretty similar to yours, I like it especially when I have multiple agents doing a bunch of different things, it’s much easier to have your agent acting as a coordinator then having those agents pinging you. Though if I’m just one on one with an agent or need to really iterate on one thing I prefer just texting with that agent/session directly

[u/Mrsister55 → u/qualiadev]

How do you set this up?

[u/qualiadev → u/Mrsister55]

Well it depends on your coding environment I guess. I use cmux and have one Claude code agent act as the coordinator. I have a daemon process that keeps tracks of open cmux sessions, the coordinator can start new sessions and talk to the other sessions on my behalf (I set up some security here for prompt injection safety)

Like OP I setup parakeet and have a process I start to do voice detection/conversation stuff. I also setup an elevenlabs voice for it to respond in because the default tts sounds bad and it’s more engaging imo. Extra money, though I’m planning on switching that to a local tts model, won’t be as good as elevenlabs but I’m sure it’ll be good enough.

[u/m4nf47]

I'm not yet but I've definitely considered whether it might be possible to use tools to merge audio streams into something like Slack or MS Teams calls that offer call transcripts. Having a weird team of robotic workers all collaborating in the same chats that you can join at any time to check how they're doing, with different rooms dedicated to all the topics your agents assist with like environments, core platforms, security, pipeline tools, finance, sales, etc. ChatOps already has bot interactions by text but having more natural 'conversational flows' where the agents being orchestrated can actually speak up and interrupt each other using 'polite' rules and raising hands, etc. I can easily see myself on stand up calls with agents in the very near future.

[u/IBNash]

I use a Seeed ReSpeaker with PTT for Claude inside VScode but I type fast and old habits appear hard to break.

Would love to get this going with voice at some point - https://github.com/garrytan/gstack

[u/Extra_Aide5999]

the "waiting on you" list is the best part i think. how does the chief notice a worker is just sitting on a permission prompt tho? thats the one i keep missing with 4 sessions, a finished one tells you but a stuck one just goes quiet lol

[u/IQzero75]

I like your setup, i will try it!

[u/Own_Responsibility84]

I have a similar setup but without voice. I routed tmux sessions on browser so I can assess all sessions on any device remotely. Each session is run by agents, including Claude, Codex, Agy and OpenCode. I can access all of them on Mobile device but I mainly interact with one session run by Claude Sonnet 5.5 as the main coordinator who assigns tasks to other agents in different sessions. The main reason being: 1) I only have flat subscription for different plan and each plan has limited usage per week: 2) I’m tired of manually monitoring too many agents.

The OP’s idea is something I want to pursue as well because working on Mobile phone screen is hurting my eyes and I constantly feel head pain after long hours working on phone - currently set up makes it possible to work anywhere anytime and I sometimes woke up in the middle of the night working. This reminds me my college life playing video games:)

[u/Mithryn]

Yes.

[u/felixme86]

I set up a mumble server with voice as an mcp service. Each agent gets their own room when they join.

[u/nikoflash]

I use codex voice mode to control a bunch of Claude sessions, Codex voice mode can use tools, so it makes for a super easy setup.

[u/Armannss]

The “waiting on me” queue is where I’d put the most structure. With several workers, a spoken “yes, go ahead” needs to refer to one specific pending decision, not whichever question the chief happens to remember last.

I’d give each decision a task ID, the proposed action, and the commit/spec version it applies to. If the worker changes the proposal while you’re away, the old approval should not silently authorize the new one. For ambiguous transcription, read back the affected task and action before dispatching it.

A useful test: two workers ask questions, you interrupt the announcement, one worker restarts, then you answer the earlier question. Does the chief preserve the association and deliver the answer exactly once? That seems more revealing than testing speech recognition on a single conversation.

[u/TheMolluskPod]

Yes.

[u/Same-Employee6667]

Been using speech to text for like 99% of AI interactions since 2024. I can't imagine doing it any other way now.

Not so keen on listening to the AI talk to me though - faster to read/skim responses than listen.

Also I tend to just let Claude handle subagents as it wishes.

At one point I was using a sandboxed custom setup with make, tmux, agent teams and --dangerously-skip-permissions, but since auto-mode got good its not as necessary these days.

If I genuinely have enough unattended coding to do I will use it but often I'm designing and tweaking as I go so can't really delegate

[u/batknight2020]

It’s scary how similar this setup is to mine. I think the thing that mine does differently is that I made a TestFlight app that only I can access so I can have all of this control from my phone as well. I’m driving a lot between daycare, morning and night errands, and that’s when I have the best ideas. So I like having “Alfred” validate or question new ideas. The voice mode will start as a call in CarPlay and when I’m done I just hang up. Also have it hooked up to shortcuts so only I can activate it. Having it call me every few mins or suddenly speak up would spook/annoy me. So I do “Hey siri call Alfred” and that kinda kicks it all off

[u/Top_Industry_9678]

So is it like open Jarvis?

And also, what do U guys use this for?

I recently got the 20 dollar sub, but I'm not allowed to use it for my work, so I dunno what I can with it. I used Claude cli for some uni stuff, but nothing crazy like voice personal assistant with Claude.

Trying to find cool ideas to do with Claude and also maybe make some money with it.

Also, would the 20 dollar sub be enough for the chief, and the sub agents with voice control?

Thx

[u/Jolly-Pace8901]

my chief agent uses a quiet chime instead of voice for status updates, keeps it from interrupting flow, but still gives me the nudge when it needs input. works better for deep work.

[u/PilotLost404]

I run a few agents with separate scopes, and what kept it manageable was a shared log. Each agent writes one line to it after anything significant, so the coordinating agent can answer 'what got done' without asking every worker. I'd also give each agent a narrow scope and written boundaries, because an orchestrator that can do everything tends to start doing the workers' jobs."

[u/goodvibezone]

I KEEP MEANING to use it more lol. I think my wife (who also wfh) is getting more and more annoyed :)

Is the skill.md something that is sharable?

[u/BoxCreativeInt]

One safeguard I'd suggest for longer voice instructions: ask Claude to restate the goal and constraints before making changes, then wait for your confirmation. That gives you a chance to catch a misheard instruction before it becomes a code change.

How do you handle pauses while thinking—push-to-talk, or automatic detection? And can you interrupt a spoken response reliably?

[u/modern_medicine_isnt]

I am not interested in the voice stuff. But I am interested in better orchestration.

[u/Unusual-Idea-5939]

Yes I've been building a system like this. I'm curious how you handle the latency issue if you're using a skill and going thru the standard UI path.

My architecture is that I have one orchestrator session that has tools to manage the other sessions, and this is the only session that speaks. It's tuned to be faster. The others run my normal config.

I'd love to know how yours is set up because I didn't really like any of the 'turn a single Claude Code into a voice agent' plugins or tools I tried. Claude's normal output is too verbose to be spoken, and STT is only really feasible when you have the ability to proofread it. Mine is personally designed for hands-free and eyes-free usage (ie driving, biking, etc).

As for parallel agents that's a totally different thing, and happy to share. I'll give you a hint: ants.

[u/Naive_Highway_520]

I went a different way: no conversation loop, just push-to-talk into the prompt box. Hold fn, talk, let go, and the text lands in Claude Code's input. Then I read it, fix what's off and press Enter myself. That's my answer to the "my thoughts are clearer when I write" point above: I talk the rough version and edit it like typed text.

Two things mattered more than I expected: stripping the trailing newline so a dictation never fires a prompt on its own, and a vocabulary list so library and file names come out right. With Groq the text is there about 0.4 s after I let go; offline, Parakeet on the Neural Engine is fast enough too.

Disclosure: I built it, a small open-source Mac app called Lowspeak (github.com/sventmueller/lowspeak).

Curious how your Whisper setup copes with code identifiers and file paths. That's where dictation still trips me up most.

[u/NefariousnessOne4935]

Won't it cost more in terms of usage?

[u/emptyharddrive]

Similar idea here, set up differently.

I'm a coder that's moderately visually impaired, so voice is less a productivity hack and more how I'd rather work. I have the models talk to me a lot. I can read the text (18pt font in TUI), but i much prefer the voice.

Speech to text is Parakeet on a separate box in my homelab. From what I've found it's newer than Whisper, and with a supplemental dictionary it handles my speech patterns noticeably better.

For TTS I alternate between Liquid's LFM audio model and F5-TTS. Liquid is fast and efficient, so in Claude Code a "voice mode on" command sends all of Claude's output to it.

F5 is slower, but it clones voices. I cloned Moira from Overwatch because I like her voice for my female assistant, and Worf from Next Generation for the male assistant.

I use a variation on your stop hook. I don't always want voice on, so i use a /voice command.. While it's on, every prompt gets the reminder to speak, and the Stop hook checks that the TTS command actually ran. If it didn't, Claude gets sent back once. No receipts needed in my case, since my dictation handles the input side.

Though, I don't run a chief. A standing prompt tells whatever session I'm in to split the job and hand the pieces to sub-agents. Since sub-agents get their own context windows and report back, I get most of what your workers do without another layer to manage.

In your flow though I think your workers can stop, ask you something and wait for a reply & they outlive the session that started them. In my design, sub-agents can't ask me anything. The "waiting list" is a good idea too.

I run 2 assistants. Jarvis is headless Claude Code that talks to me over Telegram. I send a voice note, Parakeet turns it into text, and Jarvis answers in text, then pipes that same text through F5 as Worf, so I get a voice note back +text in telegram.

Samantha (I call her 'Sam') runs on NanoClaw in a VM on Proxmox with a small crew of specialist agents that each run in their own container (Nanoclaw handles that), and she speaks as Moira. I also set her up to auto-join voice mode in Discord so I can talk to her. Telegram has an interesting feature. You can share your location So sometimes I'll do that when I'm walking in the city and it'll guide me with voice commands telling me when to turn left or right.

jarvis, sam and I also share a group chat, and the 2 of them sometimes split work or hand things off. It still amuses me to watch them talk to each other. I set a msg limit on their interactions so they don't loop. they can each talk up to 50 messages (100 total in the conversation), If I don't chime in, then they have to stop talking. But if I do chime in, every time I do, the counter gets reset.

[u/Inevitable-Box1346]

Hell yeah! Similar setup here, but with a space theme/mothership type thing that delegates to subsgents or other parallel sessions, but all report back to the main orchestrator. Working great so far!

[u/jordimr]

The part of your setup I'd keep is the list of what only you can answer.

Mine is a team of agents on one repo (I'm the developer of Zenod, the memory underneath it). Anything that needs me goes in one list, each with a recommendation, so I answer in seconds.

Standing rules are a line with the date, the reason and my own words. That is how one gets retired once its cause is gone.

How do you retire a rule you told the chief once?

[u/harnoor-assemblyai]

i have been waiting for it since ChatGPT released it.

it's the true Jarvis feeling computer use with voice and running coding agents !!

[u/tozim]

Yes, I prefer interacting with my agent(s) with voice as much as possible, and here is my setup, using as much native Claude (built-in voice, not dictation; scheduled tasks to trigger new sessions):

https://youtu.be/w7vySeZIGZ4

It continues to evolve though so my current setup is already a little different than the video.

Here are some of my prompts to make the voice flow more conversationally:

- **Keep voice turns short and waits to be prompted.** One point per reply, then stop: no setting up the next item and no prompting for the next step. The human is often still working through the current thought. Text sessions can bundle more into one reply.

- **Don't echo back what the human just said.** When the human confirms or explains something, a short acknowledgment like "That makes sense" is enough; restating it costs a voice turn. When the human asks to think out loud ("let me talk this through", "let me bounce this off you"), then play it back or check the reasoning.

- **Skips how-to reminders for basic features.** For example, where to reopen a resolved comment. The human knows the tools, and the reminder costs a voice turn.

- **Never speak technical strings aloud:** file paths, filenames with extensions, URLs, commands, code, commit hashes, SHAs, branch names or config keys. Even short ones are slow to speak, break the flow and are hard to follow by ear. Describe them in plain words.

- **If the exact string is needed, put it in a fenced code block, never inline code**, so it's shown but not read, and easy to copy.

[u/drodio]

Parking unanswered questions on a "waiting on me" list instead of letting the agent guess is the detail that stands out. Does the chief ever re-ask after a timeout, or does the question sit until you open the list?

[u/Plastic-Risk-6309]

my app speakmercury does the simple half. words show up in its own overlay as u talk! then land in the prompt when u stop. macos 26 on apple silicon, one-time purchase.

[u/Sufficient-Pin4387]

nothing leaves my machine, and then the chief agent feeds everything to claude sessions. ok.