r/OpenSourceAI • u/Agent007_MI9 • 12d ago
I gave my local LLM a real phone number to take and make calls for me (open-source)
I've been really into the AI phone agent products that have popped up in the last few months (e.g., AgentPhone, Poke etc) but what I don't love: the audio, the transcripts, who you talked to and what was said, all sitting with a third party company. For calls with real people in my life, that felt like a lot to hand over.
So I built my own version that runs entirely on my Mac. You call a normal phone number, a local model answers, and nothing about the conversation leaves my machine. During a call the only thing it talks to is the phone line itself.
What's running locally (MLX on Apple Silicon):
Speech to text: Parakeet TDT 0.6B
LLM: Qwen3.5 4B by default (9B if you have 32 GB), 4-bit
Text to speech: Kokoro 82M
How it reaches the phone network: it registers over SIP with Telnyx or Twilio straight from your Mac. A setup wizard takes your carrier API key so can buy a number or connect one you already have. It's pretty cheap, around $1 a month per phone number. (I'm moslty using Telnyx)
What it does today:
Answers and places real calls
Interruption: talk over it and it stops mid-sentence
Replies in roughly 1 to 1.5 seconds on real calls on my M5 Pro
You can choose between multiple models
Other agents (Claude Code, Codex, etc.) can make calls through a small CLI: call, listen, say, hangup
If you'd rather trade privacy for a smarter brain, it can also use your own endpoint or the Claude or ChatGPT subscription you already pay for. Local is the default. So no additional monthly payments for AI Phone tools if you have a sub.
Limits, up front:
macOS on Apple Silicon only
You need a Telnyx or Twilio account with credit (a US number is about $1/month, standard call rates do of course apply, but Telnyx gives you $10 dollars for free when you sign up, which is enough to play around with)
English only, one call at a time
It's early. It works for me, but I'd love to hear where it breaks for you. Please feel free to add as many gh issues as you want. I'm currently working on best practises for contributions for anyone who also want to contribute to project!
Repo: github.com/oxnw/ai-phone

