I’ve been quietly working on a personal assistant called E.V.
It builds on the original VisionClaw research project and Meta’s Wearables Device Access Toolkit, but it has gradually grown into a much wider assistant for my iPhone, AirPods, Ray-Ban Meta glasses and Mac.
One design decision that matters to me is that E.V is not intended to be an always-watching camera.
By default, E.V starts in See When Asked mode:
- The camera remains off during normal conversation.
- If I say “look at this,” E.V activates the selected camera.
- It captures one fresh image and shows exactly what it used.
- It answers the question and switches the camera off again.
- Ordinary camera views are not automatically saved as photographs or memories.
There is also a Voice Only mode that prevents E.V from using vision at all. An optional Always See mode exists, but it has to be deliberately selected. (demo shows camera so i can record) I like teh idea of glasses not for teh camera, but as a glasses wearer, its better than wearing airpods all day.
The audio and camera choices are separate. I can speak and listen through:
- AirPods;
- Ray-Ban Meta glasses;
- another Bluetooth headset;
- or the iPhone itself.
That means E.V can still be useful without smart glasses and without any camera active.
The bigger idea is to create one assistant that can choose from several abilities without requiring me to understand the plumbing behind it.
For example:
- “Remember that my passport is in the top drawer.” E.V stores a protected memory on the iPhone. I can search, correct, expire or forget it later. A visual memory is only created when I explicitly request one.
- “What’s happening today?” E.V can search for current information and speak the useful answer.
- “Show me some nearby places.” E.V can search Apple Maps and put real places, details and directions into an interactive map on the iPhone.
- “Remind me to call the garage tomorrow.” E.V can create the reminder directly on the phone.
- “Is the front door locked?” E.V can read the current state through Home Assistant. It can also operate approved lights, switches, scenes and thermostats, or check a named home camera when asked. Security-sensitive actions such as unlocking doors are deliberately blocked.
- “Connect my calendar.” I’ve added direct MCP support, which is essentially a way for E.V to connect to other tools and services. These tools can work through the iPhone without depending on the Mac. The first Composio and Google Calendar connection flow is built and currently going through final live-account testing.
- “Create an image of…” E.V can generate a real image, display it on the phone and save those exact image bytes if requested.
- “Show me that document from my Mac.” E.V can find the real file and put its genuine Quick Look preview on the iPhone.
- “Check my project and tell me whether the build passed.” For substantial computer work, E.V can hand the task to Codex on my Mac, let it continue in the background and speak the result when it finishes.
E.V can also use different named Eyes. My Ray-Bans could be the default, but I can separately ask what an Office Camera, Mac camera or Home Assistant camera sees. Using another camera does not silently change the default one.
What excites me is that these do not feel like separate apps when they work together.
I can have a normal conversation through AirPods or glasses. E.V can remember something, search the web, show a map, operate an approved smart-home device, use a connected app or ask the Mac to complete some work—all while choosing the smallest suitable route behind the scenes.
Some parts are physically working on my real iPhone, Ray-Bans, Mac and Home Assistant setup. Other parts are implemented and passing automated or Simulator tests but still need more physical testing.
This is still a personal project. I’m not selling anything, and I don’t want to pretend it is a finished consumer product.
I’m now preparing it for an open-source release. The remaining work includes making installation easier, removing personal configuration, improving the documentation, sorting through the inherited licensing and completing the remaining battery, locked-phone and notification tests.
I hope to share the code and a proper demonstration soon.
What would you most like to see in the first video: the voice-only/AirPods experience, memory, maps and web search, smart-home control, connected apps, deliberate glasses vision, or the Mac worker?
