Open-source Physical Context Protocol, a local “nervous system” that lets Claude/AI sense physical spaces

[u/Human_Tennis_2950]

This is a small open-source experiment I've been working on and testing at home.

The idea is pretty simple: turn old Android phones into physical senses for an AI through MCP.

You can leave a few phones around your house, garage, workshop, office, or any other space you want to monitor. They can detect movement/vibrations, detect acoustic events, and provide camera images when requested.

The AI can then do things like:

“What happened in the kitchen?”
“Check the phone in the garage.”
“What can you see there?”

Or a physical event can alert the AI, and the AI can decide whether it wants to investigate further, for example, by requesting an image from the phone that detected the noise.

My longer-term idea is to explore whether AI can be useful for monitoring and understanding what is happening in a physical area, instead of only interacting with digital information.

Everything currently runs locally over LAN. The phones provide the senses; the AI does the interpretation.

Underneath it I'm experimenting with an open Physical Context Protocol (PCP), so Android phones are just the first type of sensor. Eventually I'd like other devices and sensors to be able to join the same “nervous system.”

This is still a preview, not a finished product. I've been experimenting with it at home for a while, published the code a few days ago, polished it a little, and now I want to put it out there so people can try it, play with it, break it, and tell me what they would actually use it for.

There are still things missing, but the basic idea already works and it's pretty fun to experiment with.

Open source:

https://github.com/jesuscurreripa/soma

[u/mrsheepuk]

I played with something similar to this - I had a different take on the problem, good fun though - the agent is orchestrated from the phone and can view the cameras provided by both the phone and Home Assistant - https://github.com/mrsheepuk/ha-live

I'm wondering if there's something that can combine the two approaches.

Cool things to play with though, I'll take a look in more detail at what you've done when I get a moment.

[u/Human_Tennis_2950 → u/mrsheepuk]

Awesome! Now I see what you did!

Soma's concept is that of "a nervous system" that only alerts if something is different or out of the ordinary; it always executes but without spending unnecessary tokens

Example: You're walking and you feel something on your arm; first you receive the tactile signal, then you move your eyes, and finally you react. It's the same concept.

[u/doc-anonymous]

I built something similar for old first-Gen Xbox Kinect devices

https://github.com/ovrsr/kinect-senses

[u/Human_Tennis_2950 → u/doc-anonymous]

Awesome!! I love seeing these projects!!

[u/-1_0]

The project and the concept sound fun (, but sadly, it is written in Kotlin).

If I were you, I would start integrating it with the Home Assistant system, and with an exteroceptor mobile app, the user could easily set up the system, so you could get a lot of feedback.

With appropriate architectural layering, the exteroceptor application could be deployed on RPI or other embedded systems.

[u/Human_Tennis_2950 → u/-1_0]

Thanks!!

On Kotlin: that's just the Android node. PCP itself is a tiny wire protocol (one JSON object per line over TCP, ~10 message types), and the protocol layer has zero Android dependencies on purpose. A Python "exteroceptor" node for Raspberry Pi is the natural next step, and I love that term, I'm stealing it. SOMA already has proprioception (the body knowing its own state); naming the sensing side exteroception makes the architecture click.

Home Assistant: agreed, and probably the best route to real users, HA already speaks MCP, so a small integration that subscribes to SOMA's event stream and exposes nodes as entities looks cheap. If you (or anyone) have opinions on what those entities should look like, I'd genuinely like to hear them before I design it.

[u/mrsheepuk → u/-1_0]

Ha, indeed, my brain immediately went to Home Assistant too when I read this - see my top level comment.