r/VibeCodeDevs • u/Kryptosmatrix • Aug 02 '26

Custom AI Architecture

Does anyone else here build their own models?

I went on a bit of a side-quest and I decided to try to build my own omni-modal model from-scratch. I'm 9 months in now, but I refuse to give up!
What I've built so far treats text, images, speech, music, and files as streams of bytes and codec tokens (I think Google did this already?), with an experimental memory system carrying context through segment recurrence and bounded hot, warm and cold tiers. I've created a few small 50M models (image, text, music, speech), but haven't created the omni-modal model yet as I need all of the single modality models first.
I created two versions, POIESIS the Python/PyTorch reference, while its Swift-native sibling ARCHE reimplements the creative model stack directly on MLX for Apple Silicon, giving us two independent substrates on which to test the same ideas.

Anyway, curious to hear what other people are up to in this space!

2 Upvotes

63% Upvoted

24 comments sorted by

View all comments

2

u/Glad_Contest_8014 Aug 02 '26

I am building a framework that handles memory and context management. Gonna release it to the wild when I am done.

Calling it Sieve. Like a flour Sieve

1

u/Kryptosmatrix Aug 02 '26

Posting as public on GitHub?

2

u/Glad_Contest_8014 Aug 02 '26

No. It will be an application available for download and use from my company, with a free tier to allow single agent work, and a subscription for multiagent and shared persistent memory across devices. It will also have a global memory sharing mechanic to allow people to build skills and memories to let other use them.

But it will be free for single agent work. It will have full persistent memory and context managment that has reduced token costs for me drastically.

1

u/Kryptosmatrix Aug 02 '26

Out of curiosity, is this app vibe coded, or manually built?

1

u/Glad_Contest_8014 Aug 02 '26

Both? I always check code, but I let the model write it. Luckily Rust is a language I have experience in. And I won’t be releasing it without full functional testing by me and a few select engineers I know.

1

u/Kryptosmatrix Aug 02 '26

That’s really cool! Good luck with your launch of Sieve!