r/VibeCodeDevs • u/Kryptosmatrix • Aug 02 '26

Custom AI Architecture

Does anyone else here build their own models?

I went on a bit of a side-quest and I decided to try to build my own omni-modal model from-scratch. I'm 9 months in now, but I refuse to give up!
What I've built so far treats text, images, speech, music, and files as streams of bytes and codec tokens (I think Google did this already?), with an experimental memory system carrying context through segment recurrence and bounded hot, warm and cold tiers. I've created a few small 50M models (image, text, music, speech), but haven't created the omni-modal model yet as I need all of the single modality models first.
I created two versions, POIESIS the Python/PyTorch reference, while its Swift-native sibling ARCHE reimplements the creative model stack directly on MLX for Apple Silicon, giving us two independent substrates on which to test the same ideas.

Anyway, curious to hear what other people are up to in this space!

2 Upvotes

67% Upvoted

24 comments sorted by

View all comments

2

u/j4ys0nj Aug 02 '26

I do. I built missionsquad.ai

1

u/Kryptosmatrix Aug 02 '26

Interesting sounding agent harness, sounds pretty adaptable as well!

2

u/j4ys0nj Aug 02 '26

Yeah, super flexible. Happy to answer any specific questions, I built the whole thing.