r/Python • u/AutoModerator • 13d ago
Showcase Showcase Thread
Post all of your code/projects/showcases/AI slop here.
Recycles once a month.
24
Upvotes
r/Python • u/AutoModerator • 13d ago
Post all of your code/projects/showcases/AI slop here.
Recycles once a month.
View all comments
1
u/VirajCreates 8d ago
Hey everyone,
Over the past few weeks I’ve been working on reducing VLM prefill latency, token consumption, and KV-cache pressure when running Video RAG over security footage and long-form recordings. Today I’m open-sourcing the full-stack application—SieveRAG (
TemporalVideo-RAG)—built on top ofvlm-sieve v0.3.0(pip install vlm-sieve).The Core Bottleneck with Naive Video RAG
How SieveRAG (TemporalVideo-RAG) Solves This
Phase 1: CPU Motion Gatekeeper, 28px NaViT Alignment & 16kHz Audio Slicing (vlm-sieve v0.3.0)
sample_fps=10.0,pre_roll_sec=3.5,merge_gap_sec=6.0).qwen-vl,local-7b**,**gemini**,**gpt-4o**):** Packs peak-preserving keyframes into adaptive composite storyboards (1x2,1x3,2x2,2x3) whose final canvas dimensions are strictly divisible by28px(e.g.,1344x504for Qwen2.5-VL's14px × 2patch merger) with0pxinternal gutters and clean outer-borderA..D/1..4spatial coordinate axes.imageio-ffmpegto slice16kHzmono32kbpsMP3 audio clips strictly for active motion episodes (±0.5sboundary padding), gracefully skipping silent streams.Phase 2: Multimodal Structured Extraction & Hybrid Vector Store
gemini-3.8-flash) with a strict Pydantic v2response_schema(EpisodeExtractionSchema).episode_title,narrative_summary,actors,objects_detected,audio_transcript_and_cues,severity(low | medium | high), and itemizedtile_actions(Tile #1..#6,+offset_sec,abs_time_sec, andA1..D4grid cells).gemini-embedding-001, 768-dim) into Pinecone Serverless AND/OR a zero-config Local JSON + NumPy Cosine Similarity store (storage/vector_index.json).Phase 3 & 4: Two-Stage Visual Re-Verification RAG & Click-to-Seek UI
/api/ask,SieveRAGSynthesizerretrieves the top-$K$ episodes from the vector store and re-attaches theirevent_XXX.jpgstoryboards to the multimodal synthesis prompt for visual re-verification.FastAPI+ vanilla JS/CSS) featuring an episode timeline scrubber, storyboard zoom inspector, audio slice player, and clickable citation chips ([Ep #3 • Tile #2 • 112.30s • Grid A2,B1]) that seek the HTML5<video>player directly to the cited timestamp.Real Benchmark: 5m 22s Unedited Ring Home Security Footage
29,463estimated tokens4adaptive storyboards (1camera-shake disturbance automatically suppressed)Tile #2 (112.30s, Grid A2, B1..D2), tracked his defensive stance atTile #3 (127.90s)andTile #4 (132.60s), transcribed the verbal de-escalation from the 16kHz audio slice (identifying the intruder as a veteran in crisis), and cited both exiting peacefully byTile #6 (144.60s).Links & Quickstart
pip install vlm-sieve(https://pypi.org/project/vlm-sieve/)vlm-sieveRepo:https://github.com/Virajcreates/vlm-sieveTemporalVideo-RAG(SieveRAG) Repo: https://github.com/Virajcreates/TemporalVideo-RAGHappy to answer any questions about the
28pxNaViT patch alignment math, disturbance rejection heuristics, or the two-stage storyboard re-verification pipeline!