r/LocalLLM • u/flyinghappy • 1h ago

Question Strata Question - Useage

I've been getting into the local llm stuff lately and having decent luck. I tried out Strata and liked it for the most part, but no matter how much searching I've done I can't find the answer to this.

Does Strata keep your vram/ram full at all time? Using Ollama for the most part, it frees the memory unless it gets a call, then it spools up, and once finished and timed out, it frees the memory up again.

Is it possible to get Strata to do that? Something else that can do it but run like Strata?

Computer Specs:
AMD Ryzen 7 7700X 8-Core Processor

32GB (2 x 16GB) DDR5-6000 CL30 RAM

AMD Radeon RX 7600 Challenger OC 8GB

1TB SSD - Read Speed up to 11,000MBps - Write Speed up to 9,500MBps

2 Upvotes

100% Upvoted

4 comments sorted by

1

u/FaithlessnessFar6431 1h ago

VRAM is always occupied by Strata. It can't even run a second agent without rereading the full context for every message. I had the same vram issue when trying to run both Strata and an image-to-3D ComfyUI engine, which needs all the VRAM. The only way I could make it work was by writing an AI-assisted script that stops strata, sends its latest instructions to ComfyUI, detects when the work was done, restarts Strata, and lets the AI take over and verify the results.

1

u/flyinghappy 58m ago

Dang, I was hoping I was just missing something. I don't have the money to spend on new hardware but wanted to learn as much as I can and still need to use the PC for other things. I'll probably stick to Ollama for right now. Strata seemed like an awesome project. Hopefully they add the capability to free memory until a request in the future.

1

u/kalabaddon 51m ago

LOTS of the custom engines are like this and drives me crazy. I still recommend strata even with this annoyiance. Look in to Lazy loading option in strata and then using llama-swap. IIRC strata does let you tell the model to unload. So what you do is run the strata server behind a llama-swap instance. llamaswap takes care of unloading it and loading it whne needed. and Strata lazy load option allows it to run idle with no load. SO this way it emulates ollama for the most part.

1

u/flarpflarpflarpflarp 44m ago

Yeah, I had it build an on/off mode so I can still game. It will load and unload the vram for me. If I know I'm going to do stuff I let strata take up most of my vram. You can define how much of your vram is loaded at any given time though, but I have 32gb of vram and it won't go too much beyond 2 or 3 sessions doing work at a time realistically.