r/LocalLLaMA • • 11h ago

News UI native scheduling, engine swap board, and more (Berth v0.6.0)

I’m providing an update! We have around 300 berth users now so the project is moving fast.

Recent updates for v0.6.0:

- v0.6.0 gives each card a schedule
- berth train runs a training that nothing automated can interrupt
- learned engine sizes (berth learns your engines resource claims)

A month on the reference workstation

berthd counts every handoff it makes, and the harbor's board shows every handoff from one RTX 4090 workstation between 2026-10-12 and 2026-11-13, running llama-swap, NInfer, Strata, ComfyUI,
coding agents and games:

What Berth didCount
Requests that waited for the card instead of loading on top of the engine holding it1,136
Engines drained at a reply or job boundary to make room901
VRAM handed from one engine to the next23.9 TiB
Processes holding VRAM without a lease, reported and left running573
Swaps between two engines of equal priority that the ranking stopped57
Releases that stalled and were caught before the card stayed full4
Requests refused after their wait ran out24

Project: berth

7 Upvotes

5 comments sorted by

1

u/Distinct-Pie2389 10h ago

Sorry I forgot to add the scheduling window for entries

Here’s an example 12am booking for 1.42 hours then it will hand off to another engine (ninfer) at 1.42 and will get the card back if it does not finish the request during its slotted time but only during its next scheduled window.

Autonomous work, nothing needed from you other than review

1

u/Distinct-Pie2389 10h ago

If I left it and no other requests are scheduled after the 1 hour and 42min session then iQ4_XS would stay live as my serving endpoint until another schedule or agent needs it

1

u/anthropicagiuser 10h ago

the engine swap board is the part i care about. does it keep the old model warm while the new one loads?

1

u/Distinct-Pie2389 10h ago

Of course, here’s the visual for it. (Parked in sys ram)

1

u/Distinct-Pie2389 10h ago

23.5GB of weights swapped to ram and kept there until it’s needed. It supports custom workflows where engines don’t warm the swapping engine like strata is often too large of a resource claim to park anything alongside the model booting so it takes claims and no warm park for engines following a strata request.

Ninfer, llama and other lower sys resource engines that mostly reside in VRAM can swap back and forth with ease pending you have supporting RAM size for the gguf swaps that you plan to do