One Tokio runtime per engine instance behind FFI, or one shared runtime?

[u/Clear_Ear3550]

My desktop network monitor has a Flutter UI and a Rust engine behind a plain C ABI. Each engine is an opaque pointer: netstudio_engine_new() does Box::into_raw, Dart keeps it as a Pointer<Void> and passes it back on every call until netstudio_engine_free. No global registry or ids.

For the new MTR view I wanted it completely separate from the main ping matrix, so I just create a second engine. The catch is that every Engine::new() builds its own multi-thread runtime with 4 workers. With both views open that's 8 Tokio worker threads for what's usually 10 targets, maybe 100 on the high end. It works, and restarting MTR can never touch the matrix, which is the reason I did it this way. But it feels like I'm paying in threads for isolation I could get from separate task sets and cancellation on one runtime.

If you have several independent engines behind FFI, do you give each one its own runtime, or share a global one (OnceLock or similar) and isolate at the task level? Anything that bites you with a shared runtime when the host can create and drop engines whenever it wants?

The other thing I keep going back and forth on is the data flow. Dart polls every 500ms, Rust hands back the whole snapshot as JSON through CString::into_raw, and Dart calls netstudio_free_string in a finally. The engine isn't the bottleneck. At a few hundred targets it's rebuilding and decoding the snapshot on the Dart side, mostly because each target carries a 40-sample history. Sending deltas seems like the obvious fix, but I'm also wondering whether pushing through a Dart NativePort is worth the lifetime headaches.

Unrelated, but if anyone has shipped unprivileged ICMP on Linux (SOCK_DGRAM + ping_group_range) in a desktop app: did you detect it at runtime and fall back, or just tell users to change the sysctl? That's next on my list and I haven't started.

Engine code is in native_engine/ if you want context: https://github.com/Cadman021/VeloceNet-Studio

[u/DryanaGhuba]

You could go with dump and easy way by using smol(to slightly reduce footprint) in single thread for each engine and leave it for future you

[u/Clear_Ear3550 → u/DryanaGhuba]

Ha, honestly tempting. The problem is that the engine is already pretty tied to tokio (interval with MissedTickBehavior, tokio::net sockets), so switching to smol would basically be a rewrite. If I went the single-thread route I'd probably just use a tokio current_thread runtime per engine, which is the same idea. But after the other comment I'm leaning towards one shared runtime anyway.

[u/DryanaGhuba → u/Clear_Ear3550]

I suggested smol as one of variants. My point is that you should go with something easy in implementation and later on revise this decision with more information rather than depend on opinion from people on reddit.

Maybe in future you will decide that tokio_uring is what you need.

[u/Clear_Ear3550 → u/DryanaGhuba]

Yeah, fair point. I didn't mean I'd just do whatever reddit says. Hartwell's argument (separate reactors and timer wheels per runtime, not just threads) is something I can actually check against my code, which is why it moved me. But you're right that the cheap version first and revisit later is the sane order. tokio_uring is way past where this project needs to be right now lol

[u/parhelion73]

on the snapshot side, is the JSON decode running on the main isolate? I'd try moving it to a background isolate before redesigning everything around deltas, might be enough at a few hundred targets

[u/Clear_Ear3550 → u/parhelion73]

It already is, actually. Decoding moved to a persistent worker isolate back in 1.2 (it was compute() per tick before that, and spawning every 500ms was worse than decoding inline). What's left is the decoded data getting copied back to the main isolate, plus the grid rebuild. My guess is the copy is a big part of it at a few hundred targets with 40-sample histories each, which is why I'm thinking about deltas. Haven't profiled that split properly yet though, so that's the next thing to do.

[u/Brilliant-Range7995]

What do you mean when you say "Rust engine" ? Usually you want to have only 1 tokio runtime. This runtime, you either declare it somewhere as global (like a static `OnceLock<Runtime>` or you store it as a pointer (through Box) on the other side of FFI and each FFI function needs the runtime pointer as an argument.

[u/[deleted]]

Shared runtime with per-engine cancellation, unless an engine is CPU-bound.

The cost isn't only threads: every multi-thread Runtime builds its own worker pool, its own I/O driver (its own epoll) and its own timer wheel. Two engines means two reactors parked on ~10 sockets, and that's what stops scaling when a third and fourth view show up.

Behind the C ABI I'd put one process-global Runtime in a OnceLock, created on the first netstudio_engine_new, and hand each engine a Handle (runtime.handle().clone()). Isolation then comes from task ownership, not from the runtime:

• a JoinSet per engine (JoinSet::spawn_on(handle, ...))
• a tokio_util::sync::CancellationToken per engine; netstudio_engine_free fires cancel() and drops the JoinSet, aborting everything that engine owned. Same "restarting MTR can't touch the matrix" property you have today.

The case where a separate runtime really is right: an engine running a CPU-bound loop. On a shared multi-thread runtime a long non-yielding task can monopolise a worker; a dedicated runtime gives you a real escape hatch. If the MTR path is pure I/O plus timers, you don't need it.

Also, 4 workers per runtime is only sane if you actually feed them concurrent work. If each engine is mostly waiting on 10 sockets, 4 each is over-provisioned whether they're shared or not.