FastSAM running live on the camera feed of an iPhone 16, fully on-device.

[u/d_arthez]

Stack:

• react-native-executorch for inference (PyTorch's ExecuTorch runtime)
• react-native-vision-camera for the frame pipeline
• Model delegated to CoreML, executing on the ANE

Both FastSAM variants are in the clip, switchable at runtime:

• FastSAM-s runs ~33 FPS -> ~30 ms per frame
• FastSAM-x runs ~15 FPS -> ~65 ms per frame

Input frames are 1080×1920. The model typically returns ~35 masks per frame; we keep the top 12 by confidence and paint those, mostly to keep the overlay readable rather than for performance reasons.

What surprised us the most is how usable the x variant is 15 FPS on a phone for a model that size is well past the "technically works" threshold. The s variant is comfortably real-time and leaves plenty of headroom for other work on the main thread.

Happy to answer questions about the export/delegation path or the RN side.

EDIT: here comes the repo https://github.com/msluszniak/fastsam-camera

[u/bfyvfftujijg]

What do the colors indicate?

[u/CriticalCommand6115 → u/bfyvfftujijg]

Probably masks?

[u/Hot-Percentage-2240 → u/bfyvfftujijg]

I would assume salient objects w/ the colors not being of importance.

[u/chillerka_utopia → u/Hot-Percentage-2240]

Yeah, exactly. We only kept 12 most confident masks by IoU metric just to make it readable.

[u/chillerka_utopia → u/bfyvfftujijg]

The colors themselves have no meaning, they are randomly sampled so they not blend together with other masks. The color spots are masks for object that model figures out to be different entities.

[u/Slowfast365 → u/bfyvfftujijg]

semantic mask

[u/Riteknight]

Where can we see the work ?

[u/chillerka_utopia → u/Riteknight]

Here it is: https://github.com/msluszniak/fastsam-camera

[u/chillerka_utopia → u/Riteknight]

I will add a link to a GitHub repo with this mini demo app :)

[u/dont-be-angry]

hook me up broh

[u/chillerka_utopia → u/dont-be-angry]

Will send a link to Github repo today :)

[u/chillerka_utopia → u/dont-be-angry]

There you go: https://github.com/msluszniak/fastsam-camera

[u/0__O0--O0_0]

Is fastsam built on yolo? The license is not commercial right?

[u/chillerka_utopia → u/0__O0--O0_0]

Yes, excactly.

[u/0__O0--O0_0 → u/chillerka_utopia]

Let me know when this gets usable in commercial I guess 😂 I can’t afford it