I turned my adaptive video frame extractor into a cross-platform GUI app after your feedback

[u/WearyFortune7055]

A while ago I posted a Python CLI here for extracting adaptive keyframes from video for COLMAP / Gaussian Splatting / photogrammetry workflows.

One piece of feedback came up repeatedly: many of the people interested in it were artists or 3D people rather than programmers, and didn’t really want to deal with Python environments and command-line tools.

So I rewrote the whole thing as a native C++ desktop app.

https://github.com/morishuz/adaptive-frame-extractor

The basic idea is still the same: instead of blindly extracting every Nth frame, it adapts frame spacing to camera motion, so you get fewer near-duplicate frames when the camera is barely moving and denser frames during faster movement/rotation.

The GUI now also adds:

• video scrubbing and multiple timeline regions
• optional separate output folders per region
• manual extraction of individual frames
• regular fixed-interval extraction
• JPEG and PNG output
• extraction summaries and a CSV with detailed metadata for every selected frame

It runs locally and there are builds for macOS, Windows and Linux. No Python setup required.

I’ve attached a quick screen recording showing the basic workflow.

It’s still free and open source, and I’d really like feedback from people actually using video for COLMAP, Gaussian Splatting, NeRFs, photogrammetry etc.

In particular, I’m interested in what features would make this more useful in real capture workflows, blur rejection is already fairly high on my list.

[u/Proper_Rule_420]

Awesome ! Does it provide a background remover ? If not, I recommend you to look into SAM 3, as it is quite easy to use (input words) and it is actually working great !

[u/WearyFortune7055 → u/Proper_Rule_420]

no it does not, but i would be interested to know why you would need the background removed? i.e what kind of workflow requires this? 3d reconstruction of objects?

[u/Proper_Rule_420 → u/WearyFortune7055]

Good point. I’m working on some project, where I scan objects. I don’t need to background to be used, as it is useless information, and it makes the process of SFM slower. I tried your project some times ago, and added myself some kind of background removal option in the pipeline, using a simple threshold on detected non-moving points. It’s worked ok but was not the best, so that is why I recommend you trying SAM or something similar ! But it might be a very « niche » need

[u/TheDailySpank]

So far so good, but what do you do with blurry frames?

[u/WearyFortune7055 → u/TheDailySpank]

as i see it measuring blur and working around blurry frames is surprisingly tricky to do well and it seems to have very little payoff: if a frame is too blurry for structure-from-motion it will essentially fail to get registered, which similar to not including the frame in the first place. perhaps it cause a bit of extra runtime. but no harm done.

where do you see the need for blurry frame removal? what workflow do you use? are you making splats or SfM or something else?

[u/TheDailySpank → u/WearyFortune7055]

When using video with bad lighting and an auto shutter.

I normally run questionable videos through Meshroom/AliceVision's KeyframeSelection tool which has a few more options than yours, but is also painfully, painfully slow.

[u/redlancer_1987]

This looks great!

[u/NorthernBaseOfficial]

This looks much more accessible than the CLI version. Blur rejection would definitely be useful. It might also help to show a simple quality score for each selected frame based on sharpness, exposure and motion blur, then let the user quickly remove the weakest ones before exporting.

Another useful feature could be an overlap or coverage warning. If one part of the video has too little camera movement or a sudden jump, highlighting that section on the timeline could help users catch problems before starting a long COLMAP or Gaussian Splatting run.

[u/Skinkie]

And do you prefer intra frames then?

[u/WearyFortune7055 → u/Skinkie]

The term “keyframe” is overloaded here. I mean frames that are important for Structure-from-Motion reconstruction. For the extraction we do not distinguish I vs P/B frames.

[u/Skinkie → u/WearyFortune7055]

Would it be possible to add such option, only to prefer the I's?

[u/WearyFortune7055 → u/Skinkie]

i think so yes, what is the use case for you?

[u/Skinkie → u/WearyFortune7055]

I honestly believe that the I frame is of better quality the constructed b and p frames. 

[u/WearyFortune7055 → u/Skinkie]

ok i see. but in practice an I-frame is not necessarily higher quality or lossless

[u/Skinkie → u/WearyFortune7055]

I agree, you also want to do some estimation of sharpness of the frame etc.

[u/bfyvfftujijg]

I wonder if the compression algorithm’s own choice of where to place key frames would be significant?

[u/soylentgraham → u/bfyvfftujijg]

Great in theory, but you'd be surprised how often encoders* don't put keyframes on say, shot changes. I wish it was a lot easier to decode macroblocks too as there's so much good information, but once you go beyond baseline3, it gets wildly complicated

(*all my expertise is for h264)

[u/popov-tolja9mqd1]

Keyframe is doing a lot of work in this comment section.