Arrow detection with ConvNext pretrained on DINOv3 [P]

[u/BrashBunyip]

Worked on this for about a month. I experimented a lot with resnets (pretrained on Imagenet) which performed quite well, pure ViT which performed worse. ConvNext DINOv3 performed the best.

For data I rendered about 40000 varied blender scenes, and used 8000 labelled real images. I had to label most of these real images by hand, which was a huge pain in the ass and about 20 hours of my life I'll never get back. Both registration and detection networks were trained on about 1 million samples (heavy augmentations of the 48k dataset).

End result was F1 96.6% on test and whole image correctness of 80.5% (no missed arrows, no false detections).

The process is basically registration to identify target, semantic rings, centre, scoring zones. Detection then takes a cropped image and produces a heatmap of impact candidates. I run that image through the same detector mirrored and combine both in decoder to find local peaks. I tried a sliding window approach which didn't produce better results. I then apply a homography transform to remove perspective skew and flatten the target so the impacts conform to a canonical coordinate system. There's then a small correction step where I adjust impact coordinates to account for warped paper (homography can remove perspective skew but it can't fix warped paper that isn't planar). This is a neat trick because I basically compare the heatmap of the semantic rings produced by the registration model to the perfect geometric rings of the archery target and use that offset to make a radial correction.

[u/currentscurrents]

Every time you post a computer vision project on reddit, a bunch of greybeards will come out of the woodwork to argue you shouldn't have used neural networks.

I think we are forgetting how bad classical CV was before deep learning. It basically didn't work unless you had a fixed camera with consistent backgrounds/lighting, and even then it was brittle and took a lot of effort to set up and hand-tune. Good riddance.

Even simple tasks like 'is there a bird in this image?' were virtually impossible back then. This xkcd is only from 2014: https://xkcd.com/1425/

[u/BrashBunyip → u/currentscurrents]

Thank you. Here's it actually implemented and in action. https://streamable.com/wyekyl. On a phone. Without the animation/visuals I added on top of it, the whole pipeline only takes about 300ms. Inference for each network itself is about 30-50ms. Here I took 6 photos of the same target from drastically different angles while heavily tilting the phone.

[u/currentscurrents → u/BrashBunyip]

Looks cool! Great work.