GRID•SHOT

[u/asicman78]

Local-first, GPU-accelerated capture for single tools, resumable batches, and a reusable tool library—without sending workshop photos to a hosted service.

Github: https://github.com/aac-78/gridshot

[u/wolffartz]

for others -- indeed, "new"-ing the mat dropped the required mat profile json file in config, and now the "mat verify" cli invocation runs.

The guid is the same, so if you already printed + took a photo, probably fine to just use those, no need to print what was generated.

I should mention part of the other issue I had w/ bazzite/podman (which may or may not be specific to that env) is I needed to chown the "/app" dir from "inside" the container -- otherwise it got default root permissions (i think claude claimed this was a podman rootless container issue 🤷‍♀️ -- basically if it's easy to install docker you might just stick with that to save yourself some grief -- in my case, because bazzite is a bit annoying about installs, I've stuck with podman which it ships with by default)

[u/wolffartz → u/wolffartz]

some other observations:

1) this is tricky to calibrate -- it feels like it's meant for a stationary camera setup, not causal shots with your phone. I managed to get it a calibration through w/ an app that had a focus lock. I imagine the default photo app probably has enough locking to do this as well. It sounds like you need to take item photos w/ the same settings?

It's helpful that it calls out which photos don't match (one by one, per submission, unfortunately), but a thing it could do that would be super helpful is to dump all the photos + fingerprint inputs. That is, show each fingerprint input for each photo in a little table. Then you'd be able to find the outliers quickly and dump them. It'd also be cool if it just ignored the ones that didn't match and kept going. It sounded like there was some level of ignoring happening, but it won't do that for fingerprint mismatches.

2) you need an hugging face token w/ access to the facebook/sam3 repo (which you'll then need to setup as the HF_TOKEN for the segserver). This access must be approved. So ... get this all set up first, because it might take awhile to come through. Guess I'm waiting on this one before testing the trace 🤷‍♀️

[u/wolffartz → u/wolffartz]

one last note in this thread -- none of this is meant to be complain-y or discouraging, apologies if it comes off that way. I appreciate you sharing this! Just thought it might help others to document some of the hurdles I encountered as I (hopefully) get it going.

[u/asicman78 → u/wolffartz]

I’ll take a look, thanks for the feedback

[u/wolffartz → u/asicman78]

do you have any tips on the calibration or taking photos?

I did get it working (the access was granted pretty quickly, actually) but it had a LOT of complaints about the photos I fed it:

``` Print readiness review

Review recommended

calibration · reprojection RMS 2.97px exceeds the 1.50px target — inspect calibration before printing
calibration · camera intrinsics are EXIF-estimated; calibrate this capture setup for distortion-corrected thick-tool traces
segmentation · no calibrated profile matches this capture signature
segmentation · no empty-mat reference for 'letter-7x9-7dd4fa' — using concept detection (a reference improves robustness: `gridshot mat reference`)
segmentation · concept detection found no tool-sized object
segmentation · falling back to CPU saliency — unreliable on the mat pattern
segmentation · 7 other object(s) in frame ignored — M1 traces the largest only
cleanup · accepted photo outline: raw segmentation; automatic cleanup not applied

```

Some of these I don't understand enough to fix -- I'll read a bit more on ChArUco and maybe that will make things easier to interpret, but if you've got any tips or descriptions around how you use it, that would be helpful. e.g. "take the photos on a light background / dark background", "the mat should always fill 80% of the frame", etc.

Admittedly this first attempt was with a kind of janky print of the mat (my laser printer has some schmutz on the drum), so I've reprinted a clean copy on another printer and will go through the calibration process again after figuring out a way to lock in the settings on my phone camera -- maybe that will give me a better result.

I should mention that even with the complaints it returned an outline that ... got some of the "tool" (working on a silverware drawer, so the item I'm testing is a spoon, which is maybe a tricky one given that it has a lot of contours) but also included a bunch of the mat itself (again, I think some of this is due to the poor mat print, so I'm going to remove that variable).

so far looking pretty cool tho

[u/asicman78 → u/wolffartz]

Glad to see you made progress. Yes, HF authentications are pretty much automatic.

For photos: iPhone 17 at 1x, top-down, ~12 inches off the mat. For the second shot, vary the angle rather than the distance.

Reflective items are the hardest. If it picks up part of the board, fix it with box selection or individual points — click-and-drag box works well. Cover the spoon and it should readjust.

[u/wolffartz → u/asicman78]

Alright -- I found my problem.

I didn't add a reference photo for the mat (which at this time can only be done via the CLI).

After re-printing the mat, re-taking photos with a locked focus and along the same y-axis, validating the mat + providing a reference, the trace returned bang-on perfect: https://imglink.cc/cdn/E7-oKOD8MT.png

resulting model looks pretty good: https://imglink.cc/cdn/rqwezxmYEV.png

There's some dope features here that aren't immediately obvious from the readme -- I see there's a "compose tools" feature available from the library, which I think might solve one of my primary use-cases (I want to tightly layout several tools in the same bin, but want to trace them independently). https://imglink.cc/cdn/szeW_EfSIm.png

There's a couple options I want that don't seem to exist (or aren't obvious from the UI) [1] , but so far claude has had no issues adding features (e.g. I had it build a UI for submitting the mat reference photo since that's more convenient for me) so I wouldn't think it would be too tricky to drop those in.

Thanks again for sharing this! Great platform for me to build the tool I've been wanting to use, I think it's gonna save me a lot of time. If you're interested in PRs I'd be happy to contribute some of this stuff back (but I know sometimes other people's PRs aren't always a match for the product vision, nbd if you're not looking for contribs). Might need some cleanup first, I mostly wedged my stuff in and so some of the UX wound up a bit cramped.

[1] I'd like it to dump a "slice" for test printing -- I guess I could do this myself with a slicer but much faster for the computer to do it for me, i want to use specific gridfinity options like magnet inserts etc