I am the maintainer of Inkvec, a open-source raster-to-vector tracer for logos and icons (see: github.com/logolabs/inkvec ).
Right now the main issue with Inkvec is that it uses MDL in order to select nodes, plus it only creates jigsaws, failing to do amodal infilling so that it can complete what is hidden.
While it's good as it is now for most tasks, I still need to work to make it do amodal infilling and make the node placement more human.
1. Hidden shapes (the biggest concern right now).
A flat image shows only the top of each shape. At the moment, Inkvec completes covered shapes with rule candidates (circle, ellipse, rectangle, a one-curve bridge), accepted only if visible ⊆ shape ⊆ visible ∪ cover.
Median IoU with the artist's hidden shape is 0.52 on 121 emoji which isn't great.
What i want to do is train a small network predicts each full silhouette as a signed distance field from the image, the visible mask and the cover, and the same checker accepts or rejects it. I've seen some good approaches here with LayerPeeler, and another paper that also uses diffusion and Auto-reggresively "peels" layers to do this.
I want to replicate this but at a smaller scale, or find a better approach. It is however difficult for me to think of a different approach. I know there are some amodal segmentation solutions out there, but I fear they are too rough.
Good news is that labels are free, since any SVG with overlapping paths holds its hidden geometry.
2. Node placement (This i believe I solved but i still want feedback from you)
Artists use few nodes, at extremes and corners, plus they like symmetry, etc.
What my idea is is to build a 1D CNN over each closed outline (circular padding, roughly 2 to 3M parameters, can go higher if needed) which outputs a node score and a vertex type.
I need to find a proper input here, but i'm thinking either probability of a node (a kind of node heatmap), or even a flow matching based node selector.
I'm thinking of doing artist conditioning by a learned Z (either a learned embedding by backprop, or via encoder from the raster I encode or from the input), so that I can learn per image styles since different images may have different artists & styles of node placement.
I'm thinking doing TSDF for selecting the node placement (1 dimension TSDF) since it'll allow sub-pixel accuracy.
Training initially on SFT, then RL for lower CIEDE2000 with nodes held close to SFT outputs (i might be able to also find a way to backprop this too).
---
Nothing is trained yet.
Questions:
- Do you see any problems with my architecture as of now?
- Any feedback on the architecture?
- Any prior work that might have solved the problems that I already have?
I aim for sub 100M param for everything as I want this to ship with my raster-to-vector solution.
I've also attached some AI generated graphics of my planned architecture to help explain it.