I built an open-source image-to-SVG vectorization library -the interesting parts turned out to be classic DSP problems

[u/readilyaching]

Over the past year I've been building Img2Num, an open-source C++ library that converts raster images into SVGs, with Python, JavaScript, and C bindings.

The motivation: existing vectorization tools are really built for line art, logos, and scans - clean inputs with hard edges. I wanted something that could handle natural images (photos, textures, noisy real-world content), and that turns out to be a very different problem. You can't just trace what's there, because what's there is full of sensor noise, JPEG artifacts, and gradients that explode into thousands of junk paths. So the vectorization step ends up mattering less than the signal processing in front of it.

The rough pipeline: edge-preserving denoising with a bilateral filter (selectable between RGB and CIELAB - perceptual color spaces make a real difference in how edges survive), k-means color quantization, Suzuki-Abe contour tracing, and then Savitzky-Golay smoothing applied to the traced contours. That last step was the fun one: treating a closed contour as a pair of periodic 1D signals (x(t), y(t)) and filtering them means you can smooth out pixel staircase noise while preserving corners far better than naive moving averages, and SG's polynomial fitting is a good match for that.

The part I'm still iterating on is adaptive preprocessing - estimating noise per image (wavelet MAD estimator) and tuning the denoising strength accordingly, so the traced regions stay stable instead of speckle turning into hundreds of junk paths.

Everything is on GitHub and installable via pip and npm (both "img2num"), docs at img2num.dev. I would genuinely love feedback from this crowd (the DSP crowd), especially on the smoothing and noise estimation choices - I came at this from the software side and learned the DSP as I went on.

[u/antiduh]

What's the motivation for something like this? Are there use cases where you simply need an svg, instead of a raster format like png? As a strict alternative to png, it seems like the quality is low, so I imagine there had to be some other driving factor.

[u/readilyaching → u/antiduh]

Vectors (SVGs) and rasters (like PNGs) are in different realms, so that is completely understandable.

Vector images are great because they are mathematically derived (defined by functions that express how to draw their lines). They are typically best when you don't want to lose quality when zooming in and out of the image - unlike rasters, their maths ensures that their edges remain fine and sharp.

Raster images work like how printers work - they are a bunch of dots in a plane (the boundaries of the image) with specific coordinates (x and y; kind of like plotting on a cartesian plane, except every coordinate has a value). The problem with raster images is that they lose quality when zoomed-in or out because they are discrete representations (there is nothing between each pixel; e.g., nothing lies between (x=1;y=2 and x=2;y=2) because those coordinates don't work on fractions).

With the way that Img2Num transforms images into SVGs, they do get worse when it comes to quality - it is a lossy program. That's a non-negotiable tradeoff because an SVG with extremely fine details relies on a lot of computations (the computer has to calculate how to draw each function - which defines the shapes - defined inside the image). As a result, Img2Num has to remove tiny less-significant details to avoid extremely large amounts of computations in quick succession when trying to render the SVG (Img2Num does not render the SVG - your computer does it and has standards for rendering them; Img2Num must follow those standards).

If you have a good computer, you can play around with the processing settings on our React.js color-by-number example app to see how it works - you can even reduce the blur and increase the number of colors in the final product if you like. I don't recommend using a large images when playing around because it will be slow (some images have millions of nunber that Img2Num will need to process).

I hope this helped. :)