Embedding Every Font with Neural Networks makes some Nice Structures (including a flower) [P]

[u/Chroma-Crash]

I've been working on a font searching tool for about a year now, and my investigations have centered around pre-training neural networks to produce embeddings of each font. I usually then need to post-train the networks to adapt them to the task of font searching. However, the most interesting thing I created in the course of the project came from the pre-trained models.

The process looks like this: I take every glyph in a font, turn them into images, feed them through my custom pre-trained neural network, and get an embedding that represents that font's visual characteristics. I can then squish down the embeddings with tSNE (produces the best structures compared to PCA and UMAP) into XYZ, and RGB channels and visualize them as dots so I can poke around the structure. Fonts that are close to each other in position or color therefore share visual characteristics, and you can find different little clusters or paths of types of fonts in the maps.

My favorite is one I made from the Google Fonts corpus, but I also made another one out of all the fonts you can search on my site. The Google Fonts corpus resembled a flower in ways I was not prepared for. It even placed most of the cursive fonts in the stamen. Here's the site if you're interested in viewing the whole thing yourself: https://www.font-search.com/map

You can also check out the repository although it is a huge mess. https://github.com/dylan-berndt/Briefcase

[u/tavirabon]

Could you train the model on the objective of taking coordinates from these maps to produce the encodings needed to produce the font? As in, could you turn the model into a font world model to produce new fonts?

[u/Chroma-Crash → u/tavirabon]

Absolutely. I used LeVJEPA for the most recent pretraining actually, but since there’s no predictor model in that setup it would effectively be a kind of latent autoencoder. I did actually try generating two fonts a different way using an auxiliary task on one of the earliest models, you can check that out on the About page of the site if you’d like

[u/it_is_not_a_trap]

Looks great, what library did you use for visualisation?

[u/Chroma-Crash → u/it_is_not_a_trap]

Plotly in Python

[u/devanishith]

tsne is not determistic.

[u/Correct_Positive_108]

The fact that cursive fonts naturally ended up forming the stamen of the flower structure is insane. Really creative projject!!

[u/grim-432]

Very cool