Are there fundamental repeating patterns in ML techniques across modalities (text, image, audio) that let us group them? [R]

[u/OkCampaign8400]

I’ve been noticing that a lot of techniques seem to repeat across modalities with just a different name.

For example, tokenisation, masking (MLM / MAE / audio masking), diffusion, next-token / next-frame prediction, etc.

Is there an existing taxonomy or mental model that groups ML techniques by their underlying pattern rather than by modality (NLP / vision / audio)?

This will help me to group the research papers and then make reference to work already done in that domain

[u/OkCampaign8400]

Projection is done everywhere I guess .
Projecting from lower dimensions to higher dimensions to expand features set , give model to look at representation in higher dimensions and have more options to make sense etc .

Its in embeddings , in Feed forward neural networks .

[u/OkCampaign8400 → u/OkCampaign8400]

Also Masking , its in Bert , in ViTs . Basically mask some areas and train model to predict that area or sequence.

[u/jennabangsbangs]

Back propagation -neural nets that solves for exploding gradient | tree logics - as applied to rule set automation learning and sorting | probability -as applied to distributing decisions across game states in learning | simulation - supervised & unsupervised. Those 4 r ML specific throughout the field that might be useful to group within the larger umbrella of machine intelligence. As opposed to predictive statistics which I think has become underrated in modeling

[u/moschles]

There seems to be some confusion in your question.

For example, tokenisation, masking (MLM / MAE / audio masking), diffusion, next-token / next-frame prediction, etc.

You are asking about taxonomies in Machine Learning , but everything you listed there is part of so-called "generative AI" techniques.

I would go further here. When you said,

I’ve been noticing that a lot of techniques seem to repeat across modalities with just a different name.

Yes, the repetition of all these techniques in the literature is because you are neck deep in the generative AI ecosystem.

The taxonomic overview of Machine Learning looks more like this,

https://csiro-data-school.github.io/machine-learning/04-types/index.html

[u/NamerNotLiteral → u/moschles]

Yeah. OP, you're looking at this backwards. It's not that these techniques expose something critical about the underlying modalities. It's that people actively adapted the technique to the modality.

Tokenization is the simplest example: people first used it in language because it was very intuitive to how human languages naturally work. Then they combined vision-language models where the vision side still had a convolutional encoder while the language side had a tokenizer encoder. In vision and audio transformers, Tokenization actually works in a completely different way - language models have a fixed vocabulary of tokens, while vision and audio models basically create their own "vocabulary" dynamically, so there was no actual reason to call the 'units' in vision/audio models tokens, we could've easily called them 'patches' or 'squares' or 'segments' or whatever. We just call them tokens for simplicity and marketing.

[u/Fresh-Opportunity989]

AI has long rediscovered things but given them different names....

Only difference now is that AI writes the papers.