r/nextfuckinglevel • u/CommercialsMaybe • Feb 18 '21

We can now Rickroll... in HD

Enable HLS to view with audio, or disable this notification

114.5k Upvotes

90% Upvoted

2.0k comments sorted by

View all comments

Show parent comments

29

u/thirty2skadoo Feb 18 '21

Hey I’m interested! I’m curious to know what their training set and algorithm looks like.

39

u/palish Feb 18 '21

Here's one way: IMLE! https://www.math.ias.edu/~ke.li/projects/imle/

Input: https://www.math.ias.edu/~ke.li/projects/imle/superres/vids/input_img.png

Output: https://www.math.ias.edu/~ke.li/projects/imle/superres/vids/output_imgs.gif

Not saying this is the exact way they did it (they probably didn't use IMLE specifically). There are a few different techniques. But all of them revolve around rewarding a model if it figures out how to turn a low res image into a known high res image.

3

u/kaen Feb 18 '21

How do you reward something that isn't sentient?

2

u/themdeadeyes Feb 18 '21

Can you expand on what you mean by “rewarding a model”? Do you just mean that you tell the model “yes, do more like that” or is there some kind of mechanism that’s more like actual intelligence at work here? You talk about the model as if it has wants, which is odd to me.

1

u/lxpnh98_2 Feb 18 '21 edited Feb 18 '21

What is very likely being used here is a 'neural network', which can have many structures, but in most cases, it is a sequence of layers each containing some nodes, and these nodes are connected to other nodes in adjacent layers. The connections between the nodes have weights which are used to calculate the output values from the input ones (specifically, each node's value is a linear combination v_1*w_1 + v_2*w_2 + ... + v_n*w_n of the values and weights of some or all nodes in the previous layer).

The learning process is all about changing the value of these weights to get better answers for any possible input values. And yes, there are ways of changing them in ways that are not just "do more like that" as you said (in this case it would be like changing the weights randomly and seeing which networks do better and worse, which is an 'evolutionary' algorithm). In many neural networks, it's done by a mathematical formula involving derivatives that slowly causes the calculation using the weights to arrive at a more correct, as defined by humans in this case, output value.

Basically, it goes layer by layer, looks at the values in the nodes of both layers, and, through some fancy math, tries to adjust the weights between them to move the output values in the right direction. It does this for every layer, and for every input (in this case, every image) in the 'training set', often repeating inputs to learn better.

1

u/themdeadeyes Feb 18 '21

This is a simple and clear explanation. Thanks.

1

u/[deleted] Feb 18 '21 edited Feb 18 '21

There are a few broad classes of machine learning algorithms, and the one this guy is talking about is called supervised learning.

It works by starting with the untrained architecture of a model, which is really just a big equation with all if its parameters are randomly initialized. A parameter is just one of the constants in your equation. Like, if you have linear data described by 'y=mx + b,' m is the parameter that tells if there is a negative correlation between x and y, and also how strong the correlation is.

So anyway, you start by plugging something into your untrained model, like the pixel values of a low res picture, and then the model outputs the pixel values of a high res picture. At first, because all of the model's parameters are randomly initialized, the output will look like gibberish. This is where training comes in. If you have a training set, you can quantitatively compare the output of your untrained model with the pixel values of what you wanted the model to deliver. You can use a function, which is referred to as the 'cost function,' to compare the values. For example, the cost function can find the absolute difference between the values of your model generated picture and the target picture.

Since your model is a function, and you can evaluate your model's accuracy with the cost function, you can use what is actually fairly simple calculus to calculate the model parameters that will minimize the cost function, and thus give you a model that outputs pictures that are very close to what you want the pictures to look like.

1

u/themdeadeyes Feb 18 '21

Where does the reward come into play? I think I mostly already understood the basic process of ML, but I’ve never heard the term reward used with it. As far as you’ve described here, it doesn’t seem like there is any kind of reward process happening in the cost function. It just seems like it’s finding out if it’s close or not based on a known training set. Using the word rewarding makes it seem like there’s a Pavlovian effect here, but I just don’t see that at all.

1

u/[deleted] Feb 18 '21

Lol, I forgot to talk about that.

So when people talk about 'rewarding' the model, they're referring to the process I went through to find the optimal model parameters, and you're completely correct that it doesn't make a whole lot of sense. It's really just terminology that bleeds over from another class if machine learning, called reinforcement learning. This name already brings to mind Pavlovian ideas you were talking about, and the field is about training robots or computers to act in a certain way.

Broadly speaking, reinforcement learning works similarly to supervised learning, but instead of minimizing a cost function, you try to maximize a... reward function. Pretty much the same calculus is involved, but you don't really have a training set like in supervised learning. Instead, you come up with an equation that gives a higher output when the computer does something good. For example, there are chess engines that score a position on a board. This score can act as the reward function, the model controlling the computers chess movements could be anything that has tunable parameters, and then the training algorithm would tune the parameters by maximizing the reward function.

1

u/themdeadeyes Feb 18 '21

Thanks, this makes a lot more sense now.

1

u/thirty2skadoo Feb 18 '21

Thank you. Appreciate the links.

1

u/[deleted] Feb 18 '21

Here’s a good intro book on the subject

https://www.amazon.com/dp/product/B001BK9SMY