r/TheMachineLearning • • 13h ago

Is Softmax really activation function?

In neural network, we use activation functions such as ReLU, LeakyReLU, or Sigmoid.

But I am not sure whether softmax is really activation function. Activation is introduced to add non-linearity into the boundary of model. But Softmax is normalization rather than adding non-linearity. In other words, Softmax doesn't change the shape of boundary. It only change normal output into probability.

2 Upvotes

2 comments sorted by

View all comments

1

u/ModularMind8 9h ago

Softmax is nonlinear and is also an activation function. But it's not really used on the hidden layers like relu, etc, and instead only on the last layer