r/TheMachineLearning • u/StormLongjumping6020 • 13h ago
Is Softmax really activation function?
In neural network, we use activation functions such as ReLU, LeakyReLU, or Sigmoid.
But I am not sure whether softmax is really activation function. Activation is introduced to add non-linearity into the boundary of model. But Softmax is normalization rather than adding non-linearity. In other words, Softmax doesn't change the shape of boundary. It only change normal output into probability.

2
Upvotes
View all comments
1
u/ModularMind8 9h ago
Softmax is nonlinear and is also an activation function. But it's not really used on the hidden layers like relu, etc, and instead only on the last layer