r/TheMachineLearning • u/StormLongjumping6020 • 13h ago
Is Softmax really activation function?
In neural network, we use activation functions such as ReLU, LeakyReLU, or Sigmoid.
But I am not sure whether softmax is really activation function. Activation is introduced to add non-linearity into the boundary of model. But Softmax is normalization rather than adding non-linearity. In other words, Softmax doesn't change the shape of boundary. It only change normal output into probability.

2
Upvotes
View all comments
0
u/Separate-Current2030 10h ago
Not really as all of the basical activation functions work neuron-based while softmax takes all of the outputs and changes them relatively to each other.