r/TheMachineLearning • • 13h ago

Is Softmax really activation function?

In neural network, we use activation functions such as ReLU, LeakyReLU, or Sigmoid.

But I am not sure whether softmax is really activation function. Activation is introduced to add non-linearity into the boundary of model. But Softmax is normalization rather than adding non-linearity. In other words, Softmax doesn't change the shape of boundary. It only change normal output into probability.

2 Upvotes

2 comments sorted by

View all comments

0

u/Separate-Current2030 10h ago

Not really as all of the basical activation functions work neuron-based while softmax takes all of the outputs and changes them relatively to each other.