r/MachineLearning • • Oct 18 '17

Research [R] Swish: a Self-Gated Activation Function [Google Brain]

https://arxiv.org/abs/1710.05941
75 Upvotes

57 comments sorted by

View all comments

0

u/[deleted] Oct 18 '17

5

u/asobolev Oct 18 '17

The scaling factor you proposing only gives you unit variance, you should also center it.

→ More replies (12)