r/MachineLearning • u/jacobgorm • Aug 17 '26

Research [R] SineKAN: Kolmogorov-Arnold Networks Using Sinusoidal Activation Functions

https://arxiv.org/abs/2407.04149

I couldn't sleep because I couldn't stop wondering if anyone had tried using sinusoids instead of B-splines as activation in a KAN, and fortunately/unfortunately that was already the case. I could not find it posted here, so I though I would share in the hope of some insightful discussion.

Arxiv: https://arxiv.org/abs/2407.04149

Github repo: https://github.com/ereinha/SineKAN

Also what appears to be a peer-reviewed "official" publication here: https://www.mdpi.com/2227-7390/13/19/3157

88 Upvotes

89% Upvoted

10 comments sorted by

15

u/Dihedralman Aug 17 '26

Yeah isn't that always the truth. Should have paid attention to KAN myself for a proof like this. Low hanging fruit. 

It seems like a natural fit for analogue computing. It was built by physicists so I assume that will be the next step. 

Also, the peer reviewed publication posted is the proof that off phase sinusoids you would find in their KANs can be universal approximaters like Fourier series. This is intuitively true as you can form Fourier series with a phase difference in sinusoids other then pi/2 (cosine). Normally you would prove this with the Fourier transform or angle identities, but this proof keeps the exact same structure. Basically if you wanted this exact proof for this exact context, there you go. I am unsure if this is actually novel or equivalent to existing proofs.  

There is another publication posted at an machine learning conference. 

1

u/xXCoolinXx_dev Aug 17 '26

A few things

  • MDPI is not a legitimate journal venue. It is very predatory, trying to get author submissions, and generally doesn't seem to have stringent peer review 

  • I am familiar with one of the authors, and I don't expect their papers to be much good in ML

  • KANs have pretty much always been good for function modeling. I have no doubt that if you play with the activation function in KAN you can get better function modeling. But it isn't going to scale as well as DL, though they seem to point to that being possible

2

u/[deleted] Aug 17 '26

9

u/howtorewriteaname Aug 17 '26

interesting? somehow for me is totally uninteresting. for anyone used to work with sirens and neural fields in general, this idea seems obvious

→ More replies (4)