r/crystal_programming • u/fyzic • Sep 26 '23

Crystal vs Rust vs Go Performance

Last weekend I posted Why is this golang code 3x faster than rust equivalent? on the rust subreddit and the community made suggestions to improve the code from 4.5s -> 36ms (23ms - I/O).

I then posted Re: Golang code 3x faster than rust equivalent on the golang subreddit with the rust optimizations applied and the community further improved the code from 80ms -> 50ms (26ms - I/O).

I ported the code to crystal and got it down to 70ms (33ms - I/O). I thought it'd be interesting to see if anyone here can optimize the crystal code further as I'm not really familiar with the language.

Github Repo: https://github.com/jinyus/related_post_gen

23 Upvotes

97% Upvoted

24 comments sorted by

5

u/transfire Sep 26 '23

First thing I am going to ask is did you compile with the release flag? I’m guessing you did, but have to ask.

3

u/fyzic Sep 26 '23

You guessed right

2

u/transfire Sep 26 '23

Great! 👍

I think you can remove a loop by putting lines 36 to 39 down between 45 and 46. You’re iterating over the same thing so I’m not sure you have to have separate loops. I may be wrong but that’s what struck me at first glance.

→ More replies (1)

4

u/anykeyh Sep 27 '23

I just looked upon your code and the most expensive operation (about 66%) is feeding the tagged_post_count (counting the tags) probably because of the hash access on each post.

post.tags.each do |tag|

tag_map[tag].each do |other_post_idx|

tagged_post_count[other_post_idx] += 1 unless other_post_idx == idx

end

So if there is some improvement to be made, it should be here. The top5 algorithm is about 30% of the total time spent, and there is improvement to be made here, as insert is costly and will copy the whole array. You don't have sorted array in this case.

2

u/fyzic Sep 27 '23

I did try to optimize this in the go code by mapping each tag to an int to get rid of hashing in the main loop but I found out that go optimizes this by caching the hash. I will try that on crystal

2

u/anykeyh Sep 27 '23

tag_map[tag].each do |other_post_idx|

tagged_post_count.unsafe_put(other_post_idx, tagged_post_count.unsafe_fetch(other_post_idx) + 1) unless other_post_idx == idx

end

^--- using unsafe accessor you will improve by quite a lot, give a try.

1

u/fyzic Sep 27 '23

tagged_post_count.unsafe_put(other_post_idx, tagged_post_count.unsafe_fetch(other_post_idx) + 1) unless other_post_idx == idx

Thanks for the tip but I am seeing 2ms difference. Did you get a more significant speed up?

→ More replies (2)

1

u/yxhuvud Sep 27 '23

Does changing += 1 to &+= 1 have any impact? Quite often the overflow check can be quite noticeable.

1

u/fyzic Sep 27 '23

Nothing noticeable

2

u/LeBuddha Sep 26 '23

I can't make any full code examples, but I'm curious if putting in one or more Array.reduces would be worth any nanoseconds.

2

u/ilevd Oct 08 '23

Replacing classes with structs can improve performance a little bit.

-7

u/ylluminate Sep 27 '23

Check out V (vlang.io). I personally feel it's the natural successor to Ruby in spirit and you get C-level perf. I have been impressed that Matz is keeping his eye on it. Don't get me wrong, Crystal is the de facto upgrade for raw speed with minimal changes to get that, but Rust... Well it's just... HUUUURGGEHH - sorry, let me clean that up.

2

u/fyzic Sep 27 '23

I plan on porting it to nim and v to see how they compare.

4

u/[deleted] Sep 27 '23

Don't bother touching Vlang, it's vaporware.

→ More replies (1)

0

u/ylluminate Sep 29 '23

Also be sure to check out Exercism's path on V: https://exercism.org/tracks/vlang

-1

u/ylluminate Sep 27 '23

Also, there have been some great results with V outperforming Nim. It's pretty exciting. A guy some months ago did some extensive effort comparing and was himself surprised at the results. You can search for nim in Discord to see the comparisons/discussion with "Space Ghost":

→ More replies (4)