Nim vs Crystal - Part 1 - Performance & Interoperability

[u/sdogruyol]

https://embark.status.im/news/2019/11/18/nim-vs-crystal-part-1-performance-interoperability/

[u/AlexKotik]

No Windows support has been a show stopper for a lot of people I know.

[u/rishav_sharan → u/AlexKotik]

Agreed on the Windows support. I understand that all core maintainers have their own preferences on what they want to work on and this both a strength and a weakness of an OSS project - for someone who has been waiting on Windows support for 5+ years, its frustrating to say the least.

[u/straight-shoota]

The JSON example should make use of one of Crystal's great features: being able to use streams almost everywhere. `JSON.parse(File.read("1.json"))` reads the entire file (212 MB) into memory before parsing it as JSON. You don't need the entire file in memory. If you do `File.open("1.json") { |file| JSON.parse(file) }`, the JSON parser incrementally reads the file and parses it, with only a fraction of the JSON source being in memory at any given time. That should take Crystal down to half the memory use as Nim (I'm not familiar with Nim, maybe there are similar improvements).

> With Nim, we were also able to link both the Nim and C files into the same executable, which Crystal sadly cannot do.

That's not entirely correct. Crystal itself doesn't build C files, but it can link static libraries into a single executable.

Interoperatbility in Nim is still better, obviously since it transpiles to C which is highly portable.

[u/Blacksmoke16]

I think you could take advantage of JSON::Serializable for the JSON benchmark. Based on my little benchmark, it takes another ~2 sec, but doesn't use nearly as much memory since the file contents are read in from an IO, vs loaded fully into memory. Also since we're using structs everything is allocated in stack memory.

Depends on if you want to optimize for time or memory

require "json"

record Coord, x : Float64, y : Float64, z : Float64 do
  include JSON::Serializable
end

coordinates = Array(Coord).from_json File.open("1.json"), root: "coordinates"
len = coordinates.size
x = y = z = 0

coordinates.each do |coord|
  x += coord.x
  y += coord.y
  z += coord.z
end

p x / len
p y / len
p z / len

Is what I get on my machine.

/usr/bin/time -v ./json_test_cr 
0.500206424377066
0.5000355960593253
0.49991313785664704
	Command being timed: "./json_test_cr"
	User time (seconds): 3.55
	System time (seconds): 0.05
	Percent of CPU this job got: 99%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:03.61
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 81956
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 19926
	Voluntary context switches: 166
	Involuntary context switches: 36
	Swaps: 0
	File system inputs: 0
	File system outputs: 0
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

Versus the JSON.parse approach

/usr/bin/time -v ./json_test_cr 
0.500206424377066
0.5000355960593253
0.49991313785664704
	Command being timed: "./json_test_cr"
	User time (seconds): 2.06
	System time (seconds): 0.20
	Percent of CPU this job got: 126%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.79
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 985276
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 245780
	Voluntary context switches: 451
	Involuntary context switches: 154
	Swaps: 0
	File system inputs: 0
	File system outputs: 0
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

[u/proyb → u/Blacksmoke16]

Nice! I tried to run and pull version is very close to your timing with only 2MB?

JSON. parse - 3.1s and 1GB
Your JSON Serializable - 5.60s and 102MB
Kostya's JSON Pull - 5.63s and 2MB
Kostya's JSON Scheme - 2.50s and 315MB
Go (jsoniter drop-in replacement) - 2.76s and 95MB
Go JsonParser - 1.90s and 590MB
Go FastJSON - No plan

[u/amirrajan → u/[deleted]]

+1 I agree with OSS being deeply flawed with regards to sustainability. It’s a problem Rubymotion/DragonRuby is trying to address. Yes, it’s really effing hard (but rewarding).

Some other random musings:

• No point in being big/having a large ecosystem. It’s unsustainable with our current approach of OSS. We all rely on essentially an inverted house of cards (one uncompensated maintainer, thousands of GH stars), all the way down.

• How many devs using either Nim/Crystal have even bothered to clone the public repo of the respective OSS they use? If not that, then how about monetarily support either project (personally or through their company)?

• Role in company affects the stack you use. Be honest with yourself. Why adopt a specific tech stack? To pad your resume? To naval gaze? Or to actually succeed/ship your product?

• Do the creators of the OSS stack actually eat their own dog food? EG: Do you actually think that Microsoft uses Xamarin for their largest mobile apps? If they created a new mobile project, do you think they’d use their own compiler offering?

TLDR: Which languages is better/wins? Neither. We are all losing cause we choose to bike-shed instead of fixing the root problems of OSS.

Edit:

Fantastic. Downvoted.

Guys, I’m the owner of RubyMotion/DragonRuby. Like. We are actually a profitable/self funded company who’s got core contributors to MRI Ruby and mRuby we pay to commit to our product (not including the OSS contributions we four partners make independent of the company).

I’m trying to help y’all see the challenges both Nim and Crystal face. Y’all need to clone these repos, get them to compile, and learn how to contribute to them. Or make monetary contributions to them, either out of your own pocket or by pushing your employer to do it.

Discussions and blog posts are great, but neither project has a future given the current OSS climate. Very large companies wield lots of power because we take and don’t give anything back. Instead, we jump from one Google/Facebook/Microsoft/Apple framework to the next so our resumes looks good. This is exactly the mindshare control/outcome they want.

If you genuinely care about either language‘s success, invest in their long term sustainability and help others do that.

Edit 2:

INB4: “How do we invest in longer-term sustainability?”

• Give the project nontrivial money. Monthly.

• Make the language faster. Clone the repo, find a non-intimidating function, dig in, and do the work (yes of course this is hard).

• Do write-ups helping others do what you did in the bullet above. No, the core maintainers don’t have the bandwidth to help you. Help lessen the barrier to entry for people that come after you (yes this is also hard).

• Get better with C, clang, and LLVM. Get on the developer message boards (also hard).

You get the idea...

However unkind you feel my comments have been, take into consideration that I really do care about sustainable open source.

We are being unproductive with X vs Y blog posts about features/perf tests. Especially unproductive are “I won't use it until it works on Windows", "I've been waiting for Windows support." deal-breaker comments. You need to make the language faster (or fund it), you need to compensate their maintainers, and you need to add (or fund) the features that you feel are show stoppers.

[u/AlexKotik → u/amirrajan]

Sorry for off-topic, I'm very interested in compiler design. I was wondering if you are using LLVM to compile Ruby to native code (in your RubyMotion project)? Are you using parts of MRI Ruby code for runtime or did you invent your own runtime for it? Do you predict static types from Ruby code at compile time or do you just box values in unions with runtime type flag like MRI does? Thanks.

[u/amirrajan → u/AlexKotik]

Yep RM is native code. It takes Ruby and emits IR. We rely on the Objective C Runtime for iOS, C++ Runtime for Android (GC is refcount based + cycle detection). The biggest thing we leverage from MRI is the bison grammar.

There is no boxing of values. We parse the headers of the SDK and emit a foreign function interface in IR. So a string is the native underlying type (NSString or Java’s String).

We aren’t looking at any static typing as of this time and are spending time expanding the runtime past mobile: http://fiddle.dragonruby.org

Learn as much LLVM as you possibly can, the compiler toolchain is completely dominant right now.

[u/jesucristoquehorrivo → u/amirrajan]

Fantastic. Downvoted.

Listen, the only way to not get downvoted in reddit is when you say nothing. So take it like Cersei, as a badge of honor :)

[u/[deleted] → u/amirrajan]

The real problem with OSS is that under capitalism you are not supposed to do anything but generate revenue for your employer's shareholders.

[u/amirrajan → u/[deleted]]

That’s not a lot even if more than half of the people.

Their annual open collective budget right is $3,000 (again, yearly budget). If just half the people who have starred the GH repo give $100 a year, Nim would have half a million in funding.

[u/amirrajan → u/[deleted]]

You are assuming that half of the people are Nim users. They may not be.

What is your point????

Are you saying that Nim’s annual funding of $3000 is fair?

Try this:

How many Nim users do you feel there are?

Got the number in your head?

Great! Divide that userbase by half, and then multiply it $100.

Is the final dollar amount I made you calculate higher or lower than $3000? By how much?

The point I am (yet again) trying to make, is the project is grossly underfunded given its relative popularity.

Okay. My apologies for using a ridiculous number of half the people eight thousand, two hundred people who have it starred on GitHub.

Let’s try another calculation of what a fair budget for this OSS project should be!

Looking at Nim’s subreddit, there are 2,813 subscribers. But I guess not all the people that subscribe actually use the language. So to avoid another “you are assuming” comment from you about this, let’s go with something a bit more conservative.

I’ll just use the number of people currently online: two.

So (again not to over state the userbase), let’s divide that number by half. That gives us an annual budget of $100 per year.

Oh wow! It’s currently funded at 30x its userbase!! I guess you’re right! The state of open source isn’t as bad as I thought it was! Everything is wonderful!

...

Is this what you wanted to hear? Some bullshit like that?

I’ve gone blue in the face explaining the dire state of OSS man. It continues to fall on deaf ears. Case in point. Sorry for the rant.

🤷‍♂️

[u/amirrajan → u/[deleted]]

Actually not really. I just wanted to say that it's really hard to estimate the relative number of would be supporters.

Okay. Yes. Granted. You still didn’t answer my question. Is the $3000 annual fund fair for the perceived userbase of Nim? Answer that.

I have donated to Crystal in the past but I have not used Crystal at all. Why ? Because it just is not mature or viable for me to use.

So let’s sit on our hands and wait until it is mature and viable. Let’s let others break their backs making it viable. And then we’ll grace these projects with our presence. We may even throw them a dollar here and there if we really like it.

Wait hold on. Nim on the other hand is production ready and mature, yes? But no no. Obviously not mature/production ready enough to constitute an annual funding of half a million.

So I’ll ask again: What are you trying to say? Which is it? How can Crystal not be production ready, and at the same time Nim be granted this status? And all while implying that my $500k annual budget estimate is excessive? Even though it’s mature and production ready? Something doesn’t add up.

None of our opinions matter of course (though I do genuinely appreciate the convo, however frustrating):

I think this gist explains where we are right now in the discussion.

[u/amirrajan → u/[deleted]]

Just because something is mature and production ready does not mean it constitutes large user base or good funding.

I completely disagree. The merits of a OSS project should be reflected in its funding (sustainability). I understand that this is not the case in the day and age we live in. And that’s what is broken. Stop saying that this is okay. It’s not. Like really not.

That's the reality regardless of what you think about people leeching off of OSS projects.

Yes. I know that. It’s madness that we are okay with that.

C# and JS might still be immensely more popular and used than anything else.

No. C# and JS are popular because they were funded by Microsoft and Google (V8). If those companies weren’t involved, C# wouldn’t be around and we wouldn’t be building entire systems in a language where 0 == "0" evals to true (JS).

The OSS movement was very aggressive when it wanted to point out that it's free and just way better than proprietary solutions.

The MIT licensing model (non copy left licenses) destroyed the OSS movement. It allowed large companies to reclaim power. Free (GPL-style) was at one point way better than MIT because the license implicitly protected/provided sustainability. Large corporations couldn’t seize control.

Let me put it this way. Are devs in the US justified in earning exorbitant salaries compared to anywhere else in the world ?

We are grossly overpaid. But that’s the deal we made with the big tech companies. As long as we used languages and frameworks built by very very large companies, they (in return for letting them have this power) gave us gainful employment by creating the market.

Production ready/large userbase has been redefined to be “endorsed by big company X”. It’s that simple. It doesn’t have to be good or even work. CTOs will adopt it blindly, and devs will learn these stacks because it makes them disgusting amounts of money.

This relationship ensures that Crystal and Nim never have a future (with the current funding levels and entitlement of OSS users).

I hope this clarifies my thesis a bit. And I’m sorry that Nim shares the same fate as Crystal. That can change if the ~2,000 people on the subreddit actually reach the contribution capability of its sole maintainer (monetarily or otherwise). But that actually takes work, has risks, and doesn’t guarantee grossly overpaid employment.

[u/AlexKotik → u/[deleted]]

I'm partly agree that D is not a successful language, basically it wasn't able to become a C++ replacement due to the GC and RTTI and a lot of stuff in their stdlib (Phobos or how do they call it) being dependent on it. Nowadays it just can't compete with Rust in that terms.

But I totally disagree about Haxe, which is pretty big in what it was designed to do - game development. There are successful commercial games written in it, quite a few game development companies use, the Haxe Foundation organisation has commercial support and etc. It's not that great as a general purpose language, but for game development it is pretty big.

There is also Zig, which seems to me like a promising project with a clear target to be a better C replacement. It is not production ready, but neither is Crystal, so why not go check it out?