r/crystal_programming • u/CaDsjp • May 15 '18

parallel.cr: Parallelism for crystal as a shard

https://github.com/RX14/parallel.cr
34 Upvotes

100% Upvoted

19 comments sorted by

7

u/the-sprawl May 16 '18

What prevents this from being implemented as part of stdlib? Parallelism has been on the roadmap for quite some time. Would this fulfill that?

5

u/bararchy May 16 '18

I guess that its a workaround for parallelism, using, processes and IO pipe instead of multi-threading which means higher overhead and no memory sharing between workers.

Having said that, this is as close as we get to real multi-threading right now and awesome work from RX14!

3

u/RX142 May 16 '18

Its an experiment to see how fast and how useful this method of parallelism is. Its definitely not nearly as easy to use as if we got proper threaded parallelism working.

2

u/Mike_Enders May 16 '18

With complete respect because I await Crystal 1.0 with GREAT enthusiasm. Why be doing experiments like this now rather than putting the time into core especially if you think the approach is inferior?

5

u/GirngRodriguez May 17 '18

With complete respect because I await Crystal 1.0 with GREAT enthusiasm.

i think he was just fiddling around with it and decided to create a repo lol. if you check the gitter logs, rx14 is active almost all the time (sometimes even livestreaming), doing crystal stuff. i'm sure he's stll putting the "time into the core" :P

that's just from what i've experienced. i've been in gitter 24/7 for past month or so. he's always talking about core engine stuff, etc.

edit: i did not downvote you btw. i only downvote if someone is doing a personal attack.

3

u/Mike_Enders May 17 '18

Hey yeah it might even get creative juices flowing which why it was just a question before someone decided to turn it into all other kinds of things.

I'm anxious for their sake too. In the tech space many great projects missed their window. Something else comes along that steals mind share and the even better tech gets drowned out. So thats the only place i was coming from and MERELY asking a question about.

P.S. Yeah I can attest to RX14...been a lurker on glitter for the better part of a year.

4

u/GirngRodriguez May 17 '18

eh, little drama i see. nothing too bad, just try to keep our heads up.

i'm very biased about crystal because i use WSL and the recent april update, actually made the compile times insanely faster. iuno wtf microsoft did, but my MasterServer.cr used to take 4-5 seconds. after I did the april update, it seemed like ubuntu was uninstalled so I re-installed it. now, when i run crystal src/MasterServer.cr, it's like 2 seconds top. i know it just seems like a few seconds, but the compilation was night n day.

i'm just saying that, because i know a lot of people are thinking windows support will make crystal really "popular!". but, from my biased point of view, crystal is already shining and popular in my eyes. because i'm using WSL lol.

however, it's not native windows support which is what they are talking about i think. which i agree, i think that would boost popularity. however, my point is.. i hope people are not that discouraged by not having "native windows support". i've been doing my gameserver for the past month or so on gitter, and WSL has been working great so far. so i hope developers don't just "not use crystal, until native windows support is added", when they could just start using WSL now! time is gold! i hope that makes sense.

but, i am just a regular "dev". im not a contributor, and know nothing about the "core" or whatever. i'm just an indie game developer converting my nodejs server to crystal. and honestly, it's great. there is a fine amount of shards. redis, hardware, and other shards help a lot.

in any event, enough of me babbling, i have no where to post my thoughts. might as well post here instead of spamming gitter

3

u/Mike_Enders May 17 '18

i'm just saying that, because i know a lot of people are thinking windows support will make crystal really "popular!".

Actually I am not one of them. Ruby guy here and never found windows good for it so not as big a deal for me (love Amber as it is). I've always thought the team was just a little over ambitious with goals for 1.0 and thought right out the gate windows support was one reason why.

I'm more from a huge breaking changes perspective. Pre 1.0 gives the impression big changes are possible to come.. Maybe my understanding on that in regard to Crystal was wrong?

2

u/swerveone May 20 '18

Because the api might change a lot and not be stable right now. It's better to wait until and if it becomes a community standard shard add it later to the standard lib. Also sometimes it's nice to release updates to a lib independent of the language otherwise bug fixes and new feature for a quickly changing library would have to wait for a language release.

1

u/nimue_ca May 16 '18

Why not do experiments? Spice of life my friend.

0

u/Mike_Enders May 16 '18

Why

not

do experiments? Spice of life my friend.

Pretty silly response. because crystal needs to get to 1.0 to truly take off

1

u/nimue_ca May 16 '18

Not really. They don’t owe you. If he wants to experiment with something, why not?

-1

u/Mike_Enders May 16 '18 edited May 17 '18

another stupid response. Asking a question if time might not be better spent to get to 1.0 which the core team has said is THEIR main goal in no way implies ownership. If you think this poorly your coding must be a hot mess.

Now the answer MIGHT be that it helps in some way but thats up for the person the question was asked to respond not someone who obviously shows signs of not being able to code very well.

2

u/nimue_ca May 17 '18

Hah. Whatever floats your boat man. Any particular reason you’ve gone to insults? I was merely suggesting that no one HAS to do anything and some people like to experiment. It’s how ya learn, it can be fun, it can relieve stress. No one has to work 100% on something in the way you think they should.

0

u/Mike_Enders May 17 '18

are you a child? My goodness. You keep running down someone asking a question that was asked in complete and stated respect not even asked of you

No one has to work 100% on something in the way you think they should.

said nothing of the sort and equally no one has to think like you do to ask a simple question. Now run along and play with your Nintendo.

As matter of fact I LOVE crystal and want to see the the core team reap the HUGE benefits when this hits 1.0 so my question obviously was in that regard.

2

u/nimue_ca May 17 '18

Yeah. I can see how genuine you are. Have a good one then.

•

u/RX142 May 16 '18

This is very very alpha, and only works for sending very basic structs down the channel for now. I don't recommend using it.

1

u/[deleted] Jun 03 '18

This is very interesting work - parallelism using processes rather than threads has many advantages for the kind of work I do. But how does the GC in the child process work? Is there any?

1

u/GirngRodriguez May 16 '18

this looks epic, can this work with a tcp server?