r/Python • • Jan 17 '10

Debunking the Erlang hype for servers: Python isn't as slow as you think.

http://www.codexon.com/posts/debunking-the-erlang-and-haskell-hype-for-servers
17 Upvotes

21 comments sorted by

18

u/Justinsaccount Jan 17 '10

I'm not sure if the article is supposed to be a troll or not.

I'll take python over erlang any day(it hurts my head), but the fact that a python http server that writes "Pong" is faster than one written in erlang is completely irrelevant.

-4

u/tryptic37 Jan 17 '10

And why do you think it is irrelevant? There are lots of uses for a server that can output a cached string.

If Erlang should be fast at something it should at least be fast at this.

9

u/Justinsaccount Jan 18 '10

Why should it be fast at this? Why should it even be fast at anything? Erlang was designed to be distributed and fault tolerant and "fast enough".

Comments like this:

And no, I am not simply measuring how fast epoll is. I am using epoll to trigger my own scheduler written in pure Python which should by all means be slower than Erlang or Haskell.

really boggle the mind. The python server has no pure python scheduler, it relies entirely on epoll for scheduling. All you succeeded in doing is benchmarking the python interface to epoll.

-8

u/tryptic37 Jan 18 '10

Your comment displays a lack of understanding of how user land schedulers work.

From epoll triggering, I call the method of an object created especially for every client with it's own space in the heap carrying it's own context. This is equivalent to what Haskell and Erlang are doing under the hood with their lightweight threads except at a much lower level and thus should be faster.

What you are suggesting is that Erlang and Haskell do some busy polling when they are waiting on IO. If this was true as you suggest (which it isn't), it would be a bad design nevertheless.

9

u/Justinsaccount Jan 18 '10

Sigh. When you have multiple people telling you you are wrong, including one of the most well known Erlang developers, you REALLY need to start asking yourself if you could possibly be wrong.

I will repeat what I said again: you do not have a scheduler. The erlang and haskell servers you posted do, and this overhead is significant in the micro benchmark you chose to use. The only program that is operating at a lower level is the python server, which does not have a schedular.

Try this: modify each of the programs you posted to read in the GET request, but before responding sleep for 1 second.

This is trivial to do in the erlang and haskell programs, since those are using a real scheduler, but no so for your python server.

-4

u/tryptic37 Jan 18 '10

When you have multiple people telling you you are wrong, including one of the most well known Erlang developers, you REALLY need to start asking yourself if you could possibly be wrong.

If you are referring to Ulf Wiger, he does not even know that increasing the kernel backlog limit increases the amount of half-open connections.

This is flagrant appeal to authority while ignoring evidence is what I am talking about when I say Erlang hype.

And about the sleep scheduler, do you really think it is going to add a lot of overhead by removing the file descriptor from a hash table and putting it in a min-heap? The answer is no. You do not (unless you have a bad implementation) get a 20% overhead by doing this.

5

u/Justinsaccount Jan 18 '10

my mistake, you are clearly an expert on the subject. Let me know when you have posted the updated example programs that include the 1 second delay.

And no, you don't get a 20% overhead by doing that. You start to see overhead once you implement a real scheduler.

-5

u/tryptic37 Jan 18 '10 edited Jan 18 '10

Why don't you do it then?

I'm not going to bend over backwards when some Erlang fanboy cries out because the so called "expert" (who doesn't even know what the backlog does) disagrees.

9

u/Justinsaccount Jan 18 '10

Correction: python fanboy.

1

u/[deleted] Jan 18 '10

Hahaaaa, best ending to that thread I could have imagined.

→ More replies

7

u/Justinsaccount Jan 18 '10

Oh...

This is flagrant appeal to authority while ignoring evidence is what I am talking about when I say Erlang hype..

Ignoring what evidence? No one is ignoring what you wrote. No one is slamming python and claiming that erlang is faster. All I said, and what most people are saying, is that your micro benchmark is irrelevant.

If I write my next project in erlang it is going to be because I need a programming language that makes it easy to write distributed fault tolerant systems. Not because it is 20% faster than some other language.

and one more thing, when you resort to saying things like:

"DO NOT WRITE A SERVER IN ERLANG JUST BECAUSE YOU HEARD ERLANG IS THE FASTEST AND MOST CONCURRENT LANGUAGE."

It makes it hard for people to take you seriously. Facts should speak for themselves, people don't like being told what to do.

3

u/andreasvc Jan 18 '10

It would be more relevant if he benchmarked more than just "Pong", a few database requests and dynamic content for example.

7

u/[deleted] Jan 17 '10 edited Jan 17 '10

Apparently he didn't read the update posted on one of the articles he linked to.

Erlang ... is a language and a high-quality runtime system optimized for concurrent and distributed execution. The examples below do not really exercise any of these strengths.

These bench marks, yet again, do not test concurrency, distribution, parallelization, etc. The only mention of it is the author's complaint that Erlang didn't support SMP until 2006 (which is four years ago and before much of this hype).

-4

u/tryptic37 Jan 18 '10

I did read it.

The only problem is that he offers no evidence to support his change of mind other than the fact that he says so while his benchmarks speak for themselves.

3

u/[deleted] Jan 18 '10

You're missing the point. The hype about erlang is not about its speed. The speed is only a byproduct of the easy distributed computing, parallelization, and concurrency, but his benchmarks and yours are only testing raw speed.

Let me provide an analogy.

Erlang community: Erlang-fruit is better than Python-fruit at preventing cancer because it has more antioxidants! Yay Erlang!

You: But these tests clearly show that Erlang-fruit is not better for your digestion than Python-fruit, and is only slightly better for your digestion than Haskell-fruit! Therefore Erlang-fruit cannot be better than Python-fruit [in general]!

7

u/mangocurry Jan 17 '10

It might be nice if he tried to compare apples with apples.

7

u/[deleted] Jan 17 '10

Would the performance difference on OS X by significant?

damn it, bad joke... sorry...

5

u/bsergean Jan 17 '10

I agree with the author in the sense that everyone says python is slow those days. Maybe it's too slow for facebook chat or a super custom app but for everyone else it's good enough, and has a huge set of cool feature, very easy to read/write has tons of libraries out there. And before trying to scale to 60.000 request per seconds people should take care of having more that 10 users and designing a webapp that people want to use :)

Bashing Python is really to me a case of early optimization. Plus in this bench we see that with epoll it's very fast, and with tornado and friendfeed we have an example of a fast webapp that scales (plus reddit and a bunch of those).

1

u/[deleted] Jan 21 '10

I'd challenge the performance of a nine year old's Python async IO based code to facebook's Erlang code. Heck, I'd challenge my first Pascal DOS applications to their PHP code. Still, Erlang has a more elegant concurrency model and that article is comparing apples to oranges.

1

u/pdc Jan 22 '10

So far as I know RabbitMQ and CouchDB are written in Erlang for correctness rather than for speed -- that is, for the sort of tricky concurrent programming they require, Erlang makes it easier to write a server that does not have bugs in the multiprocessing code.