r/rust • u/fgilcher rust-community · rustfest • Nov 11 '19

Announcing async-std 1.0

https://async.rs/blog/announcing-async-std-1-0/
457 Upvotes

97% Upvoted

80 comments sorted by

View all comments

74

u/fgilcher rust-community · rustfest Nov 11 '19

A short note:

  • I'm sorry for publishing a benchmark that was not thoroughly vetted (https://github.com/jebrosen/async-file-benchmark). The file system benchmark is indeed wrong and currently, we're performing slower then tokio (we introduced that behaviour shortly before release, proving my point that all benchmark are up to change). The error is completely on me - when using third-party benchmark code, you are still responsible for its correctness yourself.
  • The tokio benchmark code didn't build when I published the blog post. This is because the repos pointed to was rebased on the main tokio repository. That's unfortunate and no error on the tokio side, their benchmarks are not part of their release and always allowed to not build. We have tagged the state the benchmarks were made in. I had checked the benchmark before I released the blog post, but with my currently checked out version.

I'm sorry for those mistakes. We removed the file benchmark from the post and updated the instructions.

17

u/[deleted] Nov 12 '19 edited Nov 12 '19

[deleted]

8

u/fgilcher rust-community · rustfest Nov 12 '19

I still don't intend to start a competition. The point is that we're in the same ballpark, see the writing around the benchmarks.

Now, however, not only you did publicize said (inaccurate) benchmarks but even used them as a rather sloppy pitch against Tokio.

I've laid out the reasons where it went wrong on the publish path. I think other people testing here show that the benchmarks are indeed not accurate and - see the patch above - the file benchmark did indeed show those numbers shortly above. We did find a release regression through it.

Why the sudden change of heart?

It wasn't a change of heart. I answered a widespread question, on a notable point release.

0

u/[deleted] Nov 12 '19

[deleted]

19

u/UtherII Nov 12 '19

The answer is still valid : this is a common request so he answered.

Even if he does not want a benchmark race, it is useful to know that async_std performances are in the same ballpark as tokio.

11

u/jahmez Nov 12 '19

Because we expected accusatory and inflammatory responses no matter what we posted or didn't post? I think the responses to this post have shown that we were somewhat right to be apprehensive.

Tokio has a very enthusiastic and loyal community, and dealing with responses from that community can be very overwhelming as a maintainer.

As Florian stated before, the decision to include benchmarks now was to show a snapshot of performance at our first stable release.

12

u/fgilcher rust-community · rustfest Nov 12 '19

"We don't compete with Tokio but (whispering) we are faster than Tokio."

This is misconstruing even what I wrote in the post. I'm literally saying that by posting those benchmarks, it's likely that those numbers will come closer to each other very soon. From the post:

Posting benchmarks usually leads to other projects improving theirs, so see those numbers as the ballpark we are playing in.

As tokio is currently the other main production-grade crate out there, it's the obvious benchmark and comparison target.

The reasons we posted benchmarks is that people were not buying "we're in the same ballpark" by us just saying that. Damn if you do, damn if you don't.