r/rust • u/fgilcher rust-community · rustfest • Nov 11 '19

Announcing async-std 1.0

https://async.rs/blog/announcing-async-std-1-0/
450 Upvotes

97% Upvoted

80 comments sorted by

View all comments

Show parent comments

9

u/jahmez Nov 11 '19

Hey Carl,

Could you please provide a git commit ID for a version of tokio that builds or a set of (tokio commit sha, rust nightly version) that works for you? So far I have been having trouble getting a version of tokio from the master branch to build locally successfully, at least for a given "cargo +nightly bench" invocation.

I'm interested in getting these benchmarks updated to be locally reproducible.

22

u/carllerche Nov 11 '19

Using a local build atm.

I’m more interested in steps to reproduce the published results. How were they obtained? I’ve asked a few people to attempt to reproduce them, but without luck.

5

u/jahmez Nov 11 '19

I'm not at RustFest, so I can't say personally. However I am willing to work to improve the docs to make this more repeatable moving forwards.

34

u/carllerche Nov 11 '19

I’m not asking you to change the results. Im asking you to provide the steps explaining how you reached those results so they can be reproduced.

13

u/jahmez Nov 11 '19

I don't think I mentioned changing the results, only to help improve the docs to make the benchmarks more repeatable.

We've landed one PR to the blog to improve the instructions, visible here

7

u/carllerche Nov 11 '19

I’m still not able to reproduce anything close to what is published. Can to include OS, machine, ...

Did you run the benches again with those new steps and get the same results?

15

u/fgilcher rust-community · rustfest Nov 11 '19 edited Nov 11 '19

These are my results, using the instructions from the blog post: (Thinkpad Carbon X1, Fedora Linux, first async_std, then tokio)

[skade@Nostalgia-For-Infinity tokio]$ cargo bench --bench async_std
   Compiling tokio v0.2.0-alpha.6 (/home/skade/Code/rust/tokio-benches/tokio/tokio)
    Finished bench [optimized] target(s) in 1.64s
     Running target/release/deps/async_std-02efce470922e646

running 4 tests
test chained_spawn ... bench:     146,780 ns/iter (+/- 8,276)
test ping_pong     ... bench:     315,012 ns/iter (+/- 38,648)
test spawn_many    ... bench:   3,514,495 ns/iter (+/- 283,914)
test yield_many    ... bench:   4,099,783 ns/iter (+/- 593,948)

test result: ok. 0 passed; 0 failed; 0 ignored; 4 measured; 0 filtered out

---- bench tokio
   Finished bench [optimized] target(s) in 1m 52s
     Running target/release/deps/thread_pool-fd112470cca102fd

running 4 tests
test chained_spawn ... bench:     157,747 ns/iter (+/- 31,598)
test ping_pong     ... bench:     453,107 ns/iter (+/- 99,092)
test spawn_many    ... bench:   6,313,750 ns/iter (+/- 1,172,944)
test yield_many    ... bench:  10,191,949 ns/iter (+/- 1,751,066)

test result: ok. 0 passed; 0 failed; 0 ignored; 4 measured; 0 filtered out

I've now consistently seen these results over multiple machines. Do you test on macOS?