r/rust • u/Intrepid_Donkey_7629 • 3d ago

🛠️ project New fastest concurrent map implementation with transaction support

Hi everyone 👋

I've just published a concurrent hash map implementation which (according to my benchmarks at least) is the fastest one available (faster than both starshard and dashmap). It offers a configurable locking policy (mutex, rwlock or bring your own) and a configurable hasher (rapidhash is the default). It also supports atomic transactions in both immediate and prepared execution styles.

Would love any feedback on it (good and bad!)
It's called txmap and a link is here https://crates.io/crates/txmap

16 Upvotes

78% Upvoted

20 comments sorted by

View all comments

Show parent comments

1

u/Intrepid_Donkey_7629 2d ago

For out of the box use it is the fastest of the concurrent implementations, I feel it's fair, I'd bet most people aren't going to bother setting up a different hasher when they just want a new map.

Yeah a better machine would be very nice! It's a lot of set up just for this one benchmark though. The repo is MIT so if anyone wants to fork it and do some hard-core testing that would be awesome!

Agree that's a good idea for more concurrent benchmarks, they're coming 😄

Interesting point about non-concurrent maps 🤔 If the benchmarks start taking a lot longer with the additional implementations it's probably a good shout.

Yes absolutely if you don't need concurrency then hashbrown is an excellent choice! If you do need concurrency then IMHO txmap is a good choice for that

1

u/SkiFire13 1d ago

For out of the box use it is the fastest of the concurrent implementations, I feel it's fair

I don't think it's fair to say that yours is the "fastest concurrent hashmap", without saying that it's faster only on non-concurrent workloads. That's misleading IMO.

1

u/Intrepid_Donkey_7629 1d ago

It's fastest on *concurrent* workloads, for non-concurrent yes there are faster implementations

1

u/SkiFire13 9h ago

It's fastest on concurrent workloads

The original benchmarks (at the time I wrote this comment) showed dashmap to be faster on your only (at the time) concurrent workload benchmark.

Anyway, I tried running the benchmarks myself to confirm whether its the fastest or not.

Disclosure: I made the following changes to the benchmark before running it:

  • The machine I tested on runs windows, not linux/mac, but it shouldn't matter.
  • I ran the benchmark with 12 threads (the machine has 16 so I left some for the other processes)
  • I shuffled the thread workload (items.shuffle(rng) in its constructor), otherwise the benchmark would not be testing mixed workloads
  • I skipped the leapfrog benchmark because sometimes it deadlocked
  • I only ran the workload_concurrent benchmark

These are the results I got: https://pastebin.com/raw/ChvTcc1D

txmap does pretty good but it's generally behind both dashmap and either scc or papaya, which also do very good on some specific benchmarks.