29K views · Aug 22, 2012 · 58:36 · 4 HN recommenders · 5 mentions · discussed 2013–2024
A 2007 Google Tech Talk in which Cliff Click of Azul Systems presents a lock-free concurrent hash table that scales to 768 CPUs. He explains why it is correct by reasoning in a different way than usual happens-before and memory-order arguments.
Why Hacker News points people here: HN regards it as one of the best technical talks, a deep and clear walkthrough of scalable lock-free hash tables built on compare-and-swap.
“I highly recommend this talk by Cliff Click about building highly scalable hash tables via Compare And Swap”HN comment, 2024
“Anything from Cliff Click is quite good.”
Best technical talk, deep understanding of compilers, VMs, CPUs
Probably this one: this video Anything from Cliff Click is quite good. This guy has a very deep understanding of compilers, virtual-machines, CPUs, and their interactions. A JVM does that? link Java on a 1000 Cores link
“I highly recommend this talk by Cliff Click about building highly scalable hash tables via Compare And Swap”
Building highly scalable hash tables via CAS
For folks interested in this topic I highly recommend this talk by Cliff Click about building highly scalable hash tables via Compare And Swap: this video
“The talk is really worth it, if you're curious about the design”
Design of lock-free hashtable and when ConcurrentHashMap suffices
Dr. Cliff Click's Google talk on Lock-Free Hashtable is here: this video Acc to Dr. Click, if you do not have 30+ CPUs in a single machine, ConcurrentHashMap (built-in to the JDK) will do the trick more often than not. The talk is really worth it, if you're curious about the design, esp. Dr. Click also had memorable debate with Rich Hickey over STMs. He works at h2o.ai and wrote about building a Key-Value store two years ago: link
“I've always admired the wizardry of Cliff Click's concurrent hash”
Admired wizardry of Click's concurrent hash
I've always admired the wizardry of Cliff Click's concurrent hash: link this video I'm not sure what to make of an "atomic" hash. I don't think OP does as well: "Doesn't this kind of defeat the point of concurrent algorithms? Well, in a sense, it does make it non-atomic, but it isn't really a spin-lock or anything like that, even though it can seem like that:" Concurrent hash seems to be more correct and clear.
“I should probably leave an example of it done well”
Example of raw atomics done well
It's worth noting that for a lot of people for a long time, Intel TSO was the memory model in practice (which IIRC makes `std::memory_order_seq_cst` pretty much free on Intel). Many clothes have been torn and many teeth gnashed porting C++ from Intel to more relaxed architectures. Back in the day that usually meant like SPARC, today that usually means ARM. "Acquire" and "Release" are relatively comprehensible in isolated examples, but the intuitions don't compose well. It's realllly easy to end up with an ABA type issue or an architecture dependency even with the simplest lock-free primitives like CAS. Combine that with how cheap `std::mutex` is on most platforms and it's just really a last resort, at least on server-class gear. Google engineers have more experience with this than probably anyone, and they've summed it up nicely here: link . Doing raw atomics isn't quite "roll your own cryptography", but it's clearly in "if you have to ask, don't use it" territory. I've lost count of how many times I've fuzzed something to Mars and back to show it "correct" and not been able to justify it compared to `pthread` when the profiles were in. With that disclaimer, people are going to do it, and so I should probably leave an example of it done well: this video
