Implement optimized linear probing

[czgdp1807]

Separate chaining in its raw form requires creating linked list for each insertion in the dict. This implies expensive malloc calls for each insertion. This becomes a major overhead when we scale the number of insertions as can be seen with C++ benchmarks here. However the advantage of separate chaining is that when the length of linked list at a given index is 1 then we know for sure that we don't need to compare keys by value as no collision has happened yet.

So, instead of creating LinkedList for each new insertion, I have updated key_mask to provide the signal whether linear probing is to be done or not. This way we can have the benefit separate chaining (avoiding comparison of keys by value while reading) and linear probing (cache efficiency, low overhead as malloc calls are made only when rehashing the table) both at one time.

I am yet to decouple the logic into a separate child class of LLVMDict so that we can switch between the two collision resolution strategies easily.

Note - The benefit of this should be visible when we will be using derived data structures as keys (such as tuple, or very long strings). For now the benchmarks don't show any slowdowns.

[czgdp1807]

@certik This is ready for review. Please let me know if anything should be changed here.

[certik]

The changes seem fine to me.

[certik]

So the current benchmarks do not show a slowdown.

Is there a benchmark that would show a speedup with this PR?

[czgdp1807]

Is there a benchmark that would show a speedup with this PR?

"Note - The benefit of this should be visible when we will be using derived data structures as keys (such as tuple, or very long strings). For now the benchmarks don't show any slowdowns." - Quoting from #990 (comment). The reason is quoted below,

So, instead of creating LinkedList for each new insertion, I have updated key_mask to provide the signal whether linear probing is to be done or not. This way we can have the benefit separate chaining (avoiding comparison of keys by value while reading) and linear probing (cache efficiency, low overhead as malloc calls are made only when rehashing the table) both at one time.

[certik]

It might be a "premature optimization" if we can measure the speedup, but it's fine with me to merge, since you believe this will help us in the future.

[czgdp1807]

Well. The choice is either this or linked lists ( a.k.a separate chaining) we want to avoid key comparison for two keys not having the same hash. Linked lists is not something I would go for as it will add costs with each insertion. The other thing left is what I implemented here.

Anyways, I will implement dict.delete and then benchmark on a bunch of hash functions for strings. So if there will be any issues I will fix them right away.

[certik]

I agree to avoid linked lists.

I thought we already had a solution in master for when two keys have the same hash.

[czgdp1807]

Yes we already had collision resolution before this PR. But that compared keys by value at least once. But if you know beforehand that at a certain key hash no collision has happened then we don’t need to compare original keys at all. So while fetching elements from the dict we will get performance benefits for keys for which no collision has happened. For those which have faced collision we already have the algorithm implemented for that and it works well.

[certik]

I see, I understand now. Thanks for implementing it.