Skip to content

bpo-44206: Add a version number to dictionary keys #26333

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
May 28, 2021

Conversation

markshannon
Copy link
Member

@markshannon markshannon commented May 24, 2021

Apart from adding a version number, this PR also:

  • Reduces the size of dict keys by 8 bytes on 64 bit systems.
  • Reduces the overall code size by ~100 lines

I'll need to run pyperformance before merging, in case this slows things downs.

https://bugs.python.org/issue44206

@markshannon markshannon requested a review from methane as a code owner May 24, 2021 14:44
@markshannon markshannon changed the title Add a version number to dictionary keys bpo-44206: Add a version number to dictionary keys May 24, 2021
@markshannon
Copy link
Member Author

Skipping news, as this is strictly internal.


CHECK(0 <= mp->ma_used && mp->ma_used <= usable);
CHECK(IS_POWER_OF_2(keys->dk_size));
CHECK(IS_POWER_OF_2(DK_SIZE(keys)));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check is now redundant.

@markshannon
Copy link
Member Author

Performance results
Faster on 40, slower on 18. I'm not claiming a speedup, just that it appears to not be slower.

typedef enum {
DICT_KEYS_GENERAL = 0,
DICT_KEYS_UNICODE = 1,
DICT_KEYS_UNICODE_NO_DUMMY = 2,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_Py_dict_lookup() don't have specialized case for DICT_KEYS_UNICODE_NO_DUMMY. Can we delete this kind?

This kind is used only here.

else if (mp->ma_keys->dk_kind == DICT_KEYS_UNICODE) {

But it can be replaced with mp->ma_used > mp->ma_keys->dk_nentries.

@markshannon markshannon merged commit f8a95df into python:main May 28, 2021
@markshannon markshannon deleted the dict-keys-version branch August 23, 2021 13:32
malor added a commit to malor/cpython-lldb that referenced this pull request Nov 17, 2024
python/cpython#26333 changed the way the size
of the hash table is stored: instead of storing the actual value, dict
now saves a log2 of that, which allowed to change the data type of the
corresponding struct field from Py_ssize_t to uint8_t making all dict
objects more compact.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants