bpo-46675: Allow object value arrays and split key dictionaries larger than 16 #31191
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
markshannon
changed the title
markshannon
added
the
🔨 test-with-buildbots
|
🤖 New build scheduled with the buildbot fleet by @markshannon for commit 3cbdf70 🤖 If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again. |
bedevere-bot
removed
the
🔨 test-with-buildbots
methane
reviewed
Feb 8, 2022
merwok
reviewed
Feb 12, 2022
| @@ -0,0 +1,2 @@ | |||
| Allow more than 16 items in a split dict before it is combined. The limit is | |||
| now 254. | |||
There was a problem hiding this comment.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
At a slight increase in overhead for for small object dictionaries, and an extra word of memory for sizes 7-15, this PR allows split-key dicts and "virtual" object dicts for size > 16.
The benchmarks suggest that this is not an improvement, but looking a bit closely reveals a couple of things.
The logging benchmarks are significantly faster. The log record object has ~20 attributes, so this is an expected improvement.
The pickle benchmarks are significantly slower, I don't know why.
Overall, this may not make things faster, but it does remove a "sharp edge". Adding an extra attribute (from 16 to 17) should not reduce performance by a significant amount.
Why are quite a few of the benchmarks slower?
I don't know, but my hypothesis is that the larger dict-keys is slowing down lookup due to poorer locality.
We may need to revisit static analysis, to provide better hints for the initial size of the dict keys.
https://bugs.python.org/issue46675