-
Notifications
You must be signed in to change notification settings - Fork 740
25-3: schemeshard: stats processing opt 1b #29885
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
25-3: schemeshard: stats processing opt 1b #29885
Conversation
Streamline and optimize datashard statistics processing. Profile guided optimizations of PersistSingleStats() (in synthetic test). Total gain is around 30%. - remove unreasonable iteration over entire ShardInfoscommits/6fefb4b0d08afdd7a9430d54d3d5d70e7bf00100) - single Now() timestamp for entire stats batch - optimize number of lookups - stop building now unnecessary storage pool kind mappings - remove table/store aggregated stats copying - collect ExternalBlobsEnabled only on PartitionConfig change - replace ETxType->CounterId map with absl::flat_hash_map - remove extra OpType->TxType lookup - remove call to GetMainTableForIndex for not-index-table shards
Protect shard statistics processing from missing channel binding information on table shards for old clusters or old databases still served by the root schemeshard.
Fix finding storage pool kind for a tablet's channel for tables created before 25-3. TChannelBindings persisted at schemeshard at earlier versions don't have StoragePoolKind field filled.
Fix disk space aggregation for column tables. Make subdomain and storage pool kind aggregation levels separation more robust. Add tests on subdomain level aggregation.
|
⚪
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
🟢 |
|
⚪ ⚪ Ya make output | Test bloat | Test bloat
⚪ Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
Cherry-pick from
main:Streamline and optimize datashard statistics processing.
Profile guided optimizations of PersistSingleStats() (in synthetic test). Total gain is around 30%.