Skip to content

Conversation

astubbs
Copy link
Contributor

@astubbs astubbs commented Mar 9, 2022

No description provided.

Addresses issues reported:

- ConcurrentModificationException in ShardManager during Rebalancing confluentinc#188
- NullPointerException in PartitionMonitor during Rebalancing confluentinc#189

Notes:

- Improve impact of no-op removed state, so it won't be retrieved for analysis
- Change from removing revoked PartitionState instances to replacing with a PartitionRemovedState subclass
-- Advantage of this is there will never be NPE from stale references of work still in flight
-- Also means we can track history of assignment
- Several refactorings that have been waiting to happen related to the change
- Improved cohesion

Relates / Future improvements on this:

Refactor: Consider a shared nothing architecture, to reduce thread complexity confluentinc#200
Cleans up warnings in logs of ignored properties.
@astubbs astubbs marked this pull request as draft March 10, 2022 21:12
astubbs and others added 3 commits March 14, 2022 15:46
- test for observing dropping average batch size
- performance: process mailbox timeout calculation not needed as poll box is interrupted either with work results, or with new messages from broker poller
- improvements to event based system and LongPollingMockConsumer
- eventing and timing improvements and simplifications / refactorings, better batch test code reuse
- only sleep for retry delay if starved
- remove clean/dirty state cache - too complex and unnecessary - stay tuned for replacement
- change close ordering of operations to fix final commit execution
- teach ProgressTracker to use time
- batch size request - always request enough to fill all batches

Co-authored-by: Antony Stubbs <[email protected]>
Co-authored-by: Benedikt Linse <[email protected]>
…ilure

Previous the container was constructed as part of a static initilialzer block for the static field. If a container failed to start, it would cause a ~Failed to initilaize class error, which was confusing and unclear in logs.
@eddyv
Copy link
Member

eddyv commented Jun 15, 2023

Closing - Stale.

@eddyv eddyv closed this Jun 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants