-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Closed
Labels
enhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomers
Description
Is your feature request related to a problem or challenge?
Historically DataFusion was one (very) large crate datafusion
, and as it grew bigger we extracted various functionality into separate crates. This leads to both faster compile times (as the crates can be compiled in parallel) as well easier to navigate code (as the crates force a cleaner dependency separation)
This project tracks moving the physical optimizers:
- The original physical optimizers are here https://github.com/apache/datafusion/tree/main/datafusion/core/src/physical_optimizer
- The new crate is here https://github.com/apache/datafusion/tree/main/datafusion/physical-optimizer
Describe the solution you'd like
Pull physical optimizer rules to datafusion-physical-optimizer
.
There are some tests that have dependencies on datasource
, which they are not possible to move out of core at this moment. We may need to leave those tests inside core for now.
- PhysicalOptimizerRule Create
datafusion-physical-optimizer
crate #11507 - OutputRequirements
- AggregateStatistics
- Move
JoinSelection
intodatafusion-physical-optimizer
crate #14073 - LimitedDistinctAggregation
- EnforceDistribution
- CombinePartialFinalAggregate
- EnforceSorting
- OptimizeAggregateOrder
- ProjectionPushdown
- CoalesceBatches
- OutputRequirements
- TopKAggregation
- Move
SanityChecker
intodatafusion-physical-optimizer
crate #14072 - Move
ProjectionPushdown
intodatafusion-physical-optimizer
crate #14184 - Move
EnforceSorting
intodatafusion-physical-optimizer
crate #14185 - Move
EnforceDistribution
intodatafusion-physical-optimizer
crate #14186 - Move
PhysicalOptimizer
and any remaining crates intodatafusion-physical-optimizer
crate, remove datafusion/core/physical-optimizer
Describe alternatives you've considered
No response
Additional context
No response
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomers