File tree Expand file tree Collapse file tree 1 file changed +8
-4
lines changed
Expand file tree Collapse file tree 1 file changed +8
-4
lines changed Original file line number Diff line number Diff line change 475475//!
476476//! The number of cores used is determined by the `target_partitions`
477477//! configuration setting, which defaults to the number of CPU cores.
478- //! During execution, DataFusion creates this many distinct `async` [`Stream`]s and
479- //! this many distinct [Tokio] [`task`]s, which drive the `Stream`s
480- //! using threads managed by the `Runtime`. Many DataFusion `Stream`s perform
481- //! CPU intensive processing.
478+ //! While preparing for execution, DataFusion tries to create this many distinct
479+ //! `async` [`Stream`]s for each `ExecutionPlan`.
480+ //! The `Stream`s for certain `ExecutionPlans`, such as as [`RepartitionExec`]
481+ //! and [`CoalescePartitionsExec`], spawn [Tokio] [`task`]s, that are run by
482+ //! threads managed by the `Runtime`.
483+ //! Many DataFusion `Stream`s perform CPU intensive processing.
482484//!
483485//! Using `async` for CPU intensive tasks makes it easy for [`TableProvider`]s
484486//! to perform network I/O using standard Rust `async` during execution.
582584//! [`Runtime`]: tokio::runtime::Runtime
583585//! [`task`]: tokio::task
584586//! [Using Rustlang’s Async Tokio Runtime for CPU-Bound Tasks]: https://thenewstack.io/using-rustlangs-async-tokio-runtime-for-cpu-bound-tasks/
587+ //! [`RepartitionExec`]: physical_plan::repartition::RepartitionExec
588+ //! [`CoalescePartitionsExec`]: physical_plan::coalesce_partitions::CoalescePartitionsExec
585589//!
586590//! ## State Management and Configuration
587591//!
You can’t perform that action at this time.
0 commit comments