Skip to content

Conversation

@comphead
Copy link
Contributor

@comphead comphead commented Dec 6, 2025

Which issue does this PR close?

  • Closes #.

This is PR instead of accidentally rebased #18985
Also

  • fixing Q30 as there is a reference to non existent column
  • adding TPCH, TPCDS scripts to compare results between branches and documentation

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions bot added the core Core DataFusion crate label Dec 7, 2025
@comphead comphead marked this pull request as ready for review December 7, 2025 00:07
@comphead comphead requested a review from alamb December 7, 2025 00:07
@comphead
Copy link
Contributor Author

comphead commented Dec 8, 2025

Thanks @Dandandan for the review

@comphead comphead added this pull request to the merge queue Dec 8, 2025
Merged via the queue into apache:main with commit e8384fb Dec 8, 2025
27 checks passed
@mbutrovich
Copy link
Contributor

I ran this on main after cloning the data from datafusion-benchmarks and a bunch of queries return 0 rows which, without looking at the TPC-DS spec yet, smells wrong to me. Is that expected behavior?

PREFER_HASH_JOIN=false DATA_DIR=/Users/matt/git/datafusion-benchmarks/tpcds/data/sf1 ./bench.sh run tpcds | grep ' 0 rows'
+ cargo run --release --bin dfbench -- tpcds --iterations 5 --path /Users/matt/git/datafusion-benchmarks/tpcds/data/sf1 --query_path ../datafusion/core/tests/tpc-ds --prefer_hash_join false -o /Users/matt/git/datafusion/benchmarks/results/main/tpcds_sf1.json
warning: unused import: `assert_or_internal_err`
  --> datafusion/expr/src/udf.rs:29:38
   |
29 |     ExprSchema, Result, ScalarValue, assert_or_internal_err, not_impl_err,
   |                                      ^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(unused_imports)]` (part of `#[warn(unused)]`) on by default

warning: `datafusion-expr` (lib) generated 1 warning (run `cargo fix --lib -p datafusion-expr` to apply 1 suggestion)
    Finished `release` profile [optimized] target(s) in 0.14s
     Running `/Users/matt/git/datafusion/target/release/dfbench tpcds --iterations 5 --path /Users/matt/git/datafusion-benchmarks/tpcds/data/sf1 --query_path ../datafusion/core/tests/tpc-ds --prefer_hash_join false -o /Users/matt/git/datafusion/benchmarks/results/main/tpcds_sf1.json`
Query 1 iteration 0 took 105.5 ms and returned 0 rows
Query 1 iteration 1 took 47.1 ms and returned 0 rows
Query 1 iteration 2 took 46.4 ms and returned 0 rows
Query 1 iteration 3 took 47.7 ms and returned 0 rows
Query 1 iteration 4 took 46.2 ms and returned 0 rows
Query 7 iteration 0 took 168.0 ms and returned 0 rows
Query 7 iteration 1 took 165.2 ms and returned 0 rows
Query 7 iteration 2 took 163.4 ms and returned 0 rows
Query 7 iteration 3 took 166.2 ms and returned 0 rows
Query 7 iteration 4 took 164.8 ms and returned 0 rows
Query 8 iteration 0 took 83.8 ms and returned 0 rows
Query 8 iteration 1 took 76.2 ms and returned 0 rows
Query 8 iteration 2 took 81.6 ms and returned 0 rows
Query 8 iteration 3 took 79.8 ms and returned 0 rows
Query 8 iteration 4 took 80.5 ms and returned 0 rows
Query 12 iteration 0 took 40.9 ms and returned 0 rows
Query 12 iteration 1 took 41.5 ms and returned 0 rows
Query 12 iteration 2 took 41.6 ms and returned 0 rows
Query 12 iteration 3 took 41.8 ms and returned 0 rows
Query 12 iteration 4 took 43.2 ms and returned 0 rows
Query 18 iteration 0 took 210.5 ms and returned 0 rows
Query 18 iteration 1 took 204.0 ms and returned 0 rows
Query 18 iteration 2 took 205.4 ms and returned 0 rows
Query 18 iteration 3 took 207.6 ms and returned 0 rows
Query 18 iteration 4 took 199.7 ms and returned 0 rows
Query 20 iteration 0 took 54.6 ms and returned 0 rows
Query 20 iteration 1 took 54.6 ms and returned 0 rows
Query 20 iteration 2 took 55.1 ms and returned 0 rows
Query 20 iteration 3 took 53.8 ms and returned 0 rows
Query 20 iteration 4 took 54.3 ms and returned 0 rows
Query 23 iteration 0 took 889.9 ms and returned 0 rows
Query 23 iteration 1 took 858.8 ms and returned 0 rows
Query 23 iteration 2 took 797.3 ms and returned 0 rows
Query 23 iteration 3 took 771.9 ms and returned 0 rows
Query 23 iteration 4 took 777.0 ms and returned 0 rows
Query 24 iteration 0 took 432.4 ms and returned 0 rows
Query 24 iteration 1 took 420.7 ms and returned 0 rows
Query 24 iteration 2 took 436.1 ms and returned 0 rows
Query 24 iteration 3 took 434.2 ms and returned 0 rows
Query 24 iteration 4 took 441.7 ms and returned 0 rows
Query 25 iteration 0 took 203.0 ms and returned 0 rows
Query 25 iteration 1 took 195.8 ms and returned 0 rows
Query 25 iteration 2 took 199.9 ms and returned 0 rows
Query 25 iteration 3 took 199.3 ms and returned 0 rows
Query 25 iteration 4 took 204.6 ms and returned 0 rows
Query 26 iteration 0 took 137.3 ms and returned 0 rows
Query 26 iteration 1 took 136.3 ms and returned 0 rows
Query 26 iteration 2 took 136.5 ms and returned 0 rows
Query 26 iteration 3 took 138.7 ms and returned 0 rows
Query 26 iteration 4 took 135.5 ms and returned 0 rows
Query 27 iteration 0 took 159.2 ms and returned 0 rows
Query 27 iteration 1 took 167.1 ms and returned 0 rows
Query 27 iteration 2 took 158.8 ms and returned 0 rows
Query 27 iteration 3 took 158.9 ms and returned 0 rows
Query 27 iteration 4 took 159.5 ms and returned 0 rows
Query 29 iteration 0 took 162.2 ms and returned 0 rows
Query 29 iteration 1 took 165.2 ms and returned 0 rows
Query 29 iteration 2 took 170.9 ms and returned 0 rows
Query 29 iteration 3 took 163.9 ms and returned 0 rows
Query 29 iteration 4 took 167.2 ms and returned 0 rows
Query 33 iteration 0 took 89.2 ms and returned 0 rows
Query 33 iteration 1 took 88.3 ms and returned 0 rows
Query 33 iteration 2 took 89.4 ms and returned 0 rows
Query 33 iteration 3 took 86.6 ms and returned 0 rows
Query 33 iteration 4 took 89.8 ms and returned 0 rows
Query 34 iteration 0 took 73.6 ms and returned 0 rows
Query 34 iteration 1 took 74.0 ms and returned 0 rows
Query 34 iteration 2 took 73.4 ms and returned 0 rows
Query 34 iteration 3 took 73.3 ms and returned 0 rows
Query 34 iteration 4 took 73.8 ms and returned 0 rows
Query 36 iteration 0 took 112.7 ms and returned 0 rows
Query 36 iteration 1 took 117.4 ms and returned 0 rows
Query 36 iteration 2 took 113.9 ms and returned 0 rows
Query 36 iteration 3 took 116.0 ms and returned 0 rows
Query 36 iteration 4 took 113.3 ms and returned 0 rows
Query 37 iteration 0 took 58.2 ms and returned 0 rows
Query 37 iteration 1 took 56.9 ms and returned 0 rows
Query 37 iteration 2 took 55.8 ms and returned 0 rows
Query 37 iteration 3 took 55.1 ms and returned 0 rows
Query 37 iteration 4 took 57.9 ms and returned 0 rows
Query 41 iteration 0 took 10.3 ms and returned 0 rows
Query 41 iteration 1 took 10.0 ms and returned 0 rows
Query 41 iteration 2 took 11.2 ms and returned 0 rows
Query 41 iteration 3 took 10.5 ms and returned 0 rows
Query 41 iteration 4 took 9.3 ms and returned 0 rows
Query 43 iteration 0 took 64.7 ms and returned 0 rows
Query 43 iteration 1 took 64.6 ms and returned 0 rows
Query 43 iteration 2 took 62.5 ms and returned 0 rows
Query 43 iteration 3 took 63.3 ms and returned 0 rows
Query 43 iteration 4 took 66.9 ms and returned 0 rows
Query 44 iteration 0 took 10.8 ms and returned 0 rows
Query 44 iteration 1 took 10.4 ms and returned 0 rows
Query 44 iteration 2 took 10.4 ms and returned 0 rows
Query 44 iteration 3 took 10.8 ms and returned 0 rows
Query 44 iteration 4 took 10.1 ms and returned 0 rows
Query 46 iteration 0 took 135.1 ms and returned 0 rows
Query 46 iteration 1 took 136.6 ms and returned 0 rows
Query 46 iteration 2 took 136.8 ms and returned 0 rows
Query 46 iteration 3 took 141.5 ms and returned 0 rows
Query 46 iteration 4 took 135.5 ms and returned 0 rows
Query 53 iteration 0 took 71.2 ms and returned 0 rows
Query 53 iteration 1 took 69.7 ms and returned 0 rows
Query 53 iteration 2 took 72.0 ms and returned 0 rows
Query 53 iteration 3 took 69.8 ms and returned 0 rows
Query 53 iteration 4 took 71.0 ms and returned 0 rows
Query 54 iteration 0 took 114.6 ms and returned 0 rows
Query 54 iteration 1 took 118.3 ms and returned 0 rows
Query 54 iteration 2 took 115.2 ms and returned 0 rows
Query 54 iteration 3 took 115.2 ms and returned 0 rows
Query 54 iteration 4 took 115.1 ms and returned 0 rows
Query 56 iteration 0 took 87.4 ms and returned 0 rows
Query 56 iteration 1 took 87.0 ms and returned 0 rows
Query 56 iteration 2 took 91.8 ms and returned 0 rows
Query 56 iteration 3 took 89.4 ms and returned 0 rows
Query 56 iteration 4 took 88.3 ms and returned 0 rows
Query 60 iteration 0 took 86.6 ms and returned 0 rows
Query 60 iteration 1 took 95.4 ms and returned 0 rows
Query 60 iteration 2 took 89.9 ms and returned 0 rows
Query 60 iteration 3 took 90.6 ms and returned 0 rows
Query 60 iteration 4 took 91.3 ms and returned 0 rows
Query 63 iteration 0 took 72.7 ms and returned 0 rows
Query 63 iteration 1 took 70.6 ms and returned 0 rows
Query 63 iteration 2 took 72.4 ms and returned 0 rows
Query 63 iteration 3 took 71.4 ms and returned 0 rows
Query 63 iteration 4 took 71.4 ms and returned 0 rows
Query 64 iteration 0 took 758.8 ms and returned 0 rows
Query 64 iteration 1 took 750.2 ms and returned 0 rows
Query 64 iteration 2 took 762.3 ms and returned 0 rows
Query 64 iteration 3 took 762.5 ms and returned 0 rows
Query 64 iteration 4 took 753.8 ms and returned 0 rows
Query 66 iteration 0 took 415.8 ms and returned 0 rows
Query 66 iteration 1 took 418.7 ms and returned 0 rows
Query 66 iteration 2 took 416.4 ms and returned 0 rows
Query 66 iteration 3 took 403.5 ms and returned 0 rows
Query 66 iteration 4 took 437.3 ms and returned 0 rows
Query 68 iteration 0 took 156.2 ms and returned 0 rows
Query 68 iteration 1 took 163.2 ms and returned 0 rows
Query 68 iteration 2 took 156.0 ms and returned 0 rows
Query 68 iteration 3 took 157.0 ms and returned 0 rows
Query 68 iteration 4 took 162.4 ms and returned 0 rows
Query 71 iteration 0 took 85.9 ms and returned 0 rows
Query 71 iteration 1 took 84.4 ms and returned 0 rows
Query 71 iteration 2 took 82.4 ms and returned 0 rows
Query 71 iteration 3 took 83.6 ms and returned 0 rows
Query 71 iteration 4 took 86.0 ms and returned 0 rows
Query 72 iteration 0 took 13939.5 ms and returned 0 rows
Query 72 iteration 1 took 16225.3 ms and returned 0 rows
Query 72 iteration 2 took 16461.0 ms and returned 0 rows
Query 72 iteration 3 took 18128.5 ms and returned 0 rows
...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants