Skip to content

Conversation

BohuTANG
Copy link
Member

@BohuTANG BohuTANG commented Mar 13, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Now, we can modify the table&column statistics and verify the TPC-DS optimized plan is expected or not, easy to DEBUG.

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-chore this PR only has small changes that no need to record, like coding styles. label Mar 13, 2025
@BohuTANG BohuTANG requested a review from zhang2014 March 13, 2025 15:48
Copy link

what-the-diff bot commented Mar 13, 2025

PR Summary

  • Enhancement of export features
    The addition of BasicColumnStatistics and TableStatistics exports in the library module allows these types to be utilized across different modules.
  • Introduction of new testing utilities
    A new test utilities module is introduced, which includes test_utils, a component for easier optimization testing.
  • Creation of test_utils.rs file
    A new file test_utils.rs has been created with the added utility TestStatisticsBuilder. This utility aids in managing and applying statistics to query plans, along with helper methods to manipulate said plans.
  • Addition of tpcds_test.rs file
    The newly introduced tpcds_test.rs file incorporates a test case structure and a specific test for TPC-DS SQL queries. This includes setting up and executing queries with statistics.
  • Exposure of optimize function
    The optimize function has been exported to allow access from other modules, enhancing its accessibility.
  • Visibility alteration of plan field
    The visibility of the plan field has been changed from pub(crate) to pub, ensuring easy access in other parts of the codebase for increased flexibility.

@BohuTANG BohuTANG requested a review from sundy-li March 13, 2025 15:50
@forsaken628
Copy link
Collaborator

Would it be easier to maintain with goldenfile::Mint?

Something like

let mut mint = Mint::new("tests/it/scalars/testdata");

@BohuTANG
Copy link
Member Author

Would it be easier to maintain with goldenfile::Mint?

Something like

let mut mint = Mint::new("tests/it/scalars/testdata");

Hmm, this is not file to file compare, the yaml file aims to store some metadata and result for more structured, need deserialize them and compare the result.
I am not sure how to use goldenfile.

@BohuTANG BohuTANG marked this pull request as ready for review March 14, 2025 11:49
@BohuTANG BohuTANG force-pushed the dev-optimizer-ut branch 2 times, most recently from 6e1883f to 9213c16 Compare March 14, 2025 12:13
@BohuTANG BohuTANG merged commit 0389734 into databendlabs:main Mar 14, 2025
76 of 77 checks passed
loloxwg pushed a commit to loloxwg/databend that referenced this pull request Apr 3, 2025
* add tpcds unit test for optimizer, easy to show and debug

* change the tpcds optimizer to file

* change snow plan from json to tree

* add q1.yaml

* fix yarm lint error

* fix foramt lint
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-chore this PR only has small changes that no need to record, like coding styles.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants