Skip to content

[SPARK-28257][SQL] Use ConfigEntry for hardcoded configs in SQL #25059

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
WangGuangxin wants to merge 5 commits into from
Closed

[SPARK-28257][SQL] Use ConfigEntry for hardcoded configs in SQL #25059

WangGuangxin wants to merge 5 commits into from
+178 −144

Conversation

@WangGuangxin
Copy link
Contributor

@WangGuangxin WangGuangxin commented Jul 5, 2019

What changes were proposed in this pull request?

There are some hardcoded configs, using config entry to replace them.

How was this patch tested?

Existing UT

@WangGuangxin WangGuangxin changed the title [SPARK-28257]Use ConfigEntry for hardcoded configs in SQL [SPARK-28257][SQL][MINOR]Use ConfigEntry for hardcoded configs in SQL Jul 5, 2019
wangyum
wangyum reviewed Jul 5, 2019
View reviewed changes
Copy link
Member

@wangyum wangyum left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you replace more configs:

spark.sql.warehouse.dir
spark.sql.sources.schemaStringLengthThreshold
spark.sql.hive.thriftServer.singleSession
spark.sql.extensions
spark.sql.streaming.streamingQueryListeners
spark.sql.ui.retainedExecutions

@jerryshao
Copy link
Contributor

jerryshao commented Jul 5, 2019

ok to test.

@SparkQA
Copy link

SparkQA commented Jul 5, 2019

Test build #107283 has finished for PR 25059 at commit 16236b7.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyun
Copy link
Member

dongjoon-hyun commented Jul 5, 2019

Hi, @WangGuangxin . We don't need to put [MINOR] if you have a JIRA ID.

@dongjoon-hyun dongjoon-hyun changed the title [SPARK-28257][SQL][MINOR]Use ConfigEntry for hardcoded configs in SQL [SPARK-28257][SQL] Use ConfigEntry for hardcoded configs in SQL Jul 5, 2019
@WangGuangxin
Copy link
Contributor Author

WangGuangxin commented Jul 6, 2019

Could you replace more configs:

spark.sql.warehouse.dir
spark.sql.sources.schemaStringLengthThreshold
spark.sql.hive.thriftServer.singleSession
spark.sql.extensions
spark.sql.streaming.streamingQueryListeners
spark.sql.ui.retainedExecutions

done

@SparkQA
Copy link

SparkQA commented Jul 6, 2019

Test build #107296 has finished for PR 25059 at commit 9198d28.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyun
Copy link
Member

dongjoon-hyun commented Jul 10, 2019

Retest this please.

@SparkQA
Copy link

SparkQA commented Jul 10, 2019

Test build #107476 has finished for PR 25059 at commit 9198d28.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyun
Copy link
Member

dongjoon-hyun commented Jul 10, 2019

This is a little huge for this kind of patch. However, it looks correct. I finished my first round review. Could you update this PR, @WangGuangxin ?

@SparkQA
Copy link

SparkQA commented Jul 11, 2019

Test build #107533 has finished for PR 25059 at commit 89da7a9.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@WangGuangxin
Copy link
Contributor Author

WangGuangxin commented Jul 11, 2019

This is a little huge for this kind of patch. However, it looks correct. I finished my first round review. Could you update this PR, @WangGuangxin ?

Thanks for your comments. I believe I've resolved all related comments & conflicts.

@SparkQA
Copy link

SparkQA commented Jul 11, 2019

Test build #107541 has finished for PR 25059 at commit e601c74.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyun
Copy link
Member

dongjoon-hyun commented Jul 11, 2019

Thank you for updating, @WangGuangxin . There are a few more remaining nits.

@SparkQA
Copy link

SparkQA commented Jul 12, 2019

Test build #107568 has finished for PR 25059 at commit b39fb62.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

dongjoon-hyun
dongjoon-hyun approved these changes Jul 12, 2019
View reviewed changes
Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Merged to master.
Thank you, @WangGuangxin and @maropu !

@WangGuangxin WangGuangxin deleted the ConfigEntry branch July 12, 2019 09:32
vinodkc pushed a commit to vinodkc/spark that referenced this pull request Jul 18, 2019
@WangGuangxin @vinodkc
[SPARK-28257][SQL] Use ConfigEntry for hardcoded configs in SQL
6cc4b8b 
## What changes were proposed in this pull request?

There are some hardcoded configs, using config entry to replace them.

## How was this patch tested?

Existing UT

Closes apache#25059 from WangGuangxin/ConfigEntry.

Authored-by: wangguangxin.cn <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@maropu maropu maropu left review comments

@wangyum wangyum wangyum left review comments

@dongjoon-hyun dongjoon-hyun dongjoon-hyun approved these changes

Assignees

No one assigned

Labels

SQL

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@WangGuangxin @jerryshao @SparkQA @dongjoon-hyun @maropu @wangyum