[SPARK-33142][SPARK-33647][SQL][FOLLOW-UP] Add docs and test cases #30666

linhongliu-db · 2020-12-08T07:43:52Z

What changes were proposed in this pull request?

Addressed comments in PR #30567, including:

add test case for SPARK-33647 and SPARK-33142
add migration guide
add getRawTempView and getRawGlobalTempView to return the raw view info (i.e. TemporaryViewRelation)
other minor code clean

Why are the changes needed?

Code clean and more test cases

Does this PR introduce any user-facing change?

No

How was this patch tested?

Existing and newly added test cases

SparkQA · 2020-12-08T07:53:10Z

Test build #132417 has finished for PR 30666 at commit fec710f.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2020-12-08T08:25:55Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37017/

SparkQA · 2020-12-08T08:52:33Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37017/

linhongliu-db · 2020-12-08T09:40:29Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala

Below changes will be removed after #30662 is merged

SparkQA · 2020-12-08T10:20:50Z

Test build #132422 has finished for PR 30666 at commit 3b71c3a.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

linhongliu-db · 2020-12-08T10:41:40Z

retest this please

SparkQA · 2020-12-08T11:29:29Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37028/

SparkQA · 2020-12-08T11:58:20Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37028/

SparkQA · 2020-12-08T12:59:48Z

Test build #132429 has finished for PR 30666 at commit 3b71c3a.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

linhongliu-db · 2020-12-08T15:25:11Z

retest this please

SparkQA · 2020-12-08T16:17:15Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37036/

SparkQA · 2020-12-08T16:43:51Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37036/

SparkQA · 2020-12-08T17:53:45Z

Test build #132435 has finished for PR 30666 at commit 3b71c3a.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

linhongliu-db · 2020-12-09T01:06:44Z

cc @cloud-fan

HyukjinKwon · 2020-12-09T01:58:34Z

@linhongliu-db can you elabourate what this PR proposes in the PR description?

SparkQA · 2020-12-10T04:29:56Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37137/

SparkQA · 2020-12-10T05:01:33Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37137/

SparkQA · 2020-12-10T05:35:03Z

Test build #132533 has finished for PR 30666 at commit a8a8f84.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2020-12-10T10:26:27Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37162/

SparkQA · 2020-12-10T10:51:52Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37162/

linhongliu-db · 2020-12-10T13:03:30Z

Thanks @HyukjinKwon , this PR is ready for review. and also cc @cloud-fan

cloud-fan · 2020-12-11T05:25:51Z

sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala

why do we remove this test?

This is same to the newly added test case in SQLViewTestSuite.
They both test the scenario:
global temp view refers to a table, create a temporary view with the same name, won't affect the result.

But SQLViewTestSuite will covert all 3 kinds of views, and also test the same table name in different database.

SparkQA · 2020-12-11T05:35:40Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37223/

SparkQA · 2020-12-11T06:02:54Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37223/

SparkQA · 2020-12-11T07:12:44Z

Test build #132612 has finished for PR 30666 at commit f578b0b.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2020-12-11T08:44:21Z

retest this please

SparkQA · 2020-12-11T09:39:10Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37242/

SparkQA · 2020-12-11T10:06:10Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37242/

SparkQA · 2020-12-11T10:56:28Z

Test build #132621 has finished for PR 30666 at commit 7e23fea.

This patch fails SparkR unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2020-12-11T11:36:20Z

Test build #132639 has finished for PR 30666 at commit 7e23fea.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

linhongliu-db · 2020-12-11T11:41:23Z

retest this please

SparkQA · 2020-12-11T12:26:19Z

Test build #132655 has finished for PR 30666 at commit 7e23fea.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2020-12-11T12:49:50Z

retest this please

SparkQA · 2020-12-11T13:43:34Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37262/

SparkQA · 2020-12-11T14:11:42Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37262/

SparkQA · 2020-12-11T14:36:21Z

Test build #132657 has finished for PR 30666 at commit 7e23fea.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2020-12-14T07:44:29Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37352/

SparkQA · 2020-12-14T08:07:47Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37352/

cloud-fan · 2020-12-14T08:31:48Z

GA passed, merging to master/3.1!

### What changes were proposed in this pull request? Addressed comments in PR #30567, including: 1. add test case for SPARK-33647 and SPARK-33142 2. add migration guide 3. add `getRawTempView` and `getRawGlobalTempView` to return the raw view info (i.e. TemporaryViewRelation) 4. other minor code clean ### Why are the changes needed? Code clean and more test cases ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Existing and newly added test cases Closes #30666 from linhongliu-db/SPARK-33142-followup. Lead-authored-by: Linhong Liu <[email protected]> Co-authored-by: Linhong Liu <[email protected]> Signed-off-by: Wenchen Fan <[email protected]> (cherry picked from commit b7c8210) Signed-off-by: Wenchen Fan <[email protected]>

SparkQA · 2020-12-14T11:50:55Z

Test build #132751 has finished for PR 30666 at commit 1cde94e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

github-actions bot added the SQL label Dec 8, 2020

linhongliu-db commented Dec 8, 2020

View reviewed changes

linhongliu-db closed this Dec 8, 2020

linhongliu-db reopened this Dec 8, 2020

HyukjinKwon changed the title ~~[SPARK-33142][SPARK-33647][SQL][FOLLOW-UP] code clean and more test cases~~ [SPARK-33142][SPARK-33647][SQL][FOLLOW-UP] Code clean and more test cases Dec 9, 2020

HyukjinKwon marked this pull request as draft December 9, 2020 01:54

linhongliu-db changed the title ~~[SPARK-33142][SPARK-33647][SQL][FOLLOW-UP] Code clean and more test cases~~ [SPARK-33142][SPARK-33647][SQL][FOLLOW-UP] Add docs and test cases Dec 10, 2020

github-actions bot added the DOCS label Dec 10, 2020

HyukjinKwon marked this pull request as ready for review December 10, 2020 07:42

linhongliu-db force-pushed the SPARK-33142-followup branch from a8a8f84 to 9d6868b Compare December 10, 2020 09:19

cloud-fan reviewed Dec 11, 2020

View reviewed changes

linhongliu-db closed this Dec 11, 2020

linhongliu-db reopened this Dec 11, 2020

linhongliu-db and others added 6 commits December 14, 2020 14:15

SPARK-33142 followup

a3cc0f7

support function

6d27c0d

add migration guide

d72e0f6

add test

8dde629

address comments

9bcf00b

Update SQLViewTestSuite.scala

1cde94e

linhongliu-db force-pushed the SPARK-33142-followup branch from 7e23fea to 1cde94e Compare December 14, 2020 06:20

cloud-fan closed this in b7c8210 Dec 14, 2020

[SPARK-33142][SPARK-33647][SQL][FOLLOW-UP] Add docs and test cases #30666

[SPARK-33142][SPARK-33647][SQL][FOLLOW-UP] Add docs and test cases #30666

Uh oh!

Conversation

linhongliu-db commented Dec 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

SparkQA commented Dec 8, 2020

Uh oh!

SparkQA commented Dec 8, 2020

Uh oh!

SparkQA commented Dec 8, 2020

Uh oh!

linhongliu-db Dec 8, 2020

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Dec 8, 2020

Uh oh!

linhongliu-db commented Dec 8, 2020

Uh oh!

SparkQA commented Dec 8, 2020

Uh oh!

SparkQA commented Dec 8, 2020

Uh oh!

SparkQA commented Dec 8, 2020

Uh oh!

linhongliu-db commented Dec 8, 2020

Uh oh!

SparkQA commented Dec 8, 2020

Uh oh!

SparkQA commented Dec 8, 2020

Uh oh!

SparkQA commented Dec 8, 2020

Uh oh!

linhongliu-db commented Dec 9, 2020

Uh oh!

HyukjinKwon commented Dec 9, 2020

Uh oh!

SparkQA commented Dec 10, 2020

Uh oh!

SparkQA commented Dec 10, 2020

Uh oh!

SparkQA commented Dec 10, 2020

Uh oh!

SparkQA commented Dec 10, 2020

Uh oh!

SparkQA commented Dec 10, 2020

Uh oh!

linhongliu-db commented Dec 10, 2020

Uh oh!

cloud-fan Dec 11, 2020

Choose a reason for hiding this comment

Uh oh!

linhongliu-db Dec 11, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Dec 11, 2020

Uh oh!

SparkQA commented Dec 11, 2020

Uh oh!

SparkQA commented Dec 11, 2020

Uh oh!

cloud-fan commented Dec 11, 2020

Uh oh!

SparkQA commented Dec 11, 2020

Uh oh!

SparkQA commented Dec 11, 2020

Uh oh!

SparkQA commented Dec 11, 2020

Uh oh!

SparkQA commented Dec 11, 2020

Uh oh!

linhongliu-db commented Dec 11, 2020

Uh oh!

SparkQA commented Dec 11, 2020

linhongliu-db commented Dec 8, 2020 •

edited

Loading

linhongliu-db Dec 11, 2020 •

edited

Loading