Skip to content

[SPARK-35034][PYTHON] Port Koalas miscellaneous unit tests into PySpark #32152

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
xinrong-meng wants to merge 2 commits into from
Closed

[SPARK-35034][PYTHON] Port Koalas miscellaneous unit tests into PySpark #32152

xinrong-meng wants to merge 2 commits into from

Conversation

@xinrong-meng
Copy link
Member

@xinrong-meng xinrong-meng commented Apr 13, 2021

What changes were proposed in this pull request?

Now that we merged the Koalas main code into the PySpark code base (#32036), we should port the Koalas miscellaneous unit tests to PySpark.

Why are the changes needed?

Currently, the pandas-on-Spark modules are not tested fully. We should enable miscellaneous unit tests.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Enable miscellaneous unit tests.

@xinrong-meng xinrong-meng changed the title [WIP]SPARK-35034][PYTHON] Port Koalas miscellaneous unit tests into PySpark [SPARK-35034][PYTHON] Port Koalas miscellaneous unit tests into PySpark Apr 13, 2021
ueshin
ueshin approved these changes Apr 13, 2021
View reviewed changes
Copy link
Member

@ueshin ueshin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, pending tests.

@SparkQA
Copy link

SparkQA commented Apr 13, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41884/

@SparkQA
Copy link

SparkQA commented Apr 13, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41884/

@SparkQA
Copy link

SparkQA commented Apr 13, 2021

Test build #137304 has finished for PR 32152 at commit 6b2d157.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@ueshin
Copy link
Member

ueshin commented Apr 13, 2021

Jenkins, retest this please.

@SparkQA
Copy link

SparkQA commented Apr 14, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41890/

@SparkQA
Copy link

SparkQA commented Apr 14, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41890/

itholic
itholic approved these changes Apr 14, 2021
View reviewed changes
Copy link
Contributor

@itholic itholic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM if test passes

@SparkQA
Copy link

SparkQA commented Apr 14, 2021

Test build #137310 has finished for PR 32152 at commit 6b2d157.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

HyukjinKwon
HyukjinKwon approved these changes Apr 14, 2021
View reviewed changes
Copy link
Member

@HyukjinKwon HyukjinKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. can we resolve the conflicts?

@xinrong-meng
Copy link
Member Author

xinrong-meng commented Apr 14, 2021

@HyukjinKwon Certainly, the conflict has been resolved!

@SparkQA
Copy link

SparkQA commented Apr 14, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41937/

@SparkQA
Copy link

SparkQA commented Apr 14, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41937/

@github-actions
Copy link

github-actions bot commented Apr 14, 2021

Test build #748860054 for PR 32152 at commit 48a17d9.

@SparkQA
Copy link

SparkQA commented Apr 14, 2021

Test build #137358 has finished for PR 32152 at commit 48a17d9.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
  • trait AnalysisOnlyCommand extends Command
  • implicit class MetadataColumnHelper(attr: Attribute)

@xinrong-meng xinrong-meng mentioned this pull request Apr 14, 2021
Closed
@HyukjinKwon
Copy link
Member

HyukjinKwon commented Apr 15, 2021

Merged to master.

This was referenced Apr 16, 2021
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ueshin ueshin ueshin approved these changes

@HyukjinKwon HyukjinKwon HyukjinKwon approved these changes

@itholic itholic itholic approved these changes

Assignees

No one assigned

Labels

BUILD CORE PYTHON

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@xinrong-meng @SparkQA @ueshin @HyukjinKwon @itholic