Skip to content

[SPARK-34996][PYTHON] Port Koalas Series-related unit tests into PySpark #32117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
xinrong-meng wants to merge 10 commits into from
Closed

[SPARK-34996][PYTHON] Port Koalas Series-related unit tests into PySpark #32117

xinrong-meng wants to merge 10 commits into from

Conversation

@xinrong-meng
Copy link
Member

@xinrong-meng xinrong-meng commented Apr 10, 2021 •
edited
Loading

What changes were proposed in this pull request?

Now that we merged the Koalas main code into the PySpark code base (#32036), we should port the Koalas Series related unit tests to PySpark.

Why are the changes needed?

Currently, the pandas-on-Spark modules are not fully tested. We should enable the Series related unit tests first.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Enable Series-related unit tests.

Keyword:SPARK-34849

@xinrong-meng
Copy link
Member Author

xinrong-meng commented Apr 10, 2021

CC @ueshin @HyukjinKwon @itholic

@SparkQA
Copy link

SparkQA commented Apr 10, 2021

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41738/

@SparkQA
Copy link

SparkQA commented Apr 10, 2021

Test build #137159 has finished for PR 32117 at commit b156c28.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@xinrong-meng xinrong-meng changed the title [SPARK-34996][PYTHON] Port Koalas Series-related unit tests into PySpark [WIP][SPARK-34996][PYTHON] Port Koalas Series-related unit tests into PySpark Apr 12, 2021
@SparkQA
Copy link

SparkQA commented Apr 12, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41813/

@SparkQA
Copy link

SparkQA commented Apr 12, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41813/

@SparkQA
Copy link

SparkQA commented Apr 12, 2021

Test build #137233 has finished for PR 32117 at commit f6b424e.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

ueshin
ueshin reviewed Apr 12, 2021
View reviewed changes
@SparkQA
Copy link

SparkQA commented Apr 12, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41828/

@SparkQA
Copy link

SparkQA commented Apr 12, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41828/

ueshin
ueshin approved these changes Apr 13, 2021
View reviewed changes
Copy link
Member

@ueshin ueshin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, pending tests.

itholic
itholic approved these changes Apr 13, 2021
View reviewed changes
Copy link
Contributor

@itholic itholic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM if tests pass

@SparkQA
Copy link

SparkQA commented Apr 13, 2021

Test build #137248 has finished for PR 32117 at commit 9dd9ef7.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Apr 13, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41836/

@SparkQA
Copy link

SparkQA commented Apr 13, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41836/

@xinrong-meng xinrong-meng changed the title [WIP][SPARK-34996][PYTHON] Port Koalas Series-related unit tests into PySpark [SPARK-34996][PYTHON] Port Koalas Series-related unit tests into PySpark Apr 13, 2021
@SparkQA
Copy link

SparkQA commented Apr 13, 2021

Test build #137257 has finished for PR 32117 at commit 0a22b38.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member

HyukjinKwon commented Apr 13, 2021

Merged to master.

@SparkQA
Copy link

SparkQA commented Apr 13, 2021

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41844/

@SparkQA
Copy link

SparkQA commented Apr 13, 2021

Test build #137265 has finished for PR 32117 at commit aa70707.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
  • case class RuleId(id: Int)
  • abstract class TreeNode[BaseType <: TreeNode[BaseType]] extends Product with TreePatternBits
  • trait TreePatternBits

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ueshin ueshin ueshin approved these changes

@itholic itholic itholic approved these changes

Assignees

No one assigned

Labels

BUILD CORE PYTHON

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@xinrong-meng @SparkQA @HyukjinKwon @ueshin @itholic