[SPARK-37018][SQL] Spark SQL should support create function with Aggregator

[beliefer]

What changes were proposed in this pull request?

Spark SQL not supports to create function of Aggregator yet and deprecated UserDefinedAggregateFunction.
If we want remove UserDefinedAggregateFunction, Spark SQL should provide a new option.
Note: This PR replaces #34303.

Why are the changes needed?

We need to provide a new way to create user defined aggregate function so as remove UserDefinedAggregateFunction in future.

Does this PR introduce any user-facing change?

Yes. Users will create user defined aggregate function by implement Aggregator.

How was this patch tested?

New tests.

[cloud-fan]

Is there any reference for the code above? The code looks obscure

[cloud-fan]

There are a few more stuff I'd like to test:

1. What if the class has type parameters? e.g. class TypedCount[IN](val f: IN => Any) extends Aggregator[IN, Long, Long]. What happens if we register it as a SQL UDAF?
2. Let's test negative cases, e.g., wrong num of parameters, wrong parameter types.

[beliefer]

1. the type parameter will be IN, we need find another way.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48969/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48969/

[SparkQA]

Test build #144497 has finished for PR 34352 at commit 89fc1c0.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49828/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49828/

[SparkQA]

Test build #145357 has finished for PR 34352 at commit 2348081.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49856/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49856/

[SparkQA]

Test build #145387 has finished for PR 34352 at commit a60af8b.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49860/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49860/

[SparkQA]

Test build #145383 has finished for PR 34352 at commit 9bc0597.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49866/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49866/

[SparkQA]

Test build #145393 has finished for PR 34352 at commit 61d9614.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50989/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50989/

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/51004/

[SparkQA]

Test build #146513 has finished for PR 34352 at commit e742552.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/51004/

[SparkQA]

Test build #146528 has finished for PR 34352 at commit f9a50e2.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[github-actions]

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!

[IceMimosa]

@beliefer Hi, is there any progress on this?