-
Notifications
You must be signed in to change notification settings - Fork 75
[Compiler plugin] Support more GroupBy shortcuts #1055
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
e970d90
[Compiler plugin] Support GroupBy.count
koperagen 4c76e17
[Compiler plugin] Support groupBy.[first | last | maxBy | minBy].into…
koperagen 7124479
Move ReducedGroupBy.concat
koperagen 981a47c
[Compiler plugin] Support GroupBy.[minOf | maxOf]
koperagen File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
35 changes: 35 additions & 0 deletions
35
...ns/kotlin-dataframe/src/org/jetbrains/kotlinx/dataframe/plugin/impl/api/ReducedGroupBy.kt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| package org.jetbrains.kotlinx.dataframe.plugin.impl.api | ||
|
|
||
| import org.jetbrains.kotlinx.dataframe.plugin.impl.AbstractInterpreter | ||
| import org.jetbrains.kotlinx.dataframe.plugin.impl.AbstractSchemaModificationInterpreter | ||
| import org.jetbrains.kotlinx.dataframe.plugin.impl.Arguments | ||
| import org.jetbrains.kotlinx.dataframe.plugin.impl.PluginDataFrameSchema | ||
| import org.jetbrains.kotlinx.dataframe.plugin.impl.SimpleColumnGroup | ||
| import org.jetbrains.kotlinx.dataframe.plugin.impl.groupBy | ||
| import org.jetbrains.kotlinx.dataframe.plugin.impl.ignore | ||
| import org.jetbrains.kotlinx.dataframe.plugin.impl.makeNullable | ||
|
|
||
| class GroupByReducePredicate : AbstractInterpreter<GroupBy>() { | ||
| val Arguments.receiver by groupBy() | ||
| val Arguments.predicate by ignore() | ||
| override fun Arguments.interpret(): GroupBy { | ||
| return receiver | ||
| } | ||
| } | ||
|
|
||
| class GroupByReduceExpression : AbstractInterpreter<GroupBy>() { | ||
| val Arguments.receiver by groupBy() | ||
| val Arguments.rowExpression by ignore() | ||
| override fun Arguments.interpret(): GroupBy { | ||
| return receiver | ||
| } | ||
| } | ||
|
|
||
| class GroupByReduceInto : AbstractSchemaModificationInterpreter() { | ||
| val Arguments.receiver by groupBy() | ||
| val Arguments.columnName: String by arg() | ||
| override fun Arguments.interpret(): PluginDataFrameSchema { | ||
| val group = makeNullable(SimpleColumnGroup(columnName, receiver.groups.columns())) | ||
| return PluginDataFrameSchema(receiver.keys.columns() + group) | ||
| } | ||
| } |
19 changes: 19 additions & 0 deletions
19
plugins/kotlin-dataframe/src/org/jetbrains/kotlinx/dataframe/plugin/impl/api/count.kt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| package org.jetbrains.kotlinx.dataframe.plugin.impl.api | ||
|
|
||
| import org.jetbrains.kotlinx.dataframe.plugin.impl.AbstractSchemaModificationInterpreter | ||
| import org.jetbrains.kotlinx.dataframe.plugin.impl.Arguments | ||
| import org.jetbrains.kotlinx.dataframe.plugin.impl.PluginDataFrameSchema | ||
| import org.jetbrains.kotlinx.dataframe.plugin.impl.Present | ||
| import org.jetbrains.kotlinx.dataframe.plugin.impl.add | ||
| import org.jetbrains.kotlinx.dataframe.plugin.impl.groupBy | ||
| import org.jetbrains.kotlinx.dataframe.plugin.impl.ignore | ||
|
|
||
| class GroupByCount0 : AbstractSchemaModificationInterpreter() { | ||
| val Arguments.receiver by groupBy() | ||
| val Arguments.resultName: String by arg(defaultValue = Present("count")) | ||
| val Arguments.predicate by ignore() | ||
|
|
||
| override fun Arguments.interpret(): PluginDataFrameSchema { | ||
| return receiver.keys.add(resultName, session.builtinTypes.intType.type, context = this) | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| import org.jetbrains.kotlinx.dataframe.* | ||
| import org.jetbrains.kotlinx.dataframe.annotations.* | ||
| import org.jetbrains.kotlinx.dataframe.api.* | ||
| import org.jetbrains.kotlinx.dataframe.io.* | ||
|
|
||
| fun box(): String { | ||
| val df = dataFrameOf("a")(1, 1, 2, 3, 3).groupBy { a }.count() | ||
koperagen marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| val i: Int = df.count[0] | ||
|
|
||
| val df1 = dataFrameOf("a")(1, 1, 2, 3, 3).groupBy { a }.count { a > 1 } | ||
| val i1: Int = df1.count[0] | ||
|
|
||
| val df2 = dataFrameOf("a")(1, 1, 2, 3, 3).groupBy { a }.count("myCol") { a > 1 } | ||
| val i2: Int = df2.myCol[0] | ||
| return "OK" | ||
| } | ||
22 changes: 22 additions & 0 deletions
22
plugins/kotlin-dataframe/testData/box/groupBy_maxOfMinOf.kt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| import org.jetbrains.kotlinx.dataframe.* | ||
| import org.jetbrains.kotlinx.dataframe.annotations.* | ||
| import org.jetbrains.kotlinx.dataframe.api.* | ||
| import org.jetbrains.kotlinx.dataframe.io.* | ||
|
|
||
| fun box(): String { | ||
| val df = dataFrameOf("a")(1, 1, 2, 3, 3).groupBy { a }.add("id") { index() }.maxOf { 123 } | ||
koperagen marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| val df1 = dataFrameOf("a")(1, 1, 2, 3, 3).groupBy { a }.add("id") { index() }.minOf { 123 } | ||
|
|
||
| val max = df.max[0] | ||
| val min = df1.min[0] | ||
|
|
||
| df.compareSchemas() | ||
| df1.compareSchemas() | ||
|
|
||
| val df2 = dataFrameOf("a")(1, 1, 2, 3, 3).groupBy { a }.add("id") { index() }.maxOf("myMax") { 123 } | ||
| val df3 = dataFrameOf("a")(1, 1, 2, 3, 3).groupBy { a }.add("id") { index() }.minOf("myMin") { 123 } | ||
|
|
||
| df2.myMax | ||
| df3.myMin | ||
| return "OK" | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| import org.jetbrains.kotlinx.dataframe.* | ||
| import org.jetbrains.kotlinx.dataframe.annotations.* | ||
| import org.jetbrains.kotlinx.dataframe.api.* | ||
| import org.jetbrains.kotlinx.dataframe.io.* | ||
|
|
||
| fun box(): String { | ||
| val groupBy = dataFrameOf("a")(1, 1, 2, 3, 3).groupBy { a }.add("id") { index() } | ||
| groupBy.maxBy { id }.into("group").compareSchemas() | ||
| groupBy.maxBy { id }.into("group").compareSchemas() | ||
| groupBy.first { id == 1 }.into("group").compareSchemas() | ||
| groupBy.first().into("group").compareSchemas() | ||
| groupBy.last { id == 1 }.into("group").compareSchemas() | ||
| groupBy.last().into("group").compareSchemas() | ||
| groupBy.minBy { id == 1 }.into("group").compareSchemas() | ||
| return "OK" | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this used anywhere? If I look for usages I cannot find any
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe we just need a junit test, we can do it later, just make sure it's in our excel sheet in the right place after moving it here :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think it was overlooked because it was not in a good place! I will likely revisit it, because sometimes it can produce empty dataframe, need to check that with compiler plugin we'd have a reasonable exception in this case