[SPARK-28390][SQL][PYTHON][TESTS] [FOLLOW-UP] Update the TODO with actual blocking JIRA IDs #25415
Conversation
|
@HyukjinKwon |
| -- | ||
| -- This test file was converted from inputs/pgSQL/select_having.sql | ||
| -- TODO: We should add UDFs in GROUP BY clause when [SPARK-28445] is resolved. | ||
| -- TODO: We should add UDFs in GROUP BY clause when [SPARK-28386] and [SPARK-26741] is resolved. |
There was a problem hiding this comment.
SPARK-28445 was wrong from the beginning, @shivusondur ?
There was a problem hiding this comment.
@dongjoon-hyun
After resolving the SPARK-28445 also, test were failing and found [SPARK-28386] and [SPARK-26741] are blocking it.
for furher details follow #25215 (comment)
| -- | ||
| -- This test file was converted from inputs/pgSQL/select_having.sql | ||
| -- TODO: We should add UDFs in GROUP BY clause when [SPARK-28445] is resolved. | ||
| -- TODO: We should add UDFs in GROUP BY clause when [SPARK-28386] and [SPARK-26741] is resolved. |
There was a problem hiding this comment.
I am a bit lost about this or I forget something.
Can't we add UDF in group-by clause (resolved in SPARK-28445)?
There was a problem hiding this comment.
@HyukjinKwon
From this
#25215 (comment)
I thought I need to update todo with blocking jira numbers
There was a problem hiding this comment.
Ah, right. I forgot. Can we enable all other tests with UDF in group-by and comment out the test?
-- !query 11
SELECT udf(b), udf(c) FROM test_having
GROUP BY udf(b), udf(c) HAVING udf(count(*)) = 1 ORDER BY udf(b), udf(c)
-- !query 11 schema
struct<>
-- !query 11 output
org.apache.spark.sql.AnalysisException
cannot resolve 'b' given input columns: [CAST(udf(cast(b as string)) AS INT), CAST(udf(cast(c as string)) AS STRING)]; line 2 pos 63
I guess we can still add some more tests?
|
BTW, @shivusondur . The follow-up PR had better have its own PR title. The current one seems to be copied from the original PR. |
shivusondur
changed the title
|
Usually, it's not worth to change it immediately because it will be resolved in 3.0.0 timeframe. But, I'm okay to keep it up-to-date, too. |
|
Yea that's fine to update comments. But @shivusondur can you confirm if you are unable to fix any test or some tests to have use in GROUP BY clause due to both JIRAs? If you can, let's add some tests and only comment out the other tests not working by both JIRAs. |
|
Test build #4829 has finished for PR 25415 at commit
|
|
|
retest this please |
|
Test build #109304 has finished for PR 25415 at commit
|
|
Merged to master. |
4 participants
What changes were proposed in this pull request?
only todo message updated. Need to add udf() for GroupBy Tests, after resolving following jira
[SPARK-28386] and [SPARK-26741]
How was this patch tested?
NA, only TODO message updated.