[SPARK-33861][SQL][FOLLOWUP] Simplify conditional in predicate should consider deterministic #31067
Conversation
| CaseWhen(Seq((UnresolvedAttribute("i") > Rand(0), FalseLiteral))) | ||
| val expectedCond = And(UnresolvedAttribute("i") > Rand(0), FalseLiteral) | ||
| // nondeterministic expressions are only allowed in Project, Filter, Aggregate or Window, | ||
| testFilter(originalCond, expectedCond = FalseLiteral) |
There was a problem hiding this comment.
expectedCond = FalseLiteral cause by
| Or(Not(cond), trueValue) | ||
| case CaseWhen(Seq((_, FalseLiteral)), Some(FalseLiteral) | None) => | ||
| FalseLiteral | ||
| case CaseWhen(Seq((cond, FalseLiteral)), Some(elseValue)) => |
There was a problem hiding this comment.
This case has handled by
| case CaseWhen(Seq((cond, TrueLiteral)), Some(FalseLiteral) | None) => | ||
| cond |
There was a problem hiding this comment.
This case has handled by
| And(cond, trueValue) | ||
| case CaseWhen(Seq((cond, trueValue)), Some(TrueLiteral)) => | ||
| Or(Not(cond), trueValue) | ||
| case CaseWhen(Seq((_, FalseLiteral)), Some(FalseLiteral) | None) => |
There was a problem hiding this comment.
can we update the classdoc of this rule?
|
Kubernetes integration test starting |
|
Kubernetes integration test status success |
|
Kubernetes integration test starting |
|
Test build #133749 has finished for PR 31067 at commit
|
|
Kubernetes integration test status success |
|
Test build #133740 has finished for PR 31067 at commit
|
HyukjinKwon
left a comment
HyukjinKwon
left a comment
There was a problem hiding this comment.
LGTM thanks for handling this
|
Merged to master. |
4 participants
What changes were proposed in this pull request?
This pr address #30865 (review) to fix simplify conditional in predicate should consider deterministic.
Why are the changes needed?
Fix bug.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Unit test.