Skip to content

Commit a4d40e8

Browse files
committed
[SPARK-45100][SQL][3.3] Fix an internal error from reflect()on NULL class and method
### What changes were proposed in this pull request? In the PR, I propose to check that the `class` and `method` arguments are not a NULL in `CallMethodViaReflection`. And if they are, throw an `AnalysisException` with new error class `DATATYPE_MISMATCH.UNEXPECTED_NULL`. This is a backport of #42849. ### Why are the changes needed? To fix the issue demonstrated by the example: ```sql $ spark-sql (default)> select reflect('java.util.UUID', CAST(NULL AS STRING)); [INTERNAL_ERROR] The Spark SQL phase analysis failed with an internal error. You hit a bug in Spark or the Spark plugins you use. Please, report this bug to the corresponding communities or vendors, and provide the full stack trace. ``` ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? By running new test: ``` $ build/sbt "test:testOnly *.MiscFunctionsSuite" ``` ### Was this patch authored or co-authored using generative AI tooling? No. Authored-by: Max Gekk <max.gekkgmail.com> (cherry picked from commit fd424ca) Closes #42856 from MaxGekk/fix-internal-error-in-reflect-3.3. Authored-by: Max Gekk <[email protected]> Signed-off-by: Max Gekk <[email protected]>
1 parent 5250ed6 commit a4d40e8

File tree

2 files changed

+10
-0
lines changed

2 files changed

+10
-0
lines changed

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/CallMethodViaReflection.scala

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,8 @@ case class CallMethodViaReflection(children: Seq[Expression])
6565
} else if (!children.take(2).forall(e => e.dataType == StringType && e.foldable)) {
6666
// The first two arguments must be string type.
6767
TypeCheckFailure("first two arguments should be string literals")
68+
} else if (children.take(2).exists(_.eval() == null)) {
69+
TypeCheckFailure("first two arguments must be non-NULL")
6870
} else if (!classExists) {
6971
TypeCheckFailure(s"class $className not found")
7072
} else if (children.slice(2, children.length)

sql/core/src/test/scala/org/apache/spark/sql/MiscFunctionsSuite.scala

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,14 @@ class MiscFunctionsSuite extends QueryTest with SharedSparkSession {
3434
s"reflect('$className', 'method1', a, b)",
3535
s"java_method('$className', 'method1', a, b)"),
3636
Row("m1one", "m1one"))
37+
val e1 = intercept[AnalysisException] {
38+
df.selectExpr("reflect(cast(null as string), 'fromString', a)")
39+
}
40+
assert(e1.getMessage.contains("first two arguments must be non-NULL"))
41+
val e2 = intercept[AnalysisException] {
42+
df.selectExpr("reflect('java.util.UUID', cast(null as string), a)")
43+
}
44+
assert(e2.getMessage.contains("first two arguments must be non-NULL"))
3745
}
3846

3947
test("version") {

0 commit comments

Comments
 (0)