Skip to content

Commit ab4e83a

Browse files
committed
[SPARK-26963][MLLIB] SizeEstimator can't make some JDK fields accessible in Java 9+
## What changes were proposed in this pull request? Don't use inaccessible fields in SizeEstimator, which comes up in Java 9+ ## How was this patch tested? Manually ran tests with Java 11; it causes these tests that failed before to pass. This ought to pass on Java 8 as there's effectively no change for Java 8. Closes #23866 from srowen/SPARK-26963. Authored-by: Sean Owen <[email protected]> Signed-off-by: Sean Owen <[email protected]>
1 parent ce3a157 commit ab4e83a

File tree

1 file changed

+14
-2
lines changed

1 file changed

+14
-2
lines changed

core/src/main/scala/org/apache/spark/util/SizeEstimator.scala

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -334,9 +334,21 @@ object SizeEstimator extends Logging {
334334
if (fieldClass.isPrimitive) {
335335
sizeCount(primitiveSize(fieldClass)) += 1
336336
} else {
337-
field.setAccessible(true) // Enable future get()'s on this field
337+
// Note: in Java 9+ this would be better with trySetAccessible and canAccess
338+
try {
339+
field.setAccessible(true) // Enable future get()'s on this field
340+
pointerFields = field :: pointerFields
341+
} catch {
342+
// If the field isn't accessible, we can still record the pointer size
343+
// but can't know more about the field, so ignore it
344+
case _: SecurityException =>
345+
// do nothing
346+
// Java 9+ can throw InaccessibleObjectException but the class is Java 9+-only
347+
case re: RuntimeException
348+
if re.getClass.getSimpleName == "InaccessibleObjectException" =>
349+
// do nothing
350+
}
338351
sizeCount(pointerSize) += 1
339-
pointerFields = field :: pointerFields
340352
}
341353
}
342354
}

0 commit comments

Comments
 (0)