Skip to content

Commit 2af5237

Browse files
jiangxb1987cloud-fan
authored andcommitted
[SPARK-29918][SQL][FOLLOWUP][TEST] Fix arrayOffset in RecordBinaryComparatorSuite
### What changes were proposed in this pull request? As mentioned in #26548 (review), some test cases in `RecordBinaryComparatorSuite` use a fixed arrayOffset when writing to long arrays, this could lead to weird stuff including crashing with a SIGSEGV. This PR fix the problem by computing the arrayOffset based on `Platform.LONG_ARRAY_OFFSET`. ### How was this patch tested? Tested locally. Previously, when we try to add `System.gc()` between write into long array and compare by RecordBinaryComparator, there is a chance to hit JVM crash with SIGSEGV like: ``` # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00007efc66970bcb, pid=11831, tid=0x00007efc0f9f9700 # # JRE version: OpenJDK Runtime Environment (8.0_222-b10) (build 1.8.0_222-8u222-b10-1ubuntu1~16.04.1-b10) # Java VM: OpenJDK 64-Bit Server VM (25.222-b10 mixed mode linux-amd64 compressed oops) # Problematic frame: # V [libjvm.so+0x5fbbcb] # # Core dump written. Default location: /home/jenkins/workspace/sql/core/core or core.11831 # # An error report file with more information is saved as: # /home/jenkins/workspace/sql/core/hs_err_pid11831.log # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp # ``` After the fix those test cases didn't crash the JVM anymore. Closes #26939 from jiangxb1987/rbc. Authored-by: Xingbo Jiang <[email protected]> Signed-off-by: Wenchen Fan <[email protected]>
1 parent ab8eb86 commit 2af5237

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

sql/core/src/test/java/test/org/apache/spark/sql/execution/sort/RecordBinaryComparatorSuite.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -325,7 +325,7 @@ public void testBinaryComparatorWhenOnlyTheLastColumnDiffers() throws Exception
325325

326326
@Test
327327
public void testCompareLongsAsLittleEndian() {
328-
long arrayOffset = 12;
328+
long arrayOffset = Platform.LONG_ARRAY_OFFSET + 4;
329329

330330
long[] arr1 = new long[2];
331331
Platform.putLong(arr1, arrayOffset, 0x0100000000000000L);
@@ -347,7 +347,7 @@ public void testCompareLongsAsLittleEndian() {
347347

348348
@Test
349349
public void testCompareLongsAsUnsigned() {
350-
long arrayOffset = 12;
350+
long arrayOffset = Platform.LONG_ARRAY_OFFSET + 4;
351351

352352
long[] arr1 = new long[2];
353353
Platform.putLong(arr1, arrayOffset + 4, 0xa000000000000000L);

0 commit comments

Comments
 (0)