Commit 9f9bf13
[SPARK-28160][CORE] Fix a bug that callback function may hang when unchecked exception missed
This is very like #23590 .
`ByteBuffer.allocate` may throw `OutOfMemoryError` when the response is large but no enough memory is available. However, when this happens, `TransportClient.sendRpcSync` will just hang forever if the timeout set to unlimited.
This PR catches `Throwable` and uses the error to complete `SettableFuture`.
I tested in my IDE by setting the value of size to -1 to verify the result. Without this patch, it won't be finished until timeout (May hang forever if timeout set to MAX_INT), or the expected `IllegalArgumentException` will be caught.
```java
Override
public void onSuccess(ByteBuffer response) {
try {
int size = response.remaining();
ByteBuffer copy = ByteBuffer.allocate(size); // set size to -1 in runtime when debug
copy.put(response);
// flip "copy" to make it readable
copy.flip();
result.set(copy);
} catch (Throwable t) {
result.setException(t);
}
}
```
Closes #24964 from LantaoJin/SPARK-28160.
Lead-authored-by: LantaoJin <[email protected]>
Co-authored-by: lajin <[email protected]>
Signed-off-by: Sean Owen <[email protected]>
(cherry picked from commit 0e42100)
Signed-off-by: Sean Owen <[email protected]>1 parent b477194 commit 9f9bf13
File tree
1 file changed
+10
-5
lines changed- common/network-common/src/main/java/org/apache/spark/network/client
1 file changed
+10
-5
lines changedLines changed: 10 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
237 | 237 | | |
238 | 238 | | |
239 | 239 | | |
240 | | - | |
241 | | - | |
242 | | - | |
243 | | - | |
244 | | - | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
245 | 250 | | |
246 | 251 | | |
247 | 252 | | |
| |||
0 commit comments