Skip to content

Commit 1e36219

Browse files
committed
Set deadline on vsock ops after exit.
This is a non-ideal fix to an issue where a task exit event could be received before all stdio has been flushed across vsock connections. Vsocks were closed when task exit occurs, so this resulted in stdio data being dropped. The fix here replaces the closing of the vsock connection with setting of a deadline on the socket. This results gives a 5 second window for any last stdio data to be flushed before closing the io pipeline down. The better fix to this issue is: 1. Don't rely on task exit events at all; instead use Wait() from the task API to monitor task exits. 2. Update Agent and Runtime to flush io before returning Wait() responses. The better fix described above will be tackled as part of upcoming Exec support, which will benefit from that change anyways. The fix here should patch the issue in the meantime. The fix was "tested" by just running e2e tests many times and verifying the error didn't occur again. However, the original issue was only encountered once so it's not possible to be sure it's gone as it may just be extremely rare. Signed-off-by: Erik Sipsma <[email protected]>
1 parent 31810cd commit 1e36219

File tree

1 file changed

+11
-18
lines changed

1 file changed

+11
-18
lines changed

internal/vm/task.go

Lines changed: 11 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ import (
1919
"net"
2020
"sync"
2121
"syscall"
22+
"time"
2223

2324
"github.com/containerd/containerd/cio"
2425
taskAPI "github.com/containerd/containerd/runtime/v2/task"
@@ -301,33 +302,25 @@ func (task *Task) proxyIO(initializeCtx context.Context, fifoPath string, port u
301302
// the FIFO and vsock have been opened, so go off proxying between them asynchronously
302303
// until Cancel is called.
303304
go func() {
305+
defer sock.Close()
306+
defer fifoFile.Close()
307+
304308
buf := make([]byte, internal.DefaultBufferSize)
305309

306310
logger.Debug("begin copying io")
307311

312+
go func() {
313+
<-task.doneCh
314+
// the task has exited, give any last data a 5 second window to be flushed
315+
// across vsock before closing down
316+
sock.SetDeadline(time.Now().Add(5 * time.Second))
317+
}()
318+
308319
switch ioDirection {
309320
case FIFOtoVSock:
310-
go func() {
311-
<-task.doneCh
312-
// If we are copying from FIFO to vsock, be sure to first close just the FIFO,
313-
// allowing any buffered data to be flushed into the vsock first.
314-
fifoFile.Close()
315-
}()
316-
317321
_, err = io.CopyBuffer(sock, fifoFile, buf)
318-
sock.Close()
319-
320322
case VSockToFIFO:
321-
go func() {
322-
<-task.doneCh
323-
// If we are copying from vsock to FIFO, be sure to first close just the vsock,
324-
// allowing any buffered data to be flushed into the FIFO first.
325-
sock.Close()
326-
}()
327-
328323
_, err = io.CopyBuffer(fifoFile, sock, buf)
329-
fifoFile.Close()
330-
331324
default:
332325
logger.Fatalf("undefined io direction") // should not be possible, direction is a bool
333326
}

0 commit comments

Comments
 (0)