std.os.ChildProcess should use clone instead of fork on linux

[andrewrk]

vfork pauses the parent process until the child process either calls execve or exit. This prevents Copy On Write pages, which makes spawning child processes avoid double the parent resident memory requirements when overcommit is off.

[andrewrk]

Interesting notes from the man page:

NOTES
       Some  consider the semantics of vfork() to be an architectural blemish,
       and the 4.2BSD man page stated: "This system call  will  be  eliminated
       when  proper  system  sharing mechanisms are implemented.  Users should
       not depend on the memory sharing semantics of vfork() as  it  will,  in
       that case, be made synonymous to fork(2)."  However, even though modern
       memory management hardware has  decreased  the  performance  difference
       between  fork(2)  and  vfork(), there are various reasons why Linux and
       other systems have retained vfork():

       *  Some performance-critical applications require the small performance
          advantage conferred by vfork().

       *  vfork()  can be implemented on systems that lack a memory-management
          unit (MMU), but  fork(2)  can't  be  implemented  on  such  systems.
          (POSIX.1-2008 removed vfork() from the standard; the POSIX rationale
          for the posix_spawn(3) function notes that that function, which pro‐
          vides functionality equivalent to fork(2)+exec(3), is designed to be
          implementable on systems that lack an MMU.)

       *  On systems where memory is constrained, vfork() avoids the  need  to
          temporarily commit memory (see the description of /proc/sys/vm/over‐
          commit_memory in proc(5)) in order to execute a new program.   (This
          can  be especially beneficial where a large parent process wishes to
          execute a small helper program in a child  process.)   By  contrast,
          using  fork(2) in this scenario requires either committing an amount
          of memory equal to the size of the parent process (if  strict  over‐
          committing  is in force) or overcommitting memory with the risk that
          a process is terminated by the out-of-memory (OOM) killer.

[bnoordhuis]

I don't think that can work. With vfork() you're only allowed to call execve() or _exit(). The setup that spawnPosix() does, like changing uid/gid and the cwd, is incompatible with vfork() - it would be visible to other threads in the parent process.

[andrewrk]

My understanding is that changes to memory would affect the parent process, but the actual pid is a new child pid. So things like current directory, uid/gid, would be independent from the parent process. I'm doing some experiments now. Currently I'm getting a confusing segfault with the following code which I am troubleshooting:

const std = @import("std");
const os = std.os;
const linux = os.linux;

pub fn main() void {
    var x: i32 = 1234;
    const rc = linux.vfork();
    if (rc == 0) {
        // we are the child
        x += 1;
        linux.exit(0);
    }
    std.debug.warn("x={}\n", x);
}

$ ./test
Segmentation fault

gdb has no stack trace, and the program prints 1234 with valgrind.

[andrewrk]

Figured it out. vfork is a function, and when the child does a return it clobber's the parent's stack. I fixed it like this:

/// This must be inline, and inline call the syscall function, because if the
/// child does a return it will clobber the parent's stack.
pub inline fn vfork() usize {
    return @inlineCall(syscall0, SYS_vfork);
}

However it's still problematic, because the example code now does this in debug builds:

x=1235

But does this in release builds:

x=1234

This is because the optimizer sees x+=1; unreachable; and then concludes that if you don't hit the if statement, the x += 1 never could have happened. The vfork breaks the compiler's understanding of control flow.

There's a better way to do this, which is to use clone followed by execve. You can see that this is how musl implements posix_spawn.

[bnoordhuis]

Sorry, I thought posix_setregid() and posix_setreuid() did the "signal other threads" dance that glibc and musl do. I see now that they only update the calling thread.

changeCurDir() and posixExecve() on the other hand allocate memory. That will be visible to other threads because the fork shares the address space with the parent.

Apropos signals, signal handlers are one reason I never used vfork() in libuv. Signals delivered to the fork will be observable in the parent when the signal handler writes to memory. And signal handlers exist for their side effects so that's probably an all too common pitfall.

[andrewrk]

Agreed, this issue is no longer to use vfork. Now it's to use clone() instead of fork() on Linux. This is how musl implements posix_spawn. That API doesn't have a way to change directory or setgid/setuid, but that's not a linux limitation; that's an API limitation.

With clone(), the memory allocations will be no problem. The parent can preallocate the memory before cloning, and then free once the child does exit or execve.

[daurnimator]

Why wouldn't you just use posix_spawn directly?