x/playground: use gVisor on linux/amd64 binaries instead of NaCl for sandboxing

[andybons]

Native Client deprecation has been announced in favor of Web Assembly. It's unclear what that means in terms of NaCl's future development and support.

I'd like us to investigate using gVisor as our sandbox mechanism for the playground. This is not a statement that we should definitively switch over to gVisor.

https://github.com/google/gvisor

@ysmolsky

[andybons]

This has come up again as attempting to upgrade the playground to 1.11 results in the following breakage during the NaCl time patching phase:

 ---> ed8ffc3e1836
Step 20/63 : RUN patch -p1 -d /usr/local/go </usr/local/playground/strict-time.patch
 ---> Running in 1f97414872b7
patching file src/runtime/os_nacl.go
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file src/runtime/os_nacl.go.rej
patching file src/runtime/sys_nacl_amd64p32.s
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file src/runtime/sys_nacl_amd64p32.s.rej
The command '/bin/sh -c patch -p1 -d /usr/local/go </usr/local/playground/strict-time.patch' returned a non-zero code: 1

/cc @bradfitz

[bradfitz]

That's @bcmills's patch. The old patch (enable-fake-time.patch) was designed to basically never bit rot but the new patch seems to have more context that makes it fragile. We should really upstream this and have it behind a bool or build tag so it's less likely to rot.

[bradfitz]

I've been playing around with this. I'm deciding between:

1. continuing to use App Engine Flex and launching gvisor's runsc "directly" (if one could consider the OCI interface direct). Unfortunately runsc still requires root today (for now: runsc doesn't work with rootless podman google/gvisor#311) and App Engine Flex doesn't give you root at serving time (only for debugging). So doing this would require modifying runsc to not require root.

2. migrating off App Engine Flex and just using Docker with vanilla runsc. We'd need to change how we serve a bit. Probably an HTTP load balancer to a GCE VM instance group (perhaps running Container-Optimized OS, which already runs docker, and supports declarative yaml configs of which containers to run, including I believe privileged ones)

I think I'm leaning towards (2).

As part of this, we'd migrate to (2) well before the Go 1.14 release (which drops nacl), so the release process doesn't involve changing playground machinery during a release. We'd still support Go 1.13 nacl mode with the new infrastructure, and then the Go 1.14 release would just be changing a single line in its Dockerfile.

/cc @dmitshur @andybons @toothrot @prattmic

[toothrot]

@bradfitz Have you investigated using Cloud Run? The documentation seems to suggest that the runtime user is root: https://cloud.google.com/run/docs/tips#container-security

[bradfitz]

@toothrot, Cloud Run is already under gVisor itself. (and it doesn't let us tweak its sandbox config to be more restrictive, as we'd need). Cloud Run gives you root, but fake-ish constrained root. We definitely couldn't use gVisor's KVM mode ("platform") under it. We could probably just the ptrace mode (ptrace syscall is at least partially supported: https://gvisor.dev/docs/user_guide/compatibility/linux/amd64/). But once runsc went to do the other root-requiring stuff (setting up namespaces, etc), I suspect we'd trip over unimplemented system calls. e.g. I see on that system call list:

Mount namespace (CLONE_NEWNS) not supported. Options CLONE_PARENT, CLONE_SYSVSEM not supported.

So basically, to use Cloud Run we'd need to do the same work hacking up runsc to run rootless that we'd have to do to run it on its current App Engine Flex environment.

That said, we should move many of our things to Cloud Run regardless, but I don't think it'd help us here.

@prattmic can correct me if indeed the ptrace mode of runsc would work unmodified under gvisor. That would be a happy surprise.

[bradfitz]

@prattmic confirms that nested gVisor doesn't work yet.