Speed up image pulls using IPFS

[csweichel]

Is your feature request related to a problem? Please describe

Today we pull all images from a central registry. While this is the standard way, it can become a performance bottleneck when starting many different, possibly large images. We already optimise for layer re-use, yet image pull remains one of most time-consuming tasks during workspace startup.

One way we could try and improve the situation is by using IPFS to share layers across nodes within a cluster. This approach

• is gaining tracking in the community
• is compatible with eStargz
• uses open standards and technology, i.e. we don't have to implement it all.

We need to solve two paths:

1. pushing layers into IPFS

• nerdctl can push to IPFS. We could make the image-builder push to our local IPFS cluster once the image-builder lives on the workspace side. Pro: we just have to call a CLI as part of the image build. Downside: we need to give the image build workspaces access to our local cluster
• registry facade can push layers on demand to IPFS (client libraries exist). Pro: straight forward caching implementation whereby IPFS becomes a mere optimisation. Downside: we have to implement this

2. pulling layers from IPFS

• the stargz snapshotter can pull from IPFS directly - registry-facade could add ipfs:// URLs to the manifest.
• we could make registry-facade try and pull from IPFS, thereby providing a standard-compliant path towards the container runtime

[kylos101]

@Furisto after you've had a chance to wind down other open PRs that you have in-flight, can you take a peek at this one? Ideally, we'd use registry-facade to push/pull into/from IPFS.

@aledbf can we use the current image in workspace-preview, or should we use one that you're working on here?

[aledbf]

@kylos101 that PR already contains most of the changes. I'm almost ready to test this change in workspace-preview https://github.com/gitpod-io/workspace-preview/commit/d59fcbd38ca7b3e15a226c9ac4b0b66f18c8bcf1

With that layout, a node will look like this:

/                        -> 300GB (pd-ssd)
/var/gitpod/workspaces   -> 750GB (RAID0 local SSD)
/var/gitpod/ipfs         -> 1TB   (RAID0 local SSD)
/var/lib/containerd      -> 700GB (RAID0 pd-ssd)

[kylos101]

Okay, cool. @Furisto needs some time to wrap a couple of his in-flight PRs, so timing wise that should work out well.