Support arbitrary Compute Engine VMs for OSS

[fkorotkov]

Right now docker_builder is essentially a VM that we maintain. But some folks requested features that can be covered by allowing any custom VMs from the users with any number of CPUs/memory. Many issues can be resolved by this, for example:

1. Run a custom kernel by @anarazel.
2. OpenBSD Support OpenBSD Support #311
3. DragonFlyBSD Support DragonFlyBSD Support #101
4. Support selectable number of CPUs for docker_builder Support selectable number of CPUs for docker_builder #741
5. Support nested virtualization. Is QMEU enabled for Linux runners for docker buildx? #793

Here is an analogue of docker_builder:

task:
  compute_engine_instance:
    image_project: cirrus-images
    image: family/docker-builder
    platform: linux
    cpu: 4
    memory: 16G
    nested_virtualization: false

This is basically a stripped version of gce_instance but for OSS and which can use any public compute engine VM.

[fkorotkov]

@anarazel, just wanted to check if you'll be up to testing this feature on PostgresQL once it is available.

[anarazel]

@anarazel, just wanted to check if you'll be up to testing this feature on PostgresQL once it is available.

Yes.

[fkorotkov]

It took a little longer then anticipated due to an ongoing refactoring but compute_engine_instance is now available. Please take a look and review #798 which has more details.

[anarazel]

It took a little longer then anticipated due to an ongoing refactoring but compute_engine_instance is now available. Please take a look and review #798 which has more details.

Seems to work well! Still need to script building an image with a bleeding edge kernel on a regular basis - but that's my issue not yours... Thanks a lot!