[Core] Support ARM architecture

[Michaelvll]

Mitigates #4793

Fixes #4601

This enables the SkyPilot clusters to run on arm architecture with the default image.

ARM support is becoming more important as the NV's GH200, GB200 (offered by Lambda clouds, GCP) come with ARM CPUs by default, but our docker image does not support ARM CPU well.

Original k8s gpu image does not support ARM architecture

sky launch --cloud lambda --gpus gh200 nvidia-smi --image-id docker:us-docker.pkg.dev/sky-dev-465/skypilotk8s/skypilot-gpu:latest -c test-lambda-arm-origin

Error:

latest: Pulling from sky-dev-465/skypilotk8s/skypilot-gpu
no matching manifest for linux/arm64/v8 in the manifest list entries

⨯ Failed to set up SkyPilot runtime on cluster.  View logs: sky api logs -l sky-2025-02-27-22-36-28-562705/provision.log
sky.exceptions.CommandError: Command docker pull us-docker.pkg.dev/sky-dev-465/skypilotk8s/skypilot-gpu:latest failed with return code 1.
Failed to run docker setup commands.

TODO (future PRs):

• Make SkyPilot runtime dependency installation compatible with ARM
• Make bucket dependency compatible with ARM
• Kubernetes GPU image does not have base env activate by default, while the CPU image does
• Make sure the API server helm chart image is arm compatible
• Make sure sky local up --ips works for remote ARM machines
• Create SkyPilot custom ARM image on different clouds, e.g. sky launch -t c6g.large works if we specify --image-id with a ARM based deep learning image, but does not work with our default image

TODO before merging:

• make the new k8s image to be the latest tag.

Tested (run the relevant ones):

• Code formatting: bash format.sh
• Any manual or new tests for this PR (please specify below)
  • cd sky/clouds/service_catalogs/images/; ./skypilot-k8s-image.sh -p -g
  • cd sky/clouds/service_catalogs/images/; ./skypilot-k8s-image.sh -p
  • sky launch --cloud kubernetes --cpus 1 --image-id docker:us-docker.pkg.dev/sky-dev-465/skypilotk8s/skypilot-gpu:20250227 echo hi
  • sky autostop sky-b694-ubuntu --down -i 0
  • sky launch --cloud kubernetes --cpus 1 --image-id docker:us-docker.pkg.dev/sky-dev-465/skypilotk8s/skypilot:20250227 echo hi
  • sky autostop sky-b694-ubuntu --down -i 0
  • docker build . on Mac and AMD64 linux
  • sky launch --cloud lambda --gpus Gh200 --image-id docker:us-docker.pkg.dev/sky-dev-465/skypilotk8s/skypilot-gpu:20250227 nvidia-smi -c test-lambda-arm, ssh into the machine and uname -m shows aarch64, i.e. arm architecture
  • Launch on ARM arch:
    • sky launch --gpus Gh200 --cloud lambda examples/using_file_mounts.yaml -c test-fm-arm --down
    • Deploy local up on the machine: sky local up --ips
    • Deploy API server on the kubernetes cluster deployed above
    • sky launch --gpus gh200 --cloud kubernetes nvidia-smi
• All smoke tests: pytest tests/test_smoke.py
  • pytest tests/test_smoke.py --kubernetes with the new images
• Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
• Backward compatibility tests: conda deactivate; bash -i tests/backward_compatibility_tests.sh

[aylei]

I'm wondering if this can be simplified as curl -LO "https://dl.k8s.io/release/v1.31.6/bin/linux/${TARGETARCH}/kubectl";

[Michaelvll]

Good point! Simplified the impl and make it more general

[romilbhardwaj]

Awesome @Michaelvll!

[Michaelvll]

/smoke-test --kubernetes

[romilbhardwaj]

Thanks @Michaelvll! Super excited to make GH200s go brrrr

[aylei]

lgtm, thanks @Michaelvll !