[k8s] Fix show-gpus when limited permissions are available

[romilbhardwaj]

Updates our minimal permissions doc to add permissions required by show-gpus, and updates show-gpus to work with limited permissions:

W 10-29 18:36:09 kubernetes_catalog.py:112] Failed to get pods in the Kubernetes cluster (forbidden). Please check if your account has necessary permissions to list pods. Realtime GPU availability information may be incorrect.
Kubernetes GPUs (context: gke_sky-dev-465_us-central1-c_gkeusc6)
GPU   REQUESTABLE_QTY_PER_NODE  TOTAL_GPUS  TOTAL_FREE_GPUS
H100  1, 2, 4, 8                8           NA

Kubernetes per node GPU availability
NODE_NAME                               GPU_NAME  TOTAL_GPUS  FREE_GPUS
gke-gkeusc6-default-pool-db727de5-62gm  H100      8           NA

Also adds a quick snippet for specifying custom resources in pod_config.

[concretevitamin]

UX: NA may suggest "not available". How about <insufficient permissions>?

[romilbhardwaj]

<insufficient permissions> might be too long... how about <no permissions>? Or even <forbidden>?

[concretevitamin]

<no permissions> sounds good.

…

On Tue, Nov 5, 2024 at 11:52 AM Romil Bhardwaj ***@***.***> wrote: <insufficient permissions> might be too long... how about <no permissions>? Or even <forbidden>? — Reply to this email directly, view it on GitHub <#4208 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEQWHWBJNM6ONYUUHJFHOTZ7EOW5AVCNFSM6AAAAABQ3CREZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJYGAZTENJSGQ> . You are receiving this because you commented.Message ID: ***@***.***>

[romilbhardwaj]

Updated:

(base) ➜  ~ sky show-gpus --cloud kubernetes
W 11-11 14:06:42 kubernetes_catalog.py:114] Failed to get pods in the Kubernetes cluster (forbidden). Please check if your account has necessary permissions to list pods. Realtime GPU availability information may be incorrect.
Kubernetes GPUs (context: kind-skypilot)
GPU   REQUESTABLE_QTY_PER_NODE  TOTAL_GPUS  TOTAL_FREE_GPUS
H100  1, 2, 4, 8                8           <no permissions>

Kubernetes per node GPU availability
NODE_NAME               GPU_NAME  TOTAL_GPUS  FREE_GPUS
skypilot-control-plane  H100      8           <no permissions>

[Michaelvll]

Thanks for the update @romilbhardwaj! LGTM.

[Michaelvll]

Is this relates to the show-gpus?

[romilbhardwaj]

No, just adding docs on how to add custom resources through pod_config. Could have been a separate PR :)