[PVC] Open a fresh new workspace, it uses large node pool. Relaunch the workspace, it uses standard node pool

[jenting]

Bug description

The user workspace class preference is standard.
The prebuild was built with large workspace class.

Open a fresh new workspace from the prebuild project, and then the workspace uses the large node pool because the prebuild was built on a large node pool. Relaunch the workspace, and then it uses a standard node pool. So, the workspace pod can't be up.

This is because the VolumeSnapshot size is 50Gi, and the PVC size is 30Gi.

Steps to reproduce

Prepare the prebuild project

• Set the user's workspace class preference as large
• Configure a prebuild project and run the prebuild

Open workspace from the prebuild project

• Set the user's workspace class preference as standard
• Enable the user's PVC feature flag
• Do not enable the project to prebuild PVC feature flag
• Open the new workspace from the prebuild project
  
• Make sure the workspace is using the large node pool g1-large-pvc by checking the label kubectl get pod <pod-name> --show-labels | grep gitpod.io/workspaceClass
• Check the corresponding PVC with size 50Gi
• Stop the workspace
• Check the VolumeSnapshot size is 50Gi
• Reopen the workspace. The workspace can't be open
  • The workspace pod is using the standard node pool g1-standard-pvc by checking the label kubectl get pod <pod-name> --show-labels | grep gitpod.io/workspaceClass
  • The workspace pvc size is 30Gi
  • The workspace pvc restored from the VolumeSnapshot, the VolumeSnapshot size is 50Gi
    

Workspace affected

No response

Expected behavior

The workspace should keep use the large node pool. So the workspace can be open.

Example repository

No response

Anything else?

#7901

Similar to #12494

[kylos101]

Reassigning to @Furisto given this thread

[jenting]

Note: it happens only with the PVC feature flag.