Recommend a workspace-class that is smaller than g1-standard and test feasibility

[kylos101]

Is your feature request related to a problem? Please describe

From a harvester preview VM or workspace-preview, we want to see if a small workspace class (smaller than g1-standard) performs well enough for simple workloads. If it is a good experience, then, we'd want to amend webapp and workspace, to support a new "small" workspace class, prior to enabling UBP and workspace classes for individuals.

Internal context
More internal context

cc: @mbrevoort @atduarte

Describe the behaviour you'd like

Idea: alter ws-manager and ws-daemon configs, and test a regular workspace with a class using a limit CPU of 1 and burst CPU of 2 (ws-daemon), memory requests of 2Gi and a limit of 4Gi (kubernetes), with 15Gi of storage, and 5Gi of ephemeral storage. Also, please be sure to enable disk IO limiting on the related ws-daemon, this way the workspace is limited similarly to how we would do in production.

Then, once you're able to start workspaces using the above config, test that the small workspace class configured is works well enough as a regular workspace to develop our website. For example:

1. Make change codes
2. How does the file watcher pick them up
3. Run tests
4. Commit code
5. Use extensions associated with the project

What about our Gitpod repo? How does it behave from a development standpoint? The assumption is that it will not a great experience.

Describe alternatives you've considered

Some extensions are resource intense. @akosyakov , can you think of any extensions that are common, but hungry, that we should include as part of this test?

Additional context

Will this be useful enough for JetBrains? @akosyakov wdyt? I assume no, because this workspace will not meet minimum requirements.

Definition of done

If feasible for users, a smaller workspace-class recommendation is shared and agreed with Product and Finance teams, and related issues are added to groundwork to support the related deployment.

[mbrevoort]

Overall we are trying to assess if there is a viable workspace resource size below g1-standard that could support development of what type of projects. The resources described above seem like a great place to start.

[mbrevoort]

In addition, how would such a resource profile affect pod scheduling, resource utilization and noisy neighbor conditions?

[akosyakov]

Some extensions are resource intense. @akosyakov , can you think of any extensions that are common, but hungry, that we should include as part of this test?

java, golang, typescript. keep in mind hungry depends on size and complexity of the project, not extensions itself

[Furisto]

Tested this with the gitpod-io/website repository. Running tests and making code changes works well but trying to build the website leads to OOM kills or network disconnects due to not being enough memory available.

[kylos101]

@mbrevoort good questions!

In addition...

how would such a resource profile affect pod scheduling

The pods would be scheduled to a dedicated node pool for the small workspace class, so, they wouldn't impact pods being scheduled to other node pools (like standard and large).

Initially we'd control density via memory requests, but, may want to consider CPU requests, too. For example, memory requests would be 2Gi, and CPU might be 1 core, or .5 cores, etc.

I do wonder how many small workspaces we could fit on a node? But, first we need to find a size that works for one workspace. And then we can talk about achieving a desired density. For example, if we run more than 18 small workspaces, perhaps 36, we'd have to reduce the related disk IO bandwidth.

resource utilization

The way this is written now, you'd get 1 CPU, and be able to burst to 2 (controlled by ws-daemon), and you wouldn't be able to use more than 4Gi of memory (controlled by Kubernetes).

and noisy neighbor conditions?

@Furisto can you think of anything new that we might have from a risk perspective, by having a higher density of workspaces?

[kylos101]

Tested this with the gitpod-io/website repository. Running tests and making code changes works well but trying to build the website leads to OOM kills or network disconnects due to not being enough memory available.

Okay, good to know, @Furisto ! Bummed to hear, though. 😉

[Furisto]

@Furisto can you think of anything new that we might have from a risk perspective, by having a higher density of workspaces?

Higher chance of WorkspaceStuckInStopping alerts because more workspaces mean more backups and we have a concurrency limit for backups. On the other hand the backups should be smaller, so they will go faster.