Load test Single Cluster Reference Architecture

[lucasvaltl]

Summary

Scale test the production reference architecture in order to be able to give info on how far it can scale

Context

We are using our reference architectures as the way to communicate what Gitpod can and cannot do given a specific set of infrastructure. It makes sense to also talk about scale in this context.

Value

• Assurance in the scale that the prod reference architecture can handle

Acceptance Criteria

• We have a measure of how far the reference architecture can scale that is relevant to our users. E.g. it could be the number of workspaces of size X (given N nodes)

Measurement

• We have a metric that tracks scale, and we can re-run these tests to see if something has changed.

Implementation Ideas:

• You can use loadgen for that https://github.com/gitpod-io/gitpod/tree/main/dev/loadgen. We use this scenario to create 55 workspaces running some stress tasks.

Additional Context

• Internal slack message

[lucasvaltl]

This is likely not just a team self-hosted thing but a Gitpod thing :)

[lucasvaltl]

We should move this to scheduled once we have the terraform scripts ( #11027) and maybe a werft command to spin up a self-hosed environment.