Skip to content

[ws-manager] Make cluster selection filter by application cluster #14000

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Oct 20, 2022

Conversation

andrew-farries
Copy link
Contributor

@andrew-farries andrew-farries commented Oct 19, 2022

Description

Context As part of #9198 we want to start syncing the `d_b_workspace_cluster` table with `db-sync`.

Currently, the table differs between US and EU regions because each table contains only the data relevant to that region. For example, in the EU table, the eu70 workspace cluster is marked as available and the us70 cluster is cordoned. In the US cluster eu70 is cordoned and us70 is available.

In order to sync the table we need to get to a point where there is no difference in the data in the table between EU and US regions.

To do that we will introduce a new field in the table called applicationCluster which records the name of the application cluster to which the record belongs. Thus, for each workspace cluster there will be two rows in Gitpod SaaS:

name applicationCluster url tls state ...
eu70 eu02 url tls info available ...
eu70 us02 url tls info cordoned ...

Effectively the new applicationCluster column gives the table an extra dimension so that we can combine both tables (EU and US) into one.

#13722 added the column to the table and made gpctl fill the value when gpctl registering a new workspace cluster. The value is taken from the GITPOD_INSTALLATION_SHORTNAME environment variable in ws-manager-bridge.

Make the cluster selection mechanism in ws-manager-api aware of the application cluster to which each workspace cluster is registered. Previously, the selection was by workspace cluster name only. Soon, as described in the context section, each region in gitpod will store records in the database for all workspace clusters - not just the ones in its region. This PR ensures that workspace clusters not in the same regions as the application cluster will not be considered.

Related Issue(s)

Part of #9198 and #13800

How to test

Edit the server deployment and change the WSMAN_CFG_MANAGERS environment variable to change the applicationCluster to which the default workspace cluster in the preview environment is registered:

[
  {
    "name": "",
    "url": "dns:///ws-manager:8080",
    "tls": {
      "ca": "/ws-manager-client-tls-certs/ca.crt",
      "crt": "/ws-manager-client-tls-certs/tls.crt",
      "key": "/ws-manager-client-tls-certs/tls.key"
    },
    "state": "available",
    "maxScore": 100,
    "score": 50,
    "govern": true,
    "admissionConstraints": null,
    "applicationCluster": "" // <- change this from "" to anything else
  }
]

The WSMAN_CFG_MANAGERS environment variable is base64 encoded.

Once the server deployment is edited it should no longer be possible to start a workspace as the only workspace cluster is now associated with a different (non-existent) application cluster.

Edit the WSMAN_CFG_MANAGERS environment variable once more and set the applicationCluster field back to "". Starting a workspace should now proceed as normal (the application cluster in preview environments is currently called "").

Release Notes

NONE

Documentation

Werft options:

  • /werft with-local-preview
    If enabled this will build install/preview
  • /werft with-preview
  • /werft with-integration-tests=all
    Valid options are all, workspace, webapp, ide

Andrew Farries added 2 commits October 19, 2022 11:23
@werft-gitpod-dev-com
Copy link

started the job as gitpod-build-af-use-app-cluster-in-ws-manager-api.8 because the annotations in the pull request description changed
(with .werft/ from main)

@github-actions github-actions bot added the team: webapp Issue belongs to the WebApp team label Oct 19, 2022
@werft-gitpod-dev-com
Copy link

started the job as gitpod-build-af-use-app-cluster-in-ws-manager-api.9 because the annotations in the pull request description changed
(with .werft/ from main)


public async getWorkspaceCluster(name: string): Promise<WorkspaceCluster | undefined> {
return this.clusters.find(m => m.name === name);
return this.clusters.find((m) => m.name === name && m.applicationCluster === this.applicationCluster);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For debugging purposes, I'd recommend logging the following:

  1. The set of clusters available
  2. The set of clusters which match this application cluster

We can add these as log fields. This would help us identify scenarios when the filtering logic doesn't work as expected.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The extra benefit of this is if we accidentally end up in a situation where there is more than 1 cluster that matches, we'd see that in the logs when debugging.

Copy link
Member

@easyCZ easyCZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Left a Q around logging but it can be tackled in a follow-up PR.

@roboquat roboquat merged commit ee54e2c into main Oct 20, 2022
@roboquat roboquat deleted the af/use-app-cluster-in-ws-manager-api branch October 20, 2022 07:52
@andrew-farries andrew-farries mentioned this pull request Oct 20, 2022
3 tasks
@roboquat roboquat added deployed: webapp Meta team change is running in production deployed Change is completely running in production labels Oct 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deployed: webapp Meta team change is running in production deployed Change is completely running in production release-note-none size/S team: webapp Issue belongs to the WebApp team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants