Skip to content

Conversation

andrew-farries
Copy link
Contributor

Description

As part of #9198, a deployment called slow-server was added, identical to server but with a higher latency connection to the database.

This PR adds the slow-server component to the image-builder-mk3 and usage component network policy ingress rules, so that ingress to those two components works for slow-server just as it does for the server component.

This means that workspace startup works correctly for workspaces started by slow-server.

Related Issue(s)

Fixes #15027

How to test

  1. Log into the preview environment.
  2. Update the configcat targeting rule for the slow_database (non-production) feature flag for your user id.
  3. Wait for 3 minutes for configcat to pull the new config or until /api/feature-flags/slow-database starts setting X-Gitpod-Slow-Database: true in the response headers.
  4. Start a workspace.

The workspace start should be slower than usual, but should complete successfully.

Release Notes

NONE

Documentation

Werft options:

  • /werft with-local-preview
    If enabled this will build install/preview
  • /werft with-preview
  • /werft with-slow-database
  • /werft with-large-vm
  • /werft with-integration-tests=all
    Valid options are all, workspace, webapp, ide, jetbrains, vscode, ssh

@werft-gitpod-dev-com
Copy link

started the job as gitpod-build-af-slow-workspace-starts.15 because the annotations in the pull request description changed
(with .werft/ from main)

@roboquat roboquat added the size/S label Dec 1, 2022
@github-actions github-actions bot added team: SID team: webapp Issue belongs to the WebApp team team: workspace Issue belongs to the Workspace team labels Dec 1, 2022
Copy link
Contributor

@Pothulapati Pothulapati left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM and sounds valid!

Just FYI, there seem to be more NetworkPolicy rules that are present with component: server to allow communication with components like proxy, etc. Might be a good idea to add now (or alter when we see the need).

Andrew Farries added 3 commits December 5, 2022 12:22
Allow ingress to `usage` from `slow-server`.
Allow ingress to `image-builder-mk3` from `slow-server`.
@andrew-farries andrew-farries force-pushed the af/slow-workspace-starts branch from 3604f30 to 5c7f28f Compare December 5, 2022 12:22
@andrew-farries
Copy link
Contributor Author

Thanks for the review @Pothulapati. server also appears as a pod selector in these other network policies:

Looking at the network policy docs, each of the above policies allows ingress from a pod called server in a namespace with a chart=monitoring label. ie not from a server pod in any other namespace. This is confusing as there doesn't appear to even be a namespace with that label in preview or live environments.

Is it possible that the policies are incorrect and we made the mistake described in the docs, ie that those parts of the policies should in fact read:

        - namespaceSelector:
            matchLabels:
              chart: monitoring
        - podSelector:
            matchLabels:
              app: gitpod
              component: server

To allow ingress from anywhere in a chart=monitoring namespace and from server? Given that Gitpod functions with the policies as they are, I think maybe these policies are redundant.

Do you have any idea what the intention is with these policies?

@andrew-farries
Copy link
Contributor Author

/unhold because the changes here are sufficient to start workspaces through slow-server.

@roboquat roboquat merged commit 9f35591 into main Dec 5, 2022
@roboquat roboquat deleted the af/slow-workspace-starts branch December 5, 2022 16:23
@roboquat roboquat added deployed: webapp Meta team change is running in production deployed: workspace Workspace team change is running in production labels Dec 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deployed: webapp Meta team change is running in production deployed: workspace Workspace team change is running in production release-note-none size/S team: SID team: webapp Issue belongs to the WebApp team team: workspace Issue belongs to the Workspace team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[db-sync] Investigate cause of slow workspace starts when working against a higher latency db
5 participants