Allow for more than one deployment per test under cloud-qa #173

lsierant · 2025-06-03T11:28:41Z

Summary

This patch allows to deploy more than one database managed by cloud-qa in a single e2e test.
Currently it's only possible under Ops Manager tests which deploys it in the same test. With cloud-qa single test always used one project for the whole test with project_name=$NAMESPACE.

Deploying more than one database resource was therefore difficult and required manually crafting my-project config maps.

Additionally, this PR allows to reuse the whole test in both OM and cloud-qa tests with little changes to the code.

How does it work for OM?
Multiple mdb resources under OM are possible thanks to resource.configure() function, which requires to pass MongoDBOpsManager resource. It creates -project config map and credentials secret by using api keys that are available because there is an OM instance deployed in the same test.

How it was for cloud-qa tests?
Our tests were able to use only one "my-project" config map and credentials, which is created before the test pod is scheduled (configure_operator.sh). It contains fixed project name, which equals to the current namespace (a random string in CI). Reusing this config map leads to attempting to deploy mdb resource in the same cloud-qa project, which then the operator rejects.

What this PR changes for cloud-qa tests?
We hook into configure function allowing for passing None as ops manager resource. In this case we clone my-project config map and overwrite projectId to '{namespace}-{resourceName}'. The operator will then create a new cloud-qa project with that name allowing for running multiple cloud-qa-managed resources in one test.

Refactored existing tests

Some tests were refactored to remove unnecessary manual project config maps:

docker/mongodb-kubernetes-tests/tests/authentication/mongodb_custom_roles.py
docker/mongodb-kubernetes-tests/tests/multicluster/multi_cluster_scale_up_cluster.py
docker/mongodb-kubernetes-tests/tests/multicluster_shardedcluster/multi_cluster_sharded_disaster_recovery.py
docker/mongodb-kubernetes-tests/tests/multicluster_shardedcluster/multi_cluster_sharded_snippets.py
docker/mongodb-kubernetes-tests/tests/replicaset/replica_set_agent_flags.py

Proof of Work

project config map was created (the default my-project is not used)
evg link

- apiVersion: v1
  data:
    baseUrl: http://om-backup-tls-svc.a-1758578405-qd641twddhz.svc.cluster.local:8080/
    orgId: ""
    projectName: sh-disaster-recovery
  kind: ConfigMap
  metadata:
    creationTimestamp: "2025-09-22T22:19:40Z"
    name: sh-disaster-recovery-config
    namespace: a-1758578405-qd641twddhz
    resourceVersion: "3609"

    uid: 86dd4283-6d2e-44cb-84dc-d87b3fc57cae

which is automatically referenced in MongoDB resource
evg link

  opsManager:
      configMapRef:
        name: sh-disaster-recovery-config

Checklist

Have you linked a jira ticket and/or is the ticket in the title?
Have you checked whether your jira ticket required DOCSP changes?
Have you checked for release_note changes?

Reminder (Please remove this when merging)

Please try to Approve or Reject Changes the PR, keep PRs in review as short as possible
Our Short Guide for PRs: Link
Remember the following Communication Standards - use comment prefixes for clarity:
- blocking: Must be addressed before approval.
- follow-up: Can be addressed in a later PR or ticket.
- q: Clarifying question.
- nit: Non-blocking suggestions.
- note: Side-note, non-actionable. Example: Praise
- --> no prefix is considered a question

lsierant · 2025-06-04T10:48:03Z

evergreen retry

# Summary This PR contains no functional changes - only moving functions around. It's extracting common bits imported from different places to their own files which resolves *some* cyclic dependencies in the code. It's by no means a complete refactor, it's an extracted refactor from the followup change (#173) which triggered cyclic dependencies in the first place. ## Proof of Work [Green EVG](https://spruce.mongodb.com/version/683fe9d44e8b7f0007877a9e/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC) ## Checklist - [ ] Have you linked a jira ticket and/or is the ticket in the title? - [ ] Have you checked whether your jira ticket required DOCSP changes? - [ ] Have you checked for release_note changes? ## Reminder (Please remove this when merging) - Please try to Approve or Reject Changes the PR, keep PRs in review as short as possible - Our Short Guide for PRs: [Link](https://docs.google.com/document/d/1T93KUtdvONq43vfTfUt8l92uo4e4SEEvFbIEKOxGr44/edit?tab=t.0) - Remember the following Communication Standards - use comment prefixes for clarity: * **blocking**: Must be addressed before approval. * **follow-up**: Can be addressed in a later PR or ticket. * **q**: Clarifying question. * **nit**: Non-blocking suggestions. * **note**: Side-note, non-actionable. Example: Praise * --> no prefix is considered a question

lsierant · 2025-06-10T12:30:00Z

evergreen refresh

lsierant · 2025-06-16T13:19:39Z

Try to use this in docker/mongodb-kubernetes-tests/tests/authentication/mongodb_custom_roles.py when #54 is merged

lsierant · 2025-07-01T13:46:19Z

Adjust it here after #206 is merged as well

lsierant · 2025-07-02T09:28:13Z

q from @nammn: can you check whether those new projects are properly cleaned up in cloudmanager?

github-actions · 2025-09-18T21:19:16Z

⚠️ (this preview might not be accurate if the PR is not rebased on current master branch)

MCK 1.4.1 Release Notes

lsierant · 2025-09-22T20:23:23Z

scripts/funcs/kubernetes

    cat > "${wrapper_script}" << EOF
 #!/bin/bash
 # Define kubectl function to include the context
-set -x


this saves tons of log spamming when deploying csi drivers while creating kind clusters

lsierant · 2025-09-22T20:25:33Z

api/v1/mdbmulti/mongodb_multi_types.go


 func (m *MongoDBMultiCluster) GetProjectConfigMapName() string {
-	return m.Spec.OpsManagerConfig.ConfigMapRef.Name
+	return m.Spec.GetProject()


not sure if this is release notes worthy. It was a bug that caused that MongoDBMultiCluster could handle only spec.opsManager field, but in MongoDB we handle both spec.opsManager and spec.cloudManager via GetProject()

yeah its a bug fix, should be in RN.

nammn

LGTM!

PTAL at my comments

nammn · 2025-09-24T12:27:24Z

api/v1/mdbmulti/mongodb_multi_types.go


 func (m *MongoDBMultiCluster) GetProjectConfigMapName() string {
-	return m.Spec.OpsManagerConfig.ConfigMapRef.Name
+	return m.Spec.GetProject()


yeah its a bug fix, should be in RN.

nammn · 2025-09-24T12:30:04Z

scripts/dev/contexts/variables/om80

 export TEST_MODE=opsmanager
 export OPS_MANAGER_REGISTRY="${BASE_REPO_URL}"
+
+export ops_manager_version="${CUSTOM_OM_VERSION}"


why only here? Was it missing on the other variables? could we just rely on CUSTOM_OM_VERSION everywhere instead and remove ops_manager_version everywhere else instead?

nammn · 2025-09-24T12:30:37Z

docker/mongodb-kubernetes-tests/kubetester/mongodb.py

+        om: Optional[MongoDBOpsManager],
+        project_name: str,
+        api_client: Optional[client.ApiClient] = None,
+    ) -> MongoDB:


wdyt about a log line -> "we are running om vs cloudqa"?

lsierant mentioned this pull request Jun 3, 2025

Resolve cyclic imports in e2e tests #172

Merged

3 tasks

lsierant force-pushed the lsierant/pytest-refactor branch from 043f6e7 to 93baf39 Compare June 3, 2025 11:29

lsierant force-pushed the lsierant/multi-cloud-qa branch from 4ba55b0 to 8515b05 Compare June 3, 2025 11:30

lsierant force-pushed the lsierant/pytest-refactor branch from 93baf39 to 35c3435 Compare June 4, 2025 06:38

lsierant force-pushed the lsierant/multi-cloud-qa branch from 8515b05 to 5ee79b4 Compare June 4, 2025 06:38

Base automatically changed from lsierant/pytest-refactor to master June 4, 2025 18:05

lsierant force-pushed the lsierant/multi-cloud-qa branch 2 times, most recently from 5bd8491 to 94a69a9 Compare June 9, 2025 21:55

lsierant mentioned this pull request Jul 1, 2025

CLOUDP-324440 Save rs member ids #206

Merged

3 tasks

lsierant mentioned this pull request Sep 18, 2025

Running search test under OM 8.2 #467

Open

3 tasks

lsierant force-pushed the lsierant/multi-cloud-qa branch from 94a69a9 to b053f96 Compare September 18, 2025 21:18

lsierant force-pushed the lsierant/multi-cloud-qa branch 2 times, most recently from 8689e2c to 5509206 Compare September 22, 2025 19:18

lsierant added the skip-changelog Use this label in Pull Request to not require new changelog entry file label Sep 22, 2025

lsierant force-pushed the lsierant/multi-cloud-qa branch 2 times, most recently from f262059 to 4964f58 Compare September 22, 2025 20:18

lsierant commented Sep 22, 2025

View reviewed changes

lsierant force-pushed the lsierant/multi-cloud-qa branch 4 times, most recently from 6245e5c to f432eb6 Compare September 23, 2025 07:19

lsierant marked this pull request as ready for review September 23, 2025 07:32

lsierant requested a review from a team as a code owner September 23, 2025 07:32

lsierant requested review from m1kola and nammn September 23, 2025 07:32

lsierant added 6 commits September 23, 2025 22:30

Allow for more than one deployment per test under cloud-qa

03c4544

Refactored mongodb_custom_roles to leverage multi-project configure

4ebcaa6

Refactored other tests to use configure

51b4680

wip

d19071a

Fixing e2e tests

2c9ced3

Fixed e2e tests

de8fd69

lsierant force-pushed the lsierant/multi-cloud-qa branch from fa19d3d to de8fd69 Compare September 23, 2025 20:30

nammn reviewed Sep 24, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Allow for more than one deployment per test under cloud-qa #173

Allow for more than one deployment per test under cloud-qa #173

Uh oh!

lsierant commented Jun 3, 2025 •

edited

Loading

Uh oh!

lsierant commented Jun 4, 2025

Uh oh!

lsierant commented Jun 10, 2025

Uh oh!

lsierant commented Jun 16, 2025

Uh oh!

lsierant commented Jul 1, 2025

Uh oh!

lsierant commented Jul 2, 2025

Uh oh!

github-actions bot commented Sep 18, 2025

Uh oh!

lsierant Sep 22, 2025

Uh oh!

lsierant Sep 22, 2025

Uh oh!

nammn Sep 24, 2025

Uh oh!

nammn left a comment

Uh oh!

nammn Sep 24, 2025

Uh oh!

nammn Sep 24, 2025

Uh oh!

nammn Sep 24, 2025

Uh oh!

Uh oh!

Allow for more than one deployment per test under cloud-qa #173

Are you sure you want to change the base?

Allow for more than one deployment per test under cloud-qa #173

Uh oh!

Conversation

lsierant commented Jun 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Refactored existing tests

Proof of Work

Checklist

Reminder (Please remove this when merging)

Uh oh!

lsierant commented Jun 4, 2025

Uh oh!

lsierant commented Jun 10, 2025

Uh oh!

lsierant commented Jun 16, 2025

Uh oh!

lsierant commented Jul 1, 2025

Uh oh!

lsierant commented Jul 2, 2025

Uh oh!

github-actions bot commented Sep 18, 2025

MCK 1.4.1 Release Notes

Uh oh!

lsierant Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

lsierant Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

nammn Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

nammn left a comment

Choose a reason for hiding this comment

Uh oh!

nammn Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

nammn Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

nammn Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lsierant commented Jun 3, 2025 •

edited

Loading