CLOUDP-295785 - release tasks integration with atomic_pipeline.py

[MaciejKaras]

Summary

⚠️ Important notice

This PR contains some changes from #336, but they are not used yet and don't impact the PRs or patches. They are included because previously this PR was stacked on the staging PR and it is much easier to include them. The changes that are included:

• latest_tag support - this is needed for staging builds, but like mentioned earlier, staging builds are not yet used
• replace 268558157000.dkr.ecr.us-east-1.amazonaws.com/dev with BASE_REPO_URL. This will be used to distinguish different repo urls: dev, staging and release. Currently hardcoded to 268558157000.dkr.ecr.us-east-1.amazonaws.com/dev

---

This change is made to unblock the release of MCK 1.3.0. It is not final state of the release mechanism and most of it will be replaced by image promotion process.

Created new .evergreen-release.yml file that contains all release tasks including integration with kubectl-mongodb plugin release task. All of the variants are triggered only when github_tag is added.

Additional changes:

• each released image will be also released with additional olm_tag that has dynamic timestamp part. It will prevent accidental overriding the tags used by OLM. The tag syntax is {version}-olm-{timestamp_suffix} where timestamp suffix is in %Y%m%d%H%M%S format
• created separate release_operator_pipeline evergreen function that uses release build scenario and version provided by git_tag
• fixed and bumped preflight script

Proof of Work

List of tasks that are triggered when doing manual patch:

⚠️ This PR was tested by running evergreen command locally:

sudo evergreen patch -p mongodb-kubernetes -a release -d "Release test" -f -y -u --browse --path .evergreen.yml --param RELEASE_OPERATOR_VERSION=1.3.0-rc

Link to evg job
-> https://spruce.mongodb.com/version/68b81b45285a950007bc8398

Checklist

• Have you linked a jira ticket and/or is the ticket in the title?
• Have you checked whether your jira ticket required DOCSP changes?
• Have you added changelog file?
  • use skip-changelog label if not needed
  • refer to Changelog files and Release Notes section in CONTRIBUTING.md for more details

[github-actions]

⚠️ (this preview might not be accurate if the PR is not rebased on current master branch)

MCK 1.3.0 Release Notes

New Features

Multi-Architecture Support

We've added comprehensive multi-architecture support for the kubernetes operator. This enhancement enables deployment on IBM Power (ppc64le) and IBM Z (s390x) architectures alongside
existing x86_64 support. Core images (operator, agent, init containers, database, readiness probe) now support multiple architectures. We do not add support IBM and ARM support for Ops-Manager and the init-ops-manager image.

• MongoDB Agent images have been migrated to new container repository: quay.io/mongodb/mongodb-agent.
  • the agents in the new repository will support the x86-64, ARM64, s390x, and ppc64le architectures. More can be read in the public docs.
  • operator running >=MCK1.3.0 and static cannot use the agent images from the old container repository quay.io/mongodb/mongodb-agent-ubi.
• quay.io/mongodb/mongodb-agent-ubi should not be used anymore, it's only there for backwards compatibility.

Bug Fixes

• This change fixes the current complex and difficult-to-maintain architecture for stateful set containers, which relies on an "agent matrix" to map operator and agent versions which led to a sheer amount of images.
• We solve this by shifting to a 3-container setup. This new design eliminates the need for the operator-version/agent-version matrix by adding one additional container containing all required binaries. This architecture maps to what we already do with the mongodb-database container.
• Fixed an issue where the readiness probe reported the node as ready even when its authentication mechanism was not in sync with the other nodes, potentially causing premature restarts.
• Fixed an issue where the MongoDB Agents did not adhere to the NO_PROXY environment variable configured on the operator.
• Changed webhook ClusterRole and ClusterRoleBinding default names to include the namespace. This ensures that multiple operator installations in different namespaces don't conflict with each other.

Other Changes

• Optional permissions for PersistentVolumeClaim moved to a separate role. When managing the operator with Helm it is possible to disable permissions for PersistentVolumeClaim resources by setting operator.enablePVCResize value to false (true by default). When enabled, previously these permissions were part of the primary operator role. With this change, permissions have a separate role.
• subresourceEnabled Helm value was removed. This setting used to be true by default and made it possible to exclude subresource permissions from the operator role by specifying false as the value. We are removing this configuration option, making the operator roles always have subresource permissions. This setting was introduced as a temporary solution for this OpenShift issue. The issue has since been resolved and the setting is no longer needed.
• We have deliberately not published the container images for OpsManager versions 7.0.16, 8.0.8, 8.0.9 and 8.0.10 due to a bug in the OpsManager which prevents MCK customers to upgrade their OpsManager deployments to those versions.

[viveksinghggits]

LGTM, can we please add the ToDos that we talked about?

[nammn]

LGTM! A few question I would like to have answered though

[nammn]

is the is_evg required? Doesn't is_patch already say that its a patch?

[MaciejKaras]

it was a bug because in next condition we check for is_evg, which would not be entered if previous condition would be is_patch or is_evg. It is still commented out, but I wanted to fix it. We can also remove the is_evg from this check, but it is maybe more verbose as is.

[nammn]

i will leave it to you, for me it reads weird as it seems redundant, but not blocking!

[lucian-tosa]

nit: I doubt we need this for the test image

[MaciejKaras]

I thought about scenario when someone interferes with our test image and it will give false positive, although the image was corrupted.