flakiness - fix cert rotation tests

[nammn]

Summary

• cherry-pick of https://github.com/10gen/ops-manager-kubernetes/pull/4298 and linked pr will be closed and cherry-picked from here instead

Proof of Work

Checklist

• Have you linked a jira ticket and/or is the ticket in the title?
• Have you checked whether your jira ticket required DOCSP changes?
• Have you checked for release_note changes?

Reminder (Please remove this when merging)

• Please try to Approve or Reject Changes the PR, keep PRs in review as short as possible
• Our Short Guide for PRs: Link
• Remember the following Communication Standards - use comment prefixes for clarity:
  • blocking: Must be addressed before approval.
  • follow-up: Can be addressed in a later PR or ticket.
  • q: Clarifying question.
  • nit: Non-blocking suggestions.
  • note: Side-note, non-actionable. Example: Praise
  • --> no prefix is considered a question

[m1kola]

Looks good overall. My comments are mostly questions. There is only one suggestion to change assert_certificate_rotation slightly.

[m1kola]

Non-blocking q: any reason not to put this to the top of the file with rest of the imports?

[m1kola]

assert_certificate_rotation gives an impression that it does not make any changes and only asserts when in fact it does.

What do you think about the following suggestion?

1. Moving these two lines into the caller functions
2. Make assert_certificate_rotation accept old_ac_version as an argument

[nammn]

renamed! bf2d183

[m1kola]

A bit of context for transparency. We discussed it in DM: while I think it is good to keep mutation (cert rotation) separate from assertions, @nammn wanted to to keep everything together to avoid forgetting to do part of the checks.

At the same time I think it is more important to address the flake than worry about semantics here. We landed on the middle ground: renaming the function to rotate_and_assert_certificates to more clearly indicate the purpose.