-
Notifications
You must be signed in to change notification settings - Fork 107
Build and run functional tests all locally #3211
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Build and run functional tests all locally #3211
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mostly, sure; this could be convenient. But a couple of questions...
979df73
to
53ab39f
Compare
53ab39f
to
d0e3372
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rather than go in the direction of trying to conditionalize the registry and repositories for the all the images, I would like to propose following the approach that I've laid out in #3215.
How we name the images is less important than how we manage the pull policy. If we use a pull policy of newer
(or never
) instead of always
(and, if we always name the images to match the remote repository), then I believe that we will be able to build and run repeatedly without anything having to be pulled (once the remote images are resident locally), and pushing the Pbench Server image is unnecessary in the build/run/repeat local scenario.
Kubernetes vs podman not withstanding (see https://kubernetes.io/docs/concepts/containers/images/#image-pull-policy, and PR #3216), I'll try not building with |
d0e3372
to
e28488b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, with the possible exception of the Pbench Server container image pull policy in run-pbench-in-a-can
. And, if that needs fixing, then there are a few other things which you might want to fix or change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like you've incorporated almost of of #3215 -- thanks!
In noting that, I found two more things to consider.
e28488b
to
563334c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rebased and ready for review.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One slightly messed up comment, but not a big deal unless you make other changes ...
Prior to this change, if one tried to use `jenkins/runlocal` without setting `BASE_IMAGE`, the default of `UBI:9` would result in an error if the system on which this script was invoked did not have a RHEL entitlement. This change simplifies things so that the RPM built is truly local to the environment, and using the installed environment, the `BASE_IMAGE` is to work with that RPM. For example, if running on a Fedora 37 box, a Fedora 37 RPM would be built and installed using the `fedora:37` base image.
If a functional test is successful, and `--cleanup` has been requested, we will have already removed the POD and container. There is not need to also run the "hard" cleanup script. Further, we emit a note when `--cleanup` is not requested after the tests have finished to alert the user that the POD and container are still running. We also emit the print statements for the clean up being performed.
All arguments to `jenkins/runlocal` are passed along to `jenkins/run-server-unit-tests` as a simple way to support the `--cleanup` option on `jenkins/runlocal` Further, we only support defaulting to a `ubi9:latest` or `centos:stream9` image in those respective environments. Lastely, we add support for `BASE_IMAGE` to be defined ahead of time.
7ea5bfa
to
09b36e1
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ship it!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The capitalization in "Support Services pod" looks a bit odd: I don't think I would capitalize either since it's not exactly a formal title. But that's trivial; let's get this in.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Worked for me locally, ship it 👍
We are having problems with the stability of the functional tests (see #3209). As a result, we need a way to iteratively build a new container and test the changes with a much shorter feedback loop.
To that end, this PR provides a way to build and run the functional tests locally without involving a container registry.
Suggested PR commit message below for review: