🌱 Migrate Rukpak #1032

dtfranz · 2024-07-10T21:56:09Z

Moves rukpak code imported by operator-controller into the internal/ folder.

Fixes #997

netlify · 2024-07-10T21:56:26Z

✅ Deploy Preview for olmv1 ready!

Name	Link
🔨 Latest commit	`bf85b3c`
🔍 Latest deploy log	https://app.netlify.com/sites/olmv1/deploys/66914a8cf0571100098ef5c7
😎 Deploy Preview	https://deploy-preview-1032--olmv1.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

dtfranz · 2024-07-10T21:57:48Z

internal/bundledeployment/bundledeployment.go

@@ -0,0 +1,244 @@
+/*


Commenting here to call specific attention to this file. How much more of the k8s stuff here can be removed? Do we need the json:"..." tags? Are we OK with the fact that I copied an autogenerated func into this file in order to maintain DeepCopy()?

dtfranz · 2024-07-10T21:59:50Z

General comment: there are likely some things copied over that are unused. I'll go through and make sure that's all removed.

codecov · 2024-07-10T22:14:16Z

Codecov Report

Attention: Patch coverage is 66.56347% with 216 lines in your changes missing coverage. Please review.

Project coverage is 73.46%. Comparing base (64f99f2) to head (bf85b3c).

Files	Patch %	Lines
internal/rukpak/source/image_registry.go	41.34%	44 Missing and 17 partials ⚠️
internal/rukpak/convert/registryv1.go	85.10%	23 Missing and 19 partials ⚠️
internal/rukpak/provisioner/plain/plain.go	60.00%	10 Missing and 10 partials ⚠️
internal/rukpak/source/unpacker.go	0.00%	19 Missing ⚠️
internal/rukpak/util/tar.go	47.05%	10 Missing and 8 partials ⚠️
...ak/preflights/crdupgradesafety/crdupgradesafety.go	70.17%	10 Missing and 7 partials ⚠️
...ternal/rukpak/bundledeployment/bundledeployment.go	0.00%	12 Missing ⚠️
internal/rukpak/util/fs.go	0.00%	10 Missing ⚠️
internal/rukpak/storage/localdir.go	73.33%	4 Missing and 4 partials ⚠️
internal/rukpak/util/util.go	80.00%	2 Missing and 2 partials ⚠️
... and 3 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1032      +/-   ##
==========================================
- Coverage   77.00%   73.46%   -3.55%     
==========================================
  Files          19       32      +13     
  Lines        1357     1986     +629     
==========================================
+ Hits         1045     1459     +414     
- Misses        222      368     +146     
- Partials       90      159      +69

Flag	Coverage Δ
e2e	`55.84% <55.72%> (-0.61%)`	⬇️
unit	`45.01% <27.39%> (-8.86%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

dtfranz · 2024-07-10T23:31:35Z

General comment: there are likely some things copied over that are unused. I'll go through and make sure that's all removed.

I've removed everything I could find that was unused; see comparison here to see what I left out.

joelanford · 2024-07-11T00:49:48Z

WDYT about dumping everything in under internal/rukpak for now so that it's more obvious in a first pass what came from rukpak and what changed in existing operator-controller code?

I'm very much in favor of moving our rukpak code directly into this repo, but I'm also feeling a little bit like we need to (in a future PR) step back and take stock of our suddenly sprawling codebase to see if there are obsolete abstractions (e.g. unpack + storage) and other ways to reduce our package count and tidy things up a bit.

bentito · 2024-07-11T08:35:25Z

internal/controllers/clusterextension_controller.go

@@ -280,7 +280,7 @@ func (r *ClusterExtensionReconciler) reconcile(ctx context.Context, ext *ocv1alp
 	case rukpaksource.StateUnpacked:
 		// TODO: Add finalizer to clean the stored bundles, after https://github.com/operator-framework/rukpak/pull/897
 		// merges.
-		if err := r.Storage.Store(ctx, ext, unpackResult.Bundle); err != nil {
+		if err := r.Storage.Store(ctx, ext.GetName(), unpackResult.Bundle); err != nil {


Was this just because you realized we only need to store the name and not the whole struct?

Yes, the thing we're actually storing is the unpackResult.Bundle, and ext was passed in purely for its identifier.

bentito · 2024-07-11T08:46:24Z

internal/handler/interfaces.go

+	bd "github.com/operator-framework/operator-controller/internal/bundledeployment"
+)
+
+type Handler interface {


Should this maybe become less generic, so RegistryHandler perhaps?

Yes, there's a good argument for that since we're only using one of the 3 types from rukpak.

bentito · 2024-07-11T09:14:05Z

internal/provisioner/plain/plain.go

+	manifestsDir = "manifests"
+)
+
+func HandleBundleDeployment(_ context.Context, fsys fs.FS, _ *bundledeployment.BundleDeployment) (*chart.Chart, chartutil.Values, error) {


Seems like a unit test on this with Bundle in/Chart out would be good. Just in case Helm bumps, etc, shift something on us

Right; and that brings up another larger issue for this PR: Rukpak was kind of infamous for testing with e2e tests instead of unit tests, so we can't bring over a lot of those tests as-is. Personally I think there's probably too much new code in this PR for adding unit tests for all of it and I'd prefer to open a new issue for that if you think that's reasonable.

sounds good.to me. just need an issue for adding rukpakian related unit tests, to do later

Created #1037

bentito · 2024-07-11T09:48:03Z

internal/source/image_registry.go

+	"path/filepath"
+	"strings"
+
+	"github.com/containerd/containerd/archive"


The focus here is just pull in the code, but maybe we should add an issue to make more use of containerd for the future? Seems like containerd has image operations that could simplify TLS and authentication handling, streamline pulling images, and optimize unpacking layers.

internal/storage/localdir.go

dtfranz · 2024-07-11T14:55:18Z

WDYT about dumping everything in under internal/rukpak for now so that it's more obvious in a first pass what came from rukpak and what changed in existing operator-controller code?

I'm very much in agreement with this and your other thought as well. Since I filtered out a lot of the types there's not currently much reason for the abstract types like storage to exist in the codebase.

For now at the very least I'll move everything into the rukpak subdir inside internal.

tmshort · 2024-07-12T13:49:57Z

For now at the very least I'll move everything into the rukpak subdir inside internal.

This is what I originally did for the original integration (before we pulled everything back)

tmshort · 2024-07-12T14:01:55Z

We might possibly want to remove rukpak references from .golangci.yaml, but not critical now. With this it appears that there are no references to the external operator-framework/rukpak repo any more.

tmshort · 2024-07-12T14:22:00Z

Looks as though a lot of the original code was not pulled over (good), and necessary changes made to only support imagev1.

tmshort

/lgtm

tmshort · 2024-07-12T15:21:45Z

@dtfranz, you'll need to rebase due to go.mod changes....

Moves rukpak code imported by operator-controller into the internal/ folder. Signed-off-by: dtfranz <[email protected]>

tmshort · 2024-07-12T15:33:40Z

/lgtm

Moves rukpak code imported by operator-controller into the internal/ folder. Signed-off-by: dtfranz <[email protected]>

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 10, 2024

dtfranz commented Jul 10, 2024

View reviewed changes

dtfranz changed the title ~~Migrate Rukpak~~ 🌱 WIP: Migrate Rukpak Jul 10, 2024

dtfranz force-pushed the migrate-rukpak branch from 6559d95 to 581e4a6 Compare July 10, 2024 22:04

dtfranz force-pushed the migrate-rukpak branch 2 times, most recently from ecacb47 to 58cd0bf Compare July 10, 2024 23:30

bentito reviewed Jul 11, 2024

View reviewed changes

internal/storage/localdir.go Outdated Show resolved Hide resolved

dtfranz force-pushed the migrate-rukpak branch from 58cd0bf to 44e18b9 Compare July 11, 2024 15:37

dtfranz marked this pull request as ready for review July 11, 2024 17:50

dtfranz requested a review from a team as a code owner July 11, 2024 17:50

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 11, 2024

dtfranz changed the title ~~🌱 WIP: Migrate Rukpak~~ 🌱 Migrate Rukpak Jul 11, 2024

dtfranz mentioned this pull request Jul 11, 2024

Add unit test coverage for migrated rukpak packages #1037

Closed

tmshort previously approved these changes Jul 12, 2024

View reviewed changes

Migrate Rukpak

bf85b3c

Moves rukpak code imported by operator-controller into the internal/ folder. Signed-off-by: dtfranz <[email protected]>

dtfranz dismissed tmshort’s stale review via bf85b3c July 12, 2024 15:23

dtfranz force-pushed the migrate-rukpak branch from 44e18b9 to bf85b3c Compare July 12, 2024 15:23

openshift-ci bot assigned tmshort Jul 12, 2024

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jul 12, 2024

tmshort enabled auto-merge July 12, 2024 15:37

tmshort approved these changes Jul 12, 2024

View reviewed changes

tmshort added this pull request to the merge queue Jul 12, 2024

Merged via the queue into operator-framework:main with commit e5470e2 Jul 12, 2024

tmshort mentioned this pull request Jul 12, 2024

⚠ Move all packages from /pkg to /internal #1043

Merged

perdasilva pushed a commit to LalatenduMohanty/operator-controller that referenced this pull request Aug 13, 2024

Migrate Rukpak (operator-framework#1032)

d1e764a

Moves rukpak code imported by operator-controller into the internal/ folder. Signed-off-by: dtfranz <[email protected]>

perdasilva pushed a commit to kevinrizza/operator-controller that referenced this pull request Aug 13, 2024

Migrate Rukpak (operator-framework#1032)

30c8d30

Moves rukpak code imported by operator-controller into the internal/ folder. Signed-off-by: dtfranz <[email protected]>

skattoju mentioned this pull request Sep 9, 2024

🌱 Descriptive Upgrade Constraint Policy constants #1233

Merged

skattoju mentioned this pull request Sep 25, 2024

✨ Remove unpacked status #1309

Merged

4 tasks

🌱 Migrate Rukpak #1032

🌱 Migrate Rukpak #1032

Uh oh!

Conversation

dtfranz commented Jul 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

netlify bot commented Jul 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for olmv1 ready!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dtfranz commented Jul 10, 2024

Uh oh!

codecov bot commented Jul 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

dtfranz commented Jul 10, 2024

Uh oh!

joelanford commented Jul 11, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bentito Jul 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dtfranz commented Jul 11, 2024

Uh oh!

tmshort commented Jul 12, 2024

Uh oh!

tmshort commented Jul 12, 2024

Uh oh!

tmshort commented Jul 12, 2024

Uh oh!

tmshort left a comment

Choose a reason for hiding this comment

Uh oh!

tmshort commented Jul 12, 2024

Uh oh!

tmshort commented Jul 12, 2024

Uh oh!

Uh oh!

dtfranz commented Jul 10, 2024 •

edited

Loading

netlify bot commented Jul 10, 2024 •

edited

Loading

codecov bot commented Jul 10, 2024 •

edited

Loading

bentito Jul 11, 2024 •

edited

Loading