You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -26,15 +26,18 @@ The steps in the ginkgo test can be summarized as follows:
26
26
- With the scale of the OLM v1 project, it is useful to have a means to test components in the operator development + lifecycle pipeline together to create a more cohesive experience for all users.
27
27
28
28
## Getting Started
29
-
29
+
- This test currently only works with the container runtime `docker`.
30
30
- Building operator-controller, deploying it into the cluster and rest of the configuration is done in the `MakeFile` of this repo under the target `operator-developer-e2e`. This includes:
31
31
32
32
- Setting up a kind cluster named `operator-controller-op-dev-e2e`.
33
33
- Installing the operator controller onto the cluster.
34
34
- Setting up `opm`, `operator-sdk` and `kustomize` using bingo.
35
35
- Setting up a local registry server for building and loading images.
36
+
### Input Values used
37
+
38
+
Below are the input values used in the test which is specified in the `operator_framework_test.go`.
36
39
37
-
- The following structs defined are required as input for both plain+v0 and registry+v1 bundles:
40
+
- The following structs defined are required, to accept input for both `plain+v0` and `registry+v1` bundles:
38
41
- For getting bundle related inputs:
39
42
```
40
43
type BundleInfo struct {
@@ -48,11 +51,11 @@ The steps in the ginkgo test can be summarized as follows:
48
51
imageRef string
49
52
}
50
53
```
51
-
- `baseFolderPath` - Base/root path of the folder for the specific bundle type input data.[path to plain-v0 or registry-v1 bundles testdata]
54
+
- `baseFolderPath` - Base/root path of the folder where the specific bundle type input data is stored.[root path to plain-v0 or registry-v1 bundles testdata]
52
55
- `bundles` - Stores the data relevant to different versions of the bundle.
53
-
- `bInputDir` - The input directory containing the specific version of the bundle data.
56
+
- `bInputDir` - The directory that stores the specific version of the bundle data. The name of the directory is formed and is of the format `<operatorName>.v<bundleVersion>`.
54
57
- `bundleVersion` - The specific version of the bundle data.
55
-
- `imageRef` - This is formed. Stores the bundle image reference which will be of the format `localhost:5001/<operator_name>-bundle:v.<bundleVersion>`
58
+
- `imageRef` - This is formed. Stores the bundle image reference which will be of the format `<registry_repo>/< operatorName>-bundle:v.<bundleVersion>`
56
59
- For getting catalog related inputs:
57
60
```
58
61
type CatalogDInfo struct {
@@ -64,11 +67,11 @@ The steps in the ginkgo test can be summarized as follows:
64
67
fbcFileName string
65
68
}
66
69
```
67
-
- `baseFolderPath` - Base/root path of the folder for the catalogs.
70
+
- `baseFolderPath` - Base/root path of the folder that stores the catalogs.
68
71
- `operatorName` - Name of the operator to be installed from the bundles.
69
72
- `desiredChannelName` - Desired channel name for the operator.
70
-
- `catalogDir` - This is formed. The directory to store the FBC. The formed value will be of the format: `<operator-name>-catalog`
71
-
- `imageRef` - This is formed. Stores the FBC image reference which will be of the format: `localhost:5001/<username>/<catalogDir>:test`
73
+
- `catalogDir` - This is formed. The directory to store the catalog/FBC. The directory name will be of the format: `<operator-name>-catalog`
74
+
- `imageRef` - This is formed. Stores the FBC image reference which will be of the format: `<registry_repo>/<catalogDir>:test`
72
75
- `fbcFileName` - Name of the FBC file. This is hard-coded as `catalog.yaml`.
73
76
- For getting information related to the install/upgrade action for operators:
74
77
```
@@ -80,9 +83,8 @@ The steps in the ginkgo test can be summarized as follows:
80
83
- `installVersion` - Version of the operator to be installed on the cluster.
81
84
- `upgradeVersion` - Version of the operator to be upgraded on the cluster.
82
85
83
-
### Plain bundles
84
-
- The plain+v0 bundles are formed using `operator-sdk` and `kustomize`.
85
-
- The below input is used to form the bundle using operator-sdk.
86
+
- The below inputs are used to form the bundle using operator-sdk.
87
+
86
88
```
87
89
type SdkProjectInfo struct {
88
90
projectName string
@@ -92,42 +94,109 @@ The steps in the ginkgo test can be summarized as follows:
92
94
kind string
93
95
}
94
96
```
97
+
## How to run
98
+
- Makefile target `operator-developer-e2e` : Runs the entire E2E setup.
99
+
- Makefile target `test-op-dev-e2e`: Runs the ginkgo test.
100
+
- Makefile target `deploy-local-registry`: Deploys local registry
101
+
- Makefile target `cleanup-local-registry` : Stops and removes local registry
102
+
- Makefile target `kind-cluster-cleanup`: Deletes the kind cluster
103
+
104
+
## Bundle Types
105
+
### Plain bundles
106
+
- The `plain+v0` bundles are formed using `operator-sdk` and `kustomize`.
107
+
- The `kustomize` organizes the different resources present in the `operator-sdk` project into a single yaml file.
108
+
- The Dockerfile for the bundle is named `plainbundle.Dockerfile` and is generated using a custom routine.
109
+
- The generated bundle is stored in the format:
110
+
```
111
+
plain-v0
112
+
└── <operatorName>.v<bundleVersion>
113
+
└── manifests
114
+
│ └── mainfest.yaml
115
+
└── plainbundle.Dockerfile
116
+
```
95
117
96
-
- After the bundle image is created and loaded, the FBC is formed by a custom routine by using the operatorName, desiredChannelName, bundle imageRefs and bundleVersions.
97
-
98
-
- The generated FBC will be present in the file path: `testdata/catalogs/<operatorName>-catalog/catalog.yaml` and the Dockerfile will be formed in the file path: `testdata/catalogs/<operatorName>-catalog.Dockerfile`
99
-
- `Default channel` is not used in forming the FBC as it is not an OLMv1 concept.
100
-
- The package schema name will be the operatorName and the bundle name will be `<operatorName>.v<version>`.
101
118
102
-
- The catalog resource is then formed with the name `<operatorName>-catalog`.
119
+
- The FBC template is formed by a custom routine by using the operatorName, desiredChannelName, bundle imageRefs and bundleVersions.
120
+
- `Default channel` is not used in forming the FBC as it is not an OLMv1 concept.
121
+
- Only one `olm.channel` is generated which is the given <desiredChannelName>.
122
+
- Upgrade graph is formed with only replaces edge.
123
+
- The generated FBC is not validated using `opm` as the tool has no support for plain bundles.
124
+
- The Dockerfile for the catalog is named `<operator-name>-catalog.Dockerfile` and is generated using a custom routine.
125
+
- The generated catalog is stored in the format:
126
+
```
127
+
catalogs
128
+
└── <operator-name>-catalog
129
+
│ └── catalog.yaml
130
+
└── <operator-name>-catalog.Dockerfile
131
+
```
132
+
- The catalog CR is then formed with the name `<operatorName>-catalog`.
103
133
104
134
- The operator is then installed and has the name `<operatorName>`.
105
135
106
136
### Registry Bundles
107
137
108
138
- The registry+v1 bundles are formed using operator-sdk.
109
-
- The below input is used to form the bundle using operator-sdk.
139
+
- The generated CSV uses the default values.
140
+
- The bundle content is formed within the operator-sdk project directory in the folder `bundle`. This is moved to the bundle directory folder.
141
+
- The generated Dockerfile has the name `bundle.Dockerfile`. The Dockerfile and bundle structure is genearted by the `operator-sdk` tool.
142
+
- The generated bundle is stored in the format:
110
143
```
111
-
type SdkProjectInfo struct {
112
-
projectName string
113
-
domainName string
114
-
group string
115
-
version string
116
-
kind string
117
-
}
144
+
registry-v1
145
+
└── <operatorName>.v<bundleVersion>
146
+
└── manifests
147
+
└── metadata
148
+
└── bundle.Dockerfile
149
+
```
150
+
151
+
- The FBC is formed using `opm alpha render-template semver` tool.
152
+
- The semver template named `registry-semver.yaml` is formed using a custom routine by passing the bundle imageRefs.
153
+
- `generatemajorchannels` and `generateminorchannels` is set to false in the semver template.
154
+
- The generated catalog is stored in the format:
155
+
```
156
+
catalogs
157
+
└── <operator-name>-catalog
158
+
│ └── catalog.yaml
159
+
└── <operator-name>-catalog.Dockerfile
118
160
```
119
-
- The generated CSV uses the default values.
120
-
- The bundle content is formed in the operator-sdk project directory in the folder `bundle`. This is moved to the folder path: `testdata/bundles/registry-v1/<operatorName>.v<version>`.
121
-
- The generated Dockerfile has the name `bundle.Dockerfile`.
122
-
- The same project directory needs to be updated for forming a different version of the `bundle`.
123
-
- After the bundle image is created and loaded, the FBC is formed using `semver` and `opm` tool.
124
-
- The semver named `registry-semver.yaml` is formed by passing the bundle imageRefs.
125
-
- The generated FBC will be present in the file path: `testdata/catalogs/<operatorName>-catalog/catalog.yaml` and the Dockerfile will be formed in the file path: `testdata/catalogs/<operatorName>-catalog.Dockerfile`
126
-
- The package schema name will be the operator-sdk projectName.
127
161
128
162
- The catalog resource is then formed with the name `<operatorName>-catalog`.
129
163
130
164
- The operator is then installed and has the name `<operatorName>`.
131
165
132
166
133
-
- After the e2e workflow, all the files formed are cleared and the catalog is deleted.
167
+
- After the e2e workflow, all the files formed are cleared.
168
+
169
+
170
+
## To-do
171
+
1. The resources are read from input manifests using universal decoder from `k8s.io/apimachinery/pkg/runtime/serializer`.
172
+
- However, in cases where a single file contains multiple YAML documents separated by `---,` the `UniversalDecoder` recognizes only the first resource. This situation is relevant as for `plain+v0` bundles generated through `kustomize,` the manifest has multiple YAML documents are combined into one file using --- separators. This is now handled by splitting the content of the YAML file and decoding each of them using the `UniversalDecoder`.
173
+
- This workaround can be improved using `YAMLToJSONDecoder` from `k8s.io/apimachinery/pkg/util/yaml`. And the kind, api version and name can be get by decoding into `Unstructured` from `k8s.io/apimachinery/pkg/apis/meta/v1/unstructured`.
174
+
2. All the tests pass and the operator is installed successfully. The bundledeployment succeeds and the resources are created. But the pod for the new operator failes due to `ImagePullBackOff`.
175
+
- This is because the `Deployment` controller-manager uses the image `controller:latest` which is not present on the cluster.
176
+
- The solution would be to replace `controller:latest` with the `busybox:latest` and then pulling and loading `busybox:latest` onto cluster.
177
+
- The replacement could possibly be achieved by adding the following to `config/default/kustomization.yaml` under `operator-sdk` project:
178
+
```
179
+
images:
180
+
- name: controller
181
+
newName: controller
182
+
newTag: latest
183
+
```
184
+
## Issues
185
+
1. This test currently only works with the container runtime `docker`.
186
+
- The default value of CONTAINER_RUNTIME defined in the Makefile is `docker`. Therefore the correct runtime has to be assigned to the variable `CONTAINER_RUNTIME` before calling the make target if docker is what is not being used. The test routine also assumes the runtime as `docker` if it is unable to retrieve the value of the environment variable.
187
+
- But this is only a partial fix to the problem. With this change the test for `plain+v0` bundles will pass but for `registry+v1` will fail for other container runtimes. This is because `registry+v1` uses `operator-sdk` support. Thus to mimic the user experience, the targets `bundle-build` and `bundle-push` from the generated Makefile by operator-sdk tool, which has docker being hard coded as the container runtime, is used to build and push the bundle images. This could be marked as an issue and addressed when hard coding docker as container runtime in the generated Makefile is addressed by operator-sdk.
188
+
189
+
2. The `opm`,`operator-sdk` and `kustomize` binaries are added in operator-controller using `bingo`.
190
+
- But based on discussions, if required test should be changed so that it has `opm` and `operator-sdk` in `PATH` and simply runs it like `exec.Command("opm", ...)`.
191
+
- This will enable in running [a matrix](https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs) for the tests and to use different versions of `opm` and `operator-sdk`.
192
+
- This might help in emulating the user experience better.
193
+
194
+
## Tooling gaps
195
+
196
+
Following are the tooling gaps identified while testing `operator-framework` end-to-end:
197
+
- `opm` doesn't have plain bundle support.
198
+
- No tool for forming FBC for plain bundles.
199
+
- No tool for generating Dockerfile for plain bundles.
200
+
- No tool for generating Dockerfile for plain catalogs.
201
+
- Since `opm` doesn't have plain bundle support, there is no means to validate the FBC generated for plain bundles.
0 commit comments