diff --git a/docs/draft/api-reference/catalogd-webserver-metas-endpoint.md b/docs/draft/api-reference/catalogd-webserver-metas-endpoint.md new file mode 100644 index 000000000..6b27ba27e --- /dev/null +++ b/docs/draft/api-reference/catalogd-webserver-metas-endpoint.md @@ -0,0 +1,111 @@ +# Catalogd web server + +[Catalogd](https://github.com/operator-framework/operator-controller/tree/main/catalogd), the OLM v1 component for making catalog contents available on cluster, includes +a web server that serves catalog contents to clients via HTTP(S) endpoints. + +The endpoints to retrieve information about installable clusterextentions can be composed from the `.status.urls.base` of a `ClusterCatalog` resource with the selected access API path. + +Currently, there are two API endpoints: + +1. `api/v1/all` endpoint that provides access to the FBC metadata in entirety. + +As an example, to access the full FBC via the v1 API endpoint (indicated by path `api/v1/all`) where `.status.urls.base` is + +```yaml + urls: + base: https://catalogd-service.olmv1-system.svc/catalogs/operatorhubio +``` + +the URL to access the service would be `https://catalogd-service.olmv1-system.svc/catalogs/operatorhubio/api/v1/all` + +2. `api/v1/metas` endpoint that allows clients to retrieve filtered portions of the FBC. + +The metas endpoint accepts parameters which are one of the sub-types of the `Meta` [definition](https://github.com/operator-framework/operator-registry/blob/e15668c933c03e229b6c80025fdadb040ab834e0/alpha/declcfg/declcfg.go#L111-L114), following the pattern `/api/v1/metas?[&...]`. + +As an example, to access only the [package schema](https://olm.operatorframework.io/docs/reference/file-based-catalogs/#olmpackage-1) blobs of the FBC via the `api/v1/metas` endpoint where `.status.urls.base` is + +```yaml + urls: + base: https://catalogd-service.olmv1-system.svc/catalogs/operatorhubio +``` + +the URL to access the service would be `https://catalogd-service.olmv1-system.svc/catalogs/operatorhubio/api/v1/metas?schema=olm.package` + +For more examples of valid queries that can be made to the `api/v1/metas` service endpoint, please see [Catalog Queries](../howto/catalog-queries.md). + +!!! note + + The values of the `.status.urls` field in a `ClusterCatalog` resource are arbitrary string values and can change at any time. + While there are no guarantees on the exact value of this field, it will always contain catalog-specific API endpoints for use + by clients to make a request from within the cluster. + +## Interacting With the Server + +### Supported HTTP Methods + +The HTTP request methods supported by the catalogd web server are: + +- [GET](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/GET) +- [HEAD](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/HEAD) + +### Response Format + +Responses are encoded as a [JSON Lines](https://jsonlines.org/) stream of [File-Based Catalog](https://olm.operatorframework.io/docs/reference/file-based-catalogs) (FBC) [Meta](https://olm.operatorframework.io/docs/reference/file-based-catalogs/#schema) objects delimited by newlines. + +??? example "Example JSON-encoded FBC snippet" + + ```json + { + "schema": "olm.package", + "name": "cockroachdb", + "defaultChannel": "stable-v6.x", + } + { + "schema": "olm.channel", + "name": "stable-v6.x", + "package": "cockroachdb", + "entries": [ + { + "name": "cockroachdb.v6.0.0", + "skipRange": "<6.0.0" + } + ] + } + { + "schema": "olm.bundle", + "name": "cockroachdb.v6.0.0", + "package": "cockroachdb", + "image": "quay.io/openshift-community-operators/cockroachdb@sha256:d3016b1507515fc7712f9c47fd9082baf9ccb070aaab58ed0ef6e5abdedde8ba", + "properties": [ + { + "type": "olm.package", + "value": { + "packageName": "cockroachdb", + "version": "6.0.0" + } + }, + ], + } + ``` + + Corresponding JSON lines response: + ```jsonlines + {"schema":"olm.package","name":"cockroachdb","defaultChannel":"stable-v6.x"} + {"schema":"olm.channel","name":"stable-v6.x","package":"cockroachdb","entries":[{"name":"cockroachdb.v6.0.0","skipRange":"<6.0.0"}]} + {"schema":"olm.bundle","name":"cockroachdb.v6.0.0","package":"cockroachdb","image":"quay.io/openshift-community-operators/cockroachdb@sha256:d3016b1507515fc7712f9c47fd9082baf9ccb070aaab58ed0ef6e5abdedde8ba","properties":[{"type":"olm.package","value":{"packageName":"cockroachdb","version":"6.0.0"}}]} + ``` + +### Compression Support + +The `catalogd` web server supports gzip compression of responses, which can significantly reduce associated network traffic. In order to signal that the client handles compressed responses, the client must include `Accept-Encoding: gzip` as a header in the HTTP request. + +The web server will include a `Content-Encoding: gzip` header in compressed responses. + +!!! note + + Only catalogs whose uncompressed response body would result in a response size greater than 1400 bytes will be compressed. + +### Cache Header Support + +For clients interested in caching the information returned from the `catalogd` web server, the `Last-Modified` header is set +on responses and the `If-Modified-Since` header is supported for requests. diff --git a/docs/draft/howto/catalog-queries-metas-endpoint.md b/docs/draft/howto/catalog-queries-metas-endpoint.md new file mode 100644 index 000000000..f723d504b --- /dev/null +++ b/docs/draft/howto/catalog-queries-metas-endpoint.md @@ -0,0 +1,93 @@ +# Catalog queries + +After you [add a catalog of extensions](../tutorials/add-catalog.md) to your cluster, you must port forward your catalog as a service. +Then you can query the catalog by using `curl` commands and the `jq` CLI tool to find extensions to install. + +## Prerequisites + +* You have added a ClusterCatalog of extensions, such as [OperatorHub.io](https://operatorhub.io), to your cluster. +* You have installed the `jq` CLI tool. + +!!! note + By default, Catalogd is installed with TLS enabled for the catalog webserver. + The following examples will show this default behavior, but for simplicity's sake will ignore TLS verification in the curl commands using the `-k` flag. + +!!! note + While using the `/api/v1/metas` endpoint shown in the below examples, it is important to note that the metas endpoint accepts parameters which are one of the sub-types of the `Meta` [definition](https://github.com/operator-framework/operator-registry/blob/e15668c933c03e229b6c80025fdadb040ab834e0/alpha/declcfg/declcfg.go#L111-L114), following the pattern `/api/v1/metas?[&...]`. e.g. `schema=&package=`, `schema=&name=`, and `package=&name=` are all valid parameter combinations. However `schema=&version=` is not a valid parameter combination, since version is not a first class FBC meta field. + +You also need to port forward the catalog server service: + +``` terminal +kubectl -n olmv1-system port-forward svc/catalogd-service 8443:443 +``` + +Now you can use the `curl` command with `jq` to query catalogs that are installed on your cluster. + +## Package queries + +* Available packages in a catalog: + ``` terminal + curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.package' + ``` + +* Packages that support `AllNamespaces` install mode and do not use webhooks: + ``` terminal + jq -cs '[.[] | select(.schema == "olm.bundle" and (.properties[] | select(.type == "olm.csv.metadata").value.installModes[] | select(.type == "AllNamespaces" and .supported == true)) and .spec.webhookdefinitions == null) | .package] | unique[]' + ``` + +* Package metadata: + ``` terminal + curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.package&name=' + ``` + + `` + : Name of the package from the catalog you are querying. + +* Blobs that belong to a package (that are not schema=olm.package): + ``` terminal + curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?package=' + ``` + + `` + : Name of the package from the catalog you are querying. + +Note: the `olm.package` schema blob does not have the `package` field set. In other words, to get all the blobs that belong to a package, along with the olm.package blob for that package, a combination of both of the above queries need to be used. + +## Channel queries + +* Channels in a package: + ``` terminal + curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.channel&package=' + ``` + + `` + : Name of the package from the catalog you are querying. + +* Versions in a channel: + ``` terminal + curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.channel&package=zoperator&name=alpha' | jq -s '.[] | .entries | .[] | .name' + ``` + + `` + : Name of the package from the catalog you are querying. + + `` + : Name of the channel for a given package. + +## Bundle queries + +* Bundles in a package: + ``` terminal + curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.bundle&package=' + ``` + + `` + : Name of the package from the catalog you are querying. + +* Bundle dependencies and available APIs: + ``` terminal + curl -k 'https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.bundle&name=' | jq -s '.[] | .properties[] | select(.type=="olm.gvk")' + ``` + + `` + : Name of the bundle for a given package. diff --git a/docs/draft/tutorials/explore-available-content-metas-endpoint.md b/docs/draft/tutorials/explore-available-content-metas-endpoint.md new file mode 100644 index 000000000..8ece0a75d --- /dev/null +++ b/docs/draft/tutorials/explore-available-content-metas-endpoint.md @@ -0,0 +1,147 @@ +--- +hide: + - toc +--- + +# Explore Available Content + +After you [add a catalog of extensions](add-catalog.md) to your cluster, you must port forward your catalog as a service. +Then you can query the catalog by using `curl` commands and the `jq` CLI tool to find extensions to install. + +## Prerequisites + +* You have added a ClusterCatalog of extensions, such as [OperatorHub.io](https://operatorhub.io), to your cluster. +* You have installed the `jq` CLI tool. + +!!! note + By default, Catalogd is installed with TLS enabled for the catalog webserver. + The following examples will show this default behavior, but for simplicity's sake will ignore TLS verification in the curl commands using the `-k` flag. + +## Procedure + +1. Port forward the catalog server service: + + ``` terminal + kubectl -n olmv1-system port-forward svc/catalogd-service 8443:443 + ``` + +2. Return a list of all the extensions in a catalog via the v1 API endpoint: + ``` terminal + curl -k https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.package' | jq -s '.[] | .name' + ``` + + ??? success + ``` text title="Example output" + "ack-acm-controller" + "ack-acmpca-controller" + "ack-apigatewayv2-controller" + "ack-applicationautoscaling-controller" + "ack-cloudfront-controller" + "ack-cloudtrail-controller" + "ack-cloudwatch-controller" + "ack-cloudwatchlogs-controller" + "ack-dynamodb-controller" + "ack-ec2-controller" + "ack-ecr-controller" + "ack-ecs-controller" + "ack-efs-controller" + "ack-eks-controller" + "ack-elasticache-controller" + "ack-emrcontainers-controller" + "ack-eventbridge-controller" + "ack-iam-controller" + "ack-kafka-controller" + "ack-keyspaces-controller" + "ack-kinesis-controller" + "ack-kms-controller" + "ack-lambda-controller" + "ack-memorydb-controller" + "ack-mq-controller" + "ack-networkfirewall-controller" + "ack-opensearchservice-controller" + "ack-pipes-controller" + "ack-prometheusservice-controller" + "ack-rds-controller" + "ack-route53-controller" + "ack-route53resolver-controller" + "ack-s3-controller" + "ack-sagemaker-controller" + "ack-secretsmanager-controller" + "ack-sfn-controller" + "ack-sns-controller" + "ack-sqs-controller" + "aerospike-kubernetes-operator" + "airflow-helm-operator" + "aiven-operator" + "akka-cluster-operator" + "alvearie-imaging-ingestion" + "anchore-engine" + "apch-operator" + "api-operator" + "api-testing-operator" + "apicast-community-operator" + "apicurio-registry" + "apimatic-kubernetes-operator" + "app-director-operator" + "appdynamics-operator" + "application-services-metering-operator" + "appranix" + "aqua" + "argocd-operator" + ... + ``` + + !!! important + Currently, OLM 1.0 does not support the installation of extensions that use webhooks or that target a single or specified set of namespaces. + +3. Return list of packages which support `AllNamespaces` install mode, do not use webhooks, and where the channel head version uses `olm.csv.metadata` format: + + ``` terminal + curl -k https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.bundle | jq -cs '[.[] | select(.properties[] | select(.type == "olm.csv.metadata").value.installModes[] | select(.type == "AllNamespaces" and .supported == true) and .spec.webhookdefinitions == null) | .package] | unique[]' + ``` + + ??? success + ``` text title="Example output" + "ack-acm-controller" + "ack-acmpca-controller" + "ack-apigateway-controller" + "ack-apigatewayv2-controller" + "ack-applicationautoscaling-controller" + "ack-athena-controller" + "ack-cloudfront-controller" + "ack-cloudtrail-controller" + "ack-cloudwatch-controller" + "ack-cloudwatchlogs-controller" + "ack-documentdb-controller" + "ack-dynamodb-controller" + "ack-ec2-controller" + "ack-ecr-controller" + "ack-ecs-controller" + ... + ``` + +4. Inspect the contents of an extension's metadata: + + ``` terminal + curl -k https://localhost:8443/catalogs/operatorhubio/api/v1/metas?schema=olm.package&name= + ``` + + `package_name` + : Specifies the name of the package you want to inspect. + + ??? success + ``` text title="Example output" + { + "defaultChannel": "stable-v6.x", + "icon": { + "base64data": "PHN2ZyB4bWxucz0ia... + "mediatype": "image/svg+xml" + }, + "name": "cockroachdb", + "schema": "olm.package" + } + ``` + +### Additional resources + +* [Catalog queries](../howto/catalog-queries.md)