Doc update for dedicated registry

product_docs/docs/edb-postgres-ai/1.3/hybrid-manager/system/synced-images.mdx

@@ -0,0 +1,116 @@
+# Sync EDB Postgres AI Platform container images into a customer owned registry
+
+The software stack of our EDB PGAI is pushed into EDB Cloudsmith registry to provide artifacts that our customers will be able to use.
+
+A requirement to use our stack will be that customers hosts their own secure and approved internal registry (in case of EKS as a platform to run PGAI that can be an ECR in the same AWS account, or can be another kind of supported container registry) and knowing the EDB PGAI version that we want to install, we can take all the artifacts from Cloudsmith and sync them internally into the local registry before installing or upgrading the software stack with the helm chart.
+
+The sync process needs to preserve the container images SHA256 to ensure images security and immutability across different environments. You can do the sync using `edbctl`, the CLI to manage PGAI resources, or by running a tool like [skopeo](https://github.com/containers/skopeo), that you can install referring to [their official docs](https://github.com/containers/skopeo/blob/main/install.md).
+
+!!! note
+    If the local registry is AWS ECR, since we want all the EDB repositories to stay under a single namespace (see related AWS docs [here](https://docs.aws.amazon.com/AmazonECR/latest/userguide/Repositories.html#repository-concepts)), we would need to create multiple repositories in the registry to allow the image copy to work, because ECR doesn’t support images with multiple slashes in their name to be saved in the same repository.
+
+## Using `edbctl` - Suggested
+
+!!! note
+    `edbctl` is still in development and we don't have yet released binaries, you will need to build it by yourself, see [here](https://github.com/EnterpriseDB/upm-beaconator-cli?tab=readme-ov-file#build-and-run-locally).
+
+```bash
+# building binary
+$ make build
+
+# Configure the EDB PGAI release to be taken
+export EDBPGAI_RELEASE=<RELEASE_VERSION>
+# Configure the EDB Cloudsmith access token
+export CS_EDB_TOKEN=<CS_EDB_TOKEN_REDACTED>
+# Configure the EDB Cloudsmith registry source
+export EDB_SOURCE_REGISTRY=pgai-platform
+# Run the sync-to-local-registry command
+build/edbctl image sync-to-local-registry \
+    --destination-registry-uri "<LOCAL_REGISTRY_URI>" \
+    --version "${EDBPGAI_RELEASE}" \
+    --source-registry-username "${EDB_SOURCE_REGISTRY}" \
+    --source-registry-password "${CS_EDB_TOKEN}" \
+    --destination-registry-username "<LOCAL_REGISTRY_USER>" \
+    --destination-registry-password "<LOCAL_REGISTRY_PWD>"
+```
+
+!!! note
+    Starting with EDB PGAI version 1.3.0, syncing the EDB PGAI Operator image to your local registry is a required step.
+
+```bash
+# Sync the EDB PGAI Operator Image to the destination registry:
+build/edbctl operator sync-to-local-registry \
+    --destination-registry-uri "<LOCAL_REGISTRY_URI>" \
+    --version "${EDBPGAI_RELEASE}" \
+    --source-registry-username "${EDB_SOURCE_REGISTRY}" \
+    --source-registry-password "${CS_EDB_TOKEN}" \
+    --destination-registry-username "<LOCAL_REGISTRY_USER>" \
+    --destination-registry-password "<LOCAL_REGISTRY_PWD>"
+```
+
+When you run the above command `edbctl image sync-to-local-registry` with a <LOCAL_REGISTRY_URI> that is AWS ECR, the CLI will ask a confirmation before proceed with they sync process and will provide a code snippet with a list of AWS CLI commands that can be used to pre-create all the repositories that ECR requires to successfully complete the sync process.
+
+## Using `skopeo`
+
+Every EDB PGAI release provides an artifact that contains the list of all the container images that are required to install/upgrade the software stack, and can be used to run a sync process to copy over all these container images from the EDB Cloudsmith registry to an internal one.
+
+The following snippet can run on Bash on Linux/MacOS/Windows WSL
+
+```bash
+# Configure the EDB PGAI release to be taken
+export EDBPGAI_RELEASE=<RELEASE_VERSION>
+# Configure the EDB Cloudsmith access token
+export CS_EDB_TOKEN=<CS_EDB_TOKEN_REDACTED>
+# Downloading the image list artifact locally
+curl -sLO "https://downloads.enterprisedb.com/${CS_EDB_TOKEN}/pgai-platform/raw/names/${EDBPGAI_RELEASE}-images.txt/versions/${EDBPGAI_RELEASE}/images.txt"
+# Configure the EDB Cloudsmith registry source
+export EDB_SOURCE_REGISTRY=docker.enterprisedb.com/pgai-platform
+# Configure the local registry destination
+export LOCAL_REGISTRY_URI=<LOCAL_REGISTRY_ADDRESS>
+# skopeo login to the source registry, provide credentials as requested
+skopeo login docker.enterprisedb.com
+# skopeo login to the destination registry, provide credentials as requested
+skopeo login <LOCAL_REGISTRY_ADDRESS>
+# Parsing the image list and syncing every image
+while read -r image; do skopeo --override-os linux copy --multi-arch all docker://$EDB_SOURCE_REGISTRY/${image/:*@/@} docker://$LOCAL_REGISTRY_URI/${image/:*@/@} --retry-times 3; done < images.txt
+```
+
+!!! note
+    Starting with EDB PGAI version 1.3.0, syncing the EDB PGAI Operator image to your local registry is a required step.
+
+```bash
+# Sync the EDB PGAI Operator Image to the destination registry:
+skopeo --override-os linux copy \
+    --multi-arch all \
+    docker://${EDB_SOURCE_REGISTRY}/edb-hcp-operator/manager:${EDBPGAI_RELEASE} \
+    docker://${LOCAL_REGISTRY_URI}/edb-hcp-operator/manager:${EDBPGAI_RELEASE} \
+    --retry-times 3
+```
+
+This is a sample run that shows an output result of the previous commands, using AWS ECR as a destination registry:
+
+```bash
+$ export EDBPGAI_RELEASE=v1.0.0-gm-appl
+$ export CS_EDB_TOKEN=<CS_EDB_TOKEN_REDACTED>
+$ export AWS_ACCOUNT_ID=123456789012 # sample value, replace with the correct one
+$ curl -sLO "https://downloads.enterprisedb.com/${CS_EDB_TOKEN}/pgai-platform/raw/names/${EDBPGAI_RELEASE}-images.txt/versions/${EDBPGAI_RELEASE}/images.txt"
+$ wc -l images.txt # shows how many images are in the release
+132 images.txt
+$ export EDB_SOURCE_REGISTRY=docker.enterprisedb.com/pgai-platform
+$ export LOCAL_REGISTRY_URI=${AWS_ACCOUNT_ID}.dkr.ecr.us-east-1.amazonaws.com/edbpgai-test-ecr
+$ skopeo login docker.enterprisedb.com
+Username: <REDACTED>
+Password:
+Login Succeeded!
+$ skopeo login ${AWS_ACCOUNT_ID}.dkr.ecr.us-east-1.amazonaws.com -u AWS -p $(aws ecr get-login-password --region us-east-1)
+Login Succeeded!
+# WE CAN IGNORE AWS RepositoryAlreadyExistsException WHILE RUNNING aws ecr create-repository
+$ while read -r image; do aws ecr create-repository --repository-name "${LOCAL_REGISTRY_URI#*/}/${image%%[:@]*}" --no-cli-pager || true; skopeo --override-os linux copy --multi-arch all docker://$EDB_SOURCE_REGISTRY/${image/:*@/@} docker://$LOCAL_REGISTRY_URI/${image/:*@/@} --retry-times 3; done < images.txt
+...the sync process will take quite a few minutes to copy the full set of images...
+# CHECKING RESULTS OF THE IMAGE SYNC
+$ aws ecr describe-repositories --query 'repositories[?starts_with(repositoryName, `edbpgai-test-ecr`)]' --output json | jq '. | length'
+93
+$ cat images.txt | awk -F'[:@]' '{print $1}' | sort -u | wc -l
+93
+# SINGLE IMAGE AND REPOS ARE MATCHING
+```

product_docs/docs/edb-postgres-ai/1.3/hybrid-manager/system/what-to-expect.mdx

@@ -13,13 +13,15 @@ After your order is confirmed, you'll complete a site readiness survey to guide
 - Power, rack, and cabling details
 - Networking and security preferences
 - Physical access and contact coordination
+- Dedicated image registry 
 
 ### What happens
 
 - Supermicro ships the full system to your selected data center.
 - EDB and Supermicro teams perform on-site racking, power-up, and validation.
 - Configuration is completed based on your preferences.
 - You receive login credentials and URL access to the Hybrid Manager portal.
+- You will have a dedicated image registry that syncs with EDB's production registry to pull all required Hybrid Manager artifacts.  More details [here](./synced-images.mdx)
 
 ## Days 21–28: Deploying your first workloads
 

product_docs/docs/edb-postgres-ai/1.3/hybrid-manager/using_hybrid_manager/cluster_management/create-clusters/cluster-settings.mdx

@@ -39,6 +39,9 @@ You set these options in the [**Data Groups**](data-groups.mdx) tab for other cl
 
 **Instance Size** — Select the number of CPUs and the amount of memory for your cluster. The number of CPUs and the amount of memory you can select depends on available resources in your Kubernetes cluster.
 
+!!! Note
+    If you **do not** have a dedicated registry setup and is relying directly on EDB production registry, you may run into issues where the system allows you to setup the cluster with operands incompatible with the installed version Hybrid Manager.  In this case, you need to pay attenion to the meta data displayed while selecting an image to ensure it's compatible.  <Add a screen shot and detailed explanation>.  To counter this issue, EDB recommends setting up a customer dedidated registry as described [here](../../../system/synced-images.mdx)
+
 ### Storage
 
 You can specify the following storage settings:

product_docs/docs/edb-postgres-ai/1.3/hybrid-manager/using_hybrid_manager/upgrading/pg_major/upgrade_major.mdx

@@ -20,6 +20,9 @@ Now check the cluster and database metrics and make sure everything is operating
 
 1.  Select your preferred new major version image.
 
+!!! Note
+    If you **do not** have a dedicated registry setup and is relying directly on EDB production registry, you may run into issues where the system allows you to setup the cluster with operands incompatible with the installed version Hybrid Manager.  In this case, you need to pay attenion to the meta data displayed while selecting an image to ensure it's compatible.  <Add a screen shot and detailed explanation>.  To counter this issue, EDB recommends setting up a customer dedidated registry as described [here](../../../system/synced-images.mdx)
+
 1.  Review the upgrade path and confirm by selecting the **Continue** button.
 
 1. The dialogue window now shows the specific package changes to be expected with the upgrade.

product_docs/docs/edb-postgres-ai/preview/hybrid-manager/system/synced-images.mdx

@@ -0,0 +1,116 @@
+# Sync EDB Postgres AI Platform container images into a customer owned registry
+
+The software stack of our EDB PGAI is pushed into EDB Cloudsmith registry to provide artifacts that our customers will be able to use.
+
+A requirement to use our stack will be that customers hosts their own secure and approved internal registry (in case of EKS as a platform to run PGAI that can be an ECR in the same AWS account, or can be another kind of supported container registry) and knowing the EDB PGAI version that we want to install, we can take all the artifacts from Cloudsmith and sync them internally into the local registry before installing or upgrading the software stack with the helm chart.
+
+The sync process needs to preserve the container images SHA256 to ensure images security and immutability across different environments. You can do the sync using `edbctl`, the CLI to manage PGAI resources, or by running a tool like [skopeo](https://github.com/containers/skopeo), that you can install referring to [their official docs](https://github.com/containers/skopeo/blob/main/install.md).
+
+!!! note
+    If the local registry is AWS ECR, since we want all the EDB repositories to stay under a single namespace (see related AWS docs [here](https://docs.aws.amazon.com/AmazonECR/latest/userguide/Repositories.html#repository-concepts)), we would need to create multiple repositories in the registry to allow the image copy to work, because ECR doesn’t support images with multiple slashes in their name to be saved in the same repository.
+
+## Using `edbctl` - Suggested
+
+!!! note
+    `edbctl` is still in development and we don't have yet released binaries, you will need to build it by yourself, see [here](https://github.com/EnterpriseDB/upm-beaconator-cli?tab=readme-ov-file#build-and-run-locally).
+
+```bash
+# building binary
+$ make build
+
+# Configure the EDB PGAI release to be taken
+export EDBPGAI_RELEASE=<RELEASE_VERSION>
+# Configure the EDB Cloudsmith access token
+export CS_EDB_TOKEN=<CS_EDB_TOKEN_REDACTED>
+# Configure the EDB Cloudsmith registry source
+export EDB_SOURCE_REGISTRY=pgai-platform
+# Run the sync-to-local-registry command
+build/edbctl image sync-to-local-registry \
+    --destination-registry-uri "<LOCAL_REGISTRY_URI>" \
+    --version "${EDBPGAI_RELEASE}" \
+    --source-registry-username "${EDB_SOURCE_REGISTRY}" \
+    --source-registry-password "${CS_EDB_TOKEN}" \
+    --destination-registry-username "<LOCAL_REGISTRY_USER>" \
+    --destination-registry-password "<LOCAL_REGISTRY_PWD>"
+```
+
+!!! note
+    Starting with EDB PGAI version 1.3.0, syncing the EDB PGAI Operator image to your local registry is a required step.
+
+```bash
+# Sync the EDB PGAI Operator Image to the destination registry:
+build/edbctl operator sync-to-local-registry \
+    --destination-registry-uri "<LOCAL_REGISTRY_URI>" \
+    --version "${EDBPGAI_RELEASE}" \
+    --source-registry-username "${EDB_SOURCE_REGISTRY}" \
+    --source-registry-password "${CS_EDB_TOKEN}" \
+    --destination-registry-username "<LOCAL_REGISTRY_USER>" \
+    --destination-registry-password "<LOCAL_REGISTRY_PWD>"
+```
+
+When you run the above command `edbctl image sync-to-local-registry` with a <LOCAL_REGISTRY_URI> that is AWS ECR, the CLI will ask a confirmation before proceed with they sync process and will provide a code snippet with a list of AWS CLI commands that can be used to pre-create all the repositories that ECR requires to successfully complete the sync process.
+
+## Using `skopeo`
+
+Every EDB PGAI release provides an artifact that contains the list of all the container images that are required to install/upgrade the software stack, and can be used to run a sync process to copy over all these container images from the EDB Cloudsmith registry to an internal one.
+
+The following snippet can run on Bash on Linux/MacOS/Windows WSL
+
+```bash
+# Configure the EDB PGAI release to be taken
+export EDBPGAI_RELEASE=<RELEASE_VERSION>
+# Configure the EDB Cloudsmith access token
+export CS_EDB_TOKEN=<CS_EDB_TOKEN_REDACTED>
+# Downloading the image list artifact locally
+curl -sLO "https://downloads.enterprisedb.com/${CS_EDB_TOKEN}/pgai-platform/raw/names/${EDBPGAI_RELEASE}-images.txt/versions/${EDBPGAI_RELEASE}/images.txt"
+# Configure the EDB Cloudsmith registry source
+export EDB_SOURCE_REGISTRY=docker.enterprisedb.com/pgai-platform
+# Configure the local registry destination
+export LOCAL_REGISTRY_URI=<LOCAL_REGISTRY_ADDRESS>
+# skopeo login to the source registry, provide credentials as requested
+skopeo login docker.enterprisedb.com
+# skopeo login to the destination registry, provide credentials as requested
+skopeo login <LOCAL_REGISTRY_ADDRESS>
+# Parsing the image list and syncing every image
+while read -r image; do skopeo --override-os linux copy --multi-arch all docker://$EDB_SOURCE_REGISTRY/${image/:*@/@} docker://$LOCAL_REGISTRY_URI/${image/:*@/@} --retry-times 3; done < images.txt
+```
+
+!!! note
+    Starting with EDB PGAI version 1.3.0, syncing the EDB PGAI Operator image to your local registry is a required step.
+
+```bash
+# Sync the EDB PGAI Operator Image to the destination registry:
+skopeo --override-os linux copy \
+    --multi-arch all \
+    docker://${EDB_SOURCE_REGISTRY}/edb-hcp-operator/manager:${EDBPGAI_RELEASE} \
+    docker://${LOCAL_REGISTRY_URI}/edb-hcp-operator/manager:${EDBPGAI_RELEASE} \
+    --retry-times 3
+```
+
+This is a sample run that shows an output result of the previous commands, using AWS ECR as a destination registry:
+
+```bash
+$ export EDBPGAI_RELEASE=v1.0.0-gm-appl
+$ export CS_EDB_TOKEN=<CS_EDB_TOKEN_REDACTED>
+$ export AWS_ACCOUNT_ID=123456789012 # sample value, replace with the correct one
+$ curl -sLO "https://downloads.enterprisedb.com/${CS_EDB_TOKEN}/pgai-platform/raw/names/${EDBPGAI_RELEASE}-images.txt/versions/${EDBPGAI_RELEASE}/images.txt"
+$ wc -l images.txt # shows how many images are in the release
+132 images.txt
+$ export EDB_SOURCE_REGISTRY=docker.enterprisedb.com/pgai-platform
+$ export LOCAL_REGISTRY_URI=${AWS_ACCOUNT_ID}.dkr.ecr.us-east-1.amazonaws.com/edbpgai-test-ecr
+$ skopeo login docker.enterprisedb.com
+Username: <REDACTED>
+Password:
+Login Succeeded!
+$ skopeo login ${AWS_ACCOUNT_ID}.dkr.ecr.us-east-1.amazonaws.com -u AWS -p $(aws ecr get-login-password --region us-east-1)
+Login Succeeded!
+# WE CAN IGNORE AWS RepositoryAlreadyExistsException WHILE RUNNING aws ecr create-repository
+$ while read -r image; do aws ecr create-repository --repository-name "${LOCAL_REGISTRY_URI#*/}/${image%%[:@]*}" --no-cli-pager || true; skopeo --override-os linux copy --multi-arch all docker://$EDB_SOURCE_REGISTRY/${image/:*@/@} docker://$LOCAL_REGISTRY_URI/${image/:*@/@} --retry-times 3; done < images.txt
+...the sync process will take quite a few minutes to copy the full set of images...
+# CHECKING RESULTS OF THE IMAGE SYNC
+$ aws ecr describe-repositories --query 'repositories[?starts_with(repositoryName, `edbpgai-test-ecr`)]' --output json | jq '. | length'
+93
+$ cat images.txt | awk -F'[:@]' '{print $1}' | sort -u | wc -l
+93
+# SINGLE IMAGE AND REPOS ARE MATCHING
+```

product_docs/docs/edb-postgres-ai/preview/hybrid-manager/system/what-to-expect.mdx

@@ -13,13 +13,15 @@ After your order is confirmed, you'll complete a site readiness survey to guide
 - Power, rack, and cabling details
 - Networking and security preferences
 - Physical access and contact coordination
+- Dedicated image registry 
 
 ### What happens
 
 - Supermicro ships the full system to your selected data center.
 - EDB and Supermicro teams perform on-site racking, power-up, and validation.
 - Configuration is completed based on your preferences.
 - You receive login credentials and URL access to the Hybrid Manager portal.
+- You will have a dedicated image registry that syncs with EDB's production registry to pull all required Hybrid Manager artifacts.  More details [here](./synced-images.mdx)
 
 ## Days 21–28: Deploying your first workloads
 

product_docs/docs/edb-postgres-ai/preview/hybrid-manager/using_hybrid_manager/cluster_management/create-clusters/cluster-settings.mdx

@@ -37,6 +37,9 @@ You set these options in the [**Data Groups**](data-groups.mdx) tab for other cl
 
 3.  HM supports multiple images of each Postgres database, stored in the image library. Each image is a configuration of the database that includes various extensions. Select the image that you want to use for your cluster. See [Asset library](../../image-management/asset-library.mdx) for more information. Generally, without `-full` at the end of the name, the image has no extensions. With `-full` at the end of the name, the image has all available extensions.
 
+!!! Note
+    If you **do not** have a dedicated registry setup and is relying directly on EDB production registry, you may run into issues where the system allows you to setup the cluster with operands incompatible with the installed version Hybrid Manager.  In this case, you need to pay attenion to the meta data displayed while selecting an image to ensure it's compatible.  <Add a screen shot and detailed explanation>.  To counter this issue, EDB recommends setting up a customer dedidated registry as described [here](../../../system/synced-images.mdx)
+
 **Instance Size** — Select the number of CPUs and the amount of memory for your cluster. The number of CPUs and the amount of memory you can select depends on available resources in your Kubernetes cluster.
 
 ### Storage

product_docs/docs/edb-postgres-ai/preview/hybrid-manager/using_hybrid_manager/upgrading/pg_major/upgrade_major.mdx

@@ -20,6 +20,9 @@ Now check the cluster and database metrics and make sure everything is operating
 
 1.  Select your preferred new major version image.
 
+!!! Note
+    If you **do not** have a dedicated registry setup and is relying directly on EDB production registry, you may run into issues where the system allows you to setup the cluster with operands incompatible with the installed version Hybrid Manager.  In this case, you need to pay attenion to the meta data displayed while selecting an image to ensure it's compatible.  <Add a screen shot and detailed explanation>.  To counter this issue, EDB recommends setting up a customer dedidated registry as described [here](../../../system/synced-images.mdx)
+
 1.  Review the upgrade path and confirm by selecting the **Continue** button.
 
 1. The dialogue window now shows the specific package changes to be expected with the upgrade.