Merge branch 'main' into rhoaeng-2285

Author: KPostOffice

.github/workflows/e2e_tests.yaml

@@ -86,7 +86,6 @@ jobs:
           make setup-e2e
           echo Deploying CodeFlare operator
           IMG="${REGISTRY_ADDRESS}"/codeflare-operator
-          sed -i 's/RayDashboardOAuthEnabled: pointer.Bool(true)/RayDashboardOAuthEnabled: pointer.Bool(false)/' main.go
           make image-push -e IMG="${IMG}"
           make deploy -e IMG="${IMG}" -e ENV="e2e"
           kubectl wait --timeout=120s --for=condition=Available=true deployment -n openshift-operators codeflare-operator-manager
@@ -97,6 +96,9 @@ jobs:
         with:
           user-name: sdk-user
 
+      - name: Add kueue resources
+        run: kubectl apply --server-side -f "https://github.com/kubernetes-sigs/kueue/releases/download/v0.6.2/manifests.yaml"
+
       - name: Configure RBAC for sdk user with limited permissions
         run: |
           kubectl create clusterrole list-ingresses --verb=get,list --resource=ingresses
@@ -105,6 +107,12 @@ jobs:
           kubectl create clusterrolebinding sdk-user-namespace-creator --clusterrole=namespace-creator --user=sdk-user
           kubectl create clusterrole raycluster-creator --verb=get,list,create,delete,patch --resource=rayclusters
           kubectl create clusterrolebinding sdk-user-raycluster-creator --clusterrole=raycluster-creator --user=sdk-user
+          kubectl create clusterrole resourceflavor-creator --verb=get,list,create,delete --resource=resourceflavors
+          kubectl create clusterrolebinding sdk-user-resourceflavor-creator --clusterrole=resourceflavor-creator --user=sdk-user
+          kubectl create clusterrole clusterqueue-creator --verb=get,list,create,delete,patch --resource=clusterqueues
+          kubectl create clusterrolebinding sdk-user-clusterqueue-creator --clusterrole=clusterqueue-creator --user=sdk-user
+          kubectl create clusterrole localqueue-creator --verb=get,list,create,delete,patch --resource=localqueues
+          kubectl create clusterrolebinding sdk-user-localqueue-creator --clusterrole=localqueue-creator --user=sdk-user
           kubectl config use-context sdk-user
 
       - name: Run e2e tests

.github/workflows/odh-notebooks-sync.yml

@@ -28,7 +28,7 @@ env:
 
 jobs:
   build:
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-20.04-4core
     steps:
       - name: Clone repository and Sync
         run: |

.github/workflows/release.yaml

@@ -106,7 +106,7 @@ jobs:
                 gh workflow run odh-notebooks-sync.yml \
                   --repo ${{ github.event.inputs.codeflare-repository-organization }}/codeflare-sdk \
                   --ref ${{ github.ref }} \
-                  --field upstream-repository-organization=opendatahub-io
+                  --field upstream-repository-organization=opendatahub-io \
                   --field codeflare-repository-organization=${{ github.event.inputs.codeflare-repository-organization }} \
                   --field codeflare_sdk_release_version=${{ github.event.inputs.release-version }}
               env:

demo-notebooks/additional-demos/hf_interactive.ipynb

@@ -68,7 +68,7 @@
    "id": "bc27f84c",
    "metadata": {},
    "source": [
-    "Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding AppWrapper).\n",
+    "Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding Ray Cluster).\n",
     "\n",
     "NOTE: We must specify the `image` which will be used in our RayCluster, we recommend you bring your own image which suits your purposes. \n",
     "The example here is a community image."
@@ -89,25 +89,28 @@
     }
    ],
    "source": [
-    "# Create our cluster and submit appwrapper\n",
+    "# Create our cluster and submit\n",
+    "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n",
     "cluster = Cluster(ClusterConfiguration(name='hfgputest', \n",
-    "                                       namespace=\"default\",\n",
+    "                                       namespace=\"default\", # Update to your namespace\n",
     "                                       num_workers=1,\n",
     "                                       min_cpus=8, \n",
     "                                       max_cpus=8, \n",
     "                                       min_memory=16, \n",
     "                                       max_memory=16, \n",
     "                                       num_gpus=4,\n",
     "                                       image=\"quay.io/project-codeflare/ray:latest-py39-cu118\",\n",
-    "                                       instascale=True, machine_types=[\"m5.xlarge\", \"p3.8xlarge\"]))"
+    "                                       write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n",
+    "                                       # local_queue=\"local-queue-name\" # Specify the local queue manually\n",
+    "                                       ))"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "12eef53c",
    "metadata": {},
    "source": [
-    "Next, we want to bring our cluster up, so we call the `up()` function below to submit our cluster AppWrapper yaml onto the MCAD queue, and begin the process of obtaining our resource cluster."
+    "Next, we want to bring our cluster up, so we call the `up()` function below to submit our Ray Cluster onto the queue, and begin the process of obtaining our resource cluster."
    ]
   },
   {

demo-notebooks/additional-demos/local_interactive.ipynb

@@ -48,13 +48,12 @@
    },
    "outputs": [],
    "source": [
-    "# Create our cluster and submit appwrapper\n",
-    "namespace = \"default\"\n",
+    "# Create and submit our cluster\n",
+    "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n",
+    "namespace = \"default\" # Update to your namespace\n",
     "cluster_name = \"hfgputest-1\"\n",
-    "local_interactive = True\n",
     "\n",
-    "cluster = Cluster(ClusterConfiguration(local_interactive=local_interactive,\n",
-    "                                       namespace=namespace,\n",
+    "cluster = Cluster(ClusterConfiguration(namespace=namespace,\n",
     "                                       name=cluster_name,\n",
     "                                       num_workers=1,\n",
     "                                       min_cpus=1,\n",
@@ -63,8 +62,9 @@
     "                                       max_memory=4,\n",
     "                                       num_gpus=0,\n",
     "                                       image=\"quay.io/project-codeflare/ray:latest-py39-cu118\",\n",
-    "                                       instascale=False,\n",
-    "                                       machine_types=[\"m5.xlarge\", \"p3.8xlarge\"]))"
+    "                                       write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n",
+    "                                       # local_queue=\"local-queue-name\" # Specify the local queue manually\n",
+    "                                       ))"
    ]
   },
   {
@@ -117,9 +117,8 @@
    "source": [
     "from codeflare_sdk import generate_cert\n",
     "\n",
-    "if local_interactive:\n",
-    "    generate_cert.generate_tls_cert(cluster_name, namespace)\n",
-    "    generate_cert.export_env(cluster_name, namespace)"
+    "generate_cert.generate_tls_cert(cluster_name, namespace)\n",
+    "generate_cert.export_env(cluster_name, namespace)"
    ]
   },
   {
@@ -339,7 +338,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.17"
+   "version": "3.9.18"
   },
   "vscode": {
    "interpreter": {

demo-notebooks/guided-demos/2_job_client.ipynb renamed to demo-notebooks/additional-demos/ray_job_client.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "In this third demo we will go over the basics of the Ray Job Submission Client in the SDK"
+    "In this demo we will go over the basics of the RayJobClient in the SDK"
    ]
   },
   {
@@ -26,7 +26,6 @@
     "# Create authentication object for user permissions\n",
     "# IF unused, SDK will automatically check for default kubeconfig, then in-cluster config\n",
     "# KubeConfigFileAuthentication can also be used to specify kubeconfig path manually\n",
-    "\n",
     "auth_token = \"XXXXX\" # The auth_token is used later for the RayJobClient\n",
     "auth = TokenAuthentication(\n",
     "    token = auth_token,\n",
@@ -43,16 +42,18 @@
    "outputs": [],
    "source": [
     "# Create and configure our cluster object\n",
+    "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n",
     "cluster = Cluster(ClusterConfiguration(\n",
     "    name='jobtest',\n",
-    "    namespace='default',\n",
+    "    namespace='default', # Update to your namespace\n",
     "    num_workers=2,\n",
     "    min_cpus=1,\n",
     "    max_cpus=1,\n",
     "    min_memory=4,\n",
     "    max_memory=4,\n",
     "    num_gpus=0,\n",
-    "    image=\"quay.io/project-codeflare/ray:latest-py39-cu118\"\n",
+    "    image=\"quay.io/project-codeflare/ray:latest-py39-cu118\",\n",
+    "    write_to_file=False # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n",
     "))"
    ]
   },

demo-notebooks/guided-demos/0_basic_ray.ipynb

@@ -5,7 +5,7 @@
    "id": "8d4a42f6",
    "metadata": {},
    "source": [
-    "In this first notebook, we will go through the basics of using the SDK to:\n",
+    "In this notebook, we will go through the basics of using the SDK to:\n",
     " - Spin up a Ray cluster with our desired resources\n",
     " - View the status and specs of our Ray cluster\n",
     " - Take down the Ray cluster when finished"
@@ -45,7 +45,7 @@
    "id": "bc27f84c",
    "metadata": {},
    "source": [
-    "Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding AppWrapper).\n",
+    "Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n",
     "\n",
     "NOTE: We must specify the `image` which will be used in our RayCluster, we recommend you bring your own image which suits your purposes. \n",
     "The example here is a community image."
@@ -58,18 +58,20 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Create and configure our cluster object (and appwrapper)\n",
+    "# Create and configure our cluster object\n",
+    "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n",
     "cluster = Cluster(ClusterConfiguration(\n",
     "    name='raytest',\n",
-    "    namespace='default',\n",
+    "    namespace='default', # Update to your namespace\n",
     "    num_workers=2,\n",
     "    min_cpus=1,\n",
     "    max_cpus=1,\n",
     "    min_memory=4,\n",
     "    max_memory=4,\n",
     "    num_gpus=0,\n",
     "    image=\"quay.io/project-codeflare/ray:latest-py39-cu118\",\n",
-    "    instascale=False\n",
+    "    write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n",
+    "    # local_queue=\"local-queue-name\" # Specify the local queue manually\n",
     "))"
    ]
   },
@@ -78,7 +80,7 @@
    "id": "12eef53c",
    "metadata": {},
    "source": [
-    "Next, we want to bring our cluster up, so we call the `up()` function below to submit our cluster AppWrapper yaml onto the MCAD queue, and begin the process of obtaining our resource cluster."
+    "Next, we want to bring our cluster up, so we call the `up()` function below to submit our Ray Cluster onto the queue, and begin the process of obtaining our resource cluster."
    ]
   },
   {