Skip to content

Commit baf4658

Browse files
committed
UI visual regression testing to cover UI widgets visibility
1 parent 1235fc8 commit baf4658

24 files changed

+4350
-2
lines changed
+110
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
name: UI notebooks tests
2+
3+
on: [pull_request]
4+
5+
concurrency:
6+
group: ${{ github.head_ref }}-${{ github.workflow }}
7+
cancel-in-progress: true
8+
9+
env:
10+
CODEFLARE_OPERATOR_IMG: "quay.io/project-codeflare/codeflare-operator:dev"
11+
12+
jobs:
13+
verify-0_basic_ray:
14+
runs-on: ubuntu-20.04-4core
15+
16+
steps:
17+
- name: Checkout code
18+
uses: actions/checkout@v4
19+
with:
20+
submodules: recursive
21+
22+
- name: Checkout common repo code
23+
uses: actions/checkout@v4
24+
with:
25+
repository: "project-codeflare/codeflare-common"
26+
ref: "main"
27+
path: "common"
28+
29+
- name: Checkout CodeFlare operator repository
30+
uses: actions/checkout@v4
31+
with:
32+
repository: project-codeflare/codeflare-operator
33+
path: codeflare-operator
34+
35+
- name: Set Go
36+
uses: actions/setup-go@v5
37+
with:
38+
go-version-file: "./codeflare-operator/go.mod"
39+
cache-dependency-path: "./codeflare-operator/go.sum"
40+
41+
- name: Set up gotestfmt
42+
uses: gotesttools/gotestfmt-action@v2
43+
with:
44+
token: ${{ secrets.GITHUB_TOKEN }}
45+
46+
- name: Set up specific Python version
47+
uses: actions/setup-python@v5
48+
with:
49+
python-version: "3.9"
50+
cache: "pip" # caching pip dependencies
51+
52+
- name: Setup and start KinD cluster
53+
uses: ./common/github-actions/kind
54+
55+
- name: Deploy CodeFlare stack
56+
id: deploy
57+
run: |
58+
cd codeflare-operator
59+
echo Setting up CodeFlare stack
60+
make setup-e2e
61+
echo Deploying CodeFlare operator
62+
make deploy -e IMG="${CODEFLARE_OPERATOR_IMG}" -e ENV="e2e"
63+
kubectl wait --timeout=120s --for=condition=Available=true deployment -n openshift-operators codeflare-operator-manager
64+
cd ..
65+
66+
- name: Setup Guided notebooks execution
67+
run: |
68+
echo "Installing papermill and dependencies..."
69+
pip install poetry ipython ipykernel
70+
poetry config virtualenvs.create false
71+
echo "Installing SDK..."
72+
poetry install --with test,docs
73+
74+
- name: Install Yarn dependencies
75+
run: |
76+
poetry run yarn install
77+
poetry run yarn playwright install chromium
78+
working-directory: ui-tests
79+
80+
- name: Fix widget_notebook_example.ipynb notebook for test
81+
run: |
82+
# Remove login/logout cells, as KinD doesn't support authentication using token
83+
jq -r 'del(.cells[] | select(.source[] | contains("Create authentication object for user permissions")))' widget_notebook_example.ipynb > 0_basic_ray.ipynb.tmp && mv 0_basic_ray.ipynb.tmp 0_basic_ray.ipynb
84+
jq -r 'del(.cells[] | select(.source[] | contains("auth.logout()")))' widget_notebook_example.ipynb > widget_notebook_example.ipynb.tmp && mv 0_basic_ray.ipynb.tmp 0_basic_ray.ipynb
85+
# Set explicit namespace as SDK need it (currently) to resolve local queues
86+
sed -i "s/head_memory_limits=2,/head_memory_limits=2, namespace='default',/" widget_notebook_example.ipynb
87+
working-directory: demo-notebooks/guided-demos
88+
89+
- name: Run UI notebook tests
90+
run: |
91+
set -euo pipefail
92+
93+
poetry run yarn test
94+
working-directory: ui-tests
95+
96+
- name: Upload Playwright Test assets
97+
if: always()
98+
uses: actions/upload-artifact@v4
99+
with:
100+
name: ipywidgets-test-assets
101+
path: |
102+
ui-tests/test-results
103+
104+
- name: Upload Playwright Test report
105+
if: always()
106+
uses: actions/upload-artifact@v4
107+
with:
108+
name: ipywidgets-test-report
109+
path: |
110+
ui-tests/playwright-report

.gitignore

+4
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,7 @@ Pipfile.lock
88
build/
99
tls-cluster-namespace
1010
quicktest.yaml
11+
node_modules
12+
.DS_Store
13+
ui-tests/playwright-report
14+
ui-tests/test-results
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,212 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "8d4a42f6",
6+
"metadata": {},
7+
"source": [
8+
"In this notebook, we will go through the basics of using the SDK to:\n",
9+
" - Spin up a Ray cluster with our desired resources\n",
10+
" - View the status and specs of our Ray cluster\n",
11+
" - Take down the Ray cluster when finished"
12+
]
13+
},
14+
{
15+
"cell_type": "code",
16+
"execution_count": null,
17+
"id": "b55bc3ea-4ce3-49bf-bb1f-e209de8ca47a",
18+
"metadata": {},
19+
"outputs": [],
20+
"source": [
21+
"# Import pieces from codeflare-sdk\n",
22+
"from codeflare_sdk import Cluster, ClusterConfiguration, TokenAuthentication"
23+
]
24+
},
25+
{
26+
"cell_type": "code",
27+
"execution_count": null,
28+
"id": "614daa0c",
29+
"metadata": {},
30+
"outputs": [],
31+
"source": [
32+
"# Create authentication object for user permissions\n",
33+
"# IF unused, SDK will automatically check for default kubeconfig, then in-cluster config\n",
34+
"# KubeConfigFileAuthentication can also be used to specify kubeconfig path manually\n",
35+
"auth = TokenAuthentication(\n",
36+
" token = \"XXXXX\",\n",
37+
" server = \"XXXXX\",\n",
38+
" skip_tls=False\n",
39+
")\n",
40+
"auth.login()"
41+
]
42+
},
43+
{
44+
"cell_type": "markdown",
45+
"id": "bc27f84c",
46+
"metadata": {},
47+
"source": [
48+
"Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n",
49+
"\n",
50+
"NOTE: 'quay.io/modh/ray:2.35.0-py39-cu121' is the default image used by the CodeFlare SDK for creating a RayCluster resource. \n",
51+
"If you have your own Ray image which suits your purposes, specify it in image field to override the default image."
52+
]
53+
},
54+
{
55+
"cell_type": "code",
56+
"execution_count": null,
57+
"id": "0f4bc870-091f-4e11-9642-cba145710159",
58+
"metadata": {},
59+
"outputs": [],
60+
"source": [
61+
"# Create and configure our cluster object\n",
62+
"# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n",
63+
"cluster = Cluster(ClusterConfiguration(\n",
64+
" name='raytest', \n",
65+
" head_cpu_requests='500m',\n",
66+
" head_cpu_limits='500m',\n",
67+
" head_memory_requests=2,\n",
68+
" head_memory_limits=2,\n",
69+
" head_extended_resource_requests={'nvidia.com/gpu':0}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n",
70+
" worker_extended_resource_requests={'nvidia.com/gpu':0},\n",
71+
" num_workers=2,\n",
72+
" worker_cpu_requests='250m',\n",
73+
" worker_cpu_limits=1,\n",
74+
" worker_memory_requests=4,\n",
75+
" worker_memory_limits=4,\n",
76+
" # image=\"\", # Optional Field \n",
77+
" write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n",
78+
" # local_queue=\"local-queue-name\" # Specify the local queue manually\n",
79+
"))"
80+
]
81+
},
82+
{
83+
"cell_type": "markdown",
84+
"id": "12eef53c",
85+
"metadata": {},
86+
"source": [
87+
"Next, we want to bring our cluster up, so we call the `up()` function below to submit our Ray Cluster onto the queue, and begin the process of obtaining our resource cluster."
88+
]
89+
},
90+
{
91+
"cell_type": "code",
92+
"execution_count": null,
93+
"id": "f0884bbc-c224-4ca0-98a0-02dfa09c2200",
94+
"metadata": {},
95+
"outputs": [],
96+
"source": [
97+
"# Bring up the cluster\n",
98+
"cluster.up()"
99+
]
100+
},
101+
{
102+
"cell_type": "markdown",
103+
"id": "657ebdfb",
104+
"metadata": {},
105+
"source": [
106+
"Now, we want to check on the status of our resource cluster, and wait until it is finally ready for use."
107+
]
108+
},
109+
{
110+
"cell_type": "code",
111+
"execution_count": null,
112+
"id": "3c1b4311-2e61-44c9-8225-87c2db11363d",
113+
"metadata": {},
114+
"outputs": [],
115+
"source": [
116+
"cluster.status()"
117+
]
118+
},
119+
{
120+
"cell_type": "code",
121+
"execution_count": null,
122+
"id": "a99d5aff",
123+
"metadata": {},
124+
"outputs": [],
125+
"source": [
126+
"cluster.wait_ready()"
127+
]
128+
},
129+
{
130+
"cell_type": "code",
131+
"execution_count": null,
132+
"id": "df71c1ed",
133+
"metadata": {},
134+
"outputs": [],
135+
"source": [
136+
"cluster.status()"
137+
]
138+
},
139+
{
140+
"cell_type": "markdown",
141+
"id": "b3a55fe4",
142+
"metadata": {},
143+
"source": [
144+
"Let's quickly verify that the specs of the cluster are as expected."
145+
]
146+
},
147+
{
148+
"cell_type": "code",
149+
"execution_count": null,
150+
"id": "7fd45bc5-03c0-4ae5-9ec5-dd1c30f1a084",
151+
"metadata": {},
152+
"outputs": [],
153+
"source": [
154+
"cluster.details()"
155+
]
156+
},
157+
{
158+
"cell_type": "markdown",
159+
"id": "5af8cd32",
160+
"metadata": {},
161+
"source": [
162+
"Finally, we bring our resource cluster down and release/terminate the associated resources, bringing everything back to the way it was before our cluster was brought up."
163+
]
164+
},
165+
{
166+
"cell_type": "code",
167+
"execution_count": null,
168+
"id": "5f36db0f-31f6-4373-9503-dc3c1c4c3f57",
169+
"metadata": {},
170+
"outputs": [],
171+
"source": [
172+
"cluster.down()"
173+
]
174+
},
175+
{
176+
"cell_type": "code",
177+
"execution_count": null,
178+
"id": "0d41b90e",
179+
"metadata": {},
180+
"outputs": [],
181+
"source": [
182+
"auth.logout()"
183+
]
184+
}
185+
],
186+
"metadata": {
187+
"kernelspec": {
188+
"display_name": "Python 3 (ipykernel)",
189+
"language": "python",
190+
"name": "python3"
191+
},
192+
"language_info": {
193+
"codemirror_mode": {
194+
"name": "ipython",
195+
"version": 3
196+
},
197+
"file_extension": ".py",
198+
"mimetype": "text/x-python",
199+
"name": "python",
200+
"nbconvert_exporter": "python",
201+
"pygments_lexer": "ipython3",
202+
"version": "3.9.19"
203+
},
204+
"vscode": {
205+
"interpreter": {
206+
"hash": "f9f85f796d01129d0dd105a088854619f454435301f6ffec2fea96ecbd9be4ac"
207+
}
208+
}
209+
},
210+
"nbformat": 4,
211+
"nbformat_minor": 5
212+
}

0 commit comments

Comments
 (0)