Skip to content

test(e2e): Add framework for canary testing #5935

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Oct 13, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/CANARY_FAILURE_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
title: Canary tests failed
labels: 'Status: Untriaged, Type: Tests'
---
Canary tests failed: {{ env.RUN_LINK }}
54 changes: 54 additions & 0 deletions .github/workflows/canary.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
name: 'Canary Tests'
on:
schedule:
# Run every day at midnight
- cron: '0 0 * * *'
workflow_dispatch:
inputs:
commit:
description: If the commit you want to test isn't the head of a branch, provide its SHA here
required: false

env:
HEAD_COMMIT: ${{ github.event.inputs.commit || github.sha }}

permissions:
contents: read
issues: write

jobs:
job_canary_test:
name: Run Canary Tests
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: 'Check out current commit'
uses: actions/checkout@v2
with:
ref: ${{ env.HEAD_COMMIT }}
- name: Set up Node
uses: actions/setup-node@v3
with:
# ember won't build under node 16, at least not with the versions of the ember build tools we use
node-version: 14
- name: Install dependencies
run: yarn install --ignore-engines --frozen-lockfile
- name: Build packages
run: yarn build
- name: Run Canary Tests
env:
E2E_TEST_PUBLISH_SCRIPT_NODE_VERSION: ${{ env.DEFAULT_NODE_VERSION }}
E2E_TEST_AUTH_TOKEN: ${{ secrets.E2E_TEST_AUTH_TOKEN }}
E2E_TEST_DSN: ${{ secrets.E2E_TEST_DSN }}
CANARY_E2E_TEST: 'yes'
run: |
cd packages/e2e-tests
yarn test:e2e
- name: Create Issue
if: failure()
uses: JasonEtco/create-an-issue@1a16035489d05041b9af40b970f02e301c52ffba
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RUN_LINK: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
with:
filename: .github/CANARY_FAILURE_TEMPLATE.md
Loading