Skip to content

Switch Nova wheel builds to OIDC #4865

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jan 11, 2024
Merged

Switch Nova wheel builds to OIDC #4865

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e428f91
Switch Nova Linux wheel build to OIDC
huydhn Jan 10, 2024
066b1ab
Remove debug message
huydhn Jan 10, 2024
8357a5f
Use gha_workflow_build_wheels from D52679407
huydhn Jan 11, 2024
eca89f3
Also apply the same role to MacOS and Win wheels
huydhn Jan 11, 2024
9f6108b
Split into roles for nightly and test
huydhn Jan 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 18 additions & 3 deletions .github/workflows/build_wheels_linux.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ on:
required: false
type: boolean
default: true
# TODO (huydhn): Remove them once all libraries using Nova has removed them
secrets:
AWS_PYTORCH_UPLOADER_ACCESS_KEY_ID:
description: "AWS Access Key passed from caller workflow"
Expand All @@ -73,6 +74,10 @@ on:
description: "AWS Secret Access Ket passed from caller workflow"
required: false

permissions:
id-token: write
contents: read

jobs:
build:
strategy:
Expand Down Expand Up @@ -230,13 +235,23 @@ jobs:
echo "${{ inputs.repository }}/${SMOKE_TEST_SCRIPT} found"
${CONDA_RUN} python "${{ inputs.repository }}/${SMOKE_TEST_SCRIPT}"
fi
# TODO (huydhn): Move the following step to a separate build job
- name: Configure aws credentials (pytorch account)
if: ${{ inputs.trigger-event == 'push' && startsWith(github.event.ref, 'refs/heads/nightly') }}
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::749337293305:role/gha_workflow_nightly_build_wheels
aws-region: us-east-1
- name: Configure aws credentials (pytorch account)
if: ${{ env.CHANNEL == 'test' && startsWith(github.event.ref, 'refs/tags/') }}
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::749337293305:role/gha_workflow_test_build_wheels
aws-region: us-east-1
- name: Upload package to pytorch.org
if: ${{ (inputs.trigger-event == 'push' && startsWith(github.event.ref, 'refs/heads/nightly')) || (env.CHANNEL == 'test' && startsWith(github.event.ref, 'refs/tags/')) }}
shell: bash -l {0}
working-directory: ${{ inputs.repository }}
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_PYTORCH_UPLOADER_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_PYTORCH_UPLOADER_SECRET_ACCESS_KEY }}
run: |
set -euxo pipefail
source "${BUILD_ENV_FILE}"
Expand Down
21 changes: 18 additions & 3 deletions .github/workflows/build_wheels_macos.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ on:
description: "The key created when saving a cache and the key used to search for a cache."
default: ""
type: string
# TODO (huydhn): Remove them once all libraries using Nova has removed them
secrets:
AWS_PYTORCH_UPLOADER_ACCESS_KEY_ID:
description: "AWS Access Key passed from caller workflow"
Expand All @@ -67,6 +68,10 @@ on:
description: "AWS Secret Access Ket passed from caller workflow"
required: false

permissions:
id-token: write
contents: read

jobs:
build:
strategy:
Expand Down Expand Up @@ -204,13 +209,23 @@ jobs:
${CONDA_RUN} python3 "${{ inputs.repository }}/${SMOKE_TEST_SCRIPT}"
fi
export PATH=${OLD_PATH}
# TODO (huydhn): Move the following step to a separate build job
- name: Configure aws credentials (pytorch account)
if: ${{ inputs.trigger-event == 'push' && startsWith(github.event.ref, 'refs/heads/nightly') }}
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::749337293305:role/gha_workflow_nightly_build_wheels
aws-region: us-east-1
- name: Configure aws credentials (pytorch account)
if: ${{ env.CHANNEL == 'test' && startsWith(github.event.ref, 'refs/tags/') }}
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::749337293305:role/gha_workflow_test_build_wheels
aws-region: us-east-1
- name: Upload package to pytorch.org
if: ${{ (inputs.trigger-event == 'push' && startsWith(github.event.ref, 'refs/heads/nightly')) || (env.CHANNEL == 'test' && startsWith(github.event.ref, 'refs/tags/')) }}
shell: bash -l {0}
working-directory: ${{ inputs.repository }}
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_PYTORCH_UPLOADER_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_PYTORCH_UPLOADER_SECRET_ACCESS_KEY }}
run: |
set -euxo pipefail
# shellcheck disable=SC1090
Expand Down
21 changes: 18 additions & 3 deletions .github/workflows/build_wheels_windows.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ on:
description: "The key created when saving a cache and the key used to search for a cache."
default: ""
type: string
# TODO (huydhn): Remove them once all libraries using Nova has removed them
secrets:
AWS_PYTORCH_UPLOADER_ACCESS_KEY_ID:
description: "AWS Access Key passed from caller workflow"
Expand All @@ -67,6 +68,10 @@ on:
description: "AWS Secret Access Ket passed from caller workflow"
required: false

permissions:
id-token: write
contents: read

jobs:
build:
strategy:
Expand Down Expand Up @@ -194,13 +199,23 @@ jobs:
echo "${{ inputs.repository }}/${SMOKE_TEST_SCRIPT} found"
${CONDA_RUN} python "${{ inputs.repository }}/${SMOKE_TEST_SCRIPT}"
fi
# TODO (huydhn): Move the following step to a separate build job
- name: Configure aws credentials (pytorch account)
if: ${{ inputs.trigger-event == 'push' && startsWith(github.event.ref, 'refs/heads/nightly') }}
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::749337293305:role/gha_workflow_nightly_build_wheels
aws-region: us-east-1
- name: Configure aws credentials (pytorch account)
if: ${{ env.CHANNEL == 'test' && startsWith(github.event.ref, 'refs/tags/') }}
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::749337293305:role/gha_workflow_test_build_wheels
aws-region: us-east-1
- name: Upload package to pytorch.org
if: ${{ (inputs.trigger-event == 'push' && startsWith(github.event.ref, 'refs/heads/nightly')) || (env.CHANNEL == 'test' && startsWith(github.event.ref, 'refs/tags/')) }}
shell: bash -l {0}
working-directory: ${{ inputs.repository }}
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_PYTORCH_UPLOADER_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_PYTORCH_UPLOADER_SECRET_ACCESS_KEY }}
run: |
source "${BUILD_ENV_FILE}"
${CONDA_RUN} pip install awscli
Expand Down