Skip to content

CI: Refactor sdist workflow and add setup-python action #46538

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions .github/actions/setup-python/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Setup Python and install requirements
inputs:
python-version:
required: true
architecture:
default: x64
runs:
using: composite
steps:
# TODO: GH#44980 https://github.com/pypa/setuptools/issues/2941
- name: Create temporary requirements.txt
run: |
# Drop cache at least once per month
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why would we cache this at all?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it saves a couple of minutes but we can also add it later.

month_today="$(date '+%Y-%m')"
cat > requirements.txt ci/deps/pip-requirements.txt - <<< "# $month_today"
shell: bash -el {0}

- name: Set up Python
uses: actions/setup-python@v3
with:
python-version: ${{ inputs.python-version }}
architecture: ${{ inputs.architecture }}
cache: pip

- name: Fix $PATH on macOS
run: |
# On macOS, the Python version we installed above is too late in $PATH
# to be effective if using "bash -l" (which we need for code that works
# with Conda envs).
cat >> ~/.bash_profile <<EOF
export PATH="\$(echo "\$PATH" | grep -Eio '[^:]+hostedtoolcache/python[^:]+bin'):\$PATH" \
EOF
shell: bash -el {0}
if: ${{ runner.os == 'macOS' }}

- name: Install dependencies
run: |
cat requirements.txt
# TODO https://github.com/numpy/numpy/issues/21196
bits32=$(python -c 'import sys; print(int(sys.maxsize > 2**32))')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

umm what is this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is required for 32 bit builds but we can add that back later

NPY_DISABLE_SVML=$bits32 pip install -r requirements.txt
pip list
shell: bash -el {0}
10 changes: 6 additions & 4 deletions .github/workflows/sdist.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ on:
- "doc/**"

jobs:
build:
if: ${{ github.event.label.name == 'Build' || contains(github.event.pull_request.labels.*.name, 'Build') || github.event_name == 'push'}}
sdist:
#if: ${{ github.event.label.name == 'Build' || contains(github.event.pull_request.labels.*.name, 'Build') || github.event_name == 'push'}}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to undo this once GHA is green.

runs-on: ubuntu-latest
timeout-minutes: 60
defaults:
Expand All @@ -26,6 +26,8 @@ jobs:
fail-fast: false
matrix:
python-version: ["3.8", "3.9", "3.10"]
name: sdist Python ${{ matrix.python-version }}

concurrency:
# https://github.community/t/concurrecy-not-work-for-push/183068/7
group: ${{ github.event_name == 'push' && github.run_number || github.ref }}-${{matrix.python-version}}-sdist
Expand All @@ -36,8 +38,8 @@ jobs:
with:
fetch-depth: 0

- name: Set up Python
uses: actions/setup-python@v3
- name: Set up Python ${{ matrix.python-version }} and install dependencies
uses: ./.github/actions/setup-python
with:
python-version: ${{ matrix.python-version }}

Expand Down
21 changes: 21 additions & 0 deletions ci/deps/pip-requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Requirements for pip-based CI environments (eg., sdist and python-dev)

# Python deps
pip
setuptools<60.0.0
wheel

# Pandas deps
# GH 39416
numpy
cython
python-dateutil
pytz

# Test deps
git+https://github.com/nedbat/coveragepy.git
hypothesis
pytest>=6.2.5
pytest-xdist
pytest-cov
pytest-asyncio>=0.17