Skip to content

Commit 6bd276c

Browse files
dcheriandcherian
committed
Merge branch 'main' into groupby-refactor-init
* main: pass kwargs through from save_mfdataset to to_netcdf (pydata#6686) Docs: indexing.rst finetuning (pydata#6685) use micromamba instead of mamba (pydata#6674) install the development version of `matplotlib` into the upstream-dev CI (pydata#6675) Add whatsnew section for v2022.06.0 release notes for 2022.06.0rc0 release notes for the pre-release (pydata#6676) more testpypi workflow fixes (pydata#6673) thin: add examples (pydata#6663) Update multidimensional-coords.ipynb (pydata#6672) try to finally fix the TestPyPI workflow (pydata#6671) pin setuptools in the modify script (pydata#6669) fix the python version for the TestPyPI workflow (pydata#6668) upload wheels from `main` to TestPyPI (pydata#6660) Set keep_attrs for flox (pydata#6667)
2 parents cac2249 + 89b1fac commit 6bd276c

File tree

16 files changed

+413
-95
lines changed

16 files changed

+413
-95
lines changed

‎.github/workflows/ci-additional.yaml

Lines changed: 47 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -39,22 +39,29 @@ jobs:
3939
run:
4040
shell: bash -l {0}
4141

42+
env:
43+
CONDA_ENV_FILE: ci/requirements/environment.yml
44+
PYTHON_VERSION: "3.10"
45+
4246
steps:
4347
- uses: actions/checkout@v3
4448
with:
4549
fetch-depth: 0 # Fetch all history for all branches and tags.
46-
- uses: conda-incubator/setup-miniconda@v2
47-
with:
48-
channels: conda-forge
49-
channel-priority: strict
50-
mamba-version: "*"
51-
activate-environment: xarray-tests
52-
auto-update-conda: false
53-
python-version: "3.9"
5450

55-
- name: Install conda dependencies
51+
- name: set environment variables
5652
run: |
57-
mamba env update -f ci/requirements/environment.yml
53+
echo "TODAY=$(date +'%Y-%m-%d')" >> $GITHUB_ENV
54+
55+
- name: Setup micromamba
56+
uses: mamba-org/provision-with-micromamba@de032af7fb3675649f3d4bbdda85178ba412ee41
57+
with:
58+
environment-file: ${{env.CONDA_ENV_FILE}}
59+
environment-name: xarray-tests
60+
extra-specs: |
61+
python=${{env.PYTHON_VERSION}}
62+
cache-env: true
63+
cache-env-key: "${{runner.os}}-${{runner.arch}}-py${{env.PYTHON_VERSION}}-${{env.TODAY}}-${{hashFiles(env.CONDA_ENV_FILE)}}"
64+
5865
- name: Install xarray
5966
run: |
6067
python -m pip install --no-deps -e .
@@ -76,23 +83,27 @@ jobs:
7683
defaults:
7784
run:
7885
shell: bash -l {0}
86+
env:
87+
CONDA_ENV_FILE: ci/requirements/environment.yml
88+
PYTHON_VERSION: "3.10"
7989

8090
steps:
8191
- uses: actions/checkout@v3
8292
with:
8393
fetch-depth: 0 # Fetch all history for all branches and tags.
84-
- uses: conda-incubator/setup-miniconda@v2
85-
with:
86-
channels: conda-forge
87-
channel-priority: strict
88-
mamba-version: "*"
89-
activate-environment: xarray-tests
90-
auto-update-conda: false
91-
python-version: "3.9"
9294

93-
- name: Install conda dependencies
95+
- name: set environment variables
9496
run: |
95-
mamba env update -f ci/requirements/environment.yml
97+
echo "TODAY=$(date +'%Y-%m-%d')" >> $GITHUB_ENV
98+
- name: Setup micromamba
99+
uses: mamba-org/provision-with-micromamba@de032af7fb3675649f3d4bbdda85178ba412ee41
100+
with:
101+
environment-file: ${{env.CONDA_ENV_FILE}}
102+
environment-name: xarray-tests
103+
extra-specs: |
104+
python=${{env.PYTHON_VERSION}}
105+
cache-env: true
106+
cache-env-key: "${{runner.os}}-${{runner.arch}}-py${{env.PYTHON_VERSION}}-${{env.TODAY}}-${{hashFiles(env.CONDA_ENV_FILE)}}"
96107
- name: Install xarray
97108
run: |
98109
python -m pip install --no-deps -e .
@@ -118,20 +129,28 @@ jobs:
118129
run:
119130
shell: bash -l {0}
120131

132+
strategy:
133+
matrix:
134+
environment-file: ["bare-minimum", "min-all-deps"]
135+
fail-fast: false
136+
121137
steps:
122138
- uses: actions/checkout@v3
123139
with:
124140
fetch-depth: 0 # Fetch all history for all branches and tags.
125-
- uses: conda-incubator/setup-miniconda@v2
141+
142+
- name: Setup micromamba
143+
uses: mamba-org/provision-with-micromamba@de032af7fb3675649f3d4bbdda85178ba412ee41
126144
with:
145+
environment-name: xarray-tests
146+
environment-file: false
147+
extra-specs: |
148+
python=3.10
149+
pyyaml
150+
conda
151+
python-dateutil
127152
channels: conda-forge
128-
channel-priority: strict
129-
mamba-version: "*"
130-
auto-update-conda: false
131-
python-version: "3.9"
132153

133154
- name: minimum versions policy
134155
run: |
135-
mamba install -y pyyaml conda python-dateutil
136-
python ci/min_deps_check.py ci/requirements/bare-minimum.yml
137-
python ci/min_deps_check.py ci/requirements/min-all-deps.yml
156+
python ci/min_deps_check.py ci/requirements/${{ matrix.environment-file }}.yml

‎.github/workflows/ci.yaml

Lines changed: 8 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,8 @@ jobs:
6565
fetch-depth: 0 # Fetch all history for all branches and tags.
6666
- name: Set environment variables
6767
run: |
68+
echo "TODAY=$(date +'%Y-%m-%d')" >> $GITHUB_ENV
69+
6870
if [[ ${{ matrix.os }} == windows* ]] ;
6971
then
7072
echo "CONDA_ENV_FILE=ci/requirements/environment-windows.yml" >> $GITHUB_ENV
@@ -83,25 +85,13 @@ jobs:
8385
8486
echo "PYTHON_VERSION=${{ matrix.python-version }}" >> $GITHUB_ENV
8587
86-
- name: Cache conda
87-
uses: actions/cache@v3
88-
with:
89-
path: ~/conda_pkgs_dir
90-
key: ${{ runner.os }}-conda-py${{ matrix.python-version }}-${{ hashFiles('ci/requirements/**.yml') }}-${{ matrix.env }}
91-
92-
- uses: conda-incubator/setup-miniconda@v2
88+
- name: Setup micromamba
89+
uses: mamba-org/provision-with-micromamba@de032af7fb3675649f3d4bbdda85178ba412ee41
9390
with:
94-
channels: conda-forge
95-
channel-priority: strict
96-
mamba-version: "*"
97-
activate-environment: xarray-tests
98-
auto-update-conda: false
99-
python-version: ${{ matrix.python-version }}
100-
use-only-tar-bz2: true
101-
102-
- name: Install conda dependencies
103-
run: |
104-
mamba env update -f $CONDA_ENV_FILE
91+
environment-file: ${{ env.CONDA_ENV_FILE }}
92+
environment-name: xarray-tests
93+
cache-env: true
94+
cache-env-key: "${{runner.os}}-${{runner.arch}}-py${{matrix.python-version}}-${{env.TODAY}}-${{hashFiles(env.CONDA_ENV_FILE)}}"
10595

10696
# We only want to install this on one run, because otherwise we'll have
10797
# duplicate annotations.

‎.github/workflows/configure-testpypi-version.py

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
import argparse
2+
import copy
3+
import pathlib
4+
5+
import tomli
6+
import tomli_w
7+
8+
9+
def split_path(path, sep="/"):
10+
if isinstance(path, str):
11+
return [part for part in path.split(sep) if part]
12+
else:
13+
return path
14+
15+
16+
def extract(mapping, path, sep="/"):
17+
parts = split_path(path, sep=sep)
18+
cur = mapping
19+
for part in parts:
20+
cur = cur[part]
21+
22+
return cur
23+
24+
25+
def update(mapping, path, value, sep="/"):
26+
new = copy.deepcopy(mapping)
27+
28+
parts = split_path(path, sep=sep)
29+
parent = extract(new, parts[:-1])
30+
parent[parts[-1]] = value
31+
32+
return new
33+
34+
35+
parser = argparse.ArgumentParser()
36+
parser.add_argument("path", type=pathlib.Path)
37+
args = parser.parse_args()
38+
39+
content = args.path.read_text()
40+
decoded = tomli.loads(content)
41+
with_local_scheme = update(
42+
decoded, "tool.setuptools_scm.local_scheme", "no-local-version", sep="."
43+
)
44+
# work around a bug in setuptools / setuptools-scm
45+
with_setuptools_pin = copy.deepcopy(with_local_scheme)
46+
requires = extract(with_setuptools_pin, "build-system.requires", sep=".")
47+
requires[0] = "setuptools>=42,<60"
48+
49+
new_content = tomli_w.dumps(with_setuptools_pin)
50+
args.path.write_text(new_content)

‎.github/workflows/testpypi-release.yaml

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
name: Build and Upload xarray to PyPI
2+
on:
3+
push:
4+
branches:
5+
- 'main'
6+
7+
# no need for concurrency limits
8+
9+
jobs:
10+
build-artifacts:
11+
runs-on: ubuntu-latest
12+
if: github.repository == 'pydata/xarray'
13+
steps:
14+
- uses: actions/checkout@v3
15+
with:
16+
fetch-depth: 0
17+
18+
- uses: actions/setup-python@v3
19+
name: Install Python
20+
with:
21+
python-version: "3.10"
22+
23+
- name: Install dependencies
24+
run: |
25+
python -m pip install --upgrade pip
26+
python -m pip install build setuptools setuptools-scm wheel twine check-manifest
27+
python -m pip install tomli tomli_w
28+
29+
- name: Disable local versions
30+
run: |
31+
python .github/workflows/configure-testpypi-version.py pyproject.toml
32+
git update-index --assume-unchanged pyproject.toml
33+
cat pyproject.toml
34+
35+
- name: Build tarball and wheels
36+
run: |
37+
git clean -xdf
38+
python -m build --sdist --wheel .
39+
40+
- name: Check built artifacts
41+
run: |
42+
python -m twine check dist/*
43+
pwd
44+
if [ -f dist/xarray-0.0.0.tar.gz ]; then
45+
echo "❌ INVALID VERSION NUMBER"
46+
exit 1
47+
else
48+
echo "✅ Looks good"
49+
fi
50+
51+
- uses: actions/upload-artifact@v3
52+
with:
53+
name: releases
54+
path: dist
55+
56+
test-built-dist:
57+
needs: build-artifacts
58+
runs-on: ubuntu-latest
59+
steps:
60+
- uses: actions/setup-python@v3
61+
name: Install Python
62+
with:
63+
python-version: "3.10"
64+
- uses: actions/download-artifact@v3
65+
with:
66+
name: releases
67+
path: dist
68+
- name: List contents of built dist
69+
run: |
70+
ls -ltrh
71+
ls -ltrh dist
72+
73+
- name: Verify the built dist/wheel is valid
74+
if: github.event_name == 'push'
75+
run: |
76+
python -m pip install --upgrade pip
77+
python -m pip install dist/xarray*.whl
78+
python -m xarray.util.print_versions
79+
80+
- name: Publish package to TestPyPI
81+
if: github.event_name == 'push'
82+
uses: pypa/[email protected]
83+
with:
84+
user: __token__
85+
password: ${{ secrets.TESTPYPI_TOKEN }}
86+
repository_url: https://test.pypi.org/legacy/
87+
verbose: true

‎.github/workflows/upstream-dev-ci.yaml

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -55,17 +55,15 @@ jobs:
5555
- uses: actions/checkout@v3
5656
with:
5757
fetch-depth: 0 # Fetch all history for all branches and tags.
58-
- uses: conda-incubator/setup-miniconda@v2
59-
with:
60-
channels: conda-forge
61-
channel-priority: strict
62-
mamba-version: "*"
63-
activate-environment: xarray-tests
64-
auto-update-conda: false
65-
python-version: ${{ matrix.python-version }}
6658
- name: Set up conda environment
59+
uses: mamba-org/provision-with-micromamba@de032af7fb3675649f3d4bbdda85178ba412ee41
60+
with:
61+
environment-file: ci/requirements/environment.yml
62+
environment-name: xarray-tests
63+
extra-specs: |
64+
python=${{ matrix.python-version }}
65+
- name: Install upstream versions
6766
run: |
68-
mamba env update -f ci/requirements/environment.yml
6967
bash ci/install-upstream-wheels.sh
7068
- name: Install xarray
7169
run: |

‎ci/install-upstream-wheels.sh

Lines changed: 4 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -18,22 +18,18 @@ conda uninstall -y --force \
1818
flox \
1919
h5netcdf \
2020
xarray
21+
# new matplotlib dependency
22+
python -m pip install --upgrade contourpy
2123
# to limit the runtime of Upstream CI
22-
python -m pip install pytest-timeout
2324
python -m pip install \
2425
-i https://pypi.anaconda.org/scipy-wheels-nightly/simple \
2526
--no-deps \
2627
--pre \
2728
--upgrade \
2829
numpy \
2930
scipy \
31+
matplotlib \
3032
pandas
31-
python -m pip install \
32-
-f https://7933911d6844c6c53a7d-47bd50c35cd79bd838daf386af554a83.ssl.cf2.rackcdn.com \
33-
--no-deps \
34-
--pre \
35-
--upgrade \
36-
matplotlib
3733
python -m pip install \
3834
--no-deps \
3935
--upgrade \
@@ -50,3 +46,4 @@ python -m pip install \
5046
git+https://github.com/SciTools/nc-time-axis \
5147
git+https://github.com/dcherian/flox \
5248
git+https://github.com/h5netcdf/h5netcdf
49+
python -m pip install pytest-timeout

‎ci/requirements/bare-minimum.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,6 @@ dependencies:
1010
- pytest-cov
1111
- pytest-env
1212
- pytest-xdist
13-
- numpy=1.18
13+
- numpy=1.19
1414
- packaging=20.0
15-
- pandas=1.1
15+
- pandas=1.2

‎doc/examples/multidimensional-coords.ipynb

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -175,7 +175,7 @@
175175
"# define a label for each bin corresponding to the central latitude\n",
176176
"lat_center = np.arange(1, 90, 2)\n",
177177
"# group according to those bins and take the mean\n",
178-
"Tair_lat_mean = ds.Tair.groupby_bins(\"xc\", lat_bins, labels=lat_center).mean(\n",
178+
"Tair_lat_mean = ds.Tair.groupby_bins(\"yc\", lat_bins, labels=lat_center).mean(\n",
179179
" dim=xr.ALL_DIMS\n",
180180
")\n",
181181
"# plot the result\n",
@@ -186,7 +186,7 @@
186186
"cell_type": "markdown",
187187
"metadata": {},
188188
"source": [
189-
"The resulting coordinate for the `groupby_bins` operation got the `_bins` suffix appended: `xc_bins`. This help us distinguish it from the original multidimensional variable `xc`.\n",
189+
"The resulting coordinate for the `groupby_bins` operation got the `_bins` suffix appended: `yc_bins`. This help us distinguish it from the original multidimensional variable `yc`.\n",
190190
"\n",
191191
"**Note**: This group-by-latitude approach does not take into account the finite-size geometry of grid cells. It simply bins each value according to the coordinates at the cell center. Xarray has no understanding of grid cells and their geometry. More precise geographic regridding for xarray data is available via the [xesmf](https://xesmf.readthedocs.io) package."
192192
]

‎doc/getting-started-guide/installing.rst

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,15 @@ installed, take a look at the ``[options.extras_require]`` section in
147147
:start-at: [options.extras_require]
148148
:end-before: [options.package_data]
149149

150+
Development versions
151+
--------------------
152+
To install the most recent development version, install from github::
153+
154+
$ python -m pip install git+https://github.com/pydata/xarray.git
155+
156+
or from TestPyPI::
157+
158+
$ python -m pip install --index-url https://test.pypi.org/simple --extra-index-url https://pypi.org/simple --pre xarray
150159

151160
Testing
152161
-------

0 commit comments

Comments
 (0)