Merge branch 'main' into get_slice_bound-kind

Author: spencerkclark

.github/workflows/ci-additional.yaml

@@ -102,7 +102,7 @@ jobs:
              $PYTEST_EXTRA_FLAGS
 
       - name: Upload code coverage to Codecov
-        uses: codecov/codecov-action@v2.0.3
+        uses: codecov/codecov-action@v2.1.0
         with:
           file: ./coverage.xml
           flags: unittests,${{ matrix.env }}

.github/workflows/ci.yaml

@@ -100,7 +100,7 @@ jobs:
           path: pytest.xml
 
       - name: Upload code coverage to Codecov
-        uses: codecov/codecov-action@v2.0.3
+        uses: codecov/codecov-action@v2.1.0
         with:
           file: ./coverage.xml
           flags: unittests

.pre-commit-config.yaml

@@ -13,7 +13,7 @@ repos:
       - id: isort
   # https://github.com/python/black#version-control-integration
   - repo: https://github.com/psf/black
-    rev: 21.7b0
+    rev: 21.9b0
     hooks:
       - id: black
   - repo: https://github.com/keewis/blackdoc
@@ -30,20 +30,21 @@ repos:
   #     - id: velin
   #       args: ["--write", "--compact"]
   - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: v0.910
+    rev: v0.910-1
     hooks:
       - id: mypy
-        # Copied from setup.cfg
-        exclude: "properties|asv_bench"
+        # `properies` & `asv_bench` are copied from setup.cfg.
+        # `_typed_ops.py` is added since otherwise mypy will complain (but notably only in pre-commit)
+        exclude: "properties|asv_bench|_typed_ops.py"
         additional_dependencies: [
             # Type stubs
             types-python-dateutil,
             types-pkg_resources,
             types-PyYAML,
             types-pytz,
+            typing-extensions==3.10.0.0,
             # Dependencies that are typed
             numpy,
-            typing-extensions==3.10.0.0,
           ]
   # run this occasionally, ref discussion https://github.com/pydata/xarray/pull/3194
   # - repo: https://github.com/asottile/pyupgrade

README.rst

@@ -105,7 +105,7 @@ to support our efforts.
 History
 -------
 
-xarray is an evolution of an internal tool developed at `The Climate
+Xarray is an evolution of an internal tool developed at `The Climate
 Corporation`__. It was originally written by Climate Corp researchers Stephan
 Hoyer, Alex Kleeman and Eugene Brevdo and was released as open source in
 May 2014. The project was renamed from "xray" in January 2016. Xarray became a
@@ -131,16 +131,16 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
 
-xarray bundles portions of pandas, NumPy and Seaborn, all of which are available
+Xarray bundles portions of pandas, NumPy and Seaborn, all of which are available
 under a "3-clause BSD" license:
 - pandas: setup.py, xarray/util/print_versions.py
 - NumPy: xarray/core/npcompat.py
 - Seaborn: _determine_cmap_params in xarray/core/plot/utils.py
 
-xarray also bundles portions of CPython, which is available under the "Python
+Xarray also bundles portions of CPython, which is available under the "Python
 Software Foundation License" in xarray/core/pycompat.py.
 
-xarray uses icons from the icomoon package (free version), which is
+Xarray uses icons from the icomoon package (free version), which is
 available under the "CC BY 4.0" license.
 
 The full text of these licenses are included in the licenses directory.

asv_bench/benchmarks/groupby.py

@@ -0,0 +1,39 @@
+import numpy as np
+
+import xarray as xr
+
+from . import parameterized, requires_dask
+
+
+class GroupBy:
+    def setup(self, *args, **kwargs):
+        self.ds = xr.Dataset(
+            {
+                "a": xr.DataArray(np.r_[np.arange(500.0), np.arange(500.0)]),
+                "b": xr.DataArray(np.arange(1000.0)),
+            }
+        )
+
+    @parameterized(["method"], [("sum", "mean")])
+    def time_agg(self, method):
+        return getattr(self.ds.groupby("a"), method)()
+
+
+class GroupByDask(GroupBy):
+    def setup(self, *args, **kwargs):
+        requires_dask()
+        super().setup(**kwargs)
+        self.ds = self.ds.chunk({"dim_0": 50})
+
+
+class GroupByDataFrame(GroupBy):
+    def setup(self, *args, **kwargs):
+        super().setup(**kwargs)
+        self.ds = self.ds.to_dataframe()
+
+
+class GroupByDaskDataFrame(GroupBy):
+    def setup(self, *args, **kwargs):
+        requires_dask()
+        super().setup(**kwargs)
+        self.ds = self.ds.chunk({"dim_0": 50}).to_dataframe()

ci/requirements/doc.yml

@@ -23,6 +23,7 @@ dependencies:
   - pooch
   - pip
   - pydata-sphinx-theme>=0.4.3
+  - pyproj
   - rasterio>=1.1
   - seaborn
   - setuptools

doc/api.rst

@@ -35,6 +35,7 @@ Top-level functions
    map_blocks
    show_versions
    set_options
+   get_options
    unify_chunks
 
 Dataset
@@ -598,8 +599,8 @@ Universal functions
 
 .. warning::
 
-   With recent versions of numpy, dask and xarray, NumPy ufuncs are now
-   supported directly on all xarray and dask objects. This obviates the need
+   With recent versions of NumPy, Dask and xarray, NumPy ufuncs are now
+   supported directly on all xarray and Dask objects. This obviates the need
    for the ``xarray.ufuncs`` module, which should not be used for new code
    unless compatibility with versions of NumPy prior to v1.13 is
    required. They will be removed once support for NumPy prior to

doc/conf.py

@@ -260,8 +260,8 @@
 ogp_image = "https://xarray.pydata.org/en/stable/_static/dataset-diagram-logo.png"
 ogp_custom_meta_tags = [
     '<meta name="twitter:card" content="summary_large_image" />',
-    '<meta property="twitter:site" content="@xarray_dev />',
-    '<meta name="image" property="og:image" content="https://xarray.pydata.org/en/stable/_static/dataset-diagram-logo.png">',
+    '<meta property="twitter:site" content="@xarray_dev" />',
+    '<meta name="image" property="og:image" content="https://xarray.pydata.org/en/stable/_static/dataset-diagram-logo.png" />',
 ]
 
 # Redirects for pages that were moved to new locations

doc/contributing.rst

@@ -257,7 +257,7 @@ Some other important things to know about the docs:
   tutorial-like overviews per topic together with some other information
   (what's new, installation, etc).
 
-- The docstrings follow the **Numpy Docstring Standard**, which is used widely
+- The docstrings follow the **NumPy Docstring Standard**, which is used widely
   in the Scientific Python community. This standard specifies the format of
   the different sections of the docstring. See `this document
   <https://numpydoc.readthedocs.io/en/latest/format.html#docstring-standard>`_

doc/ecosystem.rst

@@ -50,6 +50,7 @@ Geosciences
 Machine Learning
 ~~~~~~~~~~~~~~~~
 - `ArviZ <https://arviz-devs.github.io/arviz/>`_: Exploratory analysis of Bayesian models, built on top of xarray.
+- `Darts <https://github.com/unit8co/darts/>`_: User-friendly modern machine learning for time series in Python.
 - `Elm <https://ensemble-learning-models.readthedocs.io>`_: Parallel machine learning on xarray data structures
 - `sklearn-xarray (1) <https://phausamann.github.io/sklearn-xarray>`_: Combines scikit-learn and xarray (1).
 - `sklearn-xarray (2) <https://sklearn-xarray.readthedocs.io/en/latest/>`_: Combines scikit-learn and xarray (2).

doc/examples/ERA5-GRIB-example.ipynb

@@ -11,7 +11,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "GRIB format is commonly used to disseminate atmospheric model data. With Xarray and the cfgrib engine, GRIB data can easily be analyzed and visualized."
+    "GRIB format is commonly used to disseminate atmospheric model data. With xarray and the cfgrib engine, GRIB data can easily be analyzed and visualized."
    ]
   },
   {

doc/examples/multidimensional-coords.ipynb

@@ -184,7 +184,7 @@
    "source": [
     "The resulting coordinate for the `groupby_bins` operation got the `_bins` suffix appended: `xc_bins`. This help us distinguish it from the original multidimensional variable `xc`.\n",
     "\n",
-    "**Note**: This group-by-latitude approach does not take into account the finite-size geometry of grid cells. It simply bins each value according to the coordinates at the cell center. Xarray has no understanding of grid cells and their geometry. More precise geographic regridding for Xarray data is available via the [xesmf](https://xesmf.readthedocs.io) package."
+    "**Note**: This group-by-latitude approach does not take into account the finite-size geometry of grid cells. It simply bins each value according to the coordinates at the cell center. Xarray has no understanding of grid cells and their geometry. More precise geographic regridding for xarray data is available via the [xesmf](https://xesmf.readthedocs.io) package."
    ]
   },
   {

doc/examples/visualization_gallery.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "# Visualization Gallery\n",
     "\n",
-    "This notebook shows common visualization issues encountered in Xarray."
+    "This notebook shows common visualization issues encountered in xarray."
    ]
   },
   {
@@ -191,7 +191,10 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {},
+   "metadata": {
+    "jp-MarkdownHeadingCollapsed": true,
+    "tags": []
+   },
    "source": [
     "## `imshow()` and rasterio map projections\n",
     "\n",
@@ -213,7 +216,7 @@
     "\n",
     "# The data is in UTM projection. We have to set it manually until\n",
     "# https://github.com/SciTools/cartopy/issues/813 is implemented\n",
-    "crs = ccrs.UTM('18N')\n",
+    "crs = ccrs.UTM('18')\n",
     "\n",
     "# Plot on a map\n",
     "ax = plt.subplot(projection=crs)\n",
@@ -242,20 +245,14 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from rasterio.warp import transform\n",
+    "from pyproj import Transformer\n",
     "import numpy as np\n",
     "\n",
     "da = xr.tutorial.open_rasterio(\"RGB.byte\")\n",
     "\n",
-    "# Compute the lon/lat coordinates with rasterio.warp.transform\n",
-    "ny, nx = len(da['y']), len(da['x'])\n",
     "x, y = np.meshgrid(da['x'], da['y'])\n",
-    "\n",
-    "# Rasterio works with 1D arrays\n",
-    "lon, lat = transform(da.crs, {'init': 'EPSG:4326'},\n",
-    "                     x.flatten(), y.flatten())\n",
-    "lon = np.asarray(lon).reshape((ny, nx))\n",
-    "lat = np.asarray(lat).reshape((ny, nx))\n",
+    "transformer = Transformer.from_crs(da.crs, \"EPSG:4326\", always_xy=True)\n",
+    "lon, lat = transformer.transform(x, y)\n",
     "da.coords['lon'] = (('y', 'x'), lon)\n",
     "da.coords['lat'] = (('y', 'x'), lat)\n",
     "\n",
@@ -265,14 +262,14 @@
     "# Plot on a map\n",
     "ax = plt.subplot(projection=ccrs.PlateCarree())\n",
     "greyscale.plot(ax=ax, x='lon', y='lat', transform=ccrs.PlateCarree(),\n",
-    "               cmap='Greys_r', add_colorbar=False)\n",
+    "               cmap='Greys_r', shading=\"auto\",add_colorbar=False)\n",
     "ax.coastlines('10m', color='r')"
    ]
   }
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -286,7 +283,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.3"
+   "version": "3.9.7"
   }
  },
  "nbformat": 4,

doc/gallery/plot_rasterio.py

@@ -18,22 +18,17 @@
 import cartopy.crs as ccrs
 import matplotlib.pyplot as plt
 import numpy as np
-from rasterio.warp import transform
+from pyproj import Transformer
 
 import xarray as xr
 
 # Read the data
 url = "https://github.com/mapbox/rasterio/raw/master/tests/data/RGB.byte.tif"
 da = xr.open_rasterio(url)
 
-# Compute the lon/lat coordinates with rasterio.warp.transform
-ny, nx = len(da["y"]), len(da["x"])
-x, y = np.meshgrid(da["x"], da["y"])
-
-# Rasterio works with 1D arrays
-lon, lat = transform(da.crs, {"init": "EPSG:4326"}, x.flatten(), y.flatten())
-lon = np.asarray(lon).reshape((ny, nx))
-lat = np.asarray(lat).reshape((ny, nx))
+# Compute the lon/lat coordinates with pyproj
+transformer = Transformer.from_crs(da.crs, "EPSG:4326", always_xy=True)
+lon, lat = transformer.transform(*np.meshgrid(da["x"], da["y"]))
 da.coords["lon"] = (("y", "x"), lon)
 da.coords["lat"] = (("y", "x"), lat)
 

doc/getting-started-guide/faq.rst

@@ -55,9 +55,9 @@ rows) shouldn't really matter. For example, the images of a movie can be
 natively represented as an array with four dimensions: time, row, column and
 color.
 
-Pandas has historically supported N-dimensional panels, but deprecated them in
-version 0.20 in favor of Xarray data structures. There are now built-in methods
-on both sides to convert between pandas and Xarray, allowing for more focused
+pandas has historically supported N-dimensional panels, but deprecated them in
+version 0.20 in favor of xarray data structures. There are now built-in methods
+on both sides to convert between pandas and xarray, allowing for more focused
 development effort. Xarray objects have a much richer model of dimensionality -
 if you were using Panels:
 
@@ -72,8 +72,8 @@ if you were using Panels:
   In contrast, this sort of data structure fits very naturally in an
   xarray ``Dataset``.
 
-You can :ref:`read about switching from Panels to Xarray here <panel transition>`.
-Pandas gets a lot of things right, but many science, engineering and complex
+You can :ref:`read about switching from Panels to xarray here <panel transition>`.
+pandas gets a lot of things right, but many science, engineering and complex
 analytics use cases need fully multi-dimensional data structures.
 
 How do xarray data structures differ from those found in pandas?
@@ -92,7 +92,7 @@ of the "time" dimension. You never need to reshape arrays (e.g., with
 Why don't aggregations return Python scalars?
 ---------------------------------------------
 
-xarray tries hard to be self-consistent: operations on a ``DataArray`` (resp.
+Xarray tries hard to be self-consistent: operations on a ``DataArray`` (resp.
 ``Dataset``) return another ``DataArray`` (resp. ``Dataset``) object. In
 particular, operations returning scalar values (e.g. indexing or aggregations
 like ``mean`` or ``sum`` applied to all axes) will also return xarray objects.
@@ -152,7 +152,7 @@ What other netCDF related Python libraries should I know about?
 
 `netCDF4-python`__ provides a lower level interface for working with
 netCDF and OpenDAP datasets in Python. We use netCDF4-python internally in
-xarray, and have contributed a number of improvements and fixes upstream. xarray
+xarray, and have contributed a number of improvements and fixes upstream. Xarray
 does not yet support all of netCDF4-python's features, such as modifying files
 on-disk.
 
@@ -161,7 +161,7 @@ __ https://github.com/Unidata/netcdf4-python
 Iris_ (supported by the UK Met office) provides similar tools for in-
 memory manipulation of labeled arrays, aimed specifically at weather and
 climate data needs. Indeed, the Iris :py:class:`~iris.cube.Cube` was direct
-inspiration for xarray's :py:class:`~xarray.DataArray`. xarray and Iris take very
+inspiration for xarray's :py:class:`~xarray.DataArray`. Xarray and Iris take very
 different approaches to handling metadata: Iris strictly interprets
 `CF conventions`_. Iris particularly shines at mapping, thanks to its
 integration with Cartopy_.

doc/getting-started-guide/installing.rst

@@ -87,7 +87,7 @@ Alternative data containers
 
 Minimum dependency versions
 ---------------------------
-xarray adopts a rolling policy regarding the minimum supported version of its
+Xarray adopts a rolling policy regarding the minimum supported version of its
 dependencies:
 
 - **Python:** 24 months
@@ -110,7 +110,7 @@ You can see the actual minimum tested versions:
 Instructions
 ------------
 
-xarray itself is a pure Python package, but its dependencies are not. The
+Xarray itself is a pure Python package, but its dependencies are not. The
 easiest way to get everything installed is to use conda_. To install xarray
 with its recommended dependencies using the conda command line tool::
 

doc/getting-started-guide/quick-overview.rst

@@ -46,7 +46,7 @@ Here are the key properties for a ``DataArray``:
 Indexing
 --------
 
-xarray supports four kinds of indexing. Since we have assigned coordinate labels to the x dimension we can use label-based indexing along that dimension just like pandas. The four examples below all yield the same result (the value at `x=10`) but at varying levels of convenience and intuitiveness.
+Xarray supports four kinds of indexing. Since we have assigned coordinate labels to the x dimension we can use label-based indexing along that dimension just like pandas. The four examples below all yield the same result (the value at `x=10`) but at varying levels of convenience and intuitiveness.
 
 .. ipython:: python
 
@@ -133,7 +133,7 @@ For more, see :ref:`comput`.
 GroupBy
 -------
 
-xarray supports grouped operations using a very similar API to pandas (see :ref:`groupby`):
+Xarray supports grouped operations using a very similar API to pandas (see :ref:`groupby`):
 
 .. ipython:: python
 
@@ -225,4 +225,4 @@ You can directly read and write xarray objects to disk using :py:meth:`~xarray.D
     os.remove("example.nc")
 
 
-It is common for datasets to be distributed across multiple files (commonly one file per timestep). xarray supports this use-case by providing the :py:meth:`~xarray.open_mfdataset` and the :py:meth:`~xarray.save_mfdataset` methods. For more, see :ref:`io`.
+It is common for datasets to be distributed across multiple files (commonly one file per timestep). Xarray supports this use-case by providing the :py:meth:`~xarray.open_mfdataset` and the :py:meth:`~xarray.save_mfdataset` methods. For more, see :ref:`io`.