@@ -17,12 +17,12 @@ This NEP proposes to make all of NumPy's public API overridable via an
17
17
extensible backend mechanism.
18
18
19
19
Acceptance of this NEP means NumPy would provide global and context-local
20
- overrides, as well as a dispatch mechanism similar to NEP-18 [ 2 ]_. First
21
- experiences with ``__array_function__ `` show that it is necessary to be able
22
- to override NumPy functions that *do not take an array-like argument *, and
23
- hence aren't overridable via `` __array_function__ ``. The most pressing need is
24
- array creation and coercion functions, such as `` numpy.zeros `` or
25
- ``numpy.asarray ``; see e.g. NEP-30 [9 ]_.
20
+ overrides in a separate namespace , as well as a dispatch mechanism similar
21
+ to NEP-18 [ 2 ]_. First experiences with ``__array_function__ `` show that it
22
+ is necessary to be able to override NumPy functions that *do not take an
23
+ array-like argument *, and hence aren't overridable via
24
+ `` __array_function__ ``. The most pressing need is array creation and coercion
25
+ functions, such as `` numpy.zeros `` or ``numpy.asarray ``; see e.g. NEP-30 [9 ]_.
26
26
27
27
This NEP proposes to allow, in an opt-in fashion, overriding any part of the
28
28
NumPy API. It is intended as a comprehensive resolution to NEP-22 [3 ]_, and
@@ -32,19 +32,6 @@ type of function or object that needs to become overridable.
32
32
Motivation and Scope
33
33
--------------------
34
34
35
- The motivation behind ``uarray `` is manyfold: First, there have been several
36
- attempts to allow dispatch of parts of the NumPy API, including (most
37
- prominently), the ``__array_ufunc__ `` protocol in NEP-13 [4 ]_, and the
38
- ``__array_function__ `` protocol in NEP-18 [2 ]_, but this has shown the need
39
- for further protocols to be developed, including a protocol for coercion (see
40
- [5 ]_, [9 ]_). The reasons these overrides are needed have been extensively
41
- discussed in the references, and this NEP will not attempt to go into the
42
- details of why these are needed; but in short: It is necessary for library
43
- authors to be able to coerce arbitrary objects into arrays of their own types,
44
- such as CuPy needing to coerce to a CuPy array, for example, instead of
45
- a NumPy array. In simpler words, one needs things like ``np.asarray(...) `` or
46
- an alternative to "just work" and return duck-arrays.
47
-
48
35
The primary end-goal of this NEP is to make the following possible:
49
36
50
37
.. code :: python
@@ -94,14 +81,60 @@ vendored into a new namespace within NumPy to give users and downstream
94
81
dependencies access to these overrides. This vendoring mechanism is similar
95
82
to what SciPy decided to do for making ``scipy.fft `` overridable (see [10 ]_).
96
83
84
+ The motivation behind ``uarray `` is manyfold: First, there have been several
85
+ attempts to allow dispatch of parts of the NumPy API, including (most
86
+ prominently), the ``__array_ufunc__ `` protocol in NEP-13 [4 ]_, and the
87
+ ``__array_function__ `` protocol in NEP-18 [2 ]_, but this has shown the need
88
+ for further protocols to be developed, including a protocol for coercion (see
89
+ [5 ]_, [9 ]_). The reasons these overrides are needed have been extensively
90
+ discussed in the references, and this NEP will not attempt to go into the
91
+ details of why these are needed; but in short: It is necessary for library
92
+ authors to be able to coerce arbitrary objects into arrays of their own types,
93
+ such as CuPy needing to coerce to a CuPy array, for example, instead of
94
+ a NumPy array. In simpler words, one needs things like ``np.asarray(...) `` or
95
+ an alternative to "just work" and return duck-arrays.
97
96
98
- Detailed description
99
- --------------------
97
+ Usage and Impact
98
+ ----------------
100
99
101
- Using overrides
102
- ~~~~~~~~~~~~~~~
100
+ This NEP allows for global and context-local overrides, as well as
101
+ automatic overrides a-la `` __array_function__ ``.
103
102
104
- Here are a few examples of how an end-user would use overrides.
103
+ Here are some use-cases this NEP would enable, besides the
104
+ first one stated in the motivation section:
105
+
106
+ The first is allowing alternate dtypes to return their
107
+ respective arrays.
108
+
109
+ .. code :: python
110
+
111
+ # Returns an XND array
112
+ x = unp.ones((5 , 5 ), dtype = xnd_dtype) # Or torch dtype
113
+
114
+ The second is allowing overrides for parts of the API.
115
+ This is to allow alternate and/or optimised implementations
116
+ for ``np.linalg ``, BLAS, and ``np.random ``.
117
+
118
+ .. code :: python
119
+
120
+ import numpy as np
121
+ import pyfftw # Or mkl_fft
122
+
123
+ # Makes pyfftw the default for FFT
124
+ np.set_global_backend(pyfftw)
125
+
126
+ # Uses pyfftw without monkeypatching
127
+ np.fft.fft(numpy_array)
128
+
129
+ with np.set_backend(pyfftw) # Or mkl_fft, or numpy
130
+ # Uses the backend you specified
131
+ np.fft.fft(numpy_array)
132
+
133
+ This will allow an official way for overrides to work with NumPy without
134
+ monkeypatching or distributing a modified version of NumPy.
135
+
136
+ Here are a few other use-cases, implied but not already
137
+ stated:
105
138
106
139
.. code :: python
107
140
@@ -110,11 +143,8 @@ Here are a few examples of how an end-user would use overrides.
110
143
result = library_function(data)
111
144
result.to_zarr(' output.zarr' )
112
145
113
- This would keep on working, assuming the Dask backend was either set or
114
- registered. Registration can also be done at import-time.
115
-
116
- Now consider another function, and what would need to happen in order to
117
- make this work:
146
+ This second one would work if ``magic_library `` was built
147
+ on top of ``unumpy ``.
118
148
119
149
.. code :: python
120
150
@@ -126,15 +156,13 @@ make this work:
126
156
result = pytorch_predict(data)
127
157
result.to_zarr(' output.zarr' )
128
158
129
- This would work in two scenarios: The first is that ``pytorch_predict `` was a
130
- multimethod, and implemented by the Dask backend. Dask could provide utility
131
- functions to allow external libraries to register implementations.
159
+ Backward compatibility
160
+ ----------------------
132
161
133
- The second, and perhaps more useful way, is that ``pytorch_predict `` was defined
134
- in an idiomatic style true to NumPy in terms of other multimethods, and that Dask
135
- implemented the required multimethods itself, e.g. ``np.convolve ``. If this
136
- happened, then the above example would work without either ``magic_library ``
137
- or Dask having to do anything specific to the other.
162
+ There are no backward incompatible changes proposed in this NEP.
163
+
164
+ Detailed description
165
+ --------------------
138
166
139
167
Composing backends
140
168
~~~~~~~~~~~~~~~~~~
@@ -562,11 +590,6 @@ manner (as an example)::
562
590
def full(shape, fill_value, dtype=None, order='C'):
563
591
# Code here
564
592
565
- Backward compatibility
566
- ----------------------
567
-
568
- There are no backward incompatible changes proposed in this NEP.
569
-
570
593
Alternatives
571
594
------------
572
595
0 commit comments