Skip to content

Deprecate decode timedelta #2105

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 8 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions doc/io.rst
Original file line number Diff line number Diff line change
Expand Up @@ -335,6 +335,10 @@ like ``'days'`` for ``timedelta64`` data. ``calendar`` should be one of the cale
supported by netCDF4-python: 'standard', 'gregorian', 'proleptic_gregorian' 'noleap',
'365_day', '360_day', 'julian', 'all_leap', '366_day'.

The automatic decoding of untis like ``'days'`` to ``timedelta64`` is deprecated and
will be removed in xarray version 0.11. The default behavior will keep the same numeric
type of the original data instead of converting to ``timedelta64``.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shoyer b/c english is not my mother tongue I really appreciate your help wording this part.


By default, xarray uses the 'proleptic_gregorian' calendar and units of the smallest time
difference between values, with a reference time of the first time value.

Expand Down
4 changes: 2 additions & 2 deletions doc/time-series.rst
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ converted automatically when used as arguments in xarray objects:
import datetime
xr.Dataset({'time': datetime.datetime(2000, 1, 1)})

When reading or writing netCDF files, xarray automatically decodes datetime and
timedelta arrays using `CF conventions`_ (that is, by using a ``units``
When reading or writing netCDF files, xarray automatically decodes datetime
arrays using `CF conventions`_ (that is, by using a ``units``
Copy link
Contributor Author

@ocefpaf ocefpaf May 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that removing the mention of timedelta here is better b/c the netCDF original data is rarely a timedelta. The case of wave periods, or any other phenomena 'period,' isn't a timedelta IMO.

attribute like ``'days since 2000-01-01'``).

.. _CF conventions: http://cfconventions.org
Expand Down
13 changes: 13 additions & 0 deletions doc/whats-new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,19 @@ Enhancements
This greatly boosts speed and allows chunking on the core dims.
The function now requires dask >= 0.17.3 to work on dask-backed data
(:issue:`2074`). By `Guido Imperiale <https://github.com/crusaderky>`_.
- Deprecate the automatic decoding of time data to timedelta64 (:issue:`1621`).

.. ipython:: python

In [1]: wave_periods = xr.Variable(['t'], [0, 1, 2])
In [2]: wave_periods.attrs.update({'units': 'seconds'})
In [3]: current_behavior = xr.conventions.decode_cf_variable('t', wave_periods)
In [4]: print(current_behavior.dtype)
timedelta64[ns]
In [5]: with xr.set_options(enable_future_time_unit_decoding=True):
...: new_behavior = xr.conventions.decode_cf_variable('t', wave_periods)
In [6]: print(new_behavior.dtype)
int64

Bug fixes
~~~~~~~~~
Expand Down
14 changes: 12 additions & 2 deletions xarray/conventions.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
from .coding import times, strings, variables
from .coding.variables import SerializationWarning
from .core import duck_array_ops, indexing
from .core.options import OPTIONS
from .core.pycompat import (
OrderedDict, basestring, bytes_type, iteritems, dask_array_type,
unicode_type)
Expand Down Expand Up @@ -293,10 +294,19 @@ def decode_cf_variable(name, var, concat_characters=True, mask_and_scale=True,
variables.CFScaleOffsetCoder()]:
var = coder.decode(var, name=name)

enable_future_time_unit_decoding = OPTIONS[
'enable_future_time_unit_decoding']
if decode_times:
for coder in [times.CFTimedeltaCoder(),
times.CFDatetimeCoder()]:
if enable_future_time_unit_decoding:
coder = times.CFDatetimeCoder()
var = coder.decode(var, name=name)
else:
warnings.warn('Decoding timedeltas been deprecated and '
'will be removed in xarray v0.11.',
FutureWarning, stacklevel=2)
for coder in [times.CFTimedeltaCoder(),
times.CFDatetimeCoder()]:
var = coder.decode(var, name=name)

dimensions, data, attributes, encoding = (
variables.unpack_for_decoding(var))
Expand Down
1 change: 1 addition & 0 deletions xarray/core/options.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
OPTIONS = {
'display_width': 80,
'arithmetic_join': 'inner',
'enable_future_time_unit_decoding': False,
}


Expand Down
19 changes: 18 additions & 1 deletion xarray/tests/test_conventions.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
import pytest

from xarray import (Dataset, Variable, SerializationWarning, coding,
conventions, open_dataset)
conventions, open_dataset, set_options)
from xarray.backends.common import WritableCFDataStore
from xarray.backends.memory import InMemoryDataStore
from xarray.conventions import decode_cf
Expand Down Expand Up @@ -175,6 +175,23 @@ def test_decode_cf_with_multiple_missing_values(self):
assert_identical(expected, actual)
assert 'has multiple fill' in str(w[0].message)

def test_decode_cf_raises_timedelta_deprecation(self):
original = Variable(['t'], [0, 1, 2])
original.attrs.update({'units': 'seconds'})
with warnings.catch_warnings(record=True) as w:
actual = conventions.decode_cf_variable('t', original)
assert 'Decoding timedeltas been deprecated' in str(w[0].message)
assert np.issubdtype(actual.dtype, np.timedelta64)

def test_decode_cf_enable_future_time_unit_decoding(self):
with set_options(enable_future_time_unit_decoding=True):
original = Variable(['t'], [0, 1, 2])
original.attrs.update({'units': 'seconds'})
with warnings.catch_warnings(record=True) as w:
actual = conventions.decode_cf_variable('t', original)
assert not w
assert np.issubdtype(actual.dtype, original.dtype)

def test_decode_cf_with_drop_variables(self):
original = Dataset({
't': ('t', [0, 1, 2], {'units': 'days since 2000-01-01'}),
Expand Down