varia : calendars, time_bnds, ...

[sbiner]

Here's a few info/considerations linked to xclim

• we should take car of passing xarray.DataArray to the different indices and not xarray.Dataset. It may work but there are problems downstream.

• Nothing get's done with xarray.resample until we actually read the data. To keep in mind for the benchmark.

• xarray.resample does not support 365_day and 360_day calendars. For a reason i ignore yet (raised an issue about it), using xr.open_dataset() on a 365_day netcdf works (but it should not).

• if we aim at producing standard netCDF files, we will eventually have to take care of the time_bnds. That implies that we have to compute them and maybe/probably attach them as coordinates to the output xarray.DataArray

Feel free to comment

[huard]

Time_bnds should be handled by ocgis. I suggest defering this item to later. Calendar support is a problem. Suggestions?

[Zeitsperre]

Do we know exactly which calendars are supported by xarray? Perhaps we can implement a check for non-standard calendars that will be handled exclusively in ocgis.

Would there be ways of mapping xclim functions within ocgis's OcgOperations?

[sbiner]

xr supports all the calendars, but only standard calendar is supported now for xr.resample. Discussed this with Travis and we are thinking of bypassing xr.resample and using xr.groupby to mimic xr.resample.

[tlogan2000]

Talking to seb (if understand) the idea is to create a subfunction that tags the data (e.g. 'Year_month','Year_Season') that can be used in .groupby instead of reseample. Would probably have to limit the number of frequencies available to be used (e.g. 'M','Y','QS-DEC') only for now.

[huard]

Ok. Maybe resample implementation in xarray is something we could subcontract.

[tlogan2000]

Yes a possibility. @sbiner is working on his labeling/groupby method (says to give him a couple of days) but really an xarray library capable of dealing with various calendars would be the ideal long term solution in my opinion...

Note, other than the resample problem, simply opening a 360day nc file gives me a warning when using a 'open_dataset'. Inspection of dates indicates that they are present but non-monotonic (affects resample but could mess up other things as well most likrly).. xarray seems to use the standard calendar any how which ends up giving me 3 values of march 1 in a row....

So the contract could potentially look at xarray implmentation of non-standard calendars in general?

For now, we may want to consider always applying a decod_times=False flag:
ds = xr.open_dataset(ncfile,decode_times=False). Then building the dates ourselves with cftime..

[tlogan2000]

Uploaded some 360day and 365day .nc to the xclim testdata folder
Did some tests using xr.set_options(enable_cftimeindex=True)
Reading .nc files now seems to create correct dates (cftime instances)
resample throws an error on both datasets using this option

Using the default : enable_cftimeindex=False
data is still loaded using open_dataset (nb* warnings for 360day but not 365)
In this case resample will work with 365 data but 360 give an error of non-montonic (sse my comment above)

Code example
import xarray as xr
import glob

#Use enable_cftimeindex
xr.set_options(enable_cftimeindex=True)
test_360 = glob.glob('./tests/testdata/HadGEM2-CC_360day/*.nc')
ds360 = xr.open_mfdataset(test_360)
print(ds360.time)
print(ds360.time.encoding['calendar'])
grp = ds360.groupby('time.year')
rsmp= ds360.resample(time='QS-DEC') # gives error

test_365 = glob.glob('./tests/testdata/CanESM2_365day/*.nc')
ds365 = xr.open_mfdataset(test_365)
print(ds365.time)
print(ds365.time.encoding['calendar'])

grp = ds365.groupby('time.year')
rsmp= ds365.resample(time='QS-DEC') #gives error

#Use default options:
xr.set_options(enable_cftimeindex=False)

ds360 = xr.open_mfdataset(test_360) # gives warnings...
print(ds360.time)
print(ds360.time.encoding['calendar'])
grp = ds360.groupby('time.year')
rsmp= ds360.resample(time='QS-DEC') # gives non-montonic ... 3 march firsts (see comment above)

ds365 = xr.open_mfdataset(test_365)
print(ds365.time)
print(ds365.time.encoding['calendar'])

grp = ds365.groupby('time.year')
rsmp= ds365.resample(time='QS-DEC') # This works but probably shouldn't

[sbiner]

Setting xr.set_options(enable_cftimeindex=True) is what xarray is aiming at by default for the next version (read here).

[huard]

Note that there are two tickets in xarray related to that:

• Adding resample functionality to CFTimeIndex pydata/xarray#2191
• Implement CFPeriodIndex pydata/xarray#2481

[huard]

Opened two issues: #73 and #74 to cover outstanding issues.

[huard]

I suggest we open tickets for each outstanding issue and close this one.