Dataset class for handling Grid and Data files

[philipc2]

Separation of Grid and Data Variables

Currently, we are using ux.open_dataset to create a ux.Grid object with a single xr.Dataset containing our data variables and grid information. This lack of separation between our Grid and Data variables is source of confusion. The following figure showcases how our grid and data variables are stored within the ux.Grid.ds datset

As we can see, mesh,mesh_face_nodes and mesh_depth are listed under Data Variables, even though they are used to describe the topology of our grid.

Proposal of a ux.Dataset Class

The ux.Dataset class would be used to match ux.Grid objects to data files that lie on the same grid. The ux.Grid class would still be used to store our grid topology, however it will not be responsible for anything related to data variables.

Implementation would be an extension of the xr.Dataset class, either through inheritance or composition as mentioned in Extending Xarray

Opening Grid and Data Files

ux.open_grid(grid_file)

• grid_file: Path to file that houses unstructured grid definitions
• Returns a ux.Grid object

ux.open_dataset(grid, *args)

• grid: ux.Grid object or path to file that houses unstructured grid definitions
• *args: Data file(s) corresponding to the inputted grid
• Returns a ux.Dataset object

ux.Grid.open_dataset(*args)

• Identical to above implementation, however can be called directly from a ux.Grid object, eliminating the need for the user to pass through a grid

Usage Example

grid_path = "/path/to/grid"
data_path = "/path/to/data"

# Create a Dataset Object
grid = ux.open_grid(grid_path)

# Creating a Dataset Object
ds = ux.open_dataset(grid, data_path)
ds = ux.open_dataset(grid_path, data_path)
ds = grid.open_dataset(data_path)

[philipc2]

@erogluorhan @rajeeja @rljacob @anissa111 @dcherian @brianpm

[dcherian]

To take this further, why even have a separate function to duplicate what xarray does?

grid_ds = xr.open_dataset()  # or intake catalog thing
ds = xr.open_dataset()  # or intake catalog thing
grid = ux.Grid(grid_ds)

grid.plot(ds.variable)
grid.vorticity(ds)

See xgcm for example, which uses this model: https://xgcm.readthedocs.io/en/latest/xgcm-examples/03_MOM6.html#Vorticity-computation

This way, the grid object provides grid-aware operators that operate on a Dataset.

I guess the core question is: how does a UXarray Dataset differ from an Xarray dataset?

[philipc2]

@dcherian

This is a good point! Our current implementation of ux.open_dataset does what you stated, but combines grid_ds and ds into a single xr.Dataset accessed through ux.Grid.ds.

The motivation behind ux.Dataset would be to add separation between our grid and data files, instead of grouping them together. A ux.Datset would represent any number of data variables that lie on some unstructured grid defined by aux.Grid object. A single ux.Grid can be used to describe multiple ux.Dataset objects.

Alternatively, we could think of ux.Grid representing an unfilled unstructured grid, while ux.Dataset would represent a filled unstructured grid with any number of data variables. This "unfilled" grid would be "filled" with some data variables when we construct a ux.Dataset object.

[dcherian]

Let me rephrase:

The motivation behind ux.Dataset would be to add separation between our grid and data files, instead of grouping them together. A ux.Datset would represent any number of data variables that lie on some unstructured grid defined by a ux.Grid object. A single ux.Grid can be used to describe multiple ux.Dataset objects.

If you change ux.Dataset to xr.Dataset, woould it make any difference?

[philipc2]

After thinking about it and discussing, there wouldn't be any major difference. We are planning to implement an approach similar to what you stated: using xarray to handle loading in the data, and then calling ux.Grid with those xarray datasets

[rljacob]

To answer your question, we though we would have functions like: ux.DataArray.integrate("horiz") which would know how to do an integral on the horizontally unstructured mesh, unlike ur.DataArray.ingtegrate. But we now think its better to use a Grid.integrate(ds) approach as xgcm does.

[philipc2]

Discussion has moved to #185