-
-
Notifications
You must be signed in to change notification settings - Fork 363
Closed
Labels
Milestone
Description
This issue tracks a evaluation of the v3 runtime config
Context
The v3 branch runtime config currently looks like this:
zarr-python/src/zarr/v3/config.py
Lines 34 to 38 in 76c3450
@dataclass(frozen=True) | |
class RuntimeConfiguration: | |
order: Literal["C", "F"] = "C" | |
concurrency: Optional[int] = None | |
asyncio_loop: Optional[AbstractEventLoop] = None |
This is then attached to Array/Group classes
zarr-python/src/zarr/v3/array.py
Lines 51 to 55 in 76c3450
@dataclass(frozen=True) | |
class AsyncArray: | |
metadata: ArrayMetadata | |
store_path: StorePath | |
runtime_configuration: RuntimeConfiguration |
A few things are missing here:
- User experience
- as a user, I may want to set config settings and forget about them (e.g. order, concurrency)
- Portability
- I don't know for sure but I really doubt that putting the AsyncIO loop on the Array class is going to work when it comes to serialization
Improvements
So looking for some ideas for how to manage this better. Two ideas:
- Xarray style
set-options
: https://docs.xarray.dev/en/stable/generated/xarray.set_options.html- Pros: allows for validation and is typed
- Cons: a bit bespoke, doesn't support environment variables or a config file option
- Dask style config - https://donfig.readthedocs.io/en/latest/
- Pros: very flexible framework, support for environment variables and config files, nested namespaces, etc.
- Cons: extra dependency (though we could vendor it), no typing or validation
what do we expect to go in the runtime config?
- Order
- Concurrency
- logging settings
- what else?
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
Done