Importing zarr in R via reticulate

[mike-lawrence]

Hey all, this was simple enough that I'm sure others have devised similar solutions, but I didn't find any with my initial (possibly insufficient) searches, so I thought I'd post here for others. Scenario is simply wanting to read zarr data into R for analysis. I've seen a couple packages seeking to implement the zarr spec in R, but they seemed incomplete or unmaintained, so I figured simply using zarr-python from R via reticulate would be a fairly straightfoward option. This does mean that the data is read into memory in its entirety, I think, so this approach probably wouldn't work for bigger-than-ram datasets. Anyway, here's what I did:

library(tidyverse)
library(reticulate)
zarr = reticulate::import(module='zarr')
store = zarr$load(path_to_zarr_root)
(
	reticulate::iterate(store$array_keys())
	%>% map(
		.f = function(key){
			arr_ref = store$get(key)
			arr = arr_ref$get_basic_selection()
			attributes(arr) = append(attributes(arr),arr_ref$attrs$asdict())
			return(arr)
		}
	)
) ->
	arr_lst
str(arr_lst)

And if the any of the arrays are 2D with column names stored, easy to convert to tibbles:

(
	reticulate::iterate(store$array_keys())
	%>% map(
		.f = function(key){
			arr_ref = store$get(key)
			arr = arr_ref$get_basic_selection()
			attrs = arr_ref$attrs$asdict()
			if( 
				(length(attributes(arr)$dim)==2) 
				& ('col_names' %in% names(attrs))
			){
				arr = as_tibble(arr,.name_repair='minimal')
				names(arr) = attrs$col_names
				attrs[['col_names']] = NULL
			}
			attributes(arr) = append(attributes(arr),attrs)
			out = list()
			out[[key]] = arr
			return(out)
		}
	)
	%>% flatten()
) ->
	arr_lst
str(arr_lst)