Blosc compression in HDF5 is not designed for multiple threads call blosc library?

[MarkRivers]

hdf5Src/blosc_filter.c calls the blosc_compress function in the blosc library as follows:

    status = blosc_compress(clevel, doshuffle, typesize, nbytes,
                            *buf, outbuf, nbytes);

These comments in blosc.h suggest that in multi-threaded applications should use blosc_compress_ctx instead. This also allows directly specifying the number of threads to use:

/**
  Context interface to blosc compression. This does not require a call
  to blosc_init() and can be called from multithreaded applications
  without the global lock being used, so allowing Blosc be executed
  simultaneously in those scenarios.

  It uses the same parameters than the blosc_compress() function plus:

  `compressor`: the string representing the type of compressor to use.

  `blocksize`: the requested size of the compressed blocks.  If 0, an
   automatic blocksize will be used.

  `numinternalthreads`: the number of threads to use internally.

  A negative return value means that an internal error happened.  This
  should never happen.  If you see this, please report it back
  together with the buffer data causing this and compression settings.
*/
BLOSC_EXPORT int blosc_compress_ctx(int clevel, int doshuffle, size_t typesize,
                                    size_t nbytes, const void* src, void* dest,
                                    size_t destsize, const char* compressor,
                                    size_t blocksize, int numinternalthreads);

NDPluginCodec calls blosc_compress_ctx. Shouild hdf5Src/blosc_filter.c be changed to also call blosc_compress_ctx rather than blosc_compress?

[xiaoqiangwang]

Here is the official repo for the hdf5 filter, https://github.com/Blosc/hdf5-blosc. Maybe this issue can be raised there?

From the code, setting BLOSC_NOLOCK=1 could switch to the _ctx version.