Handling pathological xz files

[mhenriks]

Hi, KubeVirt's Containerized Data Importer project uses nbdkit internally to "import" xz compressed disk images. I just want to make you aware of an issue that has come our way: kubevirt/containerized-data-importer#2288

There are two problematic URLs, but I mostly want to bring your attention to the second: https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2.xz

It is a single stream/single block xz file. Since the goal of our project is maximum compatibility, and there is no way to realistically support such pathological cases with nbdkit xz filter as-is, we are researching alternatives.

Do you think it makes sense for xz filter to handle these pathological files? I guess you'd have to do something similar to the gzip filter where the entire file is downloaded and extracted.

[rwmjones]

nbdkit is managed as a gitlab project now: https://gitlab.com/nbdkit/nbdkit

I answered the query on the other ticket.