read jpeg linux

[r-zenine]

Hi,

As discussed earlier. This PR brings support for jpeg decoding on linux.
The link is done statically.

On open question remains though: How do we test this ?

Jpeg decoders have few differences and don't always output the same image depending on the implementation. For instance, pillow and pillow-simd will not output the same Tensors. Because libjpeg and libjpeg-turbo don't.

The issue is the following: In order to build pillow-simd we need to install zlib libpng and libjpeg-turbo on the machine. We then take the risk of linking the system libs at build time instead of our own vendored versions.

There is small package called PyTurboJPEG that is available on pip that vendors it's dependencies as we do. We could use it for our unit test ( see this commit 3599c64f64db131eccd3dffec6663eaa20e98113). However, it's not available on conda.

Thoughts ?

Have a good day.
Thanks

[cpuhrsch]

I don't think it's worth adding this only for unit testing. We're adding an libjpeg-turbo as an optimization for an existing feature and, I think, keep the PR centered around that. If we also want to evaluate and test pillow-simd we should do that separately and evaluate it as another potential optimization.

[r-zenine]

You are right.

[cpuhrsch]

Small differences in output are expected when using different JPEG libraries. In particular, when you use vectorized libraries the order in which e.g. numbers are added has influence on the result. However, those differences are small and should be within a few machine epsilons of the other libraries. We could also look at other libraries and how they test their JPEG decoding testing.

[r-zenine]

Hi @cpuhrsch,

The jpeg ISO spec seems to be behind a paywall so there is no way to get the exact precision requirements. . However, I took a look at how pillow is doing this and implemented a similar strategy.

https://github.com/python-pillow/Pillow/blob/de179eb5c6877d7da9a7759512bb5acbfbc0b76a/Tests/helper.py#L115

[r-zenine]

Hi @fmassa,

I am just leaving a small message to update you on the on going work.

The linux_wheel pipelines are working.
The conda pipelines are not. conda is doing weird things with binary relocation. I tried many different setups without success. I am out of ideas on how to make it work. Sadly, there is little to no documentation on how to do this.

Thanks

[andfoy]

Hi @r-zenine, thanks for your contribution! We have migrated the most relevant commits into #2301, where we also added your work for loading png files. Thus, we're closing this one