Update documentation to better reflect how PyArrow interoperability works

[AndreiBarsan]

In the context of #295, it seems it will be non-trivial to seamlessly support pyarrow interoperability moving forward.

It took me a bit of digging to realize why I couldn't easily interop between pyarrow and fsspec in my code base (I'm using pyarrow 4.0 but fsspec only supports pyarrow < 2.0). To this end, I added a small warning banner to the documentation to highlight this.

fsspec no longer adds pyarrow superclasses to its fs files for pyarrow >= 2.0 since this is not necessary.

Made two other minor cosmetic fixes near my main change in features.rst.

Testing plan: I built the Sphinx docs on my machine and inspected my formatting changes visually, making sure that the new link works.

Let me know what you think!

[martindurant]

Could you please clarify what doesn't work? We'd rather fix that.

[AndreiBarsan]

Oh, I believe it's because they changed the API a lot and (I think) it's no longer in pure Python.

Currently fsspec tries to detect pyarrow and make its filesystem class a base class for AbstractFileSystem, but disables this behavior on L85 of spec.py: https://github.com/intake/filesystem_spec/blob/e734622e2b837625d5c8f27477d6968d837f68b8/fsspec/spec.py#L85

Based on #295 it seems because actually inheriting from pyarrow.fs.FileSystem (the new-school 2.0+ pyarrow FS class) does not work.

[martindurant]

As far as I understand, it is no longer necessary to inherit from pyarrow's filesystem class, we expect full functionality without it.

[AndreiBarsan]

Good point. Upon closer inspection of the pyarrow docs, it seems that this is actually now still supported, just from the pyarrow side. So the user would just need to pass fsspec filesystems to, e.g., pyarrow.dataset.dataset and things should "just work".

I was originally confused because I was porting some custom code which used pyarrow's FileSystem to fsspec and was expecting the interfaces to be mostly compatible. They are in principle, except that pyarrow.fs.FileSystem just uses some different naming schemes.

I will rephrase the docs in the PR to remove the claim that things are not incompatible. Thank you!