@@ -94,20 +94,20 @@ object.
94
94
PyArrow integration
95
95
-------------------
96
96
97
- .. warning ::
98
- ``fsspec `` is only compatible with PyArrow's classic ``FileSystem ``. Starting with version
99
- 2.0 (released in October 2020), `pyarrow `_ switched to a C++ filesystem implementation that
100
- is no longer compatible with ``fsspec ``. Please refer to
101
- `this GitHub issue <https://github.com/intake/filesystem_spec/issues/295 >`_ for additional details.
102
-
103
-
104
- `pyarrow `_ has its own internal idea of what a file-system is (``pyarrow.filesystem.FileSystem ``),
97
+ `pyarrow `_ has its own internal idea of what a file-system is (``pyarrow.fs.FileSystem ``),
105
98
and some functions, particularly the loading of parquet, require that the target be compatible.
106
99
As it happens, the design of the file-system interface in ``pyarrow `` *is * compatible with ``fsspec ``
107
- (this is not by accident). Therefore at import time, ``fsspec `` checks for the existence of
108
- ``pyarrow ``, and, if found, adds it to the superclasses of the spec base-class. In this manner,
109
- all ``fsspec ``-derived file-systems are also pyarrow file-systems, and can be used by pyarrow
110
- functions.
100
+ (this is not by accident).
101
+
102
+ At import time, ``fsspec `` checks for the existence of ``pyarrow ``, and, if ``pyarrow < 2.0 `` is
103
+ found, adds its base filesystem to the superclasses of the spec base-class.
104
+ For ``pyarrow >= 2.0 ``, ``fsspec `` file systems can simply be passed to ``pyarrow `` functions
105
+ that expect ``pyarrow `` filesystems, and ``pyarrow `` `will automatically wrap them
106
+ <https://arrow.apache.org/docs/python/filesystems.html#using-fsspec-compatible-filesystems> `_.
107
+
108
+ In this manner, all ``fsspec ``-derived file-systems are also ``pyarrow `` file-systems, and can be used
109
+ by ``pyarrow `` functions.
110
+
111
111
112
112
.. _pyarrow : https://arrow.apache.org/docs/python/
113
113
0 commit comments