You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
where applying of passed encoding for the text reading\writing is now implemented:
if "b" not in mode:
mode = mode.replace("t", "") + "b"
text_kwargs = {
k: kwargs.pop(k)
for k in ["encoding", "errors", "newline"]
if k in kwargs
}
return io.TextIOWrapper(
self.open(path, mode, block_size, **kwargs), **text_kwargs
)
So, it looks that in pandas ignoring of encoding parameter happens because in the pandas.io.gcs.get_filepath_or_buffer the mode = 'rb' is passed to call of GCSFileSystem.open(filepath_or_buffer, mode)
Tracing back to the moment of the first actual setting the mode parameter we have stop on this line:
we do not pass value of mode and default mode=None works.
As I could suggest for read_csv() we need pass mode=r and for to_csv() we need pass mode=w in the call of get_filepath_or_buffer(). But I'm not sure where it's better to implement this change.
Hi, @WillAyd
In the current gcsf master
GCSFileSystem.open()
has been removed andfsspec.AbstractFileSystem.open()
has works instead:https://github.com/intake/filesystem_spec/blob/4c66e096d32dafe264e2d6707992ee6935685944/fsspec/spec.py#L717
where applying of passed
encoding
for the text reading\writing is now implemented:Note also this issue from gcsfs
So, it looks that in
pandas
ignoring ofencoding
parameter happens because in thepandas.io.gcs.get_filepath_or_buffer
the mode = 'rb' is passed to call ofGCSFileSystem.open(filepath_or_buffer, mode)
Tracing back to the moment of the first actual setting the
mode
parameter we have stop on this line:pandas.io.common.py
It is so, because in the call of
get_filepath_or_buffer()
performed from herepandas/pandas/io/parsers.py
Line 430 in 29d6b02
we do not pass value of
mode
and defaultmode=None
works.As I could suggest for read_csv() we need pass
mode=r
and for to_csv() we need passmode=w
in the call ofget_filepath_or_buffer()
. But I'm not sure where it's better to implement this change.Originally posted by @EgorBEremeev in #26124 (comment)
The text was updated successfully, but these errors were encountered: