Skip to content

ENH: Read filelike objects #1329

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Apr 20, 2020

Conversation

Sangarshanan
Copy link
Contributor

@Sangarshanan Sangarshanan commented Mar 17, 2020

Tried to support reading of file-like objects using geopandas, looking for feedback and suggestions
Ref #388 and #1323

@martinfleis
Copy link
Member

martinfleis commented Mar 17, 2020

Thanks for the PR! This looks good. What are all the possible options we might want to support (and test)? Drawing from linked issues:

  • tempfile.TemporaryFile (is it supported now? can we support it and test it?)
  • pathlib.Path
  • String IO
  • binary IO
  • raw IO
  • Bytes IO

Anything else to support?

We should also mention all these options in documentation (both User Guide and docstring).

@Sangarshanan
Copy link
Contributor Author

Yup it is possible to read tempfile.TemporaryFile added a test for it, also it is actually possible to read any object with a read() method now. Things I see missing in the test are py.path and s3 objects, url's. But py.path doc says that it's in maintenance mode and has asked to refer pathlib for path operations. I am also checking out reading s3 urls (possible with fiona) and even streaming s3 objects ref: pandas-dev/pandas#22520 but maybe that is a separate issue

Should I address the user guide changes here?

Copy link
Member

@martinfleis martinfleis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I address the user guide changes here?

Ideally yes.

S3 can go to separate PR as it might be more tricky.

@Sangarshanan
Copy link
Contributor Author

I have added some basic content to the user guide, not sure if should I add more detailed examples explaining every file like object supported?

@martinfleis
Copy link
Member

I don't think we need to add more details. Docstring clearly indicates that there are other options apart from file path.

I have pushed some minor style cleaning. I recommend using pre-commit hook to make sure that the code follows black and flake8 - https://geopandas.readthedocs.io/en/latest/contributing.html#style-guide-linting

Thanks!

@martinfleis martinfleis added this to the 0.8.0 milestone Mar 27, 2020
@martinfleis martinfleis merged commit 4e82afe into geopandas:master Apr 20, 2020
@martinfleis
Copy link
Member

Thank you @Sangarshanan!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants