BUG: read_parquet, to_parquet for s3 destinations#19135

Merged

jreback merged 25 commits intopandas-dev:masterfrom

maximveksler:master

Jan 18, 2018

Commits on Jan 11, 2018

Open S3FileSystem for writing when called from DataFrame.to_parquet
maxim veksler
committed
Adding unit test and changelog entry.
maxim veksler
committed
PEP8 E302 better 2 lines breaks in code then 1 on the tree.
maxim veksler
committed
Add mode= documentation for common.py
maxim veksler
committed
Move test_s3_roundtrip from test_s3.py to test_parquet.py
maxim veksler
committed
Use df_compat & s3_resource pytest fixtures 🤗
maxim veksler
committed
numpy compliant docstring
maxim veksler
committed
issue # in unit test code
maxim veksler
committed
Test to_parquet only when pyarrow is installed. fastparquet is unsupported.
maxim veksler
committed
Add S3 support to fastparquet (WIP)
maxim veksler
committed
Refactor test_s3_roundtrip to standalone function.
maxim veksler
committed
Revert s3 support PoC changes made in FastParquetImpl
maxim veksler
committed
Move test_s3_roundtrip into TestParquetPyArrow and add pa fixture
maxim veksler
committed
check_round_trip refactoring to trap on FastParquet wrires to s3.
maxim veksler
committed
FastParquet should fail when s3 write attempt detected.
maxim veksler
committed

Commits on Jan 13, 2018

FastParquet should fail when s3 write attempt detected.
maxim veksler
committed
Revert check_round_trip(), drop do_round_trip().
maxim veksler
committed
flake8.
maxim veksler
committed

Commits on Jan 14, 2018

flake8.
maxim veksler
committed

Commits on Jan 15, 2018

Explain why fastparquet is unimplemented.
maxim veksler
committed
pandas now knows how to play /w fastparquet on s3 (hackish..)
maxim veksler
committed

Commits on Jan 16, 2018

PR review fixes.
maxim veksler
committed
flake8
maxim veksler
committed

Commits on Jan 17, 2018

Documentation fixes.
maxim veksler
committed
PEP8 (also need to restart build, CI queue dropped)
maxim veksler
committed