Skip to content

Support for replacing date partitioned tables #85

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from

Conversation

paoloburelli
Copy link

I addressed the enhancement request in #43, now it is possible to replace data in a partition.
A few other minor changes have been added to give a full support of date partitioned tables (creation and insertion were missing mainly).
I also added a copy function, which is a convenient (and free) way to move data in BigQuery.

@codecov-io
Copy link

codecov-io commented Aug 25, 2017

Codecov Report

Merging #85 into master will decrease coverage by 47.23%.
The diff coverage is 10.32%.

Impacted file tree graph

@@             Coverage Diff             @@
##           master      #85       +/-   ##
===========================================
- Coverage   73.61%   26.37%   -47.24%     
===========================================
  Files           4        4               
  Lines        1554     1725      +171     
===========================================
- Hits         1144      455      -689     
- Misses        410     1270      +860
Impacted Files Coverage Δ
pandas_gbq/tests/test_gbq.py 26.48% <14.77%> (-56.51%) ⬇️
pandas_gbq/gbq.py 17.44% <6.25%> (-58.12%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 79c9067...ded898b. Read the comment docs.

@parthea
Copy link
Contributor

parthea commented Aug 26, 2017

@paoloburelli Firstly, thanks for this PR! I think the next step for pandas-gbq is to leverage the work done in google-cloud-bigquery. I'd like to focus on migrating to google-cloud-bigquery before reviewing this PR since it already has support for some of the features proposed including date partitioned tables and table copy operations. @jasonqng has done a great job getting this task started in PR #25. @GregCT has also expressed interest in continuing the work done in PR #25 and was recently added collaborator (based on the last comment in the PR).

I really appreciate the time that you put into this PR and I want to revisit it after #25 is merged because I agree these features would be helpful for the pandas-gbq community. If you're interested in helping out with #25, please let @jasonqng & @GregCT by commenting in PR #25.

cc @tswast
cc @jreback

@paoloburelli
Copy link
Author

Hi, my team currently needs these changes, so we will continue to develop the library on our fork.
Meanwhile I will close this pull request and I'll monitor the development of the library.

@hayato1986
Copy link

Hope this goes to next step!

@BartVanDerVurst
Copy link

Will this be merged to the master soon? Still an open topic which results in errors when running data-partioning append's

@tswast
Copy link
Collaborator

tswast commented Nov 5, 2018

@BartVanDerVurst there have been many changes to the pandas-gbq library since this PR was first created. For one, it now uses the google-cloud-bigquery library under the covers. It would need to be rebased on a more recent version of the library for us to merge it into master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants