Skip to content

QST: Replace subset of DataFrame by another subset of DataFrame #36424

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
1 of 2 tasks
zxdawn opened this issue Sep 17, 2020 · 7 comments · Fixed by #36486
Closed
1 of 2 tasks

QST: Replace subset of DataFrame by another subset of DataFrame #36424

zxdawn opened this issue Sep 17, 2020 · 7 comments · Fixed by #36486
Assignees
Labels
good first issue Indexing Related to indexing on series/frames, not to indexes themselves Needs Tests Unit test(s) needed to prevent regressions Warnings Warnings that appear or should be added to pandas
Milestone

Comments

@zxdawn
Copy link

zxdawn commented Sep 17, 2020

  • I have searched the [pandas] tag on StackOverflow for similar questions.

  • I have asked my usage related question on StackOverflow.


Question about pandas

Note: If you'd still like to submit a question, please read this guide detailing how to provide the necessary information for us to reproduce your question.

# Your code here, if applicable
import numpy as np
import pandas as pd
dic = {'A': [1, 4, 1, 4], 'B': [9, 2, 5, 3], 'C': [0, 0, 5, 3]}
df = pd.DataFrame(dic)
df.index = [0, 0, 1, 1]

df1 = df.copy()
df1[:] = np.nan

df1.loc[0]['A'] = df.loc[0]['A']

original df:

   A  B  C
0  1  9  0
0  4  2  0
1  1  5  5
1  4  3  3

I want to replace the subset of the copy with the subset of original df:

    A   B   C
0 1   NaN NaN
0 4   NaN NaN
1 NaN NaN NaN
1 NaN NaN NaN

However, it shows no change at all:

    A   B   C
0 NaN NaN NaN
0 NaN NaN NaN
1 NaN NaN NaN
1 NaN NaN NaN
@zxdawn zxdawn added Needs Triage Issue that has not been reviewed by a pandas team member Usage Question labels Sep 17, 2020
@MarcoGorelli
Copy link
Member

Hi @zxdawn - please ask usage questions on StackOverflow first

Did you not get a SettingWithCopyWarning when executing df1.loc[0]['A'] = df.loc[0]['A']?

Use

df1.loc[0, 'A'] = 

for assignment

@zxdawn
Copy link
Author

zxdawn commented Sep 17, 2020

Thanks, @MarcoGorelli. It works now.

BTW, I didn't get the warning with pandas = 1.0.5.

@MarcoGorelli
Copy link
Member

Thanks, @MarcoGorelli. It works now.

BTW, I didn't get the warning with pandas = 1.0.5.

Thanks for the report - I can confirm that 1.0.5 doesn't give this warning with this snippet, while 1.1.2 does. Reopening for now with needs tests, will check later when this was fixed

@MarcoGorelli MarcoGorelli reopened this Sep 17, 2020
@MarcoGorelli MarcoGorelli added Warnings Warnings that appear or should be added to pandas Needs Tests Unit test(s) needed to prevent regressions good first issue and removed Needs Triage Issue that has not been reviewed by a pandas team member Usage Question labels Sep 17, 2020
@MarcoGorelli
Copy link
Member

760ba37a776947694ec816eaf29ce8937e08f544 is the first bad commit
commit 760ba37a776947694ec816eaf29ce8937e08f544
Author: jbrockmendel <[email protected]>
Date:   Tue May 26 15:47:18 2020 -0700

    CLN: _consolidate_inplace less (#34389)

 pandas/core/generic.py            |  9 ---------
 pandas/core/internals/managers.py | 10 ----------
 2 files changed, 19 deletions(-)
bisect run success

@samilAyoub
Copy link
Contributor

take

@erfannariman
Copy link
Member

erfannariman commented Sep 19, 2020

Insn't this question about why it does not do the replacing?

@MarcoGorelli
Copy link
Member

MarcoGorelli commented Sep 19, 2020

Insn't this question about why it does not do the replacing?

Yes, but I believe I answered that in #36424 (comment) , as you should not use chained indexing for assignment, https://pandas.pydata.org/docs/user_guide/indexing.html

However, the OP didn't get a warning, even though they should've, so a test to make sure that such a warning will continue to be thrown would be useful

@jreback jreback added this to the 1.2 milestone Sep 21, 2020
@jreback jreback added the Indexing Related to indexing on series/frames, not to indexes themselves label Sep 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Indexing Related to indexing on series/frames, not to indexes themselves Needs Tests Unit test(s) needed to prevent regressions Warnings Warnings that appear or should be added to pandas
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants