Updating DataFrames not Case Sensitive

[smithjev]

I have two nearly identical DataFrames (same index structure and column names) with different names. One is called DF the other is df (only difference is case of the name). When updating values in columns of one DataFrame, the corresponding values in the other are also updated. I am guessing somewhere in the back end writing to DataFrames is not case sensitive where it ought to be.

[changhiskhan]

when you say they're nearly identical, do you mean that the variables 'df' and 'DF' point to the same DataFrame object? If so updating one will update "the other".
Let me know if I've misunderstood the question (if you can post a repro in that case it would be great)
THanks!

[smithjev]

To clarify a bit more, I have distilled out hopefully enough of my code to demonstrate what is happening.
Lower case df is a subset of DF that I copy over as a temporary DataFrame I am using to smooth stock data using a moving window cubic spline, with variable parameters determined by market conditions. Upper case DF is updated at each tick with the last smoothed value from the temporary DataFrame.

However, when I get to the second to last line in this sample (df.intd = d['rollspline']), both df and DF are updated identical values, where I expected each to be updated independently.

### Boiler Plate, in case you wanna see how pieces fits together in my case...###
# Moving Window Cubic Spline to remove lookahead bias
def interpNow(data):
        ii = np.array(range(1,len(data)+1))
        univariate = UnivariateSpline(ii, data, k=3, s=20)
        return Decimal(str(univariate(ii[-1])))

csfit = lambda x: interpNow(x)

def new_cols(x):
        cols = ['intd', 'dpdt', 'ddpdt', 'dddpdt', 'dpdt_sm', 'ddpdt_sm', 'dddpdt_sm', 'STD', 'pos_std', 'neg_std']
        for col in cols:
                x[col] = np.nan
        return x
### End Boiler Plate ###


DF = get_data(ticker, minDateTime, maxDateTime)
DF = DF.groupby('price').apply(new_cols)

for tick in DF.index:
        df = DF[:tick]
        rollSpline = pd.rolling_apply(df.price, Indicator_Tail, csfit)
        d={'rollspline':rollSpline}
        df.intd = d['rollspline']
        DF.intd[tick] = df.intd[-1]

Hope this was more informative.

[changhiskhan]

The issue is here: df = DF[:tick] as this gives you a view of the underlying data rather than a new copy.

You can do df = DF[:tick].copy(), or DF.intd[tick] = d['rollspline'][-1] so you don't have to copy data unnecessarily.

[smithjev]

Thanks Changhiskhan. That makes it clear why this is happening. I will try another approach as you recommended. :)