-
-
Notifications
You must be signed in to change notification settings - Fork 18.5k
BUG: pd.options.future.no_silent_downcasting is not backward compatible #59845
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Thanks for the report.
Why is it bad code? If you'd like to support multiple versions of pandas, I think you'll need to branch on the version. Though it is probably better practice to access the version via |
* Update unit_tests.yaml change python version from 3.10 to 3.12, using existing requirements.txt * Update requirements.txt update versions that were installed for Python 3.12 conda environment * Update unit_tests.yaml check out the feature_update_python branch of METcalcpy to incorporate the Python 3.12 changes * Update requirements.txt pyyaml should be 6.0.2 * Updating to Python 3.12 use "assert mock_foo.called" rather than "assert mock_foo.called_once_with(...)" Refer to this: python/cpython#100690 * Update numpy to 2.2.3 to be consistent with METcalcpy and METplotpy * Update python version * Change ?=3.12 to =3.12 for Python version * set minimum python version to 3.11 * Revert back to mimimum python of 3.10.4 * fix missing " * update escape sequence in lines 432 and 436 to be compatible with Python 3.12 * in pandas to_datetime, replace errors='ignore' to errors='raise' to address the FutureWarning * Addres FutureWarning: Downcasting behavior in the replace method. Opt-in to the future behavior and explicitly as specified in StackOverflow: https://stackoverflow.com/questions/79226735/pandas-replace-and-downcasting-deprecation-since-version-2-2-0 * update package versions to be consistent with those specified in the requirements.txt file * In addressing the FutureWarning, inadvertantly left the request for dtype in command. * use the feature_update_python branch instead of the develop branch to use the correct package versions * Explicitly cast the fill value to an int to satisfy the pandas 2.2 expectation of an int dtype * convert the OBS_LEAD andt FCST_LEAD series to int64 to make all dtypes consistent * set dtypes to OBS_LEAD and FCST_LEAD columns to facilitate merging * set dtype of the FCST_LEAD and OBS_LEAD series earlier, to fix error when using pandas merge * Revert to original code * Revert the OBS_LEAD column to match original code. Replicate same method for setting the FCST_LEAD column to int * set the FCST_LEAD and OBS_LEAD columns as strings, as the fill values in the constants.py file are supposed to be strings * Explicitly change the type of the fcst_lead and obs_lead columns prior to merging to address the FutureWarning * Address the ValueError when merging the OBS_LEAD columns between two different dataframes where one type is an int vs object * After merge, change the OBS_LEAD column to string values to address failing tests * move dtype reassignment outside of try-except code and catch only exceptions from merge validation issues * remove try-except and include changing the dtypes for the fcst_lead in both dataframes so they are consistent (and avoiding the ValueError when merging on different dtypes) * fix indentation error * fix syntax for verifying merge operation * Remove the validate option on merging two dataframes * Explicitly set the dtypes in fcst_column and obs_column for remaining merge operations * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update nco_requirements.txt updates to package versions for Python 3.12 migration * Package versions for Python 3.10 * For testing current code for backwards compatibility with Python 3.10 * Update the names of the runs to facilitate monitoring of GHA's * Remove the python 3.10 run to isolate issues from running tests * Updates for Python 3.12 * Python 3.10 version for sonarqube * Unused/ignored, removing this. * Testing for Python 3.10 * Update unit_tests.yaml updated the METcalcpy version from feature_update_python to the develop branch * Update py310_unit_tests.yaml update METcalcpy branch to develop * Update benchmark_pandas2_2.yaml update METcalcpy branch to develop * Address another deprecation with m_2d_data.obs_valid * Removing testing for Python 3.10 due to lack of backwards compatibility in Pandas: pandas-dev/pandas#59845 * Updated for Python 3.12 * Update .readthedocs.yaml update for Python 3.12 * Update sonarqube.yaml update METcalcpy branch to the develop branch
Pandas version checks
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
Issue Description
I was not able to find a code that doesn't throw warnings for pandas >= 2.0. The example will work in pandas 2.2, but will break for earlier versions. Ahem... This is probably the first time in my Python development that a backwards incompatible change has been issued with a backwards incompatible workaround.
Expected Behavior
Not throw any warning.
i could do something like this
I suppose this code would work, but it's a bad code.
Installed Versions
pandas : 2.2.2
numpy : 1.26.3
pytz : 2024.1
dateutil : 2.8.1
The text was updated successfully, but these errors were encountered: