Skip to content

Conversation

weisslj
Copy link
Contributor

@weisslj weisslj commented Jul 3, 2023

Reported also in https://datastax-oss.atlassian.net/browse/PYTHON-1359, minimal example.py:

from cassandra.cluster import Cluster
from cassandra.query import tuple_factory
from cassandra.protocol import NumpyProtocolHandler
import pandas as pd
cluster = Cluster()
session = cluster.connect()
session.row_factory = tuple_factory
session.client_protocol_handler = NumpyProtocolHandler
print(pd.DataFrame(session.execute("SELECT broadcast_port FROM system.local").one()))

Result with numpy<1.24.0:

$ python3 example.py
   broadcast_port
0            7000

Result with numpy>=1.24.0:

$ python3 example.py
/usr/lib/python3/dist-packages/cassandra/io/libevreactor.py:370: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar.  (This may have returned Python scalars in past versions.
  self.process_io_buffer()
Empty DataFrame
Columns: []
Index: []

@weisslj weisslj marked this pull request as draft July 5, 2023 05:44
@weisslj weisslj changed the title Fix for numpy>=1.24.0 PYTHON-1359 Fix for numpy>=1.24.0 Jul 9, 2023
@weisslj
Copy link
Contributor Author

weisslj commented Jul 9, 2023

The CI fails even after I have reverted my fix commit in 77892a5 for testing, see https://ci.appveyor.com/project/DataStax/python-driver/builds/47482656.

Minimal `example.py`:

```python
from cassandra.cluster import Cluster
from cassandra.query import tuple_factory
from cassandra.protocol import NumpyProtocolHandler
import pandas as pd
cluster = Cluster()
session = cluster.connect()
session.row_factory = tuple_factory
session.client_protocol_handler = NumpyProtocolHandler
print(pd.DataFrame(session.execute("SELECT broadcast_port FROM system.local").one()))
```

Result with numpy<1.24.0:
```
$ python3 example.py
   broadcast_port
0            7000
```

Result with numpy>=1.24.0:
```
$ python3 example.py
/usr/lib/python3/dist-packages/cassandra/io/libevreactor.py:370: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar.  (This may have returned Python scalars in past versions.
  self.process_io_buffer()
Empty DataFrame
Columns: []
Index: []
```
@weisslj weisslj force-pushed the fix-for-numpy-1-24-0 branch from 77892a5 to 39a937e Compare July 9, 2023 09:55
@weisslj weisslj marked this pull request as ready for review July 9, 2023 09:55
@absurdfarce
Copy link
Collaborator

Thanks for the PR @weisslj!

Have you signed the Contributor License Agreement for contributions to DataStax open source projects? If not you can find it at https://cla.datastax.com/. Thanks!

@absurdfarce absurdfarce self-requested a review July 20, 2023 05:22
@weisslj
Copy link
Contributor Author

weisslj commented Feb 28, 2024

@absurdfarce sorry for the long delay, I completely missed the notification! I know have signed the Contributor License Agreement!

@weisslj
Copy link
Contributor Author

weisslj commented Mar 22, 2024

@absurdfarce any chance to get this merged soon? It would help us a lot to have this fix included upstream 🙏

@absurdfarce
Copy link
Collaborator

This is a very nice find indeed @weisslj !

I just looked over the general context of this change and your PR; everything looks good to me. I'm running a test build on our CI infrastructure to confirm that everything behaves as expected and we get the expected improvements in our tests. Assuming there aren't any surprises there (and I'm really not expecting any) I'll merge this straightaway.

@absurdfarce
Copy link
Collaborator

Confirmed that the test_cython_protocol_handlers issues are addressed by this patch (as expected). Calling it good and ready to merge this one.

Thanks for the fix @weisslj!

@absurdfarce absurdfarce merged commit 49e1b49 into datastax:master Mar 26, 2024
@weisslj weisslj deleted the fix-for-numpy-1-24-0 branch March 26, 2024 05:43
dkropachev pushed a commit to dkropachev/python-driver that referenced this pull request Mar 5, 2025
dkropachev pushed a commit to scylladb/python-driver that referenced this pull request Mar 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants