Dk/scylla use metadata #1
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cibuildwheel==2.16.2 supports building with python 3.12.0 release
* few import needed to be ajusted/ignored * need to update cython to latest version
since we want to support python 3.12, we need to remove
this pinning, cause those versions are casueing the cython
related unittets to fail:
```
ImportError while importing test module '/project/tests/unit/cython/test_bytesio.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/python/cp312-cp312/lib/python3.12/importlib/__init__.py:90: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
/project/tests/unit/cython/test_bytesio.py:16: in <module>
bytesio_testhelper = cyimport('tests.unit.cython.bytesio_testhelper')
/project/tests/unit/cython/utils.py:29: in cyimport
import pyximport
../venv/lib/python3.12/site-packages/pyximport/__init__.py:1: in <module>
from .pyximport import *
../venv/lib/python3.12/site-packages/pyximport/pyximport.py:51: in <module>
import imp
E ModuleNotFoundError: No module named 'imp'
```
eventlet is currently broken for python 3.12, so until we have a fixed version. we'll remove those tests from the build wheel action Ref: eventlet/eventlet#795
switch from `python setup.py sdist` to `python -m build --sdist` that's now the formal way to build, and not assume we have `distutil` installed on that system.
…ersions since we need to deprecate asyncore which was the default event loop manager, we need to extend the testing of some of the other so we can select a new default
- actions/checkout@v3 - actions/setup-python@v4 the version we were using was using older node version, and github started warning us about it
since we need to run older versions of scylla with cqlsh that only support python2, we need a way to still have python2 available
This fixes an issue where thread name set by scylla would contain extra information, such as enterprise version containing current service level Fixes scylladb#228
* stop using the loop argument for `asyncio.Lock` and asyncio.Quoue` * on the lock replace `with await` with `async with`, which is the correct syntax for using that lock
since python3.8 we have this warning:
```
DeprecationWarning('The loop argument is deprecated since Python 3.8, and scheduled for removal in Python 3.10.')
```
and it's o.k. to have it since on Python 3.10 and up, we stop using that argument
asyncio can't do timeouts smaller than 1ms, as this test requires it's a limitation of `asyncio.sleep` Fixes: scylladb#263
there are some places were we don't need to pass or create the asyncio loop, and we should avoid it
running on python 3.12, we get this error, we should ignore
it until eventlet is fixed
```
ImportError while loading conftest '/home/runner/work/python-driver/python-driver/tests/integration/conftest.py'.
tests/integration/__init__.py:16: in <module>
from cassandra.cluster import Cluster
cassandra/cluster.py:103: in init cassandra.cluster
from cassandra.io.eventletreactor import EventletConnection
cassandra/io/eventletreactor.py:18: in <module>
import eventlet
.test-venv/lib/python3.12/site-packages/eventlet/__init__.py:17: in <module>
from eventlet import convenience
.test-venv/lib/python3.12/site-packages/eventlet/convenience.py:7: in <module>
from eventlet.green import socket
.test-venv/lib/python3.12/site-packages/eventlet/green/socket.py:21: in <module>
from eventlet.support import greendns
.test-venv/lib/python3.12/site-packages/eventlet/support/greendns.py:45: in <module>
from eventlet.green import ssl
.test-venv/lib/python3.12/site-packages/eventlet/green/ssl.py:25: in <module>
_original_wrap_socket = __ssl.wrap_socket
E AttributeError: module 'ssl' has no attribute 'wrap_socket'
```
Ref: eventlet/eventlet#812
this isn't being used anyhow, and breaking support for python 3.12
since asyncore isn't available in python 3.12, we should be gracfully handle it, and enable any other event loop implementions to work
since python 3.12 is deprecating asyncore, we should make asyncio the default fallback event loop when asyncore isn't available asyncio now that it's fixed and we verified it's working (passing the integration suite) in multiple python versions we support (from 3.8 - 3.12)
in push function, self._loop.create_task is called and it's return value is ignored. While the tests may pass now, this code is not correct and this example is called out in docs as a source of bugs, as python docs suggests. Ref: https://docs.python.org/3/library/asyncio-task.html#asyncio.create_task
This change makes it so newly instanced UserType during deserialization isn't immediately copied by deepcopy, which could cause huge slowdown if that UserType contains a lot of data or nested UserTypes, in which case the deepcopy calls would cascade as each to_python call would eventually clone parts of source object. As there isn't a lot of information on why this deepcopy is here in the first place this change could potentially break something. Running integration tests against this commit does not produce regressions, so this call looks safe to remove, but I'm leaving this warning here for the future reference. Fixes scylladb#152
Before this fix, the debug log would crash _connect_socket for UNIX
domain sockets. getsockname() for UNIX domain sockets returns a single
string instead of a tuple (as is the case for IPv4/IPv6). Therefore the
code could crash as it tried to get the second element of a non-tuple
(empty string):
Traceback (most recent call last):
File "/home/margdoc/Workspace/scylla/maintenance_mode_testing.py", line 5, in <module>
s = c.connect()
^^^^^^^^^^^
File "cassandra/cluster.py", line 1750, in cassandra.cluster.Cluster.connect
File "cassandra/cluster.py", line 1776, in cassandra.cluster.Cluster.connect
File "cassandra/cluster.py", line 1763, in cassandra.cluster.Cluster.connect
File "cassandra/cluster.py", line 3581, in cassandra.cluster.ControlConnection.connect
File "cassandra/cluster.py", line 3642, in cassandra.cluster.ControlConnection._reconnect_internal
cassandra.cluster.NoHostAvailable: ('Unable to connect to any servers', {'test_socket': IndexError('string index out of range')})
Fix the issue by not unpacking those values and just printing them
as-is, relying on %s formatter to print all elements of a tuple (host,
port) for IP sockets and string for UNIX domain sockets. The printed log
is not formatted as nice as before, however this is a DEBUG print
so few users will ever see it. The new approach should work with any
format of getsockname().
Fixes scylladb#278
connection: fix logging of non-IP sockets
In order for Scylla to send the tablet info, the driver must tell the database during connection handshake that it is able to interpret it. This negotation is added as a part of ProtocolFeatures class.
Fix typo in documentation
Add mechanism to parse system.tablets periodically. In TokenAwarePolicy check if keyspace uses tablets if so try to use them to find replicas. Make shard awareness work when using tablets. Everything is wrapped in experimental setting, because tablets are still experimental in ScyllaDB and changes in the tablets format are possible.
* cicd: introduce build-test.yml Mantaining file for building and publishing and building for testing is complicated. Let's split them. * cicd: move out publishing stage from reusable workflow There is bug in publishing github action: pypa/gh-action-pypi-publish#166 that does not allow to publish from reusable workflows. So we have to move it out.
before this change, when testing with cqlsh, we have warnings like: ``` <frozen importlib._bootstrap>:488: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC). ```` in this change, we replace the deprecated API with timezone-aware API, to avoid this warning. to keep the backward compatibility, `DateTime.to_python()` still returns an offset-naive timestamp. Signed-off-by: Kefu Chai <[email protected]>
… allow specifying of libev install via env vars (datastax#1223)
…x handling of variable length types (OSS C* 5.0) (datastax#1217)
According to actions/runner-images#11101 ubuntu-20.04 is going to be depricated soon, we need to move to modern version.
Commit 3ce7d7a introduced regression when instead of geven it is trying to import eventlet
Code was introduced with mistake: `col.primary_key or col.primary_key` instead of `col.primary_key or col.partition_key`
SCYLLA_USE_METADATA_ID extension allows using METADATA_ID (which was introduced in CQLv5) in CQLv4. This commit: - Introduce support for SCYLLA_USE_METADATA_ID in protocol extension negotation - Reuse CQLv5 metadata id implemnetation if use_metadata_id feature is enabled - Modify existing unit tests for introduced changes
This change restores handling of skip_metadata in _QueryMessage, that was removed in 2019 (most likely to prevent metadata inconsistencies in prepared statements). I'm not sure if any other changes are required, as there were many modifications in the codebase since the flag handling was removed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.