Test Common w/ multiple OTel versions & add compat with old OTel (#4312)

With the switch to OTel, the Common test suite is now dependent on an
otel package, so it technically fits the toxgen usecase. By letting
toxgen take care of it, we're making sure we're always testing a good
range of otel versions, including the oldest one (to catch regressions)
and the newest one (to catch incompatibilities early).

Couple things surfaced in terms of incompatibility with older versions:
- Some semantic attributes we're using weren't there from the get go
open-telemetry/opentelemetry-python@495d705.
Changed the code that uses them to handle failure.
- The signature of `span.set_status()` changed at some point
open-telemetry/opentelemetry-python@6e282d2.
Added a compat version of `set_status()` for older otel.

Also included:
- removing the `opentelemetry-experimental` extra (not used anymore)
- ❗ switching to using `opentelemetry-sdk` instead of
`opentelemetry-distro` -- the `distro` only seems to [be setting up some
defaults](https://github.com/open-telemetry/opentelemetry-python-contrib/blob/8390db35ae2062c09d4d74a08d310c7bde1912c4/opentelemetry-distro/src/opentelemetry/distro/__init__.py)
that we're not using


Closes #3241

Author: sentrivana

scripts/populate_tox/README.md

@@ -18,6 +18,7 @@ then determining which versions make sense to test to get good coverage.
 
 The lowest supported and latest version of a framework are always tested, with
 a number of releases in between:
+
 - If the package has majors, we pick the highest version of each major. For the
   latest major, we also pick the lowest version in that major.
 - If the package doesn't have multiple majors, we pick two versions in between
@@ -35,7 +36,8 @@ the main package (framework, library) to test with; any additional test
 dependencies, optionally gated behind specific conditions; and optionally
 the Python versions to test on.
 
-Constraints are defined using the format specified below. The following sections describe each key.
+Constraints are defined using the format specified below. The following sections
+describe each key.
 
 ```
 integration_name: {
@@ -46,6 +48,7 @@ integration_name: {
      },
      "python": python_version_specifier,
      "include": package_version_specifier,
+     "test_on_all_python_versions": bool,
 }
 ```
 
@@ -68,11 +71,12 @@ The test dependencies of the test suite. They're defined as a dictionary of
 in the package list of a rule will be installed as long as the rule applies.
 
 `rule`s are predefined. Each `rule` must be one of the following:
-  - `*`: packages will be always installed
-  - a version specifier on the main package (e.g. `<=0.32`): packages will only
-    be installed if the main package falls into the version bounds specified
-  - specific Python version(s) in the form `py3.8,py3.9`: packages will only be
-    installed if the Python version matches one from the list
+
+- `*`: packages will be always installed
+- a version specifier on the main package (e.g. `<=0.32`): packages will only
+  be installed if the main package falls into the version bounds specified
+- specific Python version(s) in the form `py3.8,py3.9`: packages will only be
+  installed if the Python version matches one from the list
 
 Rules can be used to specify version bounds on older versions of the main
 package's dependencies, for example. If e.g. Flask tests generally need
@@ -101,6 +105,7 @@ Python versions, you can say:
     ...
 }
 ```
+
 This key is optional.
 
 ### `python`
@@ -145,14 +150,26 @@ The `include` key can also be used to exclude a set of specific versions by usin
 `!=` version specifiers. For example, the Starlite restriction above could equivalently
 be expressed like so:
 
-
 ```python
 "starlite": {
     "include": "!=2.0.0a1,!=2.0.0a2",
     ...
 }
 ```
 
+### `test_on_all_python_versions`
+
+By default, the script will cherry-pick a few Python versions to test each
+integration on. If you want a test suite to run on all supported Python versions
+instead, set `test_on_all_python_versions` to `True`.
+
+```python
+"common": {
+    # The common test suite should run on all Python versions
+    "test_on_all_python_versions": True,
+    ...
+}
+```
 
 ## How-Tos
 
@@ -176,7 +193,8 @@ A handful of integration test suites are still hardcoded. The goal is to migrate
 them all to `populate_tox.py` over time.
 
 1. Remove the integration from the `IGNORE` list in `populate_tox.py`.
-2. Remove the hardcoded entries for the integration from the `envlist` and `deps` sections of `tox.jinja`.
+2. Remove the hardcoded entries for the integration from the `envlist` and `deps`
+   sections of `tox.jinja`.
 3. Run `scripts/generate-test-files.sh`.
 4. Run the test suite, either locally or by creating a PR.
 5. Address any test failures that happen.
@@ -185,6 +203,7 @@ You might have to introduce additional version bounds on the dependencies of the
 package. Try to determine the source of the failure and address it.
 
 Common scenarios:
+
 - An old version of the tested package installs a dependency without defining
   an upper version bound on it. A new version of the dependency is installed that
   is incompatible with the package. In this case you need to determine which

scripts/populate_tox/config.py

@@ -29,6 +29,18 @@
     "clickhouse_driver": {
         "package": "clickhouse-driver",
     },
+    "common": {
+        "package": "opentelemetry-sdk",
+        "test_on_all_python_versions": True,
+        "deps": {
+            "*": ["pytest", "pytest-asyncio"],
+            # See https://github.com/pytest-dev/pytest/issues/9621
+            # and https://github.com/pytest-dev/pytest-forked/issues/67
+            # for justification of the upper bound on pytest
+            "py3.7": ["pytest<7.0.0"],
+            "py3.8": ["hypothesis"],
+        },
+    },
     "cohere": {
         "package": "cohere",
         "python": ">=3.9",

scripts/populate_tox/populate_tox.py

@@ -61,7 +61,6 @@
     "asgi",
     "aws_lambda",
     "cloud_resource_context",
-    "common",
     "gevent",
     "opentelemetry",
     "potel",
@@ -348,22 +347,28 @@ def supported_python_versions(
     return supported
 
 
-def pick_python_versions_to_test(python_versions: list[Version]) -> list[Version]:
+def pick_python_versions_to_test(
+    python_versions: list[Version], test_all: bool = False
+) -> list[Version]:
     """
     Given a list of Python versions, pick those that make sense to test on.
 
     Currently, this is the oldest, the newest, and the second newest Python
     version.
     """
-    filtered_python_versions = {
-        python_versions[0],
-    }
+    if test_all:
+        filtered_python_versions = python_versions
 
-    filtered_python_versions.add(python_versions[-1])
-    try:
-        filtered_python_versions.add(python_versions[-2])
-    except IndexError:
-        pass
+    else:
+        filtered_python_versions = {
+            python_versions[0],
+        }
+
+        filtered_python_versions.add(python_versions[-1])
+        try:
+            filtered_python_versions.add(python_versions[-2])
+        except IndexError:
+            pass
 
     return sorted(filtered_python_versions)
 
@@ -517,6 +522,9 @@ def _add_python_versions_to_release(
 
     time.sleep(PYPI_COOLDOWN)  # give PYPI some breathing room
 
+    test_on_all_python_versions = (
+        TEST_SUITE_CONFIG[integration].get("test_on_all_python_versions") or False
+    )
     target_python_versions = TEST_SUITE_CONFIG[integration].get("python")
     if target_python_versions:
         target_python_versions = SpecifierSet(target_python_versions)
@@ -525,7 +533,8 @@ def _add_python_versions_to_release(
         supported_python_versions(
             determine_python_versions(release_pypi_data),
             target_python_versions,
-        )
+        ),
+        test_all=test_on_all_python_versions,
     )
 
     release.rendered_python_versions = _render_python_versions(release.python_versions)

scripts/populate_tox/tox.jinja

@@ -17,9 +17,6 @@ requires =
     # This version introduced using pip 24.1 which does not work with older Celery and HTTPX versions.
     virtualenv<20.26.3
 envlist =
-    # === Common ===
-    {py3.7,py3.8,py3.9,py3.10,py3.11,py3.12,py3.13}-common
-
     # === Gevent ===
     {py3.8,py3.10,py3.11,py3.12}-gevent
 
@@ -157,15 +154,6 @@ deps =
     linters: -r requirements-linting.txt
     linters: werkzeug<2.3.0
 
-    # === Common ===
-    py3.8-common: hypothesis
-    common: pytest-asyncio
-    # See https://github.com/pytest-dev/pytest/issues/9621
-    # and https://github.com/pytest-dev/pytest-forked/issues/67
-    # for justification of the upper bound on pytest
-    py3.7-common: pytest<7.0.0
-    {py3.8,py3.9,py3.10,py3.11,py3.12,py3.13}-common: pytest
-
     # === Gevent ===
     {py3.7,py3.8,py3.9,py3.10,py3.11}-gevent: gevent>=22.10.0, <22.11.0
     {py3.12}-gevent: gevent

sentry_sdk/integrations/__init__.py

@@ -131,6 +131,7 @@ def iter_default_integrations(with_auto_enabling_integrations):
     "celery": (4, 4, 7),
     "chalice": (1, 16, 0),
     "clickhouse_driver": (0, 2, 0),
+    "common": (1, 4, 0),  # opentelemetry-sdk
     "cohere": (5, 4, 0),
     "django": (2, 0),
     "dramatiq": (1, 9),

sentry_sdk/opentelemetry/utils.py

@@ -282,7 +282,13 @@ def infer_status_from_attributes(span_attributes):
 
 def get_http_status_code(span_attributes):
     # type: (Mapping[str, str | bool | int | float | Sequence[str] | Sequence[bool] | Sequence[int] | Sequence[float]]) -> Optional[int]
-    http_status = span_attributes.get(SpanAttributes.HTTP_RESPONSE_STATUS_CODE)
+    try:
+        http_status = span_attributes.get(SpanAttributes.HTTP_RESPONSE_STATUS_CODE)
+    except AttributeError:
+        # HTTP_RESPONSE_STATUS_CODE was added in 1.21, so if we're on an older
+        # OTel version SpanAttributes.HTTP_RESPONSE_STATUS_CODE will throw an
+        # AttributeError
+        http_status = None
 
     if http_status is None:
         # Fall back to the deprecated attribute

sentry_sdk/tracing.py

@@ -11,8 +11,9 @@
     get_current_span,
     INVALID_SPAN,
 )
-from opentelemetry.trace.status import StatusCode
+from opentelemetry.trace.status import Status, StatusCode
 from opentelemetry.sdk.trace import ReadableSpan
+from opentelemetry.version import __version__ as otel_version
 
 import sentry_sdk
 from sentry_sdk.consts import (
@@ -41,6 +42,7 @@
 from sentry_sdk.utils import (
     _serialize_span_attribute,
     get_current_thread_meta,
+    parse_version,
     should_be_treated_as_error,
 )
 
@@ -70,6 +72,8 @@
     from sentry_sdk.tracing_utils import Baggage
 
 
+_OTEL_VERSION = parse_version(otel_version)
+
 tracer = otel_trace.get_tracer(__name__)
 
 
@@ -531,7 +535,10 @@ def set_status(self, status):
             otel_status = StatusCode.ERROR
             otel_description = status
 
-        self._otel_span.set_status(otel_status, otel_description)
+        if _OTEL_VERSION is None or _OTEL_VERSION >= (1, 12, 0):
+            self._otel_span.set_status(otel_status, otel_description)
+        else:
+            self._otel_span.set_status(Status(otel_status, otel_description))
 
     def set_measurement(self, name, value, unit=""):
         # type: (str, float, MeasurementUnit) -> None

setup.py

@@ -41,7 +41,7 @@ def get_file_text(file_name):
     install_requires=[
         "urllib3>=1.26.11",
         "certifi",
-        "opentelemetry-distro>=0.35b0",  # XXX check lower bound
+        "opentelemetry-sdk>=1.4.0",
     ],
     extras_require={
         "aiohttp": ["aiohttp>=3.5"],
@@ -70,7 +70,6 @@ def get_file_text(file_name):
         "openai": ["openai>=1.0.0", "tiktoken>=0.3.0"],
         "openfeature": ["openfeature-sdk>=0.7.1"],
         "opentelemetry": ["opentelemetry-distro>=0.35b0"],
-        "opentelemetry-experimental": ["opentelemetry-distro"],
         "pure-eval": ["pure_eval", "executing", "asttokens"],
         "pymongo": ["pymongo>=3.1"],
         "pyspark": ["pyspark>=2.4.4"],

tests/integrations/threading/test_threading.py

@@ -2,12 +2,14 @@
 from concurrent import futures
 from textwrap import dedent
 from threading import Thread
+import sys
 
 import pytest
 
 import sentry_sdk
 from sentry_sdk import capture_message
 from sentry_sdk.integrations.threading import ThreadingIntegration
+from sentry_sdk.tracing import _OTEL_VERSION
 
 original_start = Thread.start
 original_run = Thread.run
@@ -104,13 +106,18 @@ def double(number):
         assert len(event["spans"]) == 0
 
 
+@pytest.mark.skipif(
+    sys.version[:3] == "3.8" and (1, 12) <= _OTEL_VERSION < (1, 16),
+    reason="Fails in CI on 3.8 and specific OTel versions",
+)
 def test_circular_references(sentry_init, request):
     sentry_init(default_integrations=False, integrations=[ThreadingIntegration()])
 
-    gc.collect()
     gc.disable()
     request.addfinalizer(gc.enable)
 
+    gc.collect()
+
     class MyThread(Thread):
         def run(self):
             pass