-
Notifications
You must be signed in to change notification settings - Fork 556
Closed
Description
We want to improve the data we collect for task queues.
We will start with enhancing our Celery integration. This should work with redis, and rabbitmq, and as a stretch goal amazon sqs as brokers. (Kafka should follow later.)
How spans are right now:
- Producer creates
<span op="queue.submit.celery">
when task is put into queue. (start_timestamp
is when putting into queue is started andend_timestamp
is when the item is in the queue.) - Consumer creates
<transaction op="queue.task.celery">
when task is fetched and processed. (start_timestamp
is when the item was received from the queue,end_timestamp
is when the processing of the task has finished. )- Transaction has a status:
ok
in case of successaborted
if Celery raises Retry, Ignore or Reject celery exceptionsinternal_error
if an unhandled exception was raised during execution of the task
- Transaction has a status:
New data we want to add to queue spans:
(for naming see: https://opentelemetry.io/docs/specs/semconv/attributes-registry/messaging/)
- retry count (can be found in
args
of existing_wrap_tracer
, the attached PR) - byte size of message (
messaging.message.body.size
) - byte size of message plus metadata (
messaging.message.envelope.size
) - success/failure status - already there
- host (
server.address
) - (can be found inargs
of existing_wrap_tracer
, the attached PR) - queue or topic (
messaging.destination.name
) - not sure where to get this. - message id (
messaging.message.id
) - should also be inargs
of the comments in the attached PR) - consumer id (unique id of the consumer) (
messaging.client_id
) - dont know yet where to get. - what broker is used in celery. (
messaging.system
)
See this PR for some points where to get the data: #2962
I also added a sample project that uses RabbitMQ with Celery: https://github.com/antonpirker/testing-sentry/tree/main/test-celery-queues-module
Mock of how it will look in the product:
### Tasks
- [ ] https://github.com/getsentry/sentry-python/issues/2959
Metadata
Metadata
Assignees
Labels
No labels