Inferred spans for AWS Managed Services (#204)

* Infer API Gateway spans (#172)

* Infer spans from API Gateway events

* Adding some prints. remove later

* Change some info on the API Gateway span

* Rename something

* black

* >:(

* black

* fix time

* Support various API Gateway, HTTPAPI, and Websocket events

* black

* Add DD_INFERRED_SPANS env var to turn inferred spans on and off

* infer spans in integration tests

* specify which env var to set true in order to enable inferred spans

* try setting inferred span name to inferred span URL

* s/beta/experimental/

* Correctly create spans in separate services, assuming the extension is running and the  tag is not set on the function

* Remove function_name

* Flush after closing spans

* black

* black

* update snapshots

* Make the snapshots valid json

* black

* Remove the inferredSpansFilter

* Refactor inferred-span event type detection to use the trigger event type code

* remove unused import

* lines too long >=(

* Finish refactor using _EventSource object

* lol, remove println debugging

* Update snapshots

* Inferred spans get meta.span_type:inferred

* [SLS-1594] Inferred spans for SQS & SNS (#190)

* Simplify event source parsing logic

* Add sns & sqs inferred spans

* Fix default evt source bug & update comments

* Use poetry for running unit tests

* Add additional api gateway trigger tests

* Fix bugs & refactor

* Add inferred spans unit tests

* Update integration tests

* Run all tests for all python versions

* Add missing change

* Update _dd.span_type to span_type

* Remove dependence on extension

* Additional api gateway test

* Remove log line & add todo

* Fix api gateway integration test

* Move call to get_first_record

* Revert to using pip inside docker

* [SLS-1671] Inferred Spans for Kineses, EventBridge, S3, DynamoDB (#191)

* Add kinesis inferred span

* Add dynamodb inferred span

* Add s3 inferred span

* Add eventbridge custom event inferred span

* Comment out flaky units tests (TODO)

* Add integration tests

* Remove log line

* Use timestamp instead of strftime (#194)

* [SLS-1683] Add sync/async tag and set inferred span end time based on value (#195)

* Add sync/async tag to inferred spans

* End inferred span based on sync/async tag

* Update tests

* Fix issue with comparison

* Fix small issues

* Update integration test snapshots

* [SLS-1713] Add tag for whether span should inherit service from lambda (#196)

* Use service arg rather than service.name tag
* Refactor inferred span metadata tags

* black

* Update inferred span keys

* [SLS-1674] Additional attributes for different managed services (#200)

* Additional attributes for sqs spans

* Add additional attributes for sns

* Add event subscription arn for sns

* Add additional attributes for kinesis spans

* Add additional attributes for dynamodb spans

* Add additional attributes for s3 spans

* Additional attribute for eventbridge spans

* Add additional attributes for websocket spans

* Fix websocket resource name

* Additional attributes for apigateway v1 spans

* Additional attributes for api-gateway v2 spans

* Fix api-gateway v1 resource name

* Fix merge issue

* Add space to resource name

* Update tests

* black

* Add typing_extensions to pyproject.toml

* Update snapshots

* Fix snapshots

Co-authored-by: chris.agocs <[email protected]>

* [SLS-1824] SNS trace extractor (#201)

* Extract trace context from sns payload

* Fix extractor issue

* Handle SNS message inside SQS event

* Add extractor for Eventbridge context (#202)

* Update expected breaking change date (#114)

* change inferred_span to _inferred_span

* Add extractor for eventbridge trace context

* Add another test to test eventbridge extraction

* Get tracing.Literal for pythons that don't have it already

* Add _datadog to eventbridge extractor

* Update integration tests

* Remove init complete and main start logs

Co-authored-by: Jorie Helwig <[email protected]>
Co-authored-by: Harvinder Ghotra <[email protected]>

* Remove duplicate entry

* Kinesis extractor plus small fixes

* Add snapshots

* Add tests

Co-authored-by: Christopher Agocs <[email protected]>
Co-authored-by: Jorie Helwig <[email protected]>

* Update stash

* Address feedback

* Default DD_TRACE_MANAGED_SERVICES to true

Co-authored-by: Christopher Agocs <[email protected]>
Co-authored-by: chris.agocs <[email protected]>
Co-authored-by: Jorie Helwig <[email protected]>

Author: 4 people

.github/workflows/build.yml

@@ -98,7 +98,7 @@ jobs:
           pip install .[dev]
 
       - name: Install Serverless Framework
-        run: sudo yarn global add serverless --prefix /usr/local
+        run: sudo yarn global add serverless@^2.72.2 --prefix /usr/local
       - name: Install Crossbuild Deps
         run: sudo apt install -y qemu-user-static binfmt-support
 

README.md

@@ -6,7 +6,7 @@
 [![Slack](https://chat.datadoghq.com/badge.svg?bg=632CA6)](https://chat.datadoghq.com/)
 [![License](https://img.shields.io/badge/license-Apache--2.0-blue)](https://github.com/DataDog/datadog-lambda-python/blob/main/LICENSE)
 
-Datadog Lambda Library for Python (3.6, 3.7, 3.8, and 3.9) enables enhanced Lambda metrics, distributed tracing, and custom metric submission from AWS Lambda functions.  
+Datadog Lambda Library for Python (3.6, 3.7, 3.8, and 3.9) enables enhanced Lambda metrics, distributed tracing, and custom metric submission from AWS Lambda functions.
 
 **IMPORTANT NOTE:** AWS Lambda is expected to receive a [breaking change](https://aws.amazon.com/blogs/compute/upcoming-changes-to-the-python-sdk-in-aws-lambda/) on **March 31, 2021**. If you are using Datadog Python Lambda layer version 7 or below, please upgrade to the latest.
 
@@ -103,6 +103,24 @@ Initialize the Datadog tracer when set to `true`. Defaults to `false`.
 
 Set to `true` to merge the X-Ray trace and the Datadog trace, when using both the X-Ray and Datadog tracing. Defaults to `false`.
 
+### DD_TRACE_MANAGED_SERVICES (experimental)
+
+Inferred Spans are spans that Datadog can create based on incoming event metadata.
+Set `DD_TRACE_MANAGED_SERVICES` to `true` to infer spans based on Lambda events.
+Inferring upstream spans is only supported if you are using the [Datadog Lambda Extension](https://docs.datadoghq.com/serverless/libraries_integrations/extension/).
+Defaults to `true`.
+Infers spans for:
+
+- API Gateway REST events
+- API Gateway WebSocket events
+- HTTP API events
+- SQS
+- SNS (SNS messaged delivered via SQS are also supported)
+- Kinesis Streams (if data is a JSON string or base64 encoded JSON string)
+- EventBridge (custom events, where Details is a JSON string)
+- S3
+- DynamoDB
+
 ## Opening Issues
 
 If you encounter a bug with this package, we want to hear about it. Before opening a new issue, search the existing issues to avoid duplicates.

datadog_lambda/constants.py

@@ -3,8 +3,9 @@
 # This product includes software developed at Datadog (https://www.datadoghq.com/).
 # Copyright 2019 Datadog, Inc.
 
-
 # Datadog trace sampling priority
+
+
 class SamplingPriority(object):
     USER_REJECT = -1
     AUTO_REJECT = 0