Skip to content

feat: Pretty print JSON for python. Fix missing regular expressions for masking out ephemeral information #215

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 22, 2022

Conversation

astuyve
Copy link
Contributor

@astuyve astuyve commented Mar 22, 2022

What does this PR do?

  • Prints JSON with indentations and newlines to increase readability
  • Mask out ephemeral data which caused integration tests to always fail

Motivation

Testing Guidelines

Additional Notes

Types of Changes

  • Bug fix
  • New feature
  • Breaking change
  • Misc (docs, refactoring, dependency upgrade, etc.)

Check all that apply

  • This PR's description is comprehensive
  • This PR contains breaking changes that are documented in the description
  • This PR introduces new APIs or parameters that are documented and unlikely to change in the foreseeable future
  • This PR impacts documentation, and it has been updated (or a ticket has been logged)
  • This PR's changes are covered by the automated tests
  • This PR collects user input/sensitive content into Datadog
  • This PR passes the integration tests (ask a Datadog member to run the tests)

@astuyve astuyve requested a review from a team as a code owner March 22, 2022 16:48
@@ -17,7 +17,6 @@ provider:
DD_TRACE_ENABLED: true
DD_API_KEY: ${env:DD_API_KEY}
DD_TRACE_MANAGED_SERVICES: true
DD_CAPTURE_LAMBDA_PAYLOAD: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you remove this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I inadvertently added it when I wrote the feature, but it includes all headers and payload data that is complex to mask out, so I removed it now.

I think my introduction of this is what led to the tests being in the state they're in now.

@@ -175,14 +175,15 @@ for handler_name in "${LAMBDA_HANDLERS[@]}"; do
# Replace invocation-specific data like timestamps and IDs with XXXX to normalize logs across executions
logs=$(
echo "$raw_logs" |
node parse-json.js |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙏

@nhinsch
Copy link
Contributor

nhinsch commented Mar 22, 2022

Looks good, how were tests passing before if there were so many values that needed to be masked?

@@ -17,7 +17,6 @@ provider:
DD_TRACE_ENABLED: true
DD_API_KEY: ${env:DD_API_KEY}
DD_TRACE_MANAGED_SERVICES: true
DD_CAPTURE_LAMBDA_PAYLOAD: true
lambdaHashingVersion: 20201221
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Version 3 doesn't require us to use lambdaHashingVersion anymore. Should we remove it or update it? https://www.serverless.com/framework/docs/guides/upgrading-v3#lambda-hashing-algorithm

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will remove, thanks!

Copy link
Contributor

@jcstorms1 jcstorms1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 LGTM

@astuyve
Copy link
Contributor Author

astuyve commented Mar 22, 2022

@nhinsch - RE: the tests issue. I don't think they were passing.

I don't know, I noticed 3.53 merge passed all integration tests, and so did the PR (https://github.com/DataDog/datadog-lambda-python/runs/5159541822?check_suite_focus=true#step:10:349)

But I don't know how they would, since they failed on my machine when I tried to replicate...

@astuyve astuyve merged commit fc1c286 into main Mar 22, 2022
@astuyve astuyve deleted the aj/pretty-print-snapshots branch March 22, 2022 19:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants