Skip to content

Flatten OTEL logs to ingest on Parseable #640

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

nikhilsinhaparseable
Copy link
Contributor

Fixes #312

enhancement to add feature to flatten otel logs to be able to ingest on Parseable


This PR has:

  • been tested to ensure log ingestion and log query works.
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added documentation for new or modified features or behaviors.

fixed data types of flattened log fields as per the proto
@Eshanatnight
Copy link
Contributor

I think it's better if we implement serde deserialize trait for the structs, or else it looks overwhelming

@Eshanatnight
Copy link
Contributor

I think it's better if we implement serde deserialize trait for the structs, or else it looks overwhelming

I looked over on crates.io there are crates for open telemetry but, none provide the ability to serialize into the structs.

@nikhilsinhaparseable
Copy link
Contributor Author

I also found some crates for otel but are vrl specific used by vector

@Eshanatnight
Copy link
Contributor

Eshanatnight commented Feb 3, 2024

https://crates.io/crates/opentelemetry
https://crates.io/crates/opentelemetry-otlp
https://crates.io/crates/opentelemetry-spanprocessor-any
https://crates.io/crates/opentelemetry_sdk
https://crates.io/crates/opentelemetry-contrib

There are more crates
They define the data structures and functions that we won't need to maintain. I think it's worth checking it out.

@nitisht
Copy link
Member

nitisht commented Feb 15, 2024

closing in favour of #657

@nitisht nitisht closed this Feb 15, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Feb 15, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feature: support OpenTelemetry Protocol (OTLP)
3 participants