Skip to content

Ingest internal telemetry from the OTel Collector when it is running #9928

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 25 commits into from
Oct 1, 2025
Merged

Ingest internal telemetry from the OTel Collector when it is running #9928

merged 25 commits into from
Oct 1, 2025

Conversation

faec
Copy link
Contributor

@faec faec commented Sep 12, 2025 •
edited
Loading

Add a monitoring component to ingest telemetry values from the collector's prometheus endpoint into the elastic_agent.collector dataset. This includes a remapping script processor to convert the collector's metrics to their closest ECS equivalents when possible.

This is a temporary approach that will not scale to full collector metrics in a general configuration, but gives us a backwards-compatible stopgap during the initial monitoring-only rollout.

Checklist

  • I have read and understood the pull request guidelines of this project.
  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool
  • I have added an integration test or an E2E test

How to test this PR locally

Enable agent monitoring while at least one component (including monitoring itself) uses the otel runtime. Prometheus metrics from the collector should be ingested to the elastic_agent.collector dataset.

Related issues

@faec faec self-assigned this Sep 12, 2025
@faec faec requested a review from a team as a code owner September 12, 2025 20:37
@faec faec added the enhancement New feature or request label Sep 12, 2025
@mergify Mergify
Copy link
Contributor

mergify bot commented Sep 12, 2025

This pull request does not have a backport label. Could you fix it @faec? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-./d./d is the label that automatically backports to the 8./d branch. /d is the digit
  • backport-active-all is the label that automatically backports to all active branches.
  • backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
  • backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

faec
faec commented Sep 12, 2025
View reviewed changes
@pierrehilbert pierrehilbert added the Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team label Sep 12, 2025
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

cmacknz
cmacknz reviewed Sep 15, 2025
View reviewed changes
@swiatekm swiatekm mentioned this pull request Sep 18, 2025
Merged
8 tasks
@cmacknz cmacknz mentioned this pull request Sep 19, 2025
Merged
8 tasks
@mergify Mergify
Copy link
Contributor

mergify bot commented Sep 25, 2025

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b otel-telemetry upstream/otel-telemetry
git merge upstream/main
git push upstream otel-telemetry

michalpristas
michalpristas reviewed Sep 29, 2025
View reviewed changes
@cmacknz cmacknz mentioned this pull request Sep 29, 2025
Open
@elastic-sonarqube Elastic SonarQube
Copy link

Quality Gate failed Quality Gate failed

Failed conditions
1.0% Coverage on New Code (required ≥ 40%)

See analysis details on SonarQube

ebeahan
ebeahan previously approved these changes Sep 30, 2025
View reviewed changes
Copy link
Member

@ebeahan ebeahan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with one nit.

Approving to unblock the reviewer gate, but there's still unit test failures in CI which need resolved.

@faec faec mentioned this pull request Sep 30, 2025
Open
@faec
Copy link
Contributor Author

faec commented Sep 30, 2025

If we haven't already, can we file an issue to track?

#10220

ycombinator
ycombinator previously approved these changes Sep 30, 2025
View reviewed changes
ebeahan
ebeahan previously approved these changes Sep 30, 2025
View reviewed changes
@elasticmachine
Copy link
Collaborator

elasticmachine commented Sep 30, 2025 •
edited
Loading

⏳ Build in-progress, with failures

Failed CI Steps

History

cc @faec

@faec faec dismissed stale reviews from ebeahan and ycombinator via ae03c29 September 30, 2025 23:38
@faec faec enabled auto-merge (squash) September 30, 2025 23:43
@faec faec merged commit a441ebe into elastic:main Oct 1, 2025
21 checks passed
@ebeahan ebeahan added backport-8.19 Automated backport to the 8.19 branch backport-9.1 Automated backport to the 9.1 branch labels Oct 1, 2025
mergify bot pushed a commit that referenced this pull request Oct 1, 2025
@faec @mergify
Ingest internal telemetry from the OTel Collector when it is running (#…
90d6b8a 
…9928)

Add a monitoring component to ingest telemetry values from the collector's prometheus endpoint into the `elastic_agent.collector` dataset. This includes a remapping script processor to convert the collector's metrics to their closest ECS equivalents when possible.

This is a temporary approach that will not scale to full collector metrics in a general configuration, but gives us a backwards-compatible stopgap during the initial monitoring-only rollout.

(cherry picked from commit a441ebe)

# Conflicts:
#	NOTICE-fips.txt
#	NOTICE.txt
#	go.mod
#	internal/pkg/agent/application/application.go
#	internal/pkg/agent/application/monitoring/otel_remap.js
#	internal/pkg/agent/application/monitoring/v1_monitor.go
#	internal/pkg/agent/application/monitoring/v1_monitor_test.go
#	internal/pkg/agent/cmd/inspect.go
#	testing/integration/ess/beat_receivers_test.go
mergify bot pushed a commit that referenced this pull request Oct 1, 2025
@faec @mergify
Ingest internal telemetry from the OTel Collector when it is running (#…
9a4ee0d 
…9928)

Add a monitoring component to ingest telemetry values from the collector's prometheus endpoint into the `elastic_agent.collector` dataset. This includes a remapping script processor to convert the collector's metrics to their closest ECS equivalents when possible.

This is a temporary approach that will not scale to full collector metrics in a general configuration, but gives us a backwards-compatible stopgap during the initial monitoring-only rollout.

(cherry picked from commit a441ebe)

# Conflicts:
#	NOTICE-fips.txt
#	NOTICE.txt
#	go.mod
This was referenced Oct 1, 2025
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@graphaelli graphaelli graphaelli left review comments

@michalpristas michalpristas michalpristas left review comments

@cmacknz cmacknz cmacknz left review comments

@swiatekm swiatekm swiatekm left review comments

@ebeahan ebeahan ebeahan approved these changes

@ycombinator ycombinator ycombinator left review comments

@rubenruizdegauna rubenruizdegauna Awaiting requested review from rubenruizdegauna rubenruizdegauna is a code owner automatically assigned from elastic/elastic-agent-control-plane

@leehinman leehinman Awaiting requested review from leehinman

Assignees

@faec faec

Labels
backport-8.19 Automated backport to the 8.19 branch backport-9.1 Automated backport to the 9.1 branch enhancement New feature or request Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[beats receivers] OTel based pipeline must continue to report queue metrics for the elastic_agent integration
9 participants
@faec @elasticmachine @ycombinator @graphaelli @michalpristas @cmacknz @ebeahan @swiatekm @pierrehilbert