Skip to content

[8.19] (backport #9928) Ingest internal telemetry from the OTel Collector when it is running #10232

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: 8.19
Choose a base branch
from
Open

[8.19] (backport #9928) Ingest internal telemetry from the OTel Collector when it is running #10232

wants to merge 1 commit into from
+1,431 −22

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Oct 1, 2025

Add a monitoring component to ingest telemetry values from the collector's prometheus endpoint into the elastic_agent.collector dataset. This includes a remapping script processor to convert the collector's metrics to their closest ECS equivalents when possible.

This is a temporary approach that will not scale to full collector metrics in a general configuration, but gives us a backwards-compatible stopgap during the initial monitoring-only rollout.

Checklist

  • I have read and understood the pull request guidelines of this project.
  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool
  • I have added an integration test or an E2E test

How to test this PR locally

Enable agent monitoring while at least one component (including monitoring itself) uses the otel runtime. Prometheus metrics from the collector should be ingested to the elastic_agent.collector dataset.

Related issues

This is an automatic backport of pull request #9928 done by [Mergify](https://mergify.com).

@faec @mergify
Ingest internal telemetry from the OTel Collector when it is running (#…
9a4ee0d 
…9928)

Add a monitoring component to ingest telemetry values from the collector's prometheus endpoint into the `elastic_agent.collector` dataset. This includes a remapping script processor to convert the collector's metrics to their closest ECS equivalents when possible.

This is a temporary approach that will not scale to full collector metrics in a general configuration, but gives us a backwards-compatible stopgap during the initial monitoring-only rollout.

(cherry picked from commit a441ebe)

# Conflicts:
#	NOTICE-fips.txt
#	NOTICE.txt
#	go.mod
@mergify mergify bot requested a review from a team as a code owner October 1, 2025 13:00
@mergify mergify bot added backport conflicts There is a conflict in the backported pull request labels Oct 1, 2025
@mergify mergify bot requested review from kaanyalti and removed request for a team October 1, 2025 13:00
@mergify mergify bot added the backport label Oct 1, 2025
@mergify mergify bot requested a review from swiatekm October 1, 2025 13:00
@mergify mergify bot assigned faec Oct 1, 2025
@mergify Mergify
Copy link
Contributor Author

mergify bot commented Oct 1, 2025

Cherry-pick of a441ebe has failed:

On branch mergify/bp/8.19/pr-9928
Your branch is up to date with 'origin/8.19'.

You are currently cherry-picking commit a441ebee7.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   changelog/fragments/1759257958-collector-telemetry-monitoring.yaml
	modified:   internal/pkg/agent/application/application.go
	modified:   internal/pkg/agent/application/coordinator/coordinator.go
	new file:   internal/pkg/agent/application/monitoring/component/otel_remap.js
	modified:   internal/pkg/agent/application/monitoring/component/testdata/monitoring_config_full.yaml
	modified:   internal/pkg/agent/application/monitoring/component/v1_monitor.go
	modified:   internal/pkg/agent/application/monitoring/component/v1_monitor_test.go
	modified:   internal/pkg/agent/cmd/inspect.go
	modified:   internal/pkg/otel/translate/otelconfig.go
	modified:   testing/integration/ess/beat_receivers_test.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   NOTICE-fips.txt
	both modified:   NOTICE.txt
	both modified:   go.mod

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@github-actions github-actions bot added enhancement New feature or request Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team labels Oct 1, 2025
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

@elasticmachine
Copy link
Collaborator

elasticmachine commented Oct 1, 2025 •
edited
Loading

💔 Build Failed

Failed CI Steps

History

cc @faec

@mergify Mergify
Copy link
Contributor Author

mergify bot commented Oct 6, 2025

This pull request has not been merged yet. Could you please review and merge it @faec? 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kaanyalti kaanyalti Awaiting requested review from kaanyalti kaanyalti is a code owner automatically assigned from elastic/elastic-agent-control-plane

@swiatekm swiatekm Awaiting requested review from swiatekm swiatekm is a code owner automatically assigned from elastic/elastic-agent-control-plane

At least 1 approving review is required to merge this pull request.

Assignees

@faec faec

Labels
backport conflicts There is a conflict in the backported pull request enhancement New feature or request Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@elasticmachine @faec