[9.1] (backport #10087) Fall back to process runtime if otel runtime is unsupported

[mergify]

What does this PR do?

Makes inputs use the process runtime if they're configured to use the otel runtime, and the latter cannot support them. Currently there are two possible reasons for this:

• The output is not suppored at all - logstash is an example.
• The output configuration uses unsupported config options - allow_older_versions: false for the elasticsearch output for example.

A log line is also emitted if this happens.

I've also moved the component monitoring code into its own package to avoid dependency cycles.

Why is it important?

We want to enable the otel runtime progressively without ever breaking a user's working setup. This means that a fallback is necessary, even if it may involve going against the explicit configuration. Later, we may add a flag which causes this to be an error instead.

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• [ ] I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

How to test this PR locally

Build the agent package and run it locally with either an elasticsearch output using an unsupported option (this PR uses allow_older_versions: false), or a kafka or logstash output. Then look at the status and logs. You should see a log line warning that your input was switched to the process runtime.

Related issues

• Closes [beats receivers] Only run components with supported outputs as Beats receivers #9696
• Closes [beats receivers] Conditionally enable Beats receivers only if configuration translation succeeds #9098

Questions to ask yourself

• How are we going to support this in production?
• How are we going to measure its adoption?
• How are we going to debug this?
• What are the metrics I should take care of?
• ...

---

This is an automatic backport of pull request #10087 done by [Mergify](https://mergify.com).

[mergify]

Cherry-pick of 2f0ba69 has failed:

On branch mergify/bp/9.1/pr-10087
Your branch is up to date with 'origin/9.1'.

You are currently cherry-picking commit 2f0ba69f2.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   internal/pkg/agent/application/coordinator/coordinator_unit_test.go
	renamed:    internal/pkg/agent/application/monitoring/testdata/monitoring_config_full.yaml -> internal/pkg/agent/application/monitoring/component/testdata/monitoring_config_full.yaml
	renamed:    internal/pkg/agent/application/monitoring/v1_monitor.go -> internal/pkg/agent/application/monitoring/component/v1_monitor.go
	renamed:    internal/pkg/agent/application/monitoring/v1_monitor_test.go -> internal/pkg/agent/application/monitoring/component/v1_monitor_test.go
	modified:   internal/pkg/agent/application/monitoring/server.go

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)
	both modified:   internal/pkg/agent/application/application.go
	both modified:   internal/pkg/agent/application/coordinator/coordinator.go
	both modified:   internal/pkg/agent/application/monitoring/process.go
	both modified:   internal/pkg/agent/cmd/inspect.go
	deleted by us:   internal/pkg/otel/manager/diagnostics.go
	deleted by us:   internal/pkg/otel/manager/diagnostics_test.go
	both modified:   internal/pkg/otel/translate/otelconfig.go
	both modified:   internal/pkg/otel/translate/otelconfig_test.go
	both modified:   testing/integration/ess/beat_receivers_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: 8253bee

Failed CI Steps

• check-ci
• Unit tests - Ubuntu 22.04
• Unit tests - Ubuntu 22.04
• Unit tests - Ubuntu 22.04 with requirefips build tag
• Unit tests - Ubuntu 22.04 with requirefips build tag
• Unit tests - fips140=only Ubuntu 22.04
• Unit tests - fips140=only Ubuntu 22.04
• Unit tests - Ubuntu 22.04 ARM64
• Unit tests - Ubuntu 22.04 ARM64
• Unit tests - Windows 2022
• Unit tests - Windows 2022
• Unit tests - Windows 2016
• Unit tests - Windows 2016
• Unit tests - macOS 15 ARM
• Unit tests - macOS 15 ARM
• Unit tests - Windows 10
• Unit tests - Windows 10
• Unit tests - Windows 11
• Unit tests - Windows 11
• Packaging: linux/amd64 rpm
• Packaging: linux/amd64 rpm
• Packaging: Ubuntu x86_64 FIPS
• Packaging: Ubuntu x86_64 FIPS
• Packaging: linux/arm64 tar.gz
• Packaging: linux/arm64 tar.gz
• Packaging: Ubuntu arm64 FIPS
• Packaging: Ubuntu arm64 FIPS
• Packaging: windows/amd64 zip
• Packaging: windows/amd64 zip
• Packaging: Containers linux/amd64
• Packaging: Containers linux/arm64
• Packaging: Containers linux/amd64 FIPS
• Packaging: Containers linux/arm64 FIPS

History

cc @swiatekm

[elastic-sonarqube]

Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 40%)

See analysis details on SonarQube

[swiatekm]

Backporting this change would require several additional backports, and is probably not worth it. Closing.