[ci] refactor k8s integration tests to pre-render kustomize manifest

[pkoutsovasilis]

What does this PR do?

This PR refactors the Kubernetes integration tests by pre-rendering the Kustomize manifest before test execution. The key changes include:

• Moved Kustomize rendering to TestMain: This ensures Kustomize manifests are rendered once before all tests run, avoiding repeated work in every test case.
• Introduced --integration.k8s.skip-kustomize-tests flag: Allows skipping Kustomize-based tests when necessary (e.g., local test runs or CI constraints).
• Removed inline k8sRenderKustomize logic: This logic is now centralized and retried with backoff using cenkalti/backoff/v5.
• Replaced deprecated skip logic: Now each test case uses a skip(t *testing.T) function to allow dynamic skipping logic.

Why is it important?

Reduces test flakiness: Previously, each test independently rendered the Kustomize YAML, which involved fetching resources such as kube-state-metrics manifests from GitHub. This led to transient test failures due to network instability or GitHub rate limiting. By rendering the manifest once during test setup with retry logic, we reduce (but do not eliminate) this source of flakiness.

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

Disruptive User Impact

None expected. This change only affects internal integration test infrastructure and does not impact users or production functionality.

How to test this PR locally

EXTERNAL=true SNAPSHOT=true PACKAGES=docker DOCKER_VARIANTS=basic PLATFORMS=linux/arm64 mage package

SNAPSHOT=true INSTANCE_PROVISIONER="kind" TEST_PLATFORMS="kubernetes/arm64/1.33.0/basic" mage integration:TestKubernetes

Related issues

• Relates [Flaky Test]: TestKubernetesAgentStandaloneKustomize – failed to render kustomize #8319

[elasticmachine]

⏳ Build in-progress, with failures

• Buildkite Build
• Commit: 269ac4d

Failed CI Steps

• check-ci

cc @pkoutsovasilis

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[pkoutsovasilis]

closing in favour of #8575