auto-docs: Update K8s acceptance tests #1471
Conversation
❌ Deploy Preview for redpanda-docs-preview failed. Why did it fail? →
|
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the 📝 WalkthroughWalkthroughA new Kubernetes upgrade test scenario is introduced in the upgrade feature file. The scenario validates a Redpanda cluster upgrade workflow by applying an initial manifest with image tag v25.2.11, asserting 3-node stability, then applying an upgraded manifest with tag v25.3.1-rc4, and again verifying 3-node stability. The cluster configuration uses a local operator image for sidecars throughout the test sequence. Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
modules/manage/examples/kubernetes/upgrade.feature (1)
47-47: LGTM! Consider adding a rollback scenario.The final stability check appropriately verifies the upgrade succeeded. As an optional enhancement for more comprehensive documentation, consider adding a rollback scenario to demonstrate downgrade procedures.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
modules/manage/examples/kubernetes/upgrade.feature(1 hunks)
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: Feediver1
Repo: redpanda-data/docs PR: 1153
File: antora.yml:3-5
Timestamp: 2025-07-14T19:28:43.296Z
Learning: In Redpanda docs, during beta releases, the version metadata may intentionally show inconsistencies where the header displays the beta version (e.g., 25.2 Beta) while internal attributes like full-version, latest-redpanda-tag, operator-beta-tag still reference the stable version (e.g., 25.1). This is resolved during the GA merge process when all version references are synchronized.
📚 Learning: 2025-07-14T19:28:43.296Z
Learnt from: Feediver1
Repo: redpanda-data/docs PR: 1153
File: antora.yml:3-5
Timestamp: 2025-07-14T19:28:43.296Z
Learning: In Redpanda docs, during beta releases, the version metadata may intentionally show inconsistencies where the header displays the beta version (e.g., 25.2 Beta) while internal attributes like full-version, latest-redpanda-tag, operator-beta-tag still reference the stable version (e.g., 25.1). This is resolved during the GA merge process when all version references are synchronized.
Applied to files:
modules/manage/examples/kubernetes/upgrade.feature
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Redirect rules - redpanda-docs-preview
- GitHub Check: Header rules - redpanda-docs-preview
- GitHub Check: Pages changed - redpanda-docs-preview
🔇 Additional comments (3)
modules/manage/examples/kubernetes/upgrade.feature (3)
4-24: Initial manifest configuration looks good.The initial Redpanda cluster configuration is appropriate for an upgrade test scenario. The use of a local operator image for sidecars (localhost/redpanda-operator:dev) clearly indicates this is a test/development environment setup.
25-25: LGTM!The stability check ensures the initial deployment is healthy before attempting the upgrade.
26-46: ****This file (
modules/manage/examples/kubernetes/upgrade.feature) is a Gherkin/Cucumber BDD test specification, not production documentation. The image tags (v25.2.11andv25.3.1-rc4) are intentional test placeholders used to verify upgrade scenario behavior in CI/CD, not real deployable versions. Non-existent images in test fixtures are expected—they are mocked during test execution. The repository change from stable to unstable is part of the test scenario logic to verify upgrade path handling. No changes needed.Likely an incorrect or invalid review comment.
| @skip:gke @skip:aks @skip:eks @skip:k3d | ||
| Scenario: Redpanda upgrade from 25.2.11 |
There was a problem hiding this comment.
🧩 Analysis chain
Verify the skip tags are appropriate for a documentation example.
The scenario skips all major cloud-managed Kubernetes environments (GKE, AKS, EKS) and K3D. If this test is intended as a documentation example for users, they may attempt to follow it on these platforms. Ensure these skip tags align with actual platform limitations or move this to a non-example location if it's only for internal testing.
Run the following script to check if there are other upgrade examples without these restrictions:
🏁 Script executed:
#!/bin/bash
# Description: Find other upgrade feature files to compare skip tag usage
# Search for upgrade-related feature files
fd -e feature . | xargs rg -l "upgrade" -i
# Check skip tag patterns in feature files
rg "@skip:" -A 2 -B 1 --type gherkin modules/manage/examples/Length of output: 194
🏁 Script executed:
#!/bin/bash
# Description: Extract and compare skip tags across upgrade feature files
echo "=== upgrade.feature skip tags ==="
rg "@skip:" modules/manage/examples/kubernetes/upgrade.feature -A 1 -B 1 || echo "No skip tags found"
echo ""
echo "=== operator-upgrades.feature skip tags ==="
rg "@skip:" modules/manage/examples/kubernetes/operator-upgrades.feature -A 1 -B 1 || echo "No skip tags found"
echo ""
echo "=== cluster.feature skip tags ==="
rg "@skip:" modules/manage/examples/kubernetes/cluster.feature -A 1 -B 1 || echo "No skip tags found"
echo ""
echo "=== All @skip patterns across feature files ==="
rg "@skip:" modules/manage/examples/kubernetes/ -cLength of output: 1208
🏁 Script executed:
#!/bin/bash
# Description: Examine the full content of upgrade.feature to understand k3d skip rationale
echo "=== Full upgrade.feature content ==="
cat modules/manage/examples/kubernetes/upgrade.feature
echo ""
echo "=== Check for k3d mentions in any feature files ==="
rg "k3d" modules/manage/examples/kubernetes/ -i -B 2 -A 2Length of output: 1661
Remove or document the @Skip:k3d tag—it's inconsistent with similar feature files.
The gke/aks/eks skip tags are consistently applied across all example feature files, but upgrade.feature is the only one that additionally skips k3d. No documentation explains this k3d-specific limitation. Either remove this skip tag to align with operator-upgrades.feature and cluster.feature, or add a comment explaining why upgrades specifically require k3d to be skipped.
🤖 Prompt for AI Agents
In modules/manage/examples/kubernetes/upgrade.feature around lines 2 to 3, the
file includes an extra @skip:k3d tag that is inconsistent with other example
feature files; either remove the @skip:k3d tag to match
operator-upgrades.feature and cluster.feature, or add a short comment on the
same lines explaining why k3d must be skipped for upgrades (e.g., specific k3d
limitation or known issue ID), so update the file by deleting the @skip:k3d
token if there is no k3d-specific reason, or leaving it and adding a one-line
comment documenting the reason.
vbotbuildovich
force-pushed
the
update-acceptance-tests
branch
from
132afb0 to
6726deb
Compare
vbotbuildovich
force-pushed
the
update-acceptance-tests
branch
from
6726deb to
76d2d70
Compare
This PR auto-updates the acceptance tests we use as examples in our Kubernetes docs.