Skip to content

feat: support for Spark 4 #589

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 18 commits into from
Aug 20, 2025
Merged

feat: support for Spark 4 #589

merged 18 commits into from
Aug 20, 2025

Conversation

razvan
Copy link
Member

@razvan razvan commented Jul 9, 2025 •
edited by adwk67
Loading

Description

Part of: #586

Depends on the corresponding image pr stackabletech/docker-images#1216

Spark 4 is considered experimental because of the following issues:

The integration tests have been updated to exclude spark 4 for the tests known to cause problems.

Definition of Done Checklist

  • Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
  • Please make sure all these things are done and tick the boxes

Author

  • Changes are OpenShift compatible
  • CRD changes approved
  • CRD documentation for all fields, following the style guide.
  • Helm chart can be installed and deployed operator works
  • Integration tests passed (for non trivial changes)
  • Changes need to be "offline" compatible
  • Links to generated (nightly) docs added
  • Release note snippet added

Reviewer

  • Code contains useful comments
  • Code contains useful logging statements
  • (Integration-)Test cases added
  • Documentation added or updated. Follows the style guide.
  • Changelog updated
  • Cargo.toml only contains references to git tags (not specific commits or branches)

Acceptance

  • Feature Tracker has been updated
  • Proper release label has been added
  • Links to generated (nightly) docs added
  • Release note snippet added
  • Add type/deprecation label & add to the deprecation schedule
  • Add type/experimental label & add to the experimental features tracker

@razvan razvan self-assigned this Jul 9, 2025
razvan and others added 6 commits July 10, 2025 15:33
@razvan
update ny-tlc-report
00fa81c
@stackable-bot @razvan
chore: Generated commit to update templated files since the last temp…
0b1bd1f 
…late run up to stackabletech/operator-templating@e3ffe71 (#590)

Reference-to: stackabletech/operator-templating@e3ffe71 (chore: SDP 25.7 templating updates)
@NickLarsenNZ @razvan
chore: Bump Rust dependencies pre-25.7.0 (#592)
75f9125 
* feat(helm): Add RBAC rule for automatic cluster domain detection

* chore: Bump stackable-operator to 0.94.0 and update other dependencies

* chore: Update changelog

* chore: Add sparkhistory and shs shortnames
@razvan
Merge branch 'main' into feat/spark-4
245b8f9
@razvan
fix the iceberg test
ead8e1e
@razvan
update supported versions
6d7be2e
@razvan razvan mentioned this pull request Jul 23, 2025
Closed
6 tasks
@razvan razvan marked this pull request as ready for review July 23, 2025 15:43
adwk67
adwk67 previously requested changes Jul 28, 2025
View reviewed changes
Copy link
Member

@adwk67 adwk67 left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just minor text stuff. Tests to come.
A CHANGELOG entry is missing - not sure if it is needed as this is basically all test & doc changes.

@adwk67 adwk67 moved this to Development: In Review in Stackable Engineering Jul 28, 2025
@adwk67
Copy link
Member

adwk67 commented Jul 29, 2025

🟢 Local tests (nightly suite) are all good.

@sbernauer sbernauer moved this from Development: In Review to Development: Waiting for Review in Stackable Engineering Aug 18, 2025
sbernauer
sbernauer reviewed Aug 20, 2025
View reviewed changes
sbernauer
sbernauer reviewed Aug 20, 2025
View reviewed changes
@sbernauer sbernauer moved this from Development: Waiting for Review to Development: In Review in Stackable Engineering Aug 20, 2025
sbernauer
sbernauer approved these changes Aug 20, 2025
View reviewed changes
Copy link
Member

@sbernauer sbernauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@sbernauer sbernauer dismissed adwk67’s stale review August 20, 2025 13:05

I took over review, as Andrew is out of office

@razvan razvan enabled auto-merge August 20, 2025 13:08
@razvan razvan added this pull request to the merge queue Aug 20, 2025
Merged via the queue into main with commit 0fb4fd9 Aug 20, 2025
17 checks passed
@razvan razvan deleted the feat/spark-4 branch August 20, 2025 13:14
@sbernauer sbernauer moved this from Development: In Review to Development: Done in Stackable Engineering Aug 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sbernauer sbernauer sbernauer approved these changes

@adwk67 adwk67 adwk67 left review comments

Assignees

@razvan razvan

Labels
None yet
Projects
Stackable Engineering
Status: Development: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@razvan @adwk67 @sbernauer @stackable-bot @NickLarsenNZ