Skip to content

feat: build spark 4 #1195

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 21 commits into from
Closed

feat: build spark 4 #1195

wants to merge 21 commits into from

Conversation

razvan
Copy link
Member

@razvan razvan commented Jul 8, 2025

Description

Part of: stackabletech/spark-k8s-operator#586

Definition of Done Checklist

Note

Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant.

Please make sure all these things are done and tick the boxes

  • Changes are OpenShift compatible
  • All added packages (via microdnf or otherwise) have a comment on why they are added
  • Things not downloaded from Red Hat repositories should be mirrored in the Stackable repository and downloaded from there
  • All packages should have (if available) signatures/hashes verified
  • Add an entry to the CHANGELOG.md file
  • Integration tests ran successfully
TIP: Running integration tests with a new product image

The image can be built and uploaded to the kind cluster with the following commands:

bake --product <product> --image-version <stackable-image-version>
kind load docker-image <image-tagged-with-the-major-version> --name=<name-of-your-test-cluster>

See the output of bake to retrieve the image tag for <image-tagged-with-the-major-version>.

@razvan razvan self-assigned this Jul 8, 2025
razvan and others added 15 commits July 8, 2025 18:27
* Various update: log4shell removal, testing-tools uid/gid

* address review feedback
* patches I

* patches I

* wip: patch hbase to use listener endpoints

* changed rendering

* rework patch for UI

* Consistently override the advertised ports

* add listener endpoint and info port to hbase-site.xml

* patch and entrypoint refactoring

* hbase: use listener service for region mover

* hbase: patch for 2.6.2 endpoints

* removed dead variable

* Update hbase/stackable/patches/2.6.1/0005-Allow-overriding-ipc-bind-port-and-use-alternative-p.patch

Co-authored-by: Natalie Klestrup Röijezon <[email protected]>

* Update hbase/stackable/patches/2.6.1/0005-Allow-overriding-ipc-bind-port-and-use-alternative-p.patch

Co-authored-by: Natalie Klestrup Röijezon <[email protected]>

* review comments: added constants etc.

* set RPC_CLIENT_SPECIFY_HOST to true by default

* corrected constant descriptions

* updated 2.6.2 patch

* patch 2.6.1 to reverse bound/advertised properties

* property re-working for 2.6.2

* provide advertised port as a fallback, removed changes to test class

---------

Co-authored-by: Natalie Klestrup Röijezon <[email protected]>
* Remove Hello World operator

* Fix README generation

* make pre-commit happy
* chore: Bump UBI9-minimal base image

* Bump Rust to 1.87.0

* changelog

* changelog
* fix: selectively copy items from hadoop-builder image to final image to not overwrite things from the base image

* Apply suggestion from @NickLarsenNZ

---------

Co-authored-by: Nick <[email protected]>
* Change UID & GID to 1000/1000, reverting the previous change from PR #1164

* chore: Set default GID to 0

* chore: Update changelog

---------

Co-authored-by: Nick Larsen <[email protected]>
* chore: Change GID to 1000

* chore: Update changelog link

* chore: Remove changelog entry
* fix: Update nipyapi to 0.22.0 to work with custom nifi versions

* Update testing-tools/python/requirements.txt

Co-authored-by: Sebastian Bernauer <[email protected]>

---------

Co-authored-by: Sebastian Bernauer <[email protected]>
* feat: separate Dockerfile for Hadoop

* chore: changelog / lint fix

* feat: remove unnecessary Hadoop components in distributed JARs

* chore: rename HADOOP_HADOOP to HADOOP_VERSION

* chore: rename HBASE_HBASE to HBASE_VERSION
* Backport HADOOP-18583 & fix OpenSSL native library

* Update CHANGELOG.md

Co-authored-by: Nick <[email protected]>

---------

Co-authored-by: Nick <[email protected]>
@razvan razvan closed this Jul 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants