Skip to content

Conversation

@GaganBrahmi
Copy link
Contributor

Purpose of the change

Sample application to write to Apache Hudi table format using Python/PyFlink

Verifying this change

Please test your changes both running locally, in the IDE, and in Managed Service for Apache Flink. All examples must run
in both environment without code changes.

Describe how you tested your application, show the output of the running application with screenshots.

[INFO] 
[INFO] --- assembly:3.3.0:single (make-assembly) @ managed-flink-pyfink-hudi-sink-example ---
[INFO] Reading assembly descriptor: assembly/assembly.xml
[INFO] Building zip: /Users/brahmi/git-repos/amazon-managed-service-for-apache-flink-examples/python/HudiSink/target/managed-flink-pyfink-hudi-sink-example-1.0.0.zip
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  19.858 s
[INFO] Finished at: 2025-09-29T07:33:52-07:00
[INFO] ------------------------------------------------------------------------

Significant changes

(Please check any boxes [x] if the answer is "yes". You can first publish the PR and check them afterward, for convenience.)

  • Completely new example
  • Updated an existing example to newer Flink version or dependencies versions
  • Improved an existing example
  • Modified the runtime configuration of an existing example (i.e. added/removed/modified any runtime properties)
  • Modified the expected input or output of an existing example (e.g. modified the source or sink, modified the record schema)

@nicusX
Copy link
Contributor

nicusX commented Oct 16, 2025

Please also add the entry in the main README, in the Python Examples > Connectors section

Copy link
Contributor

@nicusX nicusX left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution.
Just a couple of very small fixes.
Also, please add the entry of the new example in the top level README

* Flink Connectors: Apache Hudi & Flink S3
* Language: Python

This application demonstrates settings up Apache Hudi table as sink.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: "setting up"

| Group ID | Key | Mandatory | Example Value (default for local) | Notes |
|-----------------|-----------------|-----------|-----------------------------------|--------------------------------------------------|
| `HudiTable0` | `catalog.name` | Y | `glue_catalog` | Catalog name (used for configuration consistency, not for Glue integration) |
| `HudiTable0` | `warehouse.path`| Y | `s3a://my_bucket/my_warehouse` | Warehouse path for Hudi table. Use `s3a://` protocol (auto-converted from `s3://` if needed) |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a small note, about the fact it's the example application converting s3: to s3a:
For a second, I thought it was Flink doing it.
Alternatively, remove the auto-conversion and just say to use s3a:. This is possibly less confusion.
Up to you :)

@GaganBrahmi
Copy link
Contributor Author

Made the requested changes. Should be good to merge. The force merge was due to a rebase and squash I performed to keep the history clean.

Copy link
Contributor

@nicusX nicusX left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many thanks
Approving and merging

@nicusX nicusX merged commit ecd3380 into aws-samples:main Oct 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants