-
Notifications
You must be signed in to change notification settings - Fork 46
Hudi sink for Python #141
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hudi sink for Python #141
Conversation
|
Please also add the entry in the main README, in the Python Examples > Connectors section |
nicusX
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the contribution.
Just a couple of very small fixes.
Also, please add the entry of the new example in the top level README
python/HudiSink/README.md
Outdated
| * Flink Connectors: Apache Hudi & Flink S3 | ||
| * Language: Python | ||
|
|
||
| This application demonstrates settings up Apache Hudi table as sink. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
typo: "setting up"
python/HudiSink/README.md
Outdated
| | Group ID | Key | Mandatory | Example Value (default for local) | Notes | | ||
| |-----------------|-----------------|-----------|-----------------------------------|--------------------------------------------------| | ||
| | `HudiTable0` | `catalog.name` | Y | `glue_catalog` | Catalog name (used for configuration consistency, not for Glue integration) | | ||
| | `HudiTable0` | `warehouse.path`| Y | `s3a://my_bucket/my_warehouse` | Warehouse path for Hudi table. Use `s3a://` protocol (auto-converted from `s3://` if needed) | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe add a small note, about the fact it's the example application converting s3: to s3a:
For a second, I thought it was Flink doing it.
Alternatively, remove the auto-conversion and just say to use s3a:. This is possibly less confusion.
Up to you :)
1eea059 to
5de24ac
Compare
|
Made the requested changes. Should be good to merge. The force merge was due to a rebase and squash I performed to keep the history clean. |
nicusX
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Many thanks
Approving and merging
Purpose of the change
Sample application to write to Apache Hudi table format using Python/PyFlink
Verifying this change
Please test your changes both running locally, in the IDE, and in Managed Service for Apache Flink. All examples must run
in both environment without code changes.
Describe how you tested your application, show the output of the running application with screenshots.
Significant changes
(Please check any boxes [x] if the answer is "yes". You can first publish the PR and check them afterward, for convenience.)