[SPARK-46808][PYTHON] Refine error classes in Python with automatic sorting function #44848
Conversation
|
cc @itholic |
itholic
left a comment
itholic
left a comment
There was a problem hiding this comment.
My comment is not actually about the PR itself, so it is LGTM. Let me do some further investigation in separate ticket. Thanks for the improvement!
| "Argument `<arg_name>`(type: <arg_type>) should only contain a type in [<allowed_types>], got <return_type>" | ||
| "DISALLOWED_TYPE_FOR_CONTAINER": { | ||
| "message": [ | ||
| "Argument `<arg_name>`(type: <arg_type>) should only contain a type in [<allowed_types>], got <item_type>" |
There was a problem hiding this comment.
Oh... does it mean that the parameter name was not matched between template and actual usage but didn't raise any error so far? Let me investigate and create a ticket for it.
There was a problem hiding this comment.
Yeah. I think it wasn't being tested.
| error_class="DISALLOWED_TYPE_FOR_CONTAINER", | ||
| message_parameters={ | ||
| "arg_name": "parameters", | ||
| "arg_type": type(parameters).__name__, |
There was a problem hiding this comment.
Hmm... this also seems to be problematic. It seems that an error should have occurred if a parameter defined in the template was actually missing. Let me investigate this too.
Also we might need further effort to refine the error class (duplicated name, consistency of messages, etc.). Will add some more items into SPARK-45673. (and maybe also include this ticket into SPARK-45673 ?) |
|
Sure. let me put this ticket under that. |
| # limitations under the License. | ||
| # | ||
| # NOTE: Automatically sort this file via |
There was a problem hiding this comment.
If error_classes.py is not meant to be edited manually, I would add a clear warning here so people don't mistakenly edit the file, similar to #44847.
There was a problem hiding this comment.
Oh actually this case is slightly different. Has to be manually edited first, and then reformatted via that code :-).
|
Merged to master. |
… `test_error_classes_sorted` ### What changes were proposed in this pull request? This PR is a followup of #44848 that adds a bit of guide to sort the error classes. ### Why are the changes needed? For developers to easily sort the error classes. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? Manually. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #44874 from HyukjinKwon/minor-error-sort. Authored-by: Hyukjin Kwon <[email protected]> Signed-off-by: Ruifeng Zheng <[email protected]>
4 participants
What changes were proposed in this pull request?
This PR proposes:
error_classes.pyfile,from pyspark.errors.exceptions import _write_self; _write_self()Why are the changes needed?
Does this PR introduce any user-facing change?
Yes, it fixes a couple of typos.
How was this patch tested?
Unittests were fixed together.
Was this patch authored or co-authored using generative AI tooling?
No.