-
Notifications
You must be signed in to change notification settings - Fork 246
Labels
enhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is needed
Description
What is the problem the feature request solves?
We currently delegate to DataFusion when casting from string to timestamp and there are some differences in behavior compared to Spark.
- Spark treats many parts of the timestamp as optional
- There are timezone-related differences
- DataFusion does not throw exceptions on invalid inputs in ANSI mode
Examples of differences:
| Input | Spark | Comet |
|---|---|---|
2020 |
2020-01-01 00:00:00.0 |
null |
2020-01 |
2020-01-01 00:00:00.0 |
null |
2020-01-01 |
2020-01-01 00:00:00.0 |
2019-12-31 16:00:00.0 |
2020-01-01T12 |
2020-01-01 12:00:00.0 |
null |
2020-01-01T12:34 |
2020-01-01 12:34:00.0 |
null |
2020-01-01T12:34:56 |
2020-01-01 12:34:56.0 |
2020-01-01 04:34:56.0 |
2020-01-01T12:34:56.123456 |
2020-01-01 12:34:56.123456 |
2020-01-01 04:34:56.123456 |
T2 |
2024-04-25 02:00:00.0 |
null |
Describe the potential solution
No response
Additional context
No response
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is needed