[SPARK-28408][SQL][TEST] Restrict test values for DateType, TimestampType and CalendarIntervalType #25166
+42
−13
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Test build #107704 has finished for PR 25166 at commit
|
HyukjinKwon
approved these changes
Jul 16, 2019
dongjoon-hyun
approved these changes
Jul 16, 2019
Member
|
Merged to master. Thank you, @MaxGekk and @HyukjinKwon . |
vinodkc
pushed a commit
to vinodkc/spark
that referenced
this pull request
Jul 18, 2019
…Type and CalendarIntervalType
## What changes were proposed in this pull request?
Existing random generators in tests produce wide ranges of values that can be out of supported ranges for:
- `DateType`, the valid range is `[0001-01-01, 9999-12-31]`
- `TimestampType` supports values in `[0001-01-01T00:00:00.000000Z, 9999-12-31T23:59:59.999999Z]`
- `CalendarIntervalType` should define intervals for the ranges above.
Dates and timestamps produced by random literal generators are usually out of valid ranges for those types. And tests just check invalid values or values caused by arithmetic overflow.
In the PR, I propose to restrict tested pseudo-random values by valid ranges of `DateType`, `TimestampType` and `CalendarIntervalType`. This should allow to check valid values in test, and avoid wasting time on a priori invalid inputs.
## How was this patch tested?
The changes were checked by `DateExpressionsSuite` and modified `DateTimeUtils.dateAddMonths`:
```Scala
def dateAddMonths(days: SQLDate, months: Int): SQLDate = {
localDateToDays(LocalDate.ofEpochDay(days).plusMonths(months))
}
```
Closes apache#25166 from MaxGekk/datetime-lit-random-gen.
Authored-by: Maxim Gekk <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Existing random generators in tests produce wide ranges of values that can be out of supported ranges for:
DateType, the valid range is[0001-01-01, 9999-12-31]TimestampTypesupports values in[0001-01-01T00:00:00.000000Z, 9999-12-31T23:59:59.999999Z]CalendarIntervalTypeshould define intervals for the ranges above.Dates and timestamps produced by random literal generators are usually out of valid ranges for those types. And tests just check invalid values or values caused by arithmetic overflow.
In the PR, I propose to restrict tested pseudo-random values by valid ranges of
DateType,TimestampTypeandCalendarIntervalType. This should allow to check valid values in test, and avoid wasting time on a priori invalid inputs.How was this patch tested?
The changes were checked by
DateExpressionsSuiteand modifiedDateTimeUtils.dateAddMonths: