Implement Spark-compatible CAST between integer types

[andygrove]

We currently delegate to DataFusion for casts between integer types, which produces different results when compared to Spark.

For example, when casting from long to short in LEGACY mode, Spark will cast -9197595427105923027 to 8237 and DataFusion will cast to null because we are using DataFusion's CastOptions with safe mode disabled, which is equivalent to Spark's TRY mode.

The behavior in TRY mode is correct.

The behavior in ANSI mode is basically correct but the error message returned by DataFusion does not match Spark and it isn't practical to try and parse the error string and translate to Spark's error message.

It is probably best for now just to write our own Spark-compatible cast. The following code sample shows the two methods for casting that we would need to use (try_from for TRY and ANSI mode, and as for LEGACY mode).

fn main() {
    let v = -9197595427105923027_i64;
    println!("{:?}", i16::try_from(v));
    println!("{:}", v as i16);
}

This produces the following output:

Err(TryFromIntError(()))
8237

[ganeshkumar269]

Hi I would like to work on this, if nobody else already is.

[viirya]

Thank you @ganeshkumar269