JDBC Support

[jlvarona-aexp]

Any objections for supporting creating a dataframe directly from a result set? as opposed to have to migrate the results to CSV or JSON or using Arrow. I see the SupportedDataFrameFormat interface only allows for a file or an InputStream. Are there any impediments that would make a result set difficult to support ?

[Jolanrensen]

SupportedDataFrameFormat is only used for reading file formats. We also are able to read from other types like DataFrame.readJsonStr(), which can parse an in-memory JSON instance for example.
I don't have an objection to having an extra DataFrame.Companion.read() overload that takes a JDBC result set and converts it to a DataFrame :).
Just make sure it's in a separate module, like openapi, arrow etc. Then people can choose to include it or not (we want to move all supported formats to separate modules in the future).

[PaulWoitaschek]

Here's some hacky code for the meantime ;)

fun ResultSet.toDataFrame(): AnyFrame {
    val columnCount = metaData.columnCount
    val columns = List(columnCount) { mutableListOf<Any?>() }
    while (next()) {
        repeat(columnCount) { i ->
            columns[i].add(getObject(i + 1))
        }
    }
    return columns.mapIndexed { index, values ->
        DataColumn.createValueColumn(
            name = metaData.getColumnName(index + 1),
            values = values,
            type = getKotlinTypeForSqlType(metaData.getColumnType(index + 1))
        )
    }.toDataFrame()
}

fun getKotlinTypeForSqlType(sqlType: Int): KType {
    return when (sqlType) {
        Types.BIT -> typeOf<Boolean?>()
        Types.TINYINT -> typeOf<Byte?>()
        Types.SMALLINT -> typeOf<Short?>()
        Types.INTEGER -> typeOf<Int?>()
        Types.BIGINT -> typeOf<Long?>()
        Types.FLOAT -> typeOf<Float?>()
        Types.REAL -> typeOf<Float?>()
        Types.DOUBLE -> typeOf<Double?>()
        Types.NUMERIC -> typeOf<BigDecimal?>()
        Types.DECIMAL -> typeOf<BigDecimal?>()
        Types.CHAR -> typeOf<Char?>()
        Types.VARCHAR -> typeOf<String?>()
        Types.LONGVARCHAR -> typeOf<String?>()
        Types.DATE -> typeOf<Date?>()
        Types.TIME -> typeOf<Time?>()
        Types.TIMESTAMP -> typeOf<Timestamp?>()
        Types.BINARY -> typeOf<ByteArray?>()
        Types.VARBINARY -> typeOf<ByteArray?>()
        Types.LONGVARBINARY -> typeOf<ByteArray?>()
        Types.NULL -> typeOf<String?>()
        Types.OTHER -> typeOf<Any?>()
        Types.JAVA_OBJECT -> typeOf<Any?>()
        Types.DISTINCT -> typeOf<Any?>()
        Types.STRUCT -> typeOf<Any?>()
        Types.ARRAY -> typeOf<Array<Any?>?>()
        Types.BLOB -> typeOf<Blob?>()
        Types.CLOB -> typeOf<Clob?>()
        Types.REF -> typeOf<Ref?>()
        Types.DATALINK -> typeOf<Any?>()
        Types.BOOLEAN -> typeOf<Boolean?>()
        Types.ROWID -> typeOf<RowId?>()
        Types.NCHAR -> typeOf<Char?>()
        Types.NVARCHAR -> typeOf<String?>()
        Types.LONGNVARCHAR -> typeOf<String?>()
        Types.NCLOB -> typeOf<NClob?>()
        Types.SQLXML -> typeOf<SQLXML?>()
        else -> error("Unknown sql type: $sqlType")
    }
}