Support `DataType::Dictionary` for arrow schema conversion

[jdockerty]

The current implementation of arrow_schema_to_schema does not support the arrow Dictionary type and cannot be visited.

A small reproducer for this is as follows:

use arrow::datatypes::{DataType, Field, Schema};
use iceberg::arrow::arrow_schema_to_schema;

fn main() {
    let dict_field = Field::new(
        "my_dict",
        DataType::Dictionary(Box::new(DataType::Int32), Box::new(DataType::Utf8)),
        true,
    );
    let schema = Schema::new(vec![dict_field]);
    let _iceberg_schema = arrow_schema_to_schema(&schema).expect("Conversion works");
}

This produces the following error:

thread 'main' panicked at src/main.rs:11:59:
Conversion works: DataInvalid => Cannot visit Arrow data type: Dictionary(Int32, Utf8)

---

It looks like this Dictionary type does not exist within iceberg-rust as a native type either, nor the Java implementation.

Does this require an expansion to the Java implementation first?

[alamb]

I don't think it needs an expansion in Java -- the Parquet type system and Iceberg type systems are different than Arrow

What I think needs to happen is to extend this match expression to handle Dictionary types

iceberg-rust/crates/iceberg/src/arrow/schema.rs

Line 117 in b4bc6dd

match r#type {

I believe the visitor should probably just use the Dictionary's value type; Something like:

    match r#type {
...
        // use the inner type of the dictionary
        DataType::Dictionary(_key_type, value_type) => visit_type(&value_type, visitor)
...
}

[alamb]

• Possible dupe of Unable to convert Arrow Dictionary types to Iceberg schema #1282