Error using IN list on dictionary encoded data: InList does not support datatype Dictionary(Int32, Utf8). #3936
Description
Describe the bug
I am writing a query to select some values from a dictionary encoded string column trace_id and I get an error
select * from spans where trace_id IN ('187dcbbc68a83a0d', '335424324532');
Results in this error
This feature is not implemented: InList does not support datatype Dictionary(Int32, Utf8).
To Reproduce
Run this test:
diff --git a/datafusion/core/tests/sql/predicates.rs b/datafusion/core/tests/sql/predicates.rs
index 07e016a27..5109f0167 100644
--- a/datafusion/core/tests/sql/predicates.rs
+++ b/datafusion/core/tests/sql/predicates.rs
@@ -427,6 +427,32 @@ async fn csv_in_set_test() -> Result<()> {
Ok(())
}
+#[tokio::test]
+async fn in_set_string_dictionaries() -> Result<()> {
+ let input = vec![Some("foo"), Some("bar"), None, Some("fazzz")]
+ .into_iter()
+ .collect::<DictionaryArray<Int32Type>>();
+
+ let batch = RecordBatch::try_from_iter(vec![("c1", Arc::new(input) as _)]).unwrap();
+
+ let ctx = SessionContext::new();
+ ctx.register_batch("test", batch)?;
+
+ let sql = "SELECT * FROM test WHERE c1 IN ('foo', 'Bar', 'fazz')";
+ let actual = execute_to_batches(&ctx, sql).await;
+ let expected = vec![
+ "+-------+",
+ "| c1 |",
+ "+-------+",
+ "| foo |",
+ "| fazzz |",
+ "+-------+",
+ ];
+
+ assert_batches_eq!(expected, &actual);
+ Ok(())
+}
+Results in
thread 'sql::predicates::in_set_string_dictionaries' panicked at 'called `Result::unwrap()` on an `Err` value: "ArrowError(ExternalError(NotImplemented(\"InList does not support datatype Dictionary(Int32, Utf8).\"))) at ...
Expected behavior
Test should pass
Additional context
Found this in IOx as part of https://github.com/influxdata/influxdb_iox/issues/5959