Support Array types

[Matthieu-Tinycoaching]

Hi,

I tried to use ConnectorX to load dataframe from a postgreSQL database with the following code:

import connectorx as cx

tableName   = "semantic_search"
dataFrame = cx.read_sql('postgresql://postgres:postgres@localhost:5432/embeddings_sts_tf', "select * from " + tableName)

but got the following error:

thread '<unnamed>' panicked at 'not implemented: _float8', connectorx/src/sources/postgres/typesystem.rs:78:22
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Traceback (most recent call last):
  File "/home/matthieu/Code/Python/postgreSQL_creation.py", line 142, in <module>
    dataFrame = cx.read_sql('postgresql://{user}:{pw}@{host}:5432/{db}'.format(host=conn_info["host"], db=conn_info["database"], user=conn_info["user"], pw=conn_info["password"]), "select * from " + tableName)
  File "/home/matthieu/anaconda3/envs/sts-transformers-gpu-fresh/lib/python3.8/site-packages/connectorx/__init__.py", line 99, in read_sql
    result = _read_sql(
pyo3_runtime.PanicException: not implemented: _float8

Thanks for helping!

[wangxiaoying]

Hi @Matthieu-Tinycoaching, it seems like the type _float8 refers to the Array type in postgres (Float8Array), which we do not support yet. Currently supported types for postgres can be found here: https://github.com/sfu-db/connector-x/blob/main/Types.md#postgres-pandas-type-mapping

We will add the support of arrays in our next release (0.2.1).

[Matthieu-Tinycoaching]

Hi @wangxiaoying thanks for the feedback. Could you estimate the date of next release?

Thanks!

[wangxiaoying]

Hi @wangxiaoying thanks for the feedback. Could you estimate the date of next release?

Thanks!

@Matthieu-Tinycoaching sorry for the waiting! We just released the connectorx-0.2.1a1 which included both FloatArray and IntArray type in Postgres. You can try it by using pip install -U connectorx==0.2.1a1.

If all tests go well, we plan to release the 0.2.1 version in early October. Please feel free to have a try and let us know if you found any issue.