[V2][Example] Setup e2e DeltaCAT read & write example that runs locally #528

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

MingshiPeng opened this issue Apr 16, 2025 · 0 comments

Collaborator

MingshiPeng commented Apr 16, 2025

Currently we have an example that runs on AWS (depends on Glue and S3) [TODO: add example code link]. This causes 2 issues:

it complicates customer experience in running the example as they are forced to setup AWS account and credentials first
it also blocks DeltaCAT dev to setup e2e locally runnable tests

Known gaps for supporting running example in local mode

deltacat write_table API only supports s3 destinations, not local file systems

AC

create e2e DeltaCAT read & write example that runs locally

MingshiPeng mentioned this issue

DeltaCAT ScanOperator implementation #527

Merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment