Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions format/FlightSql.proto
Original file line number Diff line number Diff line change
Expand Up @@ -817,6 +817,17 @@ enum SqlInfo {
* - true: if invoking user-defined or vendor functions using the stored procedure escape syntax is supported.
*/
SQL_STORED_FUNCTIONS_USING_CALL_SYNTAX_SUPPORTED = 576;

/*
* Retrieves a boolean value indicating whether transactions are supported for bulk ingestion. If not, invoking
* the method commit in the context of a bulk ingestion is a noop, and the isolation level is
* `arrow.flight.protocol.sql.SqlTransactionIsolationLevel.TRANSACTION_NONE`.
*
* Returns:
* - false: if bulk ingestion transactions are unsupported;
* - true: if bulk ingestion transactions are supported.
*/
INGEST_TRANSACTIONS_SUPPORTED = 577;
}

// The level of support for Flight SQL transaction RPCs.
Expand Down Expand Up @@ -1778,6 +1789,47 @@ message CommandPreparedStatementUpdate {
bytes prepared_statement_handle = 1;
}

/*
* Represents a bulk ingestion request. Used in the command member of FlightDescriptor
* for the the RPC call DoPut to cause the server load the contents of the stream's
* FlightData into the target destination.
*/
message CommandStatementIngest {
option (experimental) = true;

// Describes the behavior for loading bulk data.
enum IngestMode {
// Ingestion behavior unspecified.
INGEST_MODE_UNSPECIFIED = 0;
// Create the target table. Fail if the target table already exists.
INGEST_MODE_CREATE = 1;
// Append to an existing target table. Fail if the target table does not exist.
INGEST_MODE_APPEND = 2;
// Drop the target table if it exists. Then follow INGEST_MODE_CREATE behavior.
INGEST_MODE_REPLACE = 3;
// Create the target table if it does not exist. Then follow INGEST_MODE_APPEND behavior.
INGEST_MODE_CREATE_APPEND = 4;
}

// The ingestion behavior.
IngestMode mode = 1;
// The table to load data into.
string table = 2;
// The db_schema of the destination table to load data into. If unset, a backend-specific default may be used.
optional string schema = 3;
// The catalog of the destination table to load data into. If unset, a backend-specific default may be used.
optional string catalog = 4;
// Use a temporary table.
optional bool temporary = 5;
// Perform the ingestion as part of this transaction (if unset, the query is auto-committed).
optional bytes transaction_id = 6;

// Future extensions to the parameters of CommandStatementIngest should be added here, at a lower index than the generic 'options' parameter.

// Backend-specific options.
map<string, string> options = 1000;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we want to use 1000 instead of 7 here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is that options can be used for any parameters not otherwise supported by the spec, which makes it different from any one parameter. By giving this a high number, it leaves room for more explicit parameters to be added to the spec at lower numbers as the spec evolves.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. Could you add a comment how to add a new explicit parameter?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I've just added a comment.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand this rationale. What would it change to add new parameters at higher numbers, rather than at lower numbers?

}

/*
* Returned from the RPC call DoPut when a CommandStatementUpdate
* CommandPreparedStatementUpdate was in the request, containing
Expand Down