-
Notifications
You must be signed in to change notification settings - Fork 136
Add sqlc templates for arbitrary text substitution #792
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
d78d533 to
b054626
Compare
bgentry
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Solid, this should help with a few different areas including unlocking full support for schemas and some other stuff in the works. Let's get it over the line ![]()
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume you're intending to fill this out a bit?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, just wanted to get your reaction before I spent too much time on it in case we decided to go a different direction.
This one aims to give us a workable resolution to one of our most common
problems with sqlc. Namely, that although it allows substitution for
parameters that work with a prepared query, it can't replace arbitrary
parts of a SQL query, leading to operations that aren't possible so that
we either don't do them or end up degrading to raw SQL that's only
checked at runtime.
Here, we add a `sqlctemplate` package that's designed to be run from
inside custom implementations of sqlc's `DBTX` interface so that it it
runs after sqlc's generated code but before the query goes to Postgres.
In sqlc code, templates look like this:
-- name: JobCountByState :one
SELECT count(*)
FROM /* TEMPLATE: schema */river_job
WHERE state = @State;
The template replacement is modeled as a comment so that it doesn't
interfere with with sqlc's parsing of SQL syntax. The above is valid SQL
with or without the template, but with it, `sqlctemplate` can add an
arbitrary schema name to the queried table.
It also supports a form of syntax where a value is required for SQL to
be valid. For example, `WHERE` and `ORDER BY` clauses both require a
value for them to be valid. Here, a stand in value is provide between
template tags. It's processed by sqlc's parser, but then replace by the
template engine before the SQL is executed:
-- name: JobList :many
SELECT *
FROM river_job
WHERE /* TEMPLATE_BEGIN: where_clause */ 1 /* TEMPLATE_END */
ORDER BY /* TEMPLATE_BEGIN: order_by_clause */ id /* TEMPLATE_END */
LIMIT @max::int;
Template values are injected via context (don't love this, but there's
no other way in getting information down to a layer below `DBTX`):
ctx = sqlctemplate.WithTemplates(ctx, map[string]sqlctemplate.Replacement{
"order_by_clause": {Value: params.OrderByClause},
"where_clause": {Value: params.WhereClause},
}, params.NamedArgs)
jobs, err := dbsqlc.New().JobList(ctx, e.dbtx, params.Max)
The template engine is written to be root out as many error as possible
by noticing if a template replacement is passed that doesn't have an
equivalent template in SQL, or if a template in SQL is present for which
there's no replacement.
Named args are support in templates similar to how sqlc supports them.
This allows pgx's prepared statement cache to continue to operate as it
did before, thereby keeping everything fast.
Lastly, I should note that templates are meant as a utility of last
resort. All effort should be made to resolve problems via mainstream
sqlc, and only bring in templates when there's no other option.
b054626 to
a45b502
Compare
|
Going to keep this around for now as reference for a while longer, but superseded by #794. |
This one aims to give us a workable resolution to one of our most common
problems with sqlc. Namely, that although it allows substitution for
parameters that work with a prepared query, it can't replace arbitrary
parts of a SQL query, leading to operations that aren't possible so that
we either don't do them or end up degrading to raw SQL that's only
checked at runtime.
Here, we add a
sqlctemplatepackage that's designed to be run frominside custom implementations of sqlc's
DBTXinterface so that it itruns after sqlc's generated code but before the query goes to Postgres.
In sqlc code, templates look like this:
The template replacement is modeled as a comment so that it doesn't
interfere with with sqlc's parsing of SQL syntax. The above is valid SQL
with or without the template, but with it,
sqlctemplatecan add anarbitrary schema name to the queried table.
It also supports a form of syntax where a value is required for SQL to
be valid. For example,
WHEREandORDER BYclauses both require avalue for them to be valid. Here, a stand in value is provide between
template tags. It's processed by sqlc's parser, but then replace by the
template engine before the SQL is executed:
Template values are injected via context (don't love this, but there's
no other way in getting information down to a layer below
DBTX):The template engine is written to be root out as many error as possible
by noticing if a template replacement is passed that doesn't have an
equivalent template in SQL, or if a template in SQL is present for which
there's no replacement.
Named args are support in templates similar to how sqlc supports them.
This allows pgx's prepared statement cache to continue to operate as it
did before, thereby keeping everything fast.
Lastly, I should note that templates are meant as a utility of last
resort. All effort should be made to resolve problems via mainstream
sqlc, and only bring in templates when there's no other option.