Skip to content

Bug: REST transport URL-encodes $alt as %24alt, causing API errors #2514

@taimo3810

Description

@taimo3810

The generated REST transport passes query parameters to the requests library via the params argument, which causes the $ character in $alt to be URL-encoded as %24. This results in API errors because the server does not recognize %24alt as a valid field.

Environment

  • Affected: All generated REST transports that use $alt parameter
  • Confirmed in: google-cloud-aiplatform (MatchService, etc.)

Reproduction

Minimal case demonstrating the encoding issue:

import requests

req = requests.Request(
    'POST',
    'https://example.com/api',
    params=[('$alt', 'json;enum-encoding=int')]
)
prepared = req.prepare()
print(prepared.url)
# Output: https://example.com/api?%24alt=json%3Benum-encoding%3Dint
#         ^ $alt is incorrectly encoded as %24alt

SDK reproduction (using google-cloud-aiplatform as example):

from google.cloud import aiplatform

aiplatform.init(api_transport="rest")
endpoint = aiplatform.MatchingEngineIndexEndpoint(endpoint_id)

# This fails with the error below
endpoint.find_neighbors(
    deployed_index_id=deployed_index_id,
    queries=[[1.0] * 768],
    num_neighbors=10,
)

Error Message

google.api_core.exceptions.BadRequest: 400 POST
https://<endpoint>.vdb.vertexai.goog/v1beta1/.../findNeighbors?%24alt=json%3Benum-encoding%3Dint:
Could not find field "%24alt" in the type "google.cloud.aiplatform.v1beta1.FindNeighborsRequest".

Root Cause

The issue is in the generated _get_response method in _shared_macros.j2:

# gapic/templates/%namespace/%name_%version/%sub/services/%service/_shared_macros.j2

response = {{ await_prefix }}getattr(session, method)(
    "{host}{uri}".format(host=host, uri=uri),
    timeout=timeout,
    headers=headers,
    params=rest_helpers.flatten_query_params(query_params, strict=True),  # <- Problem here
    ...
)

When params is passed to requests, it URL-encodes special characters including $.

Suggested Fix

Build the query string manually to prevent encoding of $:

from urllib.parse import urlencode

query_string = urlencode(
    rest_helpers.flatten_query_params(query_params, strict=True),
    safe='$'  # Do not encode $
)
url = "{host}{uri}".format(host=host, uri=uri)
if query_string:
    url = f"{url}?{query_string}"

response = {{ await_prefix }}getattr(session, method)(
    url,
    timeout=timeout,
    headers=headers,
    ...
)

Related Issues

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions