When to use types vs dicts with this library

[jceresini]

I've had a hard time using this library. I first used an example I found to create a task using this simple dictionary

I found the example by going here https://cloud.google.com/tasks/docs/quickstart which linked me to here https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/appengine/flexible/tasks/create_app_engine_queue_task.py

The example looks like this:

    task = {
            'app_engine_http_request': {  # Specify the type of request.
                'http_method': tasks_v2.HttpMethod.POST,
                'relative_uri': '/example_task_handler'
            }
    }

I modified it to be an http_request rather than app_engine_http_request after reading through the docs here https://cloud.google.com/tasks/docs/reference/rest/v2/projects.locations.queues.tasks/create. I also want to use the OAuthToken feature so I can have cloud tasks trigger a call to another google cloud API. Because the http_method above uses tasks_v2_HttpMethod.POST, I assumed I needed to use tasks_v2.OAuthToken() in my task definition. I ended up with this (I'm including a full snippet, but note the task definition):

from google.cloud import tasks_v2

client = tasks_v2.CloudTasksClient()

task = {
    "http_request": {
        "http_method": tasks_v2.HttpMethod.POST,
        "url": "https://pubsub.googleapis.com/v1/projects/my-project/topics/testtopic:publish",
        "body": b"eyJtZXNzYWdlcyI6IFt7ImRhdGEiOiAiVkdocGN5QnBjeUJoSUhSbGMzUUsifV19Cg==",
        "oauth_token": tasks_v2.OAuthToken(service_account_email='my-svc-acct@my-project.iam.gserviceaccount.com'),
    }
}

parent = 'projects/my-project/locations/us-central1/queues/my-queue'

resp = client.create_task(parent=parent, task=task)

But when I run that code I get the following error:

Traceback (most recent call last):
  File "/path/to/example_tasks.py", line 18, in <module>
    resp = client.create_task(parent=parent, task=task)
  File "/path/to/.venv/lib/python3.9/site-packages/google/cloud/tasks_v2/services/cloud_tasks/client.py", line 1700, in create_task
    request.task = task
  File "/path/to/.venv/lib/python3.9/site-packages/proto/message.py", line 632, in __setattr__
    pb_value = marshal.to_proto(pb_type, value)
  File "/path/to/.venv/lib/python3.9/site-packages/proto/marshal/marshal.py", line 208, in to_proto
    pb_value = rule.to_proto(value)
  File "/path/to/.venv/lib/python3.9/site-packages/proto/marshal/rules/message.py", line 32, in to_proto
    return self._descriptor(**value)
TypeError: Parameter to MergeFrom() must be instance of same class: expected google.cloud.tasks.v2.OAuthToken got OAuthToken.

That error seems odd, because it looks like I passed it what it expected.I suspect that is a bug, but below you'll see I got it working with a dict so I wasn't sure... so I submitted this as a support request rather than a bug.

After some fumbling around I noticed if I pass it a dictionary it seems to work. The example payload here is:

task = {
    "http_request": {
        "http_method": tasks_v2.HttpMethod.POST,
        "url": "https://pubsub.googleapis.com/v1/projects/my-project/topics/testtopic:publish",
        "body": b"eyJtZXNzYWdlcyI6IFt7ImRhdGEiOiAiVkdocGN5QnBjeUJoSUhSbGMzUUsifV19Cg==",
        "oauth_token": {"service_account_email":"my-svc-acct@my-project.iam.gserviceaccount.com"},
    }
}

The documentation here isn't very helpful either: https://googleapis.dev/python/cloudtasks/latest/

I'm working on a POC using CloudTasks, and I'm making progress by testing various things, and trying to read through the code, but it been a bit frustrating so far.

[busunkim96]

Hi @jceresini,

Thank you for taking the time to leave feedback.

The short answer is that completely sticking with one or the either (dict or types) will work 100% of the time. Mixing (dicts with types nested inside OR types with dicts nested inside) will only work sometimes because of bugs like googleapis/proto-plus-python#161 and googleapis/proto-plus-python#135.

Below I created the same task you provided in your sample 3 ways. I generally recommend folks use the generated types since IDEs are able to provide intellisense and it's possible to do better type checking. We still have rough edges with the generated types (odd/unhelpful docstrings) that we need to look into. However, I still believe that experience is better than working with untyped dictionaries.

from google.cloud import tasks_v2


# 1. Generated types via constructor
task_from_constructor = tasks_v2.Task(
    http_request=tasks_v2.HttpRequest(
        http_method=tasks_v2.HttpMethod.POST,
        url="https://pubsub.googleapis.com/v1/projects/my-project/topics/testtopic:publish",
        body=b"eyJtZXNzYWdlcyI6IFt7ImRhdGEiOiAiVkdocGN5QnBjeUJoSUhSbGMzUUsifV19Cg==",
        oauth_token=tasks_v2.OAuthToken(
            service_account_email='my-svc-acct@my-project.iam.gserviceaccount.com'
        )
    )
)

# 2. Via dictionary
task_from_dict = {
    "http_request": {
        "http_method": "POST",
        "url": "https://pubsub.googleapis.com/v1/projects/my-project/topics/testtopic:publish",
        "body": b"eyJtZXNzYWdlcyI6IFt7ImRhdGEiOiAiVkdocGN5QnBjeUJoSUhSbGMzUUsifV19Cg==",
        "oauth_token": {"service_account_email":"my-svc-acct@my-project.iam.gserviceaccount.com"},
    }
}

# 3. Instantiate object and then set attributes
http_request = tasks_v2.HttpRequest()
http_request.http_method = tasks_v2.HttpMethod.POST
http_request.url = "https://pubsub.googleapis.com/v1/projects/my-project/topics/testtopic:publish"
http_request.body = b"eyJtZXNzYWdlcyI6IFt7ImRhdGEiOiAiVkdocGN5QnBjeUJoSUhSbGMzUUsifV19Cg==",
http_request.oauth_token.service_account_email = "my-svc-acct@my-project.iam.gserviceaccount.com"

task = tasks_v2.Task()
task.http_request = http_request

Side Note: The library ultimately converts any dictionaries into the library type to create protobuf messages (see this code snippet for an example)

python-tasks/google/cloud/tasks_v2/services/cloud_tasks/client.py

Lines 427 to 432 in b8ec21e

	# Minor optimization to avoid making a copy if the user passes
	# in a cloudtasks.ListQueuesRequest.
	# There's no risk of modifying the input as we've already verified
	# there are no flattened fields.
	if not isinstance(request, cloudtasks.ListQueuesRequest):
	request = cloudtasks.ListQueuesRequest(request)

[jceresini]

Ah, that makes sense. That might make a good case for updating the samples to be one or the other. It would probably save others from similar issues.

Thanks for the details and the links to the related issues.

[parthea]

I'm going to close this as a solution was provided but please feel free to re-open it if required.