Add samples for Cloud Tasks (#1068)

* Add samples for Cloud Tasks

* Respond to tasks sample review

* Update app engine queues samples

* Address review feedback

* Address review issues and convert pull queue sample to not use API key auth

* Reform pull queues to match appengine queues changes to auth, command line input, readme

* flake8 and fix comment

Author: andrewsg

appengine/flexible/tasks/README.md

@@ -0,0 +1,87 @@
+# Google Cloud Tasks App Engine Queue Samples
+
+Sample command-line program for interacting with the Cloud Tasks API
+using App Engine queues.
+
+App Engine queues push tasks to an App Engine HTTP target. This directory
+contains both the App Engine app to deploy, as well as the snippets to run
+locally to push tasks to it, which could also be called on App Engine.
+
+`app_engine_queue_snippets.py` is a simple command-line program to create tasks
+to be pushed to the App Engine app.
+
+`main.py` is the main App Engine app. This app serves as an endpoint to receive
+App Engine task attempts.
+
+`app.yaml` configures the App Engine app.
+
+
+## Prerequisites to run locally:
+
+Please refer to [Setting Up a Python Development Environment](https://cloud.google.com/python/setup).
+
+## Authentication
+
+To set up authentication, please refer to our
+[authentication getting started guide](https://cloud.google.com/docs/authentication/getting-started).
+
+## Creating a queue
+
+To create a queue using the Cloud SDK, use the following gcloud command:
+
+    gcloud alpha tasks queues create-app-engine-queue my-appengine-queue
+
+Note: A newly created queue will route to the default App Engine service and
+version unless configured to do otherwise. Read the online help for the
+`create-app-engine-queue` or the `update-app-engine-queue` commands to learn
+about routing overrides for App Engine queues.
+
+## Deploying the App Engine app
+
+Deploy the App Engine app with gcloud:
+
+    gcloud app deploy
+
+Verify the index page is serving:
+
+    gcloud app browse
+
+The App Engine app serves as a target for the push requests. It has an
+endpoint `/log_payload` that reads the payload (i.e., the request body) of the
+HTTP POST request and logs it. The log output can be viewed with:
+
+    gcloud app logs read
+
+## Running the Samples
+
+Set environment variables:
+
+First, your project ID:
+
+    export PROJECT_ID=my-project-id
+
+Then the queue ID, as specified at queue creation time. Queue IDs already
+created can be listed with `gcloud alpha tasks queues list`.
+
+    export QUEUE_ID=my-appengine-queue
+
+And finally the location ID, which can be discovered with
+`gcloud alpha tasks queues describe $QUEUE_ID`, with the location embedded in
+the "name" value (for instance, if the name is
+"projects/my-project/locations/us-central1/queues/my-appengine-queue", then the
+location is "us-central1").
+
+    export LOCATION_ID=us-central1
+
+Create a task, targeted at the `log_payload` endpoint, with a payload specified:
+
+   python create_app_engine_queue_task.py --project=$PROJECT_ID --queue=$QUEUE_ID --location=$LOCATION_ID --payload=hello
+
+Now view that the payload was received and verify the payload:
+
+    gcloud app logs read
+
+Create a task that will be scheduled for a time in the future using the
+`--in_seconds` flag:
+
+    python create_app_engine_queue_task.py --project=$PROJECT_ID --queue=$QUEUE_ID --location=$LOCATION_ID --payload=hello --in_seconds=30

appengine/flexible/tasks/app.yaml

@@ -0,0 +1,6 @@
+runtime: python
+env: flex
+entrypoint: gunicorn -b :$PORT main:app
+
+runtime_config:
+  python_version: 3

appengine/flexible/tasks/create_app_engine_queue_task.py

@@ -0,0 +1,110 @@
+# Copyright 2017 Google Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import print_function
+
+import argparse
+import base64
+import datetime
+import json
+
+from googleapiclient import discovery
+
+
+def seconds_from_now_to_rfc3339_datetime(seconds):
+    """Return an RFC 3339 datetime string for a number of seconds from now."""
+    d = datetime.datetime.utcnow() + datetime.timedelta(seconds=seconds)
+    return d.isoformat('T') + 'Z'
+
+
+def create_task(project, queue, location, payload=None, in_seconds=None):
+    """Create a task for a given queue with an arbitrary payload."""
+
+    # Create a client.
+    DISCOVERY_URL = (
+        'https://cloudtasks.googleapis.com/$discovery/rest?version=v2beta2')
+    client = discovery.build(
+        'cloudtasks', 'v2beta2', discoveryServiceUrl=DISCOVERY_URL)
+
+    url = '/log_payload'
+    body = {
+        'task': {
+            'app_engine_task_target': {
+                'http_method': 'POST',
+                'relative_url': url
+            }
+        }
+    }
+
+    if payload is not None:
+        # Payload is a string (unicode), and must be encoded for base64.
+        # The finished request body is JSON, which requires unicode.
+        body['task']['app_engine_task_target']['payload'] = base64.b64encode(
+            payload.encode()).decode()
+
+    if in_seconds is not None:
+        scheduled_time = seconds_from_now_to_rfc3339_datetime(in_seconds)
+        body['task']['schedule_time'] = scheduled_time
+
+    queue_name = 'projects/{}/locations/{}/queues/{}'.format(
+        project, location, queue)
+
+    print('Sending task {}'.format(json.dumps(body)))
+
+    response = client.projects().locations().queues().tasks().create(
+        parent=queue_name, body=body).execute()
+
+    # By default CreateTaskRequest.responseView is BASIC, so not all
+    # information is retrieved by default because some data, such as payloads,
+    # might be desirable to return only when needed because of its large size
+    # or because of the sensitivity of data that it contains.
+    print('Created task {}'.format(response['name']))
+    return response
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(
+        description=create_task.__doc__,
+        formatter_class=argparse.RawDescriptionHelpFormatter)
+
+    parser.add_argument(
+        '--project',
+        help='Project of the queue to add the task to.'
+    )
+
+    parser.add_argument(
+        '--queue',
+        help='ID (short name) of the queue to add the task to.'
+    )
+
+    parser.add_argument(
+        '--location',
+        help='Location of the queue to add the task to.'
+    )
+
+    parser.add_argument(
+        '--payload',
+        help='Optional payload to attach to the push queue.'
+    )
+
+    parser.add_argument(
+        '--in_seconds',
+        help='The number of seconds from now to schedule task attempt.'
+    )
+
+    args = parser.parse_args()
+
+    create_task(
+        args.project, args.queue, args.location,
+        args.payload, args.in_seconds)

appengine/flexible/tasks/create_app_engine_queue_task_test.py

@@ -0,0 +1,33 @@
+# Copyright 2016 Google Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import mock
+
+import create_app_engine_queue_task
+
+TEST_PROJECT = 'mock-project'
+TEST_LOCATION = 'us-central1'
+TEST_QUEUE = 'my-appengine-queue'
+
+
+@mock.patch('googleapiclient.discovery.build')
+def test_create_task(build):
+    projects = build.return_value.projects.return_value
+    locations = projects.locations.return_value
+    create_function = locations.queues.return_value.tasks.return_value.create
+    execute_function = create_function.return_value.execute
+    execute_function.return_value = {'name': 'task_name'}
+    create_app_engine_queue_task.create_task(
+        TEST_PROJECT, TEST_QUEUE, TEST_LOCATION)
+    assert execute_function.called

appengine/flexible/tasks/main.py

@@ -0,0 +1,41 @@
+# Copyright 2016 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""App Engine app to serve as an endpoint for App Engine queue samples."""
+
+import logging
+
+from flask import Flask, request
+
+app = Flask(__name__)
+
+
+@app.route('/log_payload', methods=['POST'])
+def log_payload():
+    """Log the request payload."""
+    payload = request.data or "empty payload"
+    logging.warn(payload)
+    return 'Logged request payload: {}'.format(payload)
+
+
+@app.route('/')
+def hello():
+    """Basic index to verify app is serving."""
+    return 'Hello World!'
+
+
+if __name__ == '__main__':
+    # This is used when running locally. Gunicorn is used to run the
+    # application on Google App Engine. See entrypoint in app.yaml.
+    app.run(host='127.0.0.1', port=8080, debug=True)

appengine/flexible/tasks/main_test.py

@@ -0,0 +1,46 @@
+# Copyright 2016 Google Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import mock
+import pytest
+
+
+@pytest.fixture
+def app():
+    import main
+    main.app.testing = True
+    return main.app.test_client()
+
+
+def test_index(app):
+    r = app.get('/')
+    assert r.status_code == 200
+
+
+@mock.patch('logging.warn')
+def test_log_payload(logging_mock, app):
+    payload = 'hello'
+
+    r = app.post('/log_payload', payload)
+    assert r.status_code == 200
+
+    assert logging_mock.called
+
+
+@mock.patch('logging.warn')
+def test_empty_payload(logging_mock, app):
+    r = app.post('/log_payload')
+    assert r.status_code == 200
+
+    assert logging_mock.called

appengine/flexible/tasks/requirements.txt

@@ -0,0 +1,4 @@
+Flask==0.11.1
+google-api-python-client==1.6.0
+google-cloud-datastore==0.22.0
+gunicorn==19.6.0

tasks/README.md

@@ -0,0 +1,59 @@
+# Google Cloud Tasks Pull Queue Samples
+
+Sample command-line program for interacting with the Google Cloud Tasks API
+using pull queues.
+
+Pull queues let you add tasks to a queue, then programatically remove and
+interact with them. Tasks can be added or processed in any environment,
+such as on Google App Engine or Google Compute Engine.
+
+`pull_queue_snippets.py` is a simple command-line program to demonstrate listing queues,
+ creating tasks, and pulling and acknowledging tasks.
+
+## Prerequisites to run locally:
+
+Please refer to [Setting Up a Python Development Environment](https://cloud.google.com/python/setup).
+
+## Authentication
+
+To set up authentication, please refer to our
+[authentication getting started guide](https://cloud.google.com/docs/authentication/getting-started).
+
+## Creating a queue
+
+To create a queue using the Cloud SDK, use the following gcloud command:
+
+    gcloud alpha tasks queues create-pull-queue my-pull-queue
+
+## Running the Samples
+
+Set the environment variables:
+
+Set environment variables:
+
+First, your project ID:
+
+    export PROJECT_ID=my-project-id
+
+Then the queue ID, as specified at queue creation time. Queue IDs already
+created can be listed with `gcloud alpha tasks queues list`.
+
+    export QUEUE_ID=my-pull-queue
+
+And finally the location ID, which can be discovered with
+`gcloud alpha tasks queues describe $QUEUE_ID`, with the location embedded in
+the "name" value (for instance, if the name is
+"projects/my-project/locations/us-central1/queues/my-pull-queue", then the
+location is "us-central1").
+
+    export LOCATION_ID=us-central1
+
+Create a task for a queue:
+
+    python pull_queue_snippets.py create-task --project=$PROJECT_ID --queue=$QUEUE_ID --location=$LOCATION_ID
+
+Pull and acknowledge a task:
+
+    python pull_queue_snippets.py pull-and-ack-task --project=$PROJECT_ID --queue=$QUEUE_ID --location=$LOCATION_ID
+
+Note that usually, there would be a processing step in between pulling a task and acknowledging it.