Tutorial Creating a Task

Creating a task

In this tutorial, we will create, develop and execute a new task.

The task we create will be used as a functional test for func-y task engine itself. Creating a task consists of:

• Install python requirments
• Creating a yaml file to house our task
• Start the services the task depends on
• Defining metadata associated with the task
• Defining the states of our task
• Incrementally developing task
• Debugging Task
• Running tasks as a unit test

Install python requirements

Func-y is written in python using gevent. Unfortunately there aren't enough abstractions to insulate you from it yet. To get started install all the python requirements using pip install -r requirements.txt.

Creating a Yaml file

For this tutorial, we are creating a func-y task to test func-y task engine itself. Func-y tasks can live anywhere in your project but we'll be putting ours in:

func-y-task-engine/tests/funcy/ We are going to be cross-cutting a number of features:

• env var preprocessing
• UUID preprocessing
• nsq plugin
• postgres plugin

Because of this we'll name our test envvar-uuid-nsq-postgres.yml

Start Services

Our task will be interacting with nsq and postgres through their public interfaces. docker-compose is used to manage test dependencies. To bring up local services to test against, navigate to the func-y-task-engine root directory and run:

docker-compose up

This will start nsqd and postgres.

Define the task metadata

Metadata is stored as top level attributes in the task definition. For our task we'll specify a max_timeout for this task. A version (not used) and a task name:

---
max_timeout: 10
name: ENVVAR_UUID_nsq_postgres_assertions
version: "1"
events:

When max_timeout is reached the task engine will stop any running states and return a failure to the user.

Defining task states/ Incremental Development

Next we'll have to define each state of our task. Since we are testing nsq and postgres we'll:

• Create a nsq test topic if it doesn't exit Each item in the events yaml represents a single state in a linear state progression. The first state will always be executed and complete, before the second state; the second state will come before the third, etc.

---
max_timeout: 10
name: ENVVAR_UUID_nsq_postgres_assertions
version: "1"
events:
  - name: create_nsq_topic
    initiator:
      method: post
      type: http.HTTPInitiator
      url: "http://localhost:4151/topic/create?topic=funcy_task"

• Send a message to the test topic
• Subscribe to the test test topic
• Wait until we receive a message from the test topic
• Insert a record into postgres
• Retrieve a record from postgres