Skip to content

Updating data filters #278

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Mar 9, 2021
Merged

Updating data filters #278

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
047bf12
Updating data filters
Mar 3, 2021
deda3ec
fix sentence
Mar 3, 2021
46050a2
fix sentence
Mar 3, 2021
47b76d1
fix sentence
Mar 3, 2021
41de2fb
fix sentence
Mar 3, 2021
9894d7f
fixes per Ricardos comments
Mar 4, 2021
efdf1bd
fixes per Ricardos comments
Mar 4, 2021
1624f99
Fixes per Jorgens comments
Mar 4, 2021
0afd754
Fixes per Jorgens comments
Mar 4, 2021
578d516
Fixes per Jorgens comments
Mar 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions comparisons/comparison-argo.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -404,7 +404,7 @@ states:
actions:
- functionRef: gen-random-int-bash
actionDataFilter:
dataResultsPath: "${ .results }"
results: "${ .results }"
transition: print-message
- name: print-message
type: operation
Expand Down Expand Up @@ -581,7 +581,7 @@ states:
actions:
- functionRef: flip-coin-function
actionDataFilter:
dataResultsPath: "${ .flip.result }"
results: "${ .flip.result }"
transition: show-flip-results
- name: show-flip-results
type: switch
Expand All @@ -595,14 +595,14 @@ states:
actions:
- functionRef: echo
actionDataFilter:
dataResultsPath: it was heads
results: it was heads
end: true
- name: show-results-tails
type: operation
actions:
- functionRef: echo
actionDataFilter:
dataResultsPath: it was tails
results: it was tails
end: true
```

Expand Down Expand Up @@ -761,7 +761,7 @@ states:
actions:
- functionRef: flip-coin-function
actionDataFilter:
dataResultsPath: "${ .steps.flip-coin.outputs.result }"
results: "${ .steps.flip-coin.outputs.result }"
transition: flip-coin-check
- name: flip-coin-check
type: switch
Expand Down
12 changes: 6 additions & 6 deletions comparisons/comparison-brigade.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -403,7 +403,7 @@ states:
- eventRefs:
- execEvent
eventDataFilter:
dataOutputPath: "${ .event }"
results: "${ .event }"
actions:
- name: eventInfoAction
functionRef:
Expand Down Expand Up @@ -484,18 +484,18 @@ states:
- eventRefs:
- execEvent
eventDataFilter:
dataOutputPath: "${ .event }"
data: "${ .event }"
actions:
- name: helloAction
actionDataFilter:
dataResultsPath: "${ .helloResult }"
results: "${ .helloResult }"
functionRef:
refName: greetingFunction
arguments:
message: hello
- name: worldAction
actionDataFilter:
dataResultsPath: "${ .worldResults }"
results: "${ .worldResults }"
functionRef:
refName: greetingAction
arguments:
Expand Down Expand Up @@ -572,7 +572,7 @@ states:
- execEvent
actions: []
eventDataFilter:
dataOutputPath: "${ .execEvent }"
data: "${ .execEvent }"
transition:
nextState: NextEventState
produceEvents:
Expand All @@ -590,7 +590,7 @@ states:
- eventRefs:
- nextEvent
eventDataFilter:
dataOutputPath: "${ .nextEvent }"
data: "${ .nextEvent }"
actions:
- name: consoleLogAction
functionRef:
Expand Down
2 changes: 1 addition & 1 deletion comparisons/comparison-cadence.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,7 @@ states:
arguments:
filename: "${ .file.name }"
actionDataFilter:
dataResultsPath: "${ .processed }"
results: "${ .processed }"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit; if I'm understanding correctly, in actionFilter it's 'results', in stateFilter it's 'output' and in eventFilter it's 'data'? Should we try to use similar names for similar concepts?

Copy link
Contributor Author

@tsurdilo tsurdilo Mar 3, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think naming them the same thing is more confusing honestly. As mentioned in pr description this uses natural language concept which imo is better:
"A state data filter can filter the state input and then filter the state output"
"An event data filter can filter the event data and merge it toStateData element X"
"An action data filter can filter elements fromStateData, it can also filter the action results and merge them toStateData element Y"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a nit; I'm fine if we disagree on this so I'll only add that for actions could be considered to have 'output'.

Also, fwiw, that description appears to be referring to the wrong filter in the 3rd line. It appears to be actionDataFilter but says 'An event filter'.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops ok will fix. sorry.
so actions should have "results" / "result" or "output. I am fine with any of those. Just let me know what makes most sense

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If nobody else weighs in, let's just stick with results I guess.

- functionRef:
refName: uploadfunction
arguments:
Expand Down
4 changes: 2 additions & 2 deletions comparisons/comparison-google-cloud-workflows.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ languages.
"outputVar": "Hello ${ .firstname + \" \" + .lastname }"
},
"stateDataFilter": {
"dataOutputPath": "${ .outputVar }"
"output": "${ .outputVar }"
},
"end": true
}
Expand Down Expand Up @@ -206,7 +206,7 @@ instance is created. See the Serverless Workflow ["Workflow Data"](../specificat
]
},
"stateDataFilter": {
"dataOutputPath": "${ .array | join(\"\") }"
"output": "${ .array | join(\"\") }"
},
"end": true
}
Expand Down
52 changes: 26 additions & 26 deletions examples/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ Which is added to the states data and becomes the workflow data output.
}
},
"actionDataFilter": {
"dataResultsPath": "${ .greeting }"
"results": "${ .greeting }"
}
}
],
Expand Down Expand Up @@ -197,7 +197,7 @@ states:
arguments:
name: "${ .person.name }"
actionDataFilter:
dataResultsPath: "${ .greeting }"
results: "${ .greeting }"
end: true
```

Expand Down Expand Up @@ -243,20 +243,20 @@ Note that in the workflow definition you can see two filters defined. The event
```json
{
"eventDataFilter": {
"dataOutputPath": "${ .data.greet } "
"data": "${ .data.greet } "
}
}
```

which is triggered when the greeting event is consumed. It extracts its "data.greet" of the event and
merges it with the states data.
which is triggered when the greeting event is consumed. It extracts its "data.greet" of the event data (payload) and
merges it with the state data.

The second, a state data filter, which is defined on the event state itself:

```json
{
"stateDataFilter": {
"dataOutputPath": "${ .payload.greeting }"
"output": "${ .payload.greeting }"
}
}
```
Expand Down Expand Up @@ -310,7 +310,7 @@ filters what is selected to be the state data output which then becomes the work
"onEvents": [{
"eventRefs": ["GreetingEvent"],
"eventDataFilter": {
"dataOutputPath": "${ .data.greet }"
"data": "${ .data.greet }"
},
"actions":[
{
Expand All @@ -324,7 +324,7 @@ filters what is selected to be the state data output which then becomes the work
]
}],
"stateDataFilter": {
"dataOutputPath": "${ .payload.greeting }"
"output": "${ .payload.greeting }"
},
"end": true
}
Expand Down Expand Up @@ -355,14 +355,14 @@ states:
- eventRefs:
- GreetingEvent
eventDataFilter:
dataOutputPath: "${ .data.greet }"
data: "${ .data.greet }"
actions:
- functionRef:
refName: greetingFunction
arguments:
name: "${ .greet.name }"
stateDataFilter:
dataOutputPath: "${ .payload.greeting }"
output: "${ .payload.greeting }"
end: true
```

Expand Down Expand Up @@ -438,7 +438,7 @@ result of the workflow execution.
}
],
"stateDataFilter": {
"dataOutputPath": "${ .results }"
"output": "${ .results }"
},
"end": true
}
Expand Down Expand Up @@ -470,7 +470,7 @@ states:
arguments:
expression: "${ .singleexpression }"
stateDataFilter:
dataOutputPath: "${ .results }"
output: "${ .results }"
end: true
```

Expand Down Expand Up @@ -892,7 +892,7 @@ The data output of the workflow contains the information of the exception caught
}
],
"stateDataFilter": {
"dataOutputPath": "${ .exceptions }"
"output": "${ .exceptions }"
},
"transition": "ApplyOrder",
"onErrors": [
Expand Down Expand Up @@ -960,7 +960,7 @@ states:
arguments:
order: "${ .order }"
stateDataFilter:
dataOutputPath: "${ .exceptions }"
output: "${ .exceptions }"
transition: ApplyOrder
onErrors:
- error: Missing order id
Expand Down Expand Up @@ -1060,7 +1060,7 @@ In the case job submission raises a runtime error, we transition to a SubFlow st
}
},
"actionDataFilter": {
"dataResultsPath": "${ .jobuid }"
"results": "${ .jobuid }"
}
}
],
Expand All @@ -1071,7 +1071,7 @@ In the case job submission raises a runtime error, we transition to a SubFlow st
}
],
"stateDataFilter": {
"dataOutputPath": "${ .jobuid }"
"output": "${ .jobuid }"
},
"transition": "WaitForCompletion'"
},
Expand Down Expand Up @@ -1100,12 +1100,12 @@ In the case job submission raises a runtime error, we transition to a SubFlow st
}
},
"actionDataFilter": {
"dataResultsPath": "${ .jobstatus }"
"results": "${ .jobstatus }"
}
}
],
"stateDataFilter": {
"dataOutputPath": "${ .jobstatus }"
"output": "${ .jobstatus }"
},
"transition": "DetermineCompletion"
},
Expand Down Expand Up @@ -1190,12 +1190,12 @@ states:
arguments:
name: "${ .job.name }"
actionDataFilter:
dataResultsPath: "${ .jobuid }"
results: "${ .jobuid }"
onErrors:
- error: "*"
transition: SubmitError
stateDataFilter:
dataOutputPath: "${ .jobuid }"
output: "${ .jobuid }"
transition: WaitForCompletion
- name: SubmitError
type: subflow
Expand All @@ -1214,9 +1214,9 @@ states:
arguments:
name: "${ .jobuid }"
actionDataFilter:
dataResultsPath: "${ .jobstatus }"
results: "${ .jobstatus }"
stateDataFilter:
dataOutputPath: "${ .jobstatus }"
output: "${ .jobstatus }"
transition: DetermineCompletion
- name: DetermineCompletion
type: switch
Expand Down Expand Up @@ -2358,7 +2358,7 @@ For this example we assume that the workflow instance is started given the follo
"resultEventRef": "VetAppointmentInfo"
},
"actionDataFilter": {
"dataResultsPath": "${ .appointmentInfo }"
"results": "${ .appointmentInfo }"
},
"timeout": "PT15M"
}
Expand Down Expand Up @@ -2395,7 +2395,7 @@ states:
data: "${ .patientInfo }"
resultEventRef: VetAppointmentInfo
actionDataFilter:
dataResultsPath: "${ .appointmentInfo }"
results: "${ .appointmentInfo }"
timeout: PT15M
end: true
```
Expand Down Expand Up @@ -3126,7 +3126,7 @@ the data for an hour, send report, and so on.
}
],
"eventDataFilter": {
"dataOutputPath": "${ .readings }"
"data": "${ .readings }"
}
}
],
Expand Down Expand Up @@ -3208,7 +3208,7 @@ states:
- functionRef:
refName: LogReading
eventDataFilter:
dataOutputPath: "${ .readings }"
data: "${ .readings }"
end: true
- name: GenerateReport
type: operation
Expand Down
Binary file modified media/spec/event-data-filter-example1.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified media/spec/state-data-filter-example1.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified media/spec/state-data-filter-example2.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added media/spec/workflowdataflow.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions roadmap/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ _Status description:_
| ✔️| Replace JsonPath with jq | [spec doc](../specification.md) |
| ✔️| Update start definition (move to top-level worklow param) | [spec doc](../specification.md) |
| ✔️| Updated schedule definition | [spec doc](../specification.md) |
| ✔️| Update data filters | [spec doc](../specification.md) |
| 🚩 | Workflow invocation bindings | |
| 🚩 | CE Subscriptions & Discovery | |
| 🚩 | Error types | [issue](https://github.com/serverlessworkflow/specification/issues/200) |
Expand Down
Loading