Skip to content

Conversation

@mocsharp
Copy link
Collaborator

@mocsharp mocsharp commented Apr 8, 2022

Fixes #53.

Description

#53 Implement Task Manager with a stubbed input message queue until Workflow Manager is ready.

  • Provides ITaskRunner interface and RunnerBase base class with ExecuteTask(...) and GetStatus(...) methods for executing a task plug-in and getting the status of a task.

#54 Implements Argo plug-in that requires 2 templates, the main workflow template (kind: WorkflowTemplate), and an exit hook template (kind: WorkflowTemplate). The Argo plug-in creates a kind: Workflow with references to the 2 provided templates. It also creates any Kubernetes credentials required to connect the Argo workflow to the MinIO buckets and deletes them on completion. Due to this Argo bug, this implementation creates an Argo Workflow (kind: Workflow) based on the provided Argo workflow template (kind: WorkflowTemplate).

The Argo plug-in copies the entrypoint template and any child templates (through dags or steps) into a new Workflow and injects s3 information provided by the Workflow Manager & Task Manager. See #54 for details.

Status

Ready

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • All tests passed locally.
  • Documentation comments included/updated.
  • User guide updated.
  • I have updated the changelog

@mocsharp mocsharp added the enhancement New feature or request label Apr 8, 2022
@mocsharp mocsharp added this to the POC milestone Apr 8, 2022
@mocsharp mocsharp self-assigned this Apr 8, 2022
@codecov
Copy link

codecov bot commented Apr 8, 2022

Codecov Report

Merging #84 (0bc3926) into develop (748a5a1) will not change coverage.
The diff coverage is n/a.

❗ Current head 0bc3926 differs from pull request most recent head bd96ffd. Consider uploading reports for the commit bd96ffd to get more accurate results

Impacted file tree graph

@@      Coverage Diff      @@
##   develop   #84   +/-   ##
=============================
=============================

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 748a5a1...bd96ffd. Read the comment docs.

@mocsharp mocsharp force-pushed the feature/53-dispatch-tasks branch 4 times, most recently from e4ef86a to aa33394 Compare April 14, 2022 21:27
@mocsharp mocsharp marked this pull request as ready for review April 18, 2022 19:19
@mocsharp mocsharp force-pushed the feature/53-dispatch-tasks branch from f1b0b2f to ad1f429 Compare April 18, 2022 20:46
@mocsharp mocsharp marked this pull request as draft April 23, 2022 02:08
@mocsharp mocsharp force-pushed the feature/53-dispatch-tasks branch 2 times, most recently from e00066d to 68c442c Compare April 27, 2022 00:18
@mocsharp mocsharp marked this pull request as ready for review April 27, 2022 00:21
mocsharp added a commit that referenced this pull request May 3, 2022
mocsharp added a commit that referenced this pull request May 5, 2022
mocsharp added a commit that referenced this pull request May 5, 2022
Signed-off-by: Victor Chang <[email protected]>
@mocsharp mocsharp force-pushed the feature/53-dispatch-tasks branch from 17a15e8 to a8fcee8 Compare May 6, 2022 17:20
mocsharp added a commit that referenced this pull request May 6, 2022
mocsharp added a commit that referenced this pull request May 6, 2022
mocsharp added a commit that referenced this pull request May 6, 2022
Signed-off-by: Victor Chang <[email protected]>
mocsharp added a commit that referenced this pull request May 6, 2022
Signed-off-by: Victor Chang <[email protected]>
@mocsharp mocsharp requested review from RemakingEden, jackschofield23 and joshliberty and removed request for jackschofield23 May 6, 2022 17:20
mocsharp added 19 commits May 16, 2022 11:45
Signed-off-by: Victor Chang <[email protected]>
Signed-off-by: Victor Chang <[email protected]>
Signed-off-by: Victor Chang <[email protected]>
- include end hook template
- handle TaskCallbackEvent

Signed-off-by: Victor Chang <[email protected]>
Signed-off-by: Victor Chang <[email protected]>
Signed-off-by: Victor Chang <[email protected]>
Signed-off-by: Victor Chang <[email protected]>
Signed-off-by: Victor Chang <[email protected]>
@mocsharp mocsharp force-pushed the feature/53-dispatch-tasks branch from 5fa1767 to 4b94978 Compare May 16, 2022 19:42
mocsharp added a commit that referenced this pull request May 16, 2022
@mocsharp mocsharp force-pushed the feature/53-dispatch-tasks branch from 4b94978 to 761dffd Compare May 16, 2022 20:14
mocsharp added 2 commits May 16, 2022 13:43
Signed-off-by: Victor Chang <[email protected]>
Signed-off-by: Victor Chang <[email protected]>
@sonarqubecloud
Copy link

sonarqubecloud bot commented May 16, 2022

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 11 Code Smells

77.8% 77.8% Coverage
0.1% 0.1% Duplication

@JJFinnerty JJFinnerty self-requested a review May 17, 2022 09:33
@JJFinnerty JJFinnerty merged commit 3088b16 into develop May 17, 2022
@mocsharp mocsharp deleted the feature/53-dispatch-tasks branch May 17, 2022 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

Development

Successfully merging this pull request may close these issues.

Dispatch tasks

5 participants