feat: add a abstract layer for WorkflowNodeExcetion #18026
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ions Simplifies storage methods by eliminating try-except blocks that logged exceptions and re-raised them without additional processing. This streamlines the codebase and relies on upstream exception handling to manage errors effectively. Signed-off-by: -LAN- <[email protected]>
Introduces configuration for workflow node execution storage with options for 'rdbms', 'hybrid', and 's3'. Updates environment examples and integrates the new configuration class into the feature settings. Enhances flexibility in storage backend selection. Signed-off-by: -LAN- <[email protected]>
Replaces WorkflowNodeExecutionStatus with StrEnum for automatic string conversion. Removes redundant value_of method for cleaner and more maintainable code. Signed-off-by: -LAN- <[email protected]>
Adds repository interfaces for data access abstraction and a factory for creating repository instances. Implements initialization of repositories in the application setup. Enhances modularity and flexibility by decoupling data access logic from storage mechanisms. Signed-off-by: -LAN- <[email protected]>
Introduces criteria-based filtering capabilities for workflow node executions, allowing for more flexible querying. Adds new methods for retrieving and deleting based on criteria, and removes abstract base class in favor of Protocol for interface definition. Updates ordering options for retrieved instances, enabling better control over query results. Improves repository interface to better handle domain-specific concepts at the implementation level, maintaining a clean core domain model. Signed-off-by: -LAN- <[email protected]>
Includes TODO comments suggesting a transition to a repository pattern for database interactions. Provides examples for future implementation but retains current direct database access. Facilitates future refactoring towards cleaner code architecture. Signed-off-by: -LAN- <[email protected]>
Eliminates WorkflowNodeExecutionStorageConfig class to simplify storage configuration. Updates WorkflowNodeExecutionConfig to directly define storage backend options. This change reduces complexity by consolidating settings and avoiding duplication. Signed-off-by: -LAN- <[email protected]>
Replaces direct database access with repository pattern for managing workflow node executions, enhancing modularity and maintainability. Updates cache handling to ensure consistency with database operations. Streamlines execution queries and deletions across various services by leveraging the new repository. Removes outdated TODO comments related to repository migration. Improves code clarity by removing redundant code blocks. Relates to performance improvement and codebase modernization efforts. Signed-off-by: -LAN- <[email protected]>
Eliminates get_by_id and delete_by_criteria methods from WorkflowNodeExecutionRepository as they are not used anywhere in the codebase, simplifying the interface and reducing maintenance overhead. Signed-off-by: -LAN- <[email protected]>
Introduces SQLAlchemy implementation for WorkflowNodeExecution repository with CRUD operations, supporting multi-tenancy. Registers the repository based on RDBMS storage configuration. Includes unit tests to ensure repository functionality. Signed-off-by: -LAN- <[email protected]>
Simplifies code by consolidating line breaks in the registration and error handling for repository storage types. Improves readability and maintains functionality. No functionality changes. Signed-off-by: -LAN- <[email protected]>
Updates repository methods to flush without committing, shifting transaction responsibility to callers. Adds explicit commit calls after batch deletions and saves to ensure data persistence. Improves database operation consistency and control. Signed-off-by: -LAN- <[email protected]>
Replaces direct database queries with a repository pattern to fetch workflow node executions. This change enhances code maintainability and aligns with best practices by abstracting data access logic into a repository. No issue or ticket reference provided. Signed-off-by: -LAN- <[email protected]>
|
I'll review the PR later, after completing #16317. |
Signed-off-by: -LAN- <[email protected]>
dosubot
bot
added
size:XXL
…torage-for-workflow-node-execution
Introduces OrderConfig dataclass to encapsulate ordering parameters, enhancing code readability and maintainability. Updates repository methods to use OrderConfig for ordering, allowing multiple fields to be specified. Adjusts tests to reflect these changes for consistency and accuracy. Signed-off-by: -LAN- <[email protected]>
Removes `get_by_id` and `delete_by_criteria` methods as they are not used anywhere in the codebase, simplifying the repository interface and improving maintainability. Signed-off-by: -LAN- <[email protected]>
Simplifies repository creation by introducing a dedicated method in the RepositoryFactory for workflow node execution. Replaces previous generic repository creation calls with the new method for better clarity and type safety. Removes unnecessary imports and type casting, streamlining the codebase. This update enhances maintainability and reduces the potential for errors in repository initialization across the application. Signed-off-by: -LAN- <[email protected]>
api/repositories/workflow_node_execution/sqlalchemy_repository.py
Outdated
Show resolved
Hide resolved
api/repositories/workflow_node_execution/sqlalchemy_repository.py
Outdated
Show resolved
Hide resolved
Eliminates the WorkflowNodeExecutionCriteria class and related methods, simplifying the querying process by using direct SQLAlchemy queries. This change reduces complexity in repository interactions and enhances code maintainability by removing redundant logic. Refactors services to use direct queries instead of repository methods, improving performance and clarity in workflow node execution management. Signed-off-by: -LAN- <[email protected]>
Changes enum class inheritance to StrEnum for consistency across the codebase. Removes the unused value_of method to simplify code and reduce redundancy. Signed-off-by: -LAN- <[email protected]>
Refactors workflow node execution handling to use a session factory for creating new sessions, eliminating direct session passing and merging. This change aims to improve session management and prevent long-running connections. Updates tests to accommodate session factory usage, ensuring consistent test behavior. Signed-off-by: -LAN- <[email protected]>
Replaces repository pattern with direct SQLAlchemy queries for fetching and saving workflow node executions. Enhances code simplicity and maintainability by removing redundant repository usage and leveraging native SQLAlchemy session methods for database operations. Signed-off-by: -LAN- <[email protected]>
There was a problem hiding this comment.
Copilot reviewed 27 out of 28 changed files in this pull request and generated no comments.
Files not reviewed (1)
- api/.env.example: Language not supported
Comments suppressed due to low confidence (2)
api/services/clear_free_plan_tenant_expired_logs.py:141
- Row-by-row deletion of workflow node executions may impact performance when processing large batches. Consider using a bulk delete operation if transactional consistency can be maintained.
for execution in workflow_node_executions:
api/extensions/ext_storage.py:75
- Removal of the try/except block here eliminates error logging, which might hinder debugging of storage failures; ensure that error tracking is handled elsewhere or consider retaining logging.
self.storage_runner.save(filename, data)
laipz8200
force-pushed
the
feat/support-hybrid-storage-for-workflow-node-execution
branch
2 times, most recently
from
20fbce7 to
d4fac76
Compare
Signed-off-by: -LAN- <[email protected]>
laipz8200
force-pushed
the
feat/support-hybrid-storage-for-workflow-node-execution
branch
from
d4fac76 to
ff42e85
Compare
laipz8200
deleted the
feat/support-hybrid-storage-for-workflow-node-execution
branch
This was referenced Apr 17, 2025
|
When step by step workflow node, raise exception: 'Engine' object does not support the context manager protocol
|
|
When trace workflow run log, the same error. @hjlarry
|
5 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: -LAN- [email protected]
Summary
close #18025
Tip
Close issue syntax:
Fixes #<issue number>orResolves #<issue number>, see documentation for more details.Screenshots
Checklist
Important
Please review the checklist below before submitting your pull request.
dev/reformat(backend) andcd web && npx lint-staged(frontend) to appease the lint gods