Adding new Dell EMC ECS Object Storage Plugin for CloudStack #12124
Conversation
|
Congratulations on your first Pull Request and welcome to the Apache CloudStack community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md)
|
jbampton
left a comment
jbampton
left a comment
There was a problem hiding this comment.
You have failing tests for pre-commit and license checks.
Have a quick read up on pre-commit it really is a series of basic checks we run:
https://github.com/apache/cloudstack/blob/main/PRE-COMMIT.md
Also we normally add the ASF license header to all files or add an exclude
Codecov Report✅ All modified and coverable lines are covered by tests.
Additional details and impacted files@@ Coverage Diff @@
## main #12124 +/- ##
=============================================
- Coverage 17.56% 3.58% -13.98%
=============================================
Files 5912 445 -5467
Lines 529383 37571 -491812
Branches 64660 6915 -57745
=============================================
- Hits 92984 1347 -91637
+ Misses 425941 36060 -389881
+ Partials 10458 164 -10294
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
@mhkadhum can you check the build errors? |
| @@ -0,0 +1,1490 @@ | |||
| /* ... license header unchanged ... */ | |||
There was a problem hiding this comment.
this is not a valid apache license
| @@ -0,0 +1,14 @@ | |||
| <beans xmlns="http://www.springframework.org/schema/beans" | |||
| for lifecycle and driver, same as Ceph. --> | ||
| <bean id="EcsObjectStoreProviderImpl" | ||
| class="org.apache.cloudstack.storage.datastore.provider.EcsObjectStoreProviderImpl" /> | ||
| </beans> No newline at end of file |
There was a problem hiding this comment.
| </beans> | |
| </beans> | |
There was a problem hiding this comment.
i know, lint can be picky at times
4 participants
Description
This PR adds support for a Dell EMC ECS S3 object storage plugin for Apache CloudStack. ECS is a software-defined object storage platform that supports both object and file-system protocols, with a focus on scalable and reliable object storage. We have been using ECS in production for three years, and this work extends CloudStack’s existing MinIO plugin to provide similar functionality for ECS.
The implementation supports the full lifecycle of S3-compatible buckets on ECS, including user provisioning, bucket creation, policy management, versioning, creation-time encryption, and integration with CloudStack’s S3 Browser. All functionality has been tested in a lab environment on Ubuntu using a CloudStack development setup based on the official installation guidelines.
Key architectural differences from MinIO:
Management API Integration
ECS requires use of the ECS Management API (port 4443, or 443 when fronted by HAProxy). CloudStack authenticates with management-user credentials to perform bucket and user operations.
Namespace Requirements
A dedicated ECS namespace is required for CloudStack-managed buckets. Multiple namespaces allow different CloudStack environments to share the same ECS cluster.
S3 Endpoints (Public and Private)
ECS exposes S3 services on ports 9020/9021 (non-TLS/TLS). In our deployment, these are routed through HAProxy and exposed externally on port 443. The Public URL is displayed to CloudStack users, while the Private URL is used internally.
TLS Handling
The “Allow Insecure HTTPS” option controls whether CloudStack accepts untrusted certificates when communicating with the ECS Management API.
User Provisioning Workflow
When a CloudStack user creates their first bucket, CloudStack provisions a corresponding ECS object user using the CloudStack UUID with a
cs-prefix. ECS generates access and secret keys once, which CloudStack securely stores and reuses for subsequent bucket operations.Bucket features:
Bucket modification supports quota changes, versioning updates, and policy changes. Encryption is excluded because ECS does not allow changing it after creation.
CloudStack’s S3 Browser supports upload, download, delete, listing, and prefix filtering through the ECS S3 endpoint. ECS prevents deletion of non-empty buckets, and CloudStack surfaces these errors accordingly.
We welcome review and feedback. The development fork is available here:
https://github.com/mhkadhum/cloudstack
Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
Bug Severity
Screenshots (if appropriate):
How Has This Been Tested?
The plugin was tested in an Ubuntu-based CloudStack development environment created by cloning the CloudStack source and following the official installation instructions. Testing included: