Skip to content
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
2e38e76
feat: apprunner poc
penguinjournals Mar 24, 2022
f5182be
WIP
penguinjournals Mar 30, 2022
a0548a0
feat: doc update
penguinjournals Apr 4, 2022
d95d779
WIP
penguinjournals Apr 7, 2022
374407d
chore: add name to apprunner service name to privent name colision
hayk99 Apr 12, 2022
6e0ae78
Merge branch 'master' into feat-apprunner-poc
penguinjournals Apr 20, 2022
939f247
feat: cloudconnector deploy on apprunner
penguinjournals Apr 21, 2022
0baaeba
Merge branch 'feat-apprunner-poc' of github.com:sysdiglabs/terraform-…
penguinjournals Apr 21, 2022
50ef9e3
fix: changed name for apprunner task to fix aws requirements of length
penguinjournals Apr 22, 2022
c3a077d
Merge branch 'master' into feat-apprunner-poc
penguinjournals Apr 22, 2022
96afdd2
fix: task name to avoid confusion
penguinjournals Apr 22, 2022
9a9ac9b
fix: precommit
penguinjournals Apr 22, 2022
2c6b241
retrigger checks
penguinjournals Apr 25, 2022
97a697e
chore: linter fixing
penguinjournals Apr 25, 2022
02135cf
doc: apprunner diagram updated
penguinjournals Apr 25, 2022
ff8430a
chore: sequential test
Apr 25, 2022
26befc9
chore: sequential test
Apr 25, 2022
553099f
chore: sequential test
Apr 25, 2022
6a90df8
chorus: timeout increased to pass tests on CI
penguinjournals Apr 25, 2022
1d9be6d
Merge branch 'feat-apprunner-poc' of github.com:sysdiglabs/terraform-…
penguinjournals Apr 25, 2022
0cce02d
chorus: ci test region changed to fit apprunner
penguinjournals Apr 25, 2022
4aef63f
chore: name per test
Apr 26, 2022
0873ae8
Merge branch 'feat-apprunner-poc' of github.com:sysdiglabs/terraform-…
Apr 26, 2022
0c16f8b
docs: snippet fixed
penguinjournals Apr 26, 2022
58024c3
Merge branch 'feat-apprunner-poc' of github.com:sysdiglabs/terraform-…
penguinjournals Apr 26, 2022
ef309b1
chore: fix naming length
Apr 26, 2022
1a9c8b0
feat: retrieve cloud connector from new registry
penguinjournals Apr 26, 2022
67a415a
docs: readme updated to explain the apprunner option
penguinjournals Apr 26, 2022
83fd08b
Merge branch 'feat-apprunner-poc' of github.com:sysdiglabs/terraform-…
penguinjournals Apr 26, 2022
b16654f
docs: updated broken image links
penguinjournals Apr 27, 2022
d347c20
Merge branch 'master' into feat-apprunner-poc
penguinjournals Apr 27, 2022
d7dcb1d
docs: fixed typo error
penguinjournals Apr 29, 2022
07bf221
docs: several naming fixes
penguinjournals Apr 29, 2022
5f5146d
feat: versions upgraded
penguinjournals Apr 29, 2022
cb28b13
docs: updated url for diagram to fit master
penguinjournals May 3, 2022
3278a0c
docs: versions updated
penguinjournals May 3, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 41 additions & 6 deletions .github/workflows/ci-integration-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ on:
concurrency: terraform

jobs:
integration_test-eks:
integration_test_eks:
concurrency: terraform-account
continue-on-error: true

Expand Down Expand Up @@ -99,7 +99,7 @@ jobs:
run: bundle exec kitchen destroy "organizational-k8s-aws"

integration_test_ecs:
needs: integration_test-eks
needs: integration_test_eks
concurrency: terraform-account
continue-on-error: true

Expand All @@ -118,20 +118,20 @@ jobs:
ruby-version: 2.7
bundler-cache: true

- name: Run single-account test
- name: Run single-account-ecs test
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_QA_CLOUDNATIVE_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_QA_CLOUDNATIVE_SECRET_ACCESS_KEY }}
AWS_REGION: ${{ secrets.AWS_REGION }}
run: bundle exec kitchen test "single-account-aws"
run: bundle exec kitchen test "single-account-ecs-aws"

- name: Destroy single-account resources
- name: Destroy single-account-ecs resources
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_QA_CLOUDNATIVE_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_QA_CLOUDNATIVE_SECRET_ACCESS_KEY }}
AWS_REGION: ${{ secrets.AWS_REGION }}
if: ${{ failure() }}
run: bundle exec kitchen destroy "single-account-aws"
run: bundle exec kitchen destroy "single-account-ecs-aws"

- name: Run organizational test
env:
Expand All @@ -149,3 +149,38 @@ jobs:
TF_VAR_sysdig_secure_for_cloud_member_account_id: ${{ secrets.AWS_QA_CLOUDNATIVE_ACCOUNT_ID }}
if: ${{ failure() }}
run: bundle exec kitchen destroy "organizational-aws"

integration_test_app_runner:
needs: integration_test_ecs
concurrency: terraform-account
continue-on-error: true

name: Test-Kitchen-AppRunner
runs-on: ubuntu-latest
env:
TF_VAR_sysdig_secure_url: https://secure.sysdig.com
TF_VAR_sysdig_secure_api_token: ${{secrets.KUBELAB_SECURE_API_TOKEN}}

steps:
- name: Checkout
uses: actions/checkout@v2

- uses: ruby/setup-ruby@v1
with:
ruby-version: 2.7
bundler-cache: true

- name: Run single-account-apprunner test
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_QA_CLOUDNATIVE_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_QA_CLOUDNATIVE_SECRET_ACCESS_KEY }}
AWS_REGION: "eu-west-1" #APPRUNNER ONLY RUNS ON SOME REGIONS https://docs.aws.amazon.com/general/latest/gr/apprunner.html
run: bundle exec kitchen test "single-account-apprunner-aws"

- name: Destroy single-account-apprunner resources
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_QA_CLOUDNATIVE_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_QA_CLOUDNATIVE_SECRET_ACCESS_KEY }}
AWS_REGION: "eu-west-1" #APPRUNNER ONLY RUNS ON SOME REGIONS https://docs.aws.amazon.com/general/latest/gr/apprunner.html
if: ${{ failure() }}
run: bundle exec kitchen destroy "single-account-apprunner-aws"
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -42,5 +42,5 @@ override.tf.json

# test
.kitchen/*
test/fixtures/single-account/.kitchen/
test/fixtures/single-account-ecs/.kitchen/
test/snippets/*
8 changes: 6 additions & 2 deletions .kitchen.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ driver:
root_module_directory: test/fixtures
parallelism: 4
verify_version: false
command_timeout: 1200

provisioner:
name: terraform
Expand All @@ -12,9 +13,9 @@ platforms:
- name: "aws"

suites:
- name: single-account
- name: single-account-ecs
driver:
root_module_directory: test/fixtures/single-account
root_module_directory: test/fixtures/single-account-ecs
- name: single-account-k8s
driver:
root_module_directory: test/fixtures/single-account-k8s
Expand All @@ -24,3 +25,6 @@ suites:
- name: organizational-k8s
driver:
root_module_directory: test/fixtures/organizational-k8s
- name: single-account-apprunner
driver:
root_module_directory: test/fixtures/single-account-apprunner
2 changes: 1 addition & 1 deletion CONTRIBUTE.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@

- [ ] **modules** (infra or services) have been modified?
- [ ] a `README.md` file has been added to the folder
- [ ] if modules are relevant to usage-case understanding `diagram.py/png` have been updated accodingly
- [ ] if modules are relevant to usage-case understanding `diagram.py/png` have been updated accordingly. To re-generate diagrams yo need to run `python diagram.py` and need diagram installed `pip install diagrams`.
- [ ] if pre-requirements have been modified, update accordingly on
- [ ] README's
- [ ] Sysdig docs
Expand Down
18 changes: 15 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,25 +51,37 @@ For other Cloud providers check: [GCP](https://github.com/sysdiglabs/terraform-g
## Usage

- There are several ways to deploy this in you AWS infrastructure, gathered under **[`/examples`](./examples)**
- [Single Account on ECS](#--single-account-on-ecs)
- [Single Account on AppRunner](#--single-account-on-apprunner)
- [Single-Account with a pre-existing Kubernetes Cluster](#--single-account-with-a-pre-existing-kubernetes-cluster)
- [Organizational](#--organizational)
- Many module,examples and use-cases provide ways to **re-use existing resources (as optionals)** in your infrastructure (cloudtrail, ecs, vpc, k8s cluster,...)
- Find some real **use-case scenario explanations** under [`/examples-internal/use-cases*`](./examples-internal)
- [Single Account - Existing Cloudtrail](./examples-internal/use-cases-reuse-resources/single-existing-cloudtrail.md)
- [Organizational - Existing Cloudtrail, ECS, VPC, Subnet](./examples-internal/use-cases-reuse-resources/org-existing-cloudtrail-ecs-vpc-subnet.md)
- [Organizational - Existing Cloudtrail withouth SNS, but with S3 configuration, with K8s Cluster and Filtered Cloudtrail Event Account](./examples-internal/use-cases-self-baked/org-s3-k8s-filtered-account.md)

### - Single-Account
### - Single-Account on ECS

Sysdig workload will be deployed in the same account where user's resources will be watched.<br/>
More info in [`./examples/single-account`](https://github.com/sysdiglabs/terraform-aws-secure-for-cloud/tree/master/examples/single-account)
More info in [`./examples/single-account-ecs`](https://github.com/sysdiglabs/terraform-aws-secure-for-cloud/tree/master/examples/single-account-ecs)

![single-account diagram](https://raw.githubusercontent.com/sysdiglabs/terraform-aws-secure-for-cloud/master/examples/single-account/diagram-single.png)
![single-account diagram](https://raw.githubusercontent.com/sysdiglabs/terraform-aws-secure-for-cloud/master/examples/single-account-ecs/diagram-single.png)

### - Single-Account on AppRunner

Sysdig workload will be deployed using AppRunner in the same account where user's resources will be watched.<br/>
More info in [`./examples/single-account-apprunner`](https://github.com/sysdiglabs/terraform-aws-secure-for-cloud/tree/master/examples/single-account-apprunner)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️
i would point all diagrams to master already, or you're gonna need an extra PR for that :D


![single-account diagram on apprunner](https://raw.githubusercontent.com/sysdiglabs/terraform-aws-secure-for-cloud/master/examples/single-account-apprunner/diagram-single.png)

### - Single-Account with a pre-existing Kubernetes Cluster

If you already own a Kubernetes Cluster on AWS, you can use it to deploy Sysdig Secure for Cloud, instead of default ECS cluster.<br/>
More info in [`./examples/single-account-k8s`](https://github.com/sysdiglabs/terraform-aws-secure-for-cloud/tree/master/examples/single-account-k8s)

![single-account with pre-existing kubernetes cluster](https://raw.githubusercontent.com/sysdiglabs/terraform-aws-secure-for-cloud/feat-apprunner-poc/examples/single-account-k8s/diagram.png)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this one to master too :P


### - Organizational

Using an organizational configuration Cloudtrail.<br/>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@

## Suggested setup

For this use-case we're going to use the [`./examples/single-account`](../../examples/single-account/README.md) setup.
For this use-case we're going to use the [`./examples/single-account-ecs`](../../examples/single-account-ecs/README.md) setup.
In order for this setup to work, all resources must be in the same AWS account and region.
Before proceeding, please read the example README and check whether you comply with requirements.

Expand Down Expand Up @@ -86,7 +86,7 @@ provider "aws" {
}

module "sysdig-s4c" {
source = "sysdiglabs/secure-for-cloud/aws//examples/single-account"
source = "sysdiglabs/secure-for-cloud/aws//examples/single-account-ecs"
name = "sysdig-s4c"

cloudtrail_sns_arn = "<CLOUDRAIL_SNS_TOPIC_ARN>"
Expand Down
2 changes: 1 addition & 1 deletion examples/organizational/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ $ terraform apply
| Name | Source | Version |
|------|--------|---------|
| <a name="module_cloud_bench"></a> [cloud\_bench](#module\_cloud\_bench) | ../../modules/services/cloud-bench | n/a |
| <a name="module_cloud_connector"></a> [cloud\_connector](#module\_cloud\_connector) | ../../modules/services/cloud-connector | n/a |
| <a name="module_cloud_connector"></a> [cloud\_connector](#module\_cloud\_connector) | ../../modules/services/cloud-connector-ecs | n/a |
| <a name="module_cloudtrail"></a> [cloudtrail](#module\_cloudtrail) | ../../modules/infrastructure/cloudtrail | n/a |
| <a name="module_codebuild"></a> [codebuild](#module\_codebuild) | ../../modules/infrastructure/codebuild | n/a |
| <a name="module_ecs_vpc"></a> [ecs\_vpc](#module\_ecs\_vpc) | ../../modules/infrastructure/ecs-vpc | n/a |
Expand Down
2 changes: 1 addition & 1 deletion examples/organizational/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ module "cloud_connector" {
providers = {
aws = aws.member
}
source = "../../modules/services/cloud-connector"
source = "../../modules/services/cloud-connector-ecs"
name = "${var.name}-cloudconnector"

secure_api_token_secret_name = module.ssm.secure_api_token_secret_name
Expand Down
2 changes: 1 addition & 1 deletion examples/organizational/permissions.tf
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ module "secure_for_cloud_role" {
# secure_for_cloud_role <-> ecs_role trust relationship
# note:
# - definition of a ROOT lvl secure_for_cloud_connector_ecs_tas_role to avoid cyclic dependencies
# - duplicated in ../../modules/services/cloud-connector/ecs-service-security.tf
# - duplicated in ../../modules/services/cloud-connector-ecs/ecs-service-security.tf
# -----------------------------------------------------------------
resource "aws_iam_role" "connector_ecs_task" {
provider = aws.member
Expand Down
128 changes: 128 additions & 0 deletions examples/single-account-apprunner/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
# Sysdig Secure for Cloud in AWS<br/>[ Example :: App Runner ]

Deploy Sysdig Secure for Cloud in a single AWS account using App Runner.<br/>
All the required resources and workloads will be run under the same account.

## Prerequisites

Minimum requirements:

1. Configure [Terraform **AWS** Provider](https://registry.terraform.io/providers/hashicorp/aws/latest/docs)
1. Secure requirements, as input variable value
```
sysdig_secure_api_token=<SECURE_API_TOKEN>
```
## Notice
* **Resource creation inventory** Find all the resources created by Sysdig examples in the resource-group `sysdig-secure-for-cloud` (AWS Resource Group & Tag Editor) <br/><br/>
* **Deployment cost** This example will create resources that cost money.<br/>Run `terraform destroy` when you don't need them anymore <br/><br/>
* **AppRunner enabled zones** AppRunner isn't available in all AWS zones, check [AppRunner Service endpoints](https://docs.aws.amazon.com/general/latest/gr/apprunner.html) for enabled zones.
## Usage
For quick testing, use this snippet on your terraform files
```terraform
terraform {
required_providers {
sysdig = {
source = "sysdiglabs/sysdig"
version = ">=0.5.33"
}
}
}
provider "sysdig" {
sysdig_secure_api_token = "<SYSDIG_SECURE_URL>"
sysdig_secure_url = "<SYSDIG_SECURE_API_TOKEN"
}
provider "aws" {
region = "<AWS_REGION> Take care of AppRunner available zones: https://docs.aws.amazon.com/general/latest/gr/apprunner.html"
}
module "cloudvision_aws_apprunner_single_account" {
source = "sysdiglabs/secure-for-cloud/aws//examples/single-account-apprunner"
name = "<APPRUNNER_SERVICE_NAME>"
}
```

See [inputs summary](#inputs) or module module [`variables.tf`](https://github.com/sysdiglabs/terraform-aws-secure-for-cloud/blob/master/examples/single-account/variables.tf) file for more optional configuration.

To run this example you need have your [aws account profile configured in CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html) and to execute:
```terraform
$ terraform init
$ terraform plan
$ terraform apply
```

<!-- BEGIN_TF_DOCS -->
## Requirements

| Name | Version |
|------|---------|
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.15.0 |
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.0.0 |
| <a name="requirement_sysdig"></a> [sysdig](#requirement\_sysdig) | >= 0.5.33 |

## Providers

| Name | Version |
|------|---------|
| <a name="provider_sysdig"></a> [sysdig](#provider\_sysdig) | >= 0.5.33 |

## Modules

| Name | Source | Version |
|------|--------|---------|
| <a name="module_cloud_bench"></a> [cloud\_bench](#module\_cloud\_bench) | ../../modules/services/cloud-bench | n/a |
| <a name="module_cloud_connector"></a> [cloud\_connector](#module\_cloud\_connector) | ../../modules/services/cloud-connector | n/a |
| <a name="module_cloudtrail"></a> [cloudtrail](#module\_cloudtrail) | ../../modules/infrastructure/cloudtrail | n/a |
| <a name="module_codebuild"></a> [codebuild](#module\_codebuild) | ../../modules/infrastructure/codebuild | n/a |
| <a name="module_ecs_vpc"></a> [ecs\_vpc](#module\_ecs\_vpc) | ../../modules/infrastructure/ecs-vpc | n/a |
| <a name="module_resource_group"></a> [resource\_group](#module\_resource\_group) | ../../modules/infrastructure/resource-group | n/a |
| <a name="module_ssm"></a> [ssm](#module\_ssm) | ../../modules/infrastructure/ssm | n/a |

## Resources

| Name | Type |
|------|------|
| [sysdig_secure_connection.current](https://registry.terraform.io/providers/sysdiglabs/sysdig/latest/docs/data-sources/secure_connection) | data source |

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| <a name="input_benchmark_regions"></a> [benchmark\_regions](#input\_benchmark\_regions) | List of regions in which to run the benchmark. If empty, the task will contain all aws regions by default. | `list(string)` | `[]` | no |
| <a name="input_cloudtrail_is_multi_region_trail"></a> [cloudtrail\_is\_multi\_region\_trail](#input\_cloudtrail\_is\_multi\_region\_trail) | true/false whether cloudtrail will ingest multiregional events | `bool` | `true` | no |
| <a name="input_cloudtrail_kms_enable"></a> [cloudtrail\_kms\_enable](#input\_cloudtrail\_kms\_enable) | true/false whether cloudtrail delivered events to S3 should persist encrypted | `bool` | `true` | no |
| <a name="input_cloudtrail_sns_arn"></a> [cloudtrail\_sns\_arn](#input\_cloudtrail\_sns\_arn) | ARN of a pre-existing cloudtrail\_sns. If defaulted, a new cloudtrail will be created | `string` | `"create"` | no |
| <a name="input_deploy_benchmark"></a> [deploy\_benchmark](#input\_deploy\_benchmark) | Whether to deploy or not the cloud benchmarking | `bool` | `true` | no |
| <a name="input_deploy_image_scanning_ecr"></a> [deploy\_image\_scanning\_ecr](#input\_deploy\_image\_scanning\_ecr) | true/false whether to deploy the image scanning on ECR pushed images | `bool` | `true` | no |
| <a name="input_deploy_image_scanning_ecs"></a> [deploy\_image\_scanning\_ecs](#input\_deploy\_image\_scanning\_ecs) | true/false whether to deploy the image scanning on ECS running images | `bool` | `true` | no |
| <a name="input_ecs_cluster_name"></a> [ecs\_cluster\_name](#input\_ecs\_cluster\_name) | Name of a pre-existing ECS (elastic container service) cluster. If defaulted, a new ECS cluster/VPC/Security Group will be created | `string` | `"create"` | no |
| <a name="input_ecs_task_cpu"></a> [ecs\_task\_cpu](#input\_ecs\_task\_cpu) | Amount of CPU (in CPU units) to reserve for cloud-connector task | `string` | `"256"` | no |
| <a name="input_ecs_task_memory"></a> [ecs\_task\_memory](#input\_ecs\_task\_memory) | Amount of memory (in megabytes) to reserve for cloud-connector task | `string` | `"512"` | no |
| <a name="input_ecs_vpc_id"></a> [ecs\_vpc\_id](#input\_ecs\_vpc\_id) | ID of the VPC where the workload is to be deployed. Defaulted to be created when 'ecs\_cluster\_name' is not provided. | `string` | `"create"` | no |
| <a name="input_ecs_vpc_region_azs"></a> [ecs\_vpc\_region\_azs](#input\_ecs\_vpc\_region\_azs) | List of Availability Zones for ECS VPC creation. e.g.: ["apne1-az1", "apne1-az2"]. If defaulted, two of the default 'aws\_availability\_zones' datasource will be taken | `list(string)` | `[]` | no |
| <a name="input_ecs_vpc_subnets_private_ids"></a> [ecs\_vpc\_subnets\_private\_ids](#input\_ecs\_vpc\_subnets\_private\_ids) | List of VPC subnets where workload is to be deployed. Defaulted to be created when 'ecs\_cluster\_name' is not provided. | `list(string)` | `[]` | no |
| <a name="input_name"></a> [name](#input\_name) | Name to be assigned to all child resources. A suffix may be added internally when required. Use default value unless you need to install multiple instances | `string` | `"sfc"` | no |
| <a name="input_tags"></a> [tags](#input\_tags) | sysdig secure-for-cloud tags | `map(string)` | <pre>{<br> "product": "sysdig-secure-for-cloud"<br>}</pre> | no |

## Outputs

| Name | Description |
|------|-------------|
| <a name="output_cloudtrail_sns_topic_arn"></a> [cloudtrail\_sns\_topic\_arn](#output\_cloudtrail\_sns\_topic\_arn) | ARN of cloudtrail\_sns topic |
<!-- END_TF_DOCS -->


## Authors

Module is maintained and supported by [Sysdig](https://sysdig.com).

## License

Apache 2 Licensed. See LICENSE for full details.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading