From f551f4d5197658a4950fe5a2b10ab90acef37d92 Mon Sep 17 00:00:00 2001 From: Cizer Pereira Date: Sun, 5 Jun 2022 18:35:37 +0200 Subject: [PATCH 1/4] refactor readme upgrade ts-node and typescript version use default synthesizer for custom qualifer Signed-off-by: Cizer Pereira --- README.md | 120 ++++++++++++++++++++++++++++++++----------- bin/code_pipeline.ts | 10 +++- package.json | 4 +- 3 files changed, 99 insertions(+), 35 deletions(-) diff --git a/README.md b/README.md index de61c64..acf383a 100644 --- a/README.md +++ b/README.md @@ -1,62 +1,77 @@ # AWS CodePipeline with CI/CD best practices + The intention of this sample is to put together [DevOps](https://aws.amazon.com/training/learn-about/devops/) CI/CD best practices and provide a sample for the [AWS CodePipeline](https://aws.amazon.com/codepipeline/). After implementing this sample, you will get an AWS CodePipeline with linting, testing, security check, deployment and post-deployment process. ## Overview + This project is based on [AWS CDK v2](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-construct-library.html) and uses TypeScript as a primary language. + ![pipepline](./docs/pipeline.png) + The resulting pipeline deploy solution to 3 stages: * dev - active development stage * test - for advanced testing and integration check * prod - production environment -In the dev stage there are 3 steps `Liniting`, `Security` and `UnitTests`. These stups runs in parallel to speedup the process. -The pipeline will stop execution on each faled step. +In the dev stage there are 3 steps `Liniting`, `Security` and `UnitTests`. These steps runs in parallel to speedup the process. +The pipeline will stop execution on each failed step. ## Prerequsites -This project use AWS CDK v2 based on typescript. The developer laptop/computer should have following software. -* [cnf_nag](https://github.com/stelligent/cfn_nag) +This project use AWS CDK v2 based on [Typescript](https://docs.aws.amazon.com/cdk/v2/guide/work-with-cdk-typescript.html). Make sure you have installed all of the following prerequisites on your development machine: + +* [cfn_nag](https://github.com/stelligent/cfn_nag) * [git-remote-codecommit](https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-git-remote-codecommit.html) * [node](https://github.com/nvm-sh/nvm) v16.3.0 * [npm](https://github.com/nvm-sh/nvm) 7.15.1 -### MacOS installing +### Installation -Use following command to install required software for MacOS: -``` +#### MacOS or Linux + +If you are using MacOS, you can install the prerequisites by running the following command in your preferred terminal or also on Linux using [homebrew for Linux](https://docs.brew.sh/Homebrew-on-Linux): + +```bash nvm install 16.3 brew install git-remote-codecommit brew install ruby brew-gem brew gem install cfn-nag ``` +#### AWS Cloud9 -### AWS Cloud9 installing +If you are using AWS [Cloud9](https://aws.amazon.com/cloud9/), you can use following command to install the prerequisites: -Use following command to install required software for [Cloud9](https://aws.amazon.com/cloud9/): -``` +```bash gem install cfn-nag ``` +Note: Cloud9 should have node and npm installed. You can check the installation or version by running the following command: -To check installed versions use following commands -``` -cfn_nag -v +```bash node -v npm -v ``` -## Inital deployment in the clean AWS account -Below you may find instructions for initial deployment in your AWS account -1. Clone this repository with following command. +## Initial Deployment -``` +For the initial deployment in your AWS account, you can run the following command: + +### Cloning The GitHub Repository + +```bash git clone --depth 1 https://github.com/aws-samples/aws-codepipeline-cicd.git +``` +You should remove `.git` dir from the cloned repository as [later](#set-up-remote-codecommit-repository-and-branch) we will use newly created codecommit repository as a remote origin. + +```bash cd ./aws-codepipeline-cicd rm -rf ./.git ``` -2. Connect to AWS account. It could be temporary security token or landing zone auth. Check that you are in the right account by following command: +### Connect to AWS Account + +It could be temporary security token or landing zone auth. Make sure you are using the correct account and region by running the following command: ``` AWS_REGION="eu-west-1" @@ -64,33 +79,78 @@ ACCOUNT_NUMBER=$(aws sts get-caller-identity --query Account --output text) echo "${ACCOUNT_NUMBER}" ``` -3. Prepare the AWS Account by following command +### Bootstrapping an environment -``` -npm i +```bash +npm install +npm run build npm run cdk bootstrap "aws://${ACCOUNT_NUMBER}/${AWS_REGION}" ``` -Confirm installation and complete the account preparation -4. Build stacks using following command +If you are facing issues with cdk staging bucket conflict, as cdk uses `hnb659fds` as default qualifier, you can specify customize qualifier using context in `cdk.json` file: + +```json +"context": { + "@aws-cdk/core:bootstrapQualifier": "cdkdemo", + +} +``` + + +After successful bootstrap, you should see the following output: +```bash + ⏳ Bootstrapping environment aws://{account#}/eu-west-1... + ✅ Environment aws://{account#}/eu-west-1 bootstrapped +``` + +For more details refer CDK Bootstraping section in [AWS CDK](https://docs.aws.amazon.com/cdk/v2/guide/bootstrapping.html). + + +### Synthesize a template + +To synthesize a CDK app, use the `cdk synth` command: ``` npm run cdk synth ``` +You should see following output: -You should see following output ``` -Successfully synthesized to /Users/user/CodePipeline/cdk.out +Successfully synthesized to /aws-codepipeline-cicd/cdk.out Supply a stack id (CodePipeline, Dev-MainStack) to display its template. ``` -5. Deploy the CodePipeline stack to create repository and pipeline for further execution +### Deploy CodePipeline stack + +Now, you can deploy the CodePipeline stack by running following command. This is required as it will create the necessary prerequisites(CodeCommit Repository) for the CodePipeline. ``` npm run cdk -- deploy CodePipeline --require-approval never ``` -6. After deployment collect the repository name from the pipeline stack and set it as original for your folder. +You should see output like following: +```bash +CodePipeline: deploying... +CodePipeline: creating CloudFormation changeset... + + + ✅ CodePipeline + +Outputs: +CodePipeline.RepositoryName = SampleRepository +Stack ARN: +arn:aws:cloudformation:REGION:ACCOUNT-ID:stack/CodePipeline/STACK-ID +``` + +### Set up remote CodeCommit Repository and branch + +After successful deployment of CodePipeline stack, you should see the newly created CodeCommit repository and CodePipeline. + +You can see the CodePipeline initial execution in [AWS CodePipeline console](https://eu-west-1.console.aws.amazon.com/codesuite/codepipeline/pipelines). By default when you create a repository in CodeCommit via CDK, no branch is created and thats why CodePipeline is failing to execute with below error: + +`The action failed because no branch named main was found in the selected AWS CodeCommit repository SampleRepository. Make sure you are using the correct branch name, and then try again. Error: null` + +You can set up remote origin as a `SampleRepository` and create required main branch by running the following command: ``` RepoName=$(aws cloudformation describe-stacks --stack-name CodePipeline --query "Stacks[0].Outputs[?OutputKey=='RepositoryName'].OutputValue" --output text) @@ -105,11 +165,10 @@ git commit -m "Initial commit" git push -u origin main ``` -Open [AWS CodePipeline](https://console.aws.amazon.com/codesuite/codepipeline/pipelines) page and follow the AWS CodePipeline execution. +## CodePipeline in Action -## Development process via Pipeline +After successful initial deployment, you should have complete CI/CD pipeline with a `main` branch of `SampleRepository` as a Source branch. As soon as you commit changes to the `main` branch the AWS CodePipeline will trigger and execute following sequence of actions: -After initialisation you have complete CI/CD process with target to the `main` branch. As soon as you commit changes to the main branch the AWS CodePipeline will trigger and execute following sequence of actions: 1. Get your code from the AWS CodeCommit repository 2. Build your code 3. Update the pipeline itself (SelfMutate) @@ -126,7 +185,6 @@ To simplify development process and provide an ability to run tests locally we u * Deploy to the current account: `make deploy` * Cleanup the environment: `make clean` - ## Code of Conduct This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact diff --git a/bin/code_pipeline.ts b/bin/code_pipeline.ts index a489286..c97cf0c 100644 --- a/bin/code_pipeline.ts +++ b/bin/code_pipeline.ts @@ -3,8 +3,14 @@ import 'source-map-support/register' import * as cdk from 'aws-cdk-lib' import { CodePipelineStack } from '../lib/pipeline-stack' import { MainStack } from '../lib/main-stack' +import { DefaultStackSynthesizer } from 'aws-cdk-lib' +const defaultStackSynthesizer = new DefaultStackSynthesizer({ + fileAssetsBucketName: 'cdk-${Qualifier}-assets-${AWS::AccountId}-${AWS::Region}', + bootstrapStackVersionSsmParameter: '/cdk-bootstrap/${Qualifier}/version', +}) const app = new cdk.App() -new CodePipelineStack(app, 'CodePipeline') - +new CodePipelineStack(app, 'CodePipeline', { + synthesizer: defaultStackSynthesizer, +}) new MainStack(app, 'Dev-MainStack') diff --git a/package.json b/package.json index 4a72a0e..14abc30 100644 --- a/package.json +++ b/package.json @@ -25,8 +25,8 @@ "jest": "^26.4.2", "jest-junit": "^13.2.0", "ts-jest": "^26.2.0", - "ts-node": "^9.0.0", - "typescript": "^4.6.2" + "ts-node": "^10.8.1", + "typescript": "^4.7.3" }, "dependencies": { "aws-cdk-lib": "2.22.0", From d95a8ea738cb03f2529f19f7a37c0c3a3b6ed259 Mon Sep 17 00:00:00 2001 From: Cizer Pereira Date: Tue, 7 Jun 2022 15:55:14 +0200 Subject: [PATCH 2/4] fix: linting issues --- bin/code_pipeline.ts | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/bin/code_pipeline.ts b/bin/code_pipeline.ts index c97cf0c..4a6de56 100644 --- a/bin/code_pipeline.ts +++ b/bin/code_pipeline.ts @@ -6,11 +6,14 @@ import { MainStack } from '../lib/main-stack' import { DefaultStackSynthesizer } from 'aws-cdk-lib' const defaultStackSynthesizer = new DefaultStackSynthesizer({ - fileAssetsBucketName: 'cdk-${Qualifier}-assets-${AWS::AccountId}-${AWS::Region}', - bootstrapStackVersionSsmParameter: '/cdk-bootstrap/${Qualifier}/version', + /* eslint-disable */ + // Disable eslint no-template-curly-in-string + fileAssetsBucketName: 'cdk-${Qualifier}-assets-${AWS::AccountId}-${AWS::Region}', + bootstrapStackVersionSsmParameter: '/cdk-bootstrap/${Qualifier}/version' + /* eslint-enable */ }) const app = new cdk.App() new CodePipelineStack(app, 'CodePipeline', { - synthesizer: defaultStackSynthesizer, + synthesizer: defaultStackSynthesizer }) new MainStack(app, 'Dev-MainStack') From c16f75278344aaa9deab6a95e69c83912892349b Mon Sep 17 00:00:00 2001 From: Cizer Pereira Date: Mon, 13 Jun 2022 10:10:06 +0200 Subject: [PATCH 3/4] remove defaultsynthesizer --- README.md | 9 --------- bin/code_pipeline.ts | 13 ++----------- 2 files changed, 2 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index acf383a..f52c10d 100644 --- a/README.md +++ b/README.md @@ -87,15 +87,6 @@ npm run build npm run cdk bootstrap "aws://${ACCOUNT_NUMBER}/${AWS_REGION}" ``` -If you are facing issues with cdk staging bucket conflict, as cdk uses `hnb659fds` as default qualifier, you can specify customize qualifier using context in `cdk.json` file: - -```json -"context": { - "@aws-cdk/core:bootstrapQualifier": "cdkdemo", - -} -``` - After successful bootstrap, you should see the following output: ```bash diff --git a/bin/code_pipeline.ts b/bin/code_pipeline.ts index 4a6de56..a489286 100644 --- a/bin/code_pipeline.ts +++ b/bin/code_pipeline.ts @@ -3,17 +3,8 @@ import 'source-map-support/register' import * as cdk from 'aws-cdk-lib' import { CodePipelineStack } from '../lib/pipeline-stack' import { MainStack } from '../lib/main-stack' -import { DefaultStackSynthesizer } from 'aws-cdk-lib' -const defaultStackSynthesizer = new DefaultStackSynthesizer({ - /* eslint-disable */ - // Disable eslint no-template-curly-in-string - fileAssetsBucketName: 'cdk-${Qualifier}-assets-${AWS::AccountId}-${AWS::Region}', - bootstrapStackVersionSsmParameter: '/cdk-bootstrap/${Qualifier}/version' - /* eslint-enable */ -}) const app = new cdk.App() -new CodePipelineStack(app, 'CodePipeline', { - synthesizer: defaultStackSynthesizer -}) +new CodePipelineStack(app, 'CodePipeline') + new MainStack(app, 'Dev-MainStack') From 0efda297baabd735788ca638bfffd369eaf68bdb Mon Sep 17 00:00:00 2001 From: Cizer Pereira Date: Mon, 13 Jun 2022 10:21:24 +0200 Subject: [PATCH 4/4] tiny beautification in readme --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index d88beca..ced3f79 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ The intention of this sample is to put together [DevOps](https://aws.amazon.com/training/learn-about/devops/) CI/CD best practices and provide a sample for the [AWS CodePipeline](https://aws.amazon.com/codepipeline/). After implementing this sample, you will get an AWS CodePipeline with linting, testing, security check, deployment and post-deployment process. -Target technology stack +### Target technology stack After execution of the CDK code, following type of resources gets generated: @@ -66,8 +66,8 @@ npm -v ``` ### AWS CLI SetUp -[Windows:Configure for HTTPS connections to your CodeCommit repositories](https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-https-windows.html) -[Linux, macOS, Unix:Configure for HTTPS connections to your CodeCommit repositories](https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-https-unixes.html) +- Windows: [Configure for HTTPS connections to your CodeCommit repositories](https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-https-windows.html) +- Linux, macOS, Unix: [Configure for HTTPS connections to your CodeCommit repositories](https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-https-unixes.html) ## Initial Deployment