Skip to content

Commit 61bd559

Browse files
mbaretambareta
authored
Add CirclCI recipe (#5)
1 parent c1f71e1 commit 61bd559

File tree

1 file changed

+344
-0
lines changed

1 file changed

+344
-0
lines changed

‎recipes/circleci-build-guide.md‎

Lines changed: 344 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,344 @@
1+
# CircleCI Build Guide Setup
2+
3+
The following page provides a "getting started" example of CircleCI config.
4+
This config is used on a few active Studion projects. However, there are plans
5+
to make this guide obsolete by making Studion orb and a CLI script to generate
6+
config.yml file automatically. We can use this as a reference and to learn how to
7+
set up CircleCI. Especially as Studion orb(s) will be based off this setup.
8+
9+
## The application
10+
11+
This guide assumes the application has the following components:
12+
- SPA frontend
13+
- Dockerfile for building server
14+
- Pulumi config for infrastructure
15+
16+
This guide also assumes we are hosting the app on AWS.
17+
18+
## `./.circleci/config.yml`
19+
20+
CircleCI uses `config.yml` file to define the tasks it will perform.
21+
The file should be placed inside `.circleci` directory in project root.
22+
Knowing YAML syntax is a prerequisite for writing CircleCI config.
23+
24+
For better clarity, we will look at separate config blocks and describe what they do.
25+
26+
### Config Init
27+
28+
Here we define the config version and dependencies.
29+
30+
Orbs are CircleCI packages that allow us to define build process
31+
in a simple and easy way. Read more about orbs here https://circleci.com/orbs/.
32+
33+
For this app, we need `aws-cli`, `aws-ecr`, `node` and `pulumi` orbs.
34+
35+
36+
```yaml
37+
version: 2.1
38+
orbs:
39+
aws-cli: circleci/[email protected]
40+
aws-ecr: circleci/[email protected]
41+
node: circleci/[email protected]
42+
pulumi: pulumi/[email protected]
43+
44+
executors:
45+
node:
46+
docker:
47+
- image: cimg/node:16.20.2
48+
base:
49+
docker:
50+
- image: cimg/base:stable-20.04
51+
```
52+
53+
### AWS Credentials
54+
55+
In case we have multiple AWS credentials, we can define them at the beginning and
56+
reuse them where applicable. In this example, we have Studion AWS account and client
57+
AWS account credentials.
58+
59+
```yaml
60+
studion-aws-credentials: &studion-aws-credentials
61+
access_key: STUDION_AWS_ACCESS_KEY
62+
secret_key: STUDION_AWS_SECRET_KEY
63+
region: ${STUDION_AWS_REGION}
64+
65+
client-aws-credentials: &client-aws-credentials
66+
access_key: CLIENT_AWS_ACCESS_KEY
67+
secret_key: CLIENT_AWS_SECRET_KEY
68+
region: ${CLIENT_AWS_REGION}
69+
```
70+
71+
Note that we used YAML anchor here so we can reuse the credentials objects.
72+
Also, note that `access_key` and `secret_key` just contain the name of the env
73+
variable while `region` contains the actual value of the env variable.
74+
75+
Environment variables are configured in CircleCI project settings
76+
within the CircleCI application.
77+
78+
79+
### Job 1: Build Frontend
80+
81+
This step pulls the code, injects secret .npmrc file, installs npm packages and
82+
runs build process. Finally, the output is persisted to workspace so we can upload
83+
it to S3 later in the build process.
84+
85+
```yaml
86+
jobs:
87+
build-frontend:
88+
working_directory: ~/app
89+
executor: node
90+
steps:
91+
- checkout
92+
- run:
93+
command: echo "@fortawesome:registry=https://npm.fontawesome.com/" > ~/app/.npmrc
94+
- run:
95+
command: echo "//npm.fontawesome.com/:_authToken=${FA_TOKEN}" >> ~/app/.npmrc
96+
- node/install-packages:
97+
override-ci-command: npm ci
98+
- run:
99+
name: Build frontend
100+
command: npm run build
101+
- persist_to_workspace:
102+
root: .
103+
paths:
104+
- dist
105+
```
106+
107+
In this example, we have .npmrc file that contains the auth token for Font Awesome Pro
108+
package. This is how we can construct that file so `npm install` can install all
109+
required packages.
110+
111+
112+
### Job 2: Build server
113+
114+
This step pulls the code and uses AWS ECR orb to build Docker image and push it to
115+
private AWS registry.
116+
117+
```yaml
118+
build-server:
119+
working_directory: ~/app
120+
executor:
121+
name: aws-ecr/default
122+
docker_layer_caching: true
123+
parameters:
124+
access_key:
125+
type: string
126+
secret_key:
127+
type: string
128+
region:
129+
type: string
130+
account_id:
131+
type: string
132+
ecr_repo:
133+
type: string
134+
resource_class: medium
135+
steps:
136+
- checkout
137+
- run:
138+
command: echo "@fortawesome:registry=https://npm.fontawesome.com/" > ~/app/.npmrc
139+
- run:
140+
command: echo "//npm.fontawesome.com/:_authToken=${FA_TOKEN}" >> ~/app/.npmrc
141+
- aws-ecr/build_and_push_image:
142+
auth:
143+
- aws-cli/setup:
144+
aws_access_key_id: << parameters.access_key >>
145+
aws_secret_access_key: << parameters.secret_key >>
146+
region: << parameters.region >>
147+
account_id: << parameters.account_id >>
148+
attach_workspace: true
149+
checkout: false
150+
extra_build_args: "--secret id=npmrc_secret,src=.npmrc --target server"
151+
region: << parameters.region >>
152+
repo: << parameters.ecr_repo >>
153+
repo_encryption_type: KMS
154+
tag: latest,${CIRCLE_SHA1}
155+
```
156+
157+
Note that this step accepts parameters which will be passed later when we define
158+
the complete workflow.
159+
160+
More info about AWS ECR orb can be found here:
161+
https://circleci.com/developer/orbs/orb/circleci/aws-ecr
162+
163+
164+
### Job 3: Deploy infrastructure
165+
166+
This part calls Pulumi to set up AWS resources.
167+
168+
```yaml
169+
deploy-aws:
170+
working_directory: ~/app
171+
executor: node
172+
parameters:
173+
access_key:
174+
type: string
175+
secret_key:
176+
type: string
177+
region:
178+
type: string
179+
account_id:
180+
type: string
181+
ecr_repo:
182+
type: string
183+
stack:
184+
type: string
185+
steps:
186+
- checkout
187+
- aws-cli/setup:
188+
aws_access_key_id: << parameters.access_key >>
189+
aws_secret_access_key: << parameters.secret_key >>
190+
region: << parameters.region >>
191+
- pulumi/login
192+
- node/install-packages:
193+
app-dir: ./infrastructure
194+
- run:
195+
name: Configure envs
196+
command: |
197+
echo 'export SERVER_IMAGE="<< parameters.account_id >>.dkr.ecr.<< parameters.region >>.amazonaws.com/<< parameters.ecr_repo >>:${CIRCLE_SHA1}"' >> "$BASH_ENV"
198+
source "$BASH_ENV"
199+
- pulumi/update:
200+
stack: "<< parameters.stack >>"
201+
working_directory: ./infrastructure
202+
skip-preview: true
203+
- pulumi/stack_output:
204+
stack: "<< parameters.stack >>"
205+
property_name: frontendBucketName
206+
env_var: S3_SITE_BUCKET
207+
working_directory: ./infrastructure
208+
- pulumi/stack_output:
209+
stack: "<< parameters.stack >>"
210+
property_name: cloudfrontId
211+
env_var: CF_DISTRIBUTION_ID
212+
working_directory: ./infrastructure
213+
- run:
214+
name: Store pulumi output as env file
215+
command: cp $BASH_ENV bash.env
216+
- persist_to_workspace:
217+
root: .
218+
paths:
219+
- bash.env
220+
```
221+
222+
Note that this step assumes that Pulumi files are located in `infrastructure`
223+
directory in project root.
224+
225+
We export `SERVER_IMAGE` env variable which is used in Pulumi to create an ECS
226+
service with that image. Notice we're missing .env files. That is because we put
227+
all secrets in AWS SSM Parameter Store and we configured our Pulumi ECS service
228+
to pull the secrets from there.
229+
230+
Pulumi needs to be configured so it outputs at least two variables:
231+
232+
1. S3 bucket name where we will upload built frontend from job 1
233+
2. Cloudfront Distribution ID which we'll use to invalidate its cache
234+
235+
236+
Both variables are stored in `bash.env` file and that file is persisted to workspace
237+
because that is the easiest way of carrying those variables over to the next step.
238+
239+
### Job 4: Deploy Frontend
240+
241+
This is the step where we upload the frontend dist files from job 1 to S3 bucket
242+
that was created in job 3.
243+
244+
```yaml
245+
deploy-frontend:
246+
working_directory: ~/app
247+
parameters:
248+
access_key:
249+
type: string
250+
secret_key:
251+
type: string
252+
region:
253+
type: string
254+
executor: base
255+
steps:
256+
- attach_workspace:
257+
at: .
258+
- aws-cli/setup:
259+
aws_access_key_id: << parameters.access_key >>
260+
aws_secret_access_key: << parameters.secret_key >>
261+
region: << parameters.region >>
262+
- run:
263+
name: Set environment variables
264+
command: cat bash.env >> $BASH_ENV
265+
- run:
266+
name: Deploy to S3
267+
command: |
268+
aws s3 sync dist s3://${S3_SITE_BUCKET} --no-progress --delete
269+
aws cloudfront create-invalidation --distribution-id ${CF_DISTRIBUTION_ID} --paths "/*"
270+
271+
```
272+
273+
### Workflow definition
274+
275+
Workflow is used to orchestrate different jobs and configure job dependencies,
276+
for example: we need to wait for the infrastructure deployment before we can upload
277+
files to S3 (which is supposed to be created in that job).
278+
In this example we can see that we run this workflow only when the branch name is
279+
`develop`.
280+
281+
```yaml
282+
workflows:
283+
version: 2
284+
build-and-deploy-dev:
285+
when:
286+
and:
287+
- equal: [develop, << pipeline.git.branch >>]
288+
jobs:
289+
- build-frontend
290+
- build-server:
291+
<<: *studion-aws-credentials
292+
account_id: ${STUDION_AWS_ACCOUNT_ID}
293+
ecr_repo: app_server
294+
- deploy-aws:
295+
<<: *studion-aws-credentials
296+
account_id: ${STUDION_AWS_ACCOUNT_ID}
297+
ecr_repo: app_server
298+
stack: dev
299+
requires:
300+
- build-server
301+
- deploy-frontend:
302+
<<: *studion-aws-credentials
303+
requires:
304+
- build-frontend
305+
- deploy-aws
306+
```
307+
308+
Note that job params are set for each job and here we can use AWS credentials which
309+
we defined at the beginning of the file. We can also see that some jobs can run in
310+
parallel, for example: frontend and backend builds don't depend on each other and
311+
that is how we can speed up the build process.
312+
313+
### Workflow definition part 2
314+
315+
In the previous steps we defined job parameters that allow us to
316+
easily build different environments, for example: staging.
317+
318+
Everything remains the same, we just need to change some variables and we can
319+
easily deploy to as many environments as we want.
320+
321+
```yaml
322+
build-and-deploy-stage:
323+
when:
324+
and:
325+
- equal: [stage, << pipeline.git.branch >>]
326+
jobs:
327+
- build-frontend
328+
- build-server:
329+
<<: *client-aws-credentials
330+
account_id: ${CLIENT_AWS_ACCOUNT_ID}
331+
ecr_repo: app_server
332+
- deploy-aws:
333+
<<: *client-aws-credentials
334+
account_id: ${CLIENT_AWS_ACCOUNT_ID}
335+
ecr_repo: app_server
336+
stack: stage
337+
requires:
338+
- build-server
339+
- deploy-frontend:
340+
<<: *client-aws-credentials
341+
requires:
342+
- build-frontend
343+
- deploy-aws
344+
```

0 commit comments

Comments
 (0)