Skip to content

Commit 561cbc5

Browse files
authored
chore: build noctilucent WASM library in a container (#26123)
Due to the addition of noctilucent to cdk, contributors needed to download rust/rustup to be able to build the cdk. This uses the pre-existing dependency on Docker/Finch to containerize the process in order to not incur any further dependencies for contributors to manage. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
1 parent 2fec6a4 commit 561cbc5

File tree

7 files changed

+68
-49
lines changed

7 files changed

+68
-49
lines changed

CONTRIBUTING.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -207,7 +207,7 @@ eval $(gp env -e)
207207

208208
### Amazon CodeCatalyst Dev Environments
209209

210-
Dev Environments are cloud-based development environments.
210+
Dev Environments are cloud-based development environments.
211211
[Amazon CodeCatalyst](https://aws.amazon.com/codecatalyst/) allows you to checkout your linked Github
212212
repositories in your Dev Environments with your favorite local IDEs such as VSCode or JetBrains.
213213

@@ -220,7 +220,7 @@ $ NODE_OPTIONS=--max-old-space-size=8192 npx lerna run build --scope=aws-cdk-lib
220220

221221
You may [configure your Dev Env](https://docs.aws.amazon.com/codecatalyst/latest/userguide/devenvironment-devfile.html) with the `devfile.yaml` to further customize your Dev Env for CDK development.
222222

223-
Read the links below for more details:
223+
Read the links below for more details:
224224
- [Dev Environments in CodeCatalyst](https://docs.aws.amazon.com/codecatalyst/latest/userguide/devenvironment.html)
225225
- [Using GitHub repositories in CodeCatalyst](https://docs.aws.amazon.com/codecatalyst/latest/userguide/extensions-github.html)
226226
- [Setting up to use the AWS CLI with CodeCatalyst](https://docs.aws.amazon.com/codecatalyst/latest/userguide/set-up-cli.html)
@@ -429,7 +429,7 @@ $ npx cdk -a test/aws-eks/test/sample.js deploy
429429

430430
This allows you to iterate your development and ensure a minimal sample app would successfully deploy as you expect.
431431
You have the freedom to interact with it just as a common CDK app such as viewing differences with `npx cdk diff`
432-
or pass context variables with `npx cdk deploy -c`. You can rapidly iterate your testing with repeated deployments
432+
or pass context variables with `npx cdk deploy -c`. You can rapidly iterate your testing with repeated deployments
433433
by importing existing resource such as existing VPC. This can save a lot of time and help you focus on the core changes.
434434

435435
```ts
@@ -439,7 +439,7 @@ const vpc = ec2.Vpc.fromLookup(stack, 'Vpc', { isDefault: true });
439439
As this is for testing only, do not commit `sample.ts` and `sample.js` to your PR branch.
440440

441441
Alternatively, you can write this test as a new integration test like `integ.my-test.ts` and deploy it
442-
using `yarn integ --no-clean`. This may be useful when you need to publish a new
442+
using `yarn integ --no-clean`. This may be useful when you need to publish a new
443443
integration test:
444444

445445
```console

packages/aws-cdk/.gitignore

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,3 @@ test/integ/cli/*.d.ts
3939
.DS_Store
4040

4141
junit.xml
42-
43-
# Exclude the noctilucent WASM package
44-
lib/vendor/noctilucent/

packages/aws-cdk/.npmignore

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,5 +29,5 @@ tsconfig.json
2929
**/cdk.out
3030
junit.xml
3131

32-
# exclude noctilucent source
33-
/vendor/noctilucent/
32+
generate.sh
33+
lib/vendor/noctilucent/Dockerfile

packages/aws-cdk/generate.sh

Lines changed: 17 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -15,30 +15,20 @@ cat > build-info.json <<HERE
1515
}
1616
HERE
1717

18-
# Build noctilucent package
19-
(
20-
# Check out the submodule if it's not there already
21-
if [ ! -f "vendor/noctilucent/Cargo.toml" ]; then
22-
git -C ./vendor clone https://github.com/iph/noctilucent.git
23-
fi
24-
25-
# update the package to the pinned commit hash
26-
git -C vendor/noctilucent reset --hard HEAD
27-
git -C vendor/noctilucent fetch && git -C vendor/noctilucent checkout 6da7c9fade55f8443bba7b8fdfcd4ebfe5208fb1
28-
29-
# Install wasm-pack if it's not there already
30-
if ! command -v wasm-pack >/dev/null 2>/dev/null; then
31-
echo "installing wasm-pack, this may take a while..."
32-
cargo install wasm-pack
33-
fi
34-
35-
pkgroot=$(cd $(dirname -- "$0") && pwd)
36-
37-
cd vendor/noctilucent
38-
wasm-pack build --target nodejs \
39-
--out-dir="${pkgroot}/lib/vendor/noctilucent" \
40-
--out-name=index
41-
42-
cd ../../lib/vendor/noctilucent
43-
rm package.json
44-
)
18+
# Build noctilucent package in a Docker/Finch VM
19+
NOCTILUCENT_GIT="https://github.com/iph/noctilucent.git"
20+
NOCTILUCENT_COMMIT_ID="6da7c9fade55f8443bba7b8fdfcd4ebfe5208fb1"
21+
if [ "$(cat lib/vendor/noctilucent/.version 2>/dev/null || echo '')" == "${NOCTILUCENT_GIT}:${NOCTILUCENT_COMMIT_ID}" ]
22+
then
23+
echo "⏭️ Noctilucent WASM binary is up-to date, skipping build..."
24+
echo "ℹ️ Delete lib/vendor/noctilucent/.version to force a rebuild."
25+
else
26+
echo "⏳ Building Noctilucent WASM binary for embedding... This will take a while..."
27+
${CDK_DOCKER:-docker} build --rm \
28+
--build-arg NOCTILUCENT_GIT="${NOCTILUCENT_GIT}" \
29+
--build-arg NOCTILUCENT_COMMIT_ID="${NOCTILUCENT_COMMIT_ID}" \
30+
--file lib/vendor/noctilucent/Dockerfile \
31+
--target wasm \
32+
--output type=local,dest=lib/vendor/noctilucent \
33+
lib/vendor/noctilucent
34+
fi
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# Ignore all files in this directory except the Dockerfile
2+
/*
3+
!/.gitignore
4+
!/Dockerfile
Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
FROM public.ecr.aws/debian/debian:buster-slim as build
2+
3+
# Install basic pre-requisites
4+
RUN apt-get update \
5+
&& apt-get install -y build-essential curl git libssl-dev openssl pkg-config zsh
6+
7+
# Make sure we use the correct shell going forward
8+
SHELL ["/bin/zsh", "-c"]
9+
10+
# Install Rustup
11+
ENV RUSTUP_HOME=/usr/local/rustup
12+
ENV CARGO_HOME=/usr/local/cargo
13+
RUN set -eo pipefail \
14+
&& curl -fSsL "https://sh.rustup.rs" | sh -s -- -y --no-modify-path --profile=minimal \
15+
&& echo "source ${CARGO_HOME}/env" >> /etc/profile.d/cargo.sh \
16+
&& chmod -R a+rw ${CARGO_HOME}
17+
ENV PATH=$PATH:${CARGO_HOME}/bin
18+
19+
# Install Node
20+
RUN curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
21+
&& apt-get install -y nodejs
22+
23+
# Install wasm-pack
24+
RUN cargo install wasm-opt wasm-pack
25+
26+
ARG NOCTILUCENT_GIT
27+
ARG NOCTILUCENT_COMMIT_ID
28+
29+
# Check out noctilucent
30+
RUN git clone "${NOCTILUCENT_GIT}" "${TMPDIR}/noctilucent" \
31+
&& git -C "${TMPDIR}/noctilucent" checkout -b wasm "${NOCTILUCENT_COMMIT_ID}"
32+
33+
# Build noctilucent to WASM
34+
RUN cd "${TMPDIR}/noctilucent" \
35+
&& wasm-pack build --target=nodejs --out-name=index --out-dir=/wasm-out \
36+
&& rm --force /wasm-out/.gitignore /wasm-out/README.md /wasm-out/package.json \
37+
&& echo "${NOCTILUCENT_GIT}:${NOCTILUCENT_COMMIT_ID}" > /wasm-out/.version
38+
39+
####################################################################################################
40+
FROM scratch as wasm
41+
COPY --from=build /wasm-out /

packages/aws-cdk/vendor/README.md

Lines changed: 0 additions & 13 deletions
This file was deleted.

0 commit comments

Comments
 (0)