Skip to content

Commit 1e3021a

Browse files
per1234cmaglie
authored andcommitted
Use npm to manage tool dependencies
Some of the project infrastructure uses tools sourced from the npm software registry. Previously, the version of the tools used was not controlled. This was problematic because: - A different version of the tool may be used on the contributor's machine than on the CI runner, resulting in confusing failures. - The project is immediately subject to disruption or breakage resulting from a release of the tool. --- These tools were installed via either of the following methods: `npx <pkg>` This approach has the following behaviors of interest: https://docs.npmjs.com/cli/v8/commands/npx#description > If any requested packages are not present in the local project dependencies, then they are installed to a folder in the npm cache, which is added to the PATH environment variable in the executed process. > Package names provided without a specifier will be matched with whatever version exists in the local project. Package names with a specifier will only be considered a match if they have the exact same name and version as the local dependency. This means that the version used was: 1. Whatever happens to be present in the local cache 2. The latest available version if it is not already present `npm install --global <pkg>` The latest available version of the package is used. --- ` The new approach is to specify the version of the tools via the standard npm metadata files (package.json + package-lock.json). This approach was chosen over the `npx <pkg>@<version>` alternative for the following reasons: - Enables automated updates via Dependabot PRs - Enables automated vulnerability alerts - Separates dependency management from the asset contents (i.e., no need to mess with the taskfile or workflow on every update) - Matches how we are already managing Python dependencies (pyproject.toml + poetry.lock)
1 parent 93a6283 commit 1e3021a

11 files changed

+3468
-31
lines changed

.github/dependabot.yml

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,12 @@ updates:
1111
interval: daily
1212
labels:
1313
- "topic: infrastructure"
14+
15+
- package-ecosystem: npm
16+
directory: /
17+
schedule:
18+
interval: daily
19+
labels:
20+
- "topic: infrastructure"
21+
assignees:
22+
- per1234

.github/workflows/check-markdown-task.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,8 @@ on:
1111
paths:
1212
- ".github/workflows/check-markdown-task.ya?ml"
1313
- ".markdown-link-check.json"
14+
- "package.json"
15+
- "package-lock.json"
1416
- "Taskfile.ya?ml"
1517
- "**/.markdownlint*"
1618
- "**.mdx?"
@@ -21,6 +23,8 @@ on:
2123
paths:
2224
- ".github/workflows/check-markdown-task.ya?ml"
2325
- ".markdown-link-check.json"
26+
- "package.json"
27+
- "package-lock.json"
2428
- "Taskfile.ya?ml"
2529
- "**/.markdownlint*"
2630
- "**.mdx?"

.github/workflows/check-taskfiles.yml

Lines changed: 14 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -10,11 +10,15 @@ on:
1010
push:
1111
paths:
1212
- ".github/workflows/check-taskfiles.ya?ml"
13+
- "package.json"
14+
- "package-lock.json"
1315
- "**/Taskfile.ya?ml"
1416
- "**/DistTasks.ya?ml"
1517
pull_request:
1618
paths:
1719
- ".github/workflows/check-taskfiles.ya?ml"
20+
- "package.json"
21+
- "package-lock.json"
1822
- "**/Taskfile.ya?ml"
1923
- "**/DistTasks.ya?ml"
2024
schedule:
@@ -55,18 +59,17 @@ jobs:
5559
location: ${{ runner.temp }}/taskfile-schema
5660

5761
- name: Install JSON schema validator
58-
run: |
59-
sudo npm install \
60-
--global \
61-
ajv-cli \
62-
ajv-formats
62+
run: npm install
6363

6464
- name: Validate ${{ matrix.file }}
6565
run: |
6666
# See: https://github.com/ajv-validator/ajv-cli#readme
67-
ajv validate \
68-
--all-errors \
69-
--strict=false \
70-
-c ajv-formats \
71-
-s "${{ steps.download-schema.outputs.file-path }}" \
72-
-d "${{ matrix.file }}"
67+
npx \
68+
--package=ajv-cli \
69+
--package=ajv-formats \
70+
ajv validate \
71+
--all-errors \
72+
--strict=false \
73+
-c ajv-formats \
74+
-s "${{ steps.download-schema.outputs.file-path }}" \
75+
-d "${{ matrix.file }}"

.github/workflows/check-workflows-task.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,14 @@ on:
1010
push:
1111
paths:
1212
- ".github/workflows/*.ya?ml"
13+
- "package.json"
14+
- "package-lock.json"
1315
- "Taskfile.ya?ml"
1416
pull_request:
1517
paths:
1618
- ".github/workflows/*.ya?ml"
19+
- "package.json"
20+
- "package-lock.json"
1721
- "Taskfile.ya?ml"
1822
schedule:
1923
# Run every Tuesday at 8 AM UTC to catch breakage resulting from changes to the JSON schema.

.github/workflows/sync-labels.yml renamed to .github/workflows/sync-labels-npm.yml

Lines changed: 22 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/sync-labels.md
1+
# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/sync-labels-npm.md
22
name: Sync Labels
33

44
env:
@@ -11,12 +11,16 @@ env:
1111
on:
1212
push:
1313
paths:
14-
- ".github/workflows/sync-labels.ya?ml"
14+
- ".github/workflows/sync-labels-npm.ya?ml"
1515
- ".github/label-configuration-files/*.ya?ml"
16+
- "package.json"
17+
- "package-lock.json"
1618
pull_request:
1719
paths:
18-
- ".github/workflows/sync-labels.ya?ml"
20+
- ".github/workflows/sync-labels-npm.ya?ml"
1921
- ".github/label-configuration-files/*.ya?ml"
22+
- "package.json"
23+
- "package-lock.json"
2024
schedule:
2125
# Run daily at 8 AM UTC to sync with changes to shared label configurations.
2226
- cron: "0 8 * * *"
@@ -44,20 +48,19 @@ jobs:
4448
location: ${{ runner.temp }}/label-configuration-schema
4549

4650
- name: Install JSON schema validator
47-
run: |
48-
sudo npm install \
49-
--global \
50-
ajv-cli \
51-
ajv-formats
51+
run: npm install
5252

5353
- name: Validate local labels configuration
5454
run: |
5555
# See: https://github.com/ajv-validator/ajv-cli#readme
56-
ajv validate \
57-
--all-errors \
58-
-c ajv-formats \
59-
-s "${{ steps.download-schema.outputs.file-path }}" \
60-
-d "${{ env.CONFIGURATIONS_FOLDER }}/*.{yml,yaml}"
56+
npx \
57+
--package=ajv-cli \
58+
--package=ajv-formats \
59+
ajv validate \
60+
--all-errors \
61+
-c ajv-formats \
62+
-s "${{ steps.download-schema.outputs.file-path }}" \
63+
-d "${{ env.CONFIGURATIONS_FOLDER }}/*.{yml,yaml}"
6164
6265
download:
6366
needs: check
@@ -137,14 +140,15 @@ jobs:
137140
cat "${{ env.CONFIGURATIONS_FOLDER }}"/*.@(yml|yaml) > "${{ env.MERGED_CONFIGURATION_PATH }}"
138141
139142
- name: Install github-label-sync
140-
run: sudo npm install --global github-label-sync
143+
run: npm install
141144

142145
- name: Sync labels
143146
env:
144147
GITHUB_ACCESS_TOKEN: ${{ secrets.GITHUB_TOKEN }}
145148
run: |
146149
# See: https://github.com/Financial-Times/github-label-sync
147-
github-label-sync \
148-
--labels "${{ env.MERGED_CONFIGURATION_PATH }}" \
149-
${{ steps.dry-run.outputs.flag }} \
150-
${{ github.repository }}
150+
npx \
151+
github-label-sync \
152+
--labels "${{ env.MERGED_CONFIGURATION_PATH }}" \
153+
${{ steps.dry-run.outputs.flag }} \
154+
${{ github.repository }}

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,3 +12,4 @@ __pycache__/
1212
*.sublime-workspace
1313

1414
/dist
15+
/node_modules/

.markdownlintignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/check-markdown/.markdownlintignore
2+
.licenses/
3+
__pycache__/
4+
node_modules/

.prettierignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
11
/.licenses/
2+
node_modules/

Taskfile.yml

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,8 @@ tasks:
7575
# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/check-prettier-formatting-task/Taskfile.yml
7676
general:format-prettier:
7777
desc: Format all supported files with Prettier
78+
deps:
79+
- task: npm:install-deps
7880
cmds:
7981
- npx prettier --write .
8082

@@ -98,12 +100,16 @@ tasks:
98100
# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/check-markdown-task/Taskfile.yml
99101
markdown:lint:
100102
desc: Check for problems in Markdown files
103+
deps:
104+
- task: npm:install-deps
101105
cmds:
102106
- npx markdownlint-cli "**/*.md"
103107

104108
# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/check-markdown-task/Taskfile.yml
105109
markdown:fix:
106110
desc: Automatically correct linting violations in Markdown files where possible
111+
deps:
112+
- task: npm:install-deps
107113
cmds:
108114
- npx markdownlint-cli --fix "**/*.md"
109115

@@ -112,6 +118,7 @@ tasks:
112118
desc: Check for broken links
113119
deps:
114120
- task: docs:generate
121+
- task: npm:install-deps
115122
cmds:
116123
- |
117124
if [[ "{{.OS}}" == "Windows_NT" ]]; then
@@ -128,7 +135,14 @@ tasks:
128135
# Using -regex instead of -name to avoid Task's behavior of globbing even when quoted on Windows
129136
# The odd method for escaping . in the regex is required for windows compatibility because mvdan.cc/sh gives
130137
# \ characters special treatment on Windows in an attempt to support them as path separators.
131-
for file in $(find . -regex ".*[.]md"); do
138+
for file in $(
139+
find . \
140+
-type d -name '.git' -prune -o \
141+
-type d -name '.licenses' -prune -o \
142+
-type d -name '__pycache__' -prune -o \
143+
-type d -name 'node_modules' -prune -o \
144+
-regex ".*[.]md" -print
145+
); do
132146
markdown-link-check \
133147
--quiet \
134148
--config "./.markdown-link-check.json" \
@@ -139,7 +153,14 @@ tasks:
139153
else
140154
npx --package=markdown-link-check --call='
141155
STATUS=0
142-
for file in $(find . -regex ".*[.]md"); do
156+
for file in $(
157+
find . \
158+
-type d -name '.git' -prune -o \
159+
-type d -name '.licenses' -prune -o \
160+
-type d -name '__pycache__' -prune -o \
161+
-type d -name 'node_modules' -prune -o \
162+
-regex ".*[.]md" -print
163+
); do
143164
markdown-link-check \
144165
--quiet \
145166
--config "./.markdown-link-check.json" \
@@ -150,6 +171,12 @@ tasks:
150171
'
151172
fi
152173
174+
# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/npm-task/Taskfile.yml
175+
npm:install-deps:
176+
desc: Install dependencies managed by npm
177+
cmds:
178+
- npm install
179+
153180
# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/check-workflows-task/Taskfile.yml
154181
ci:validate:
155182
desc: Validate GitHub Actions workflows against their JSON schema
@@ -159,6 +186,8 @@ tasks:
159186
WORKFLOW_SCHEMA_PATH:
160187
sh: mktemp -t workflow-schema-XXXXXXXXXX.json
161188
WORKFLOWS_DATA_PATH: "./.github/workflows/*.{yml,yaml}"
189+
deps:
190+
- task: npm:install-deps
162191
cmds:
163192
- |
164193
wget \

0 commit comments

Comments
 (0)