Skip to content

Use npm to manage tool dependencies #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Sep 2, 2022
Merged

Use npm to manage tool dependencies #22

merged 2 commits into from
Sep 2, 2022

Conversation

per1234
Copy link
Contributor

@per1234 per1234 commented Sep 2, 2022

This is a sync of a change made in the upstream assets, validated there and already in use in other Arduino Tooling projects: arduino/tooling-project-assets#240

Some of the project infrastructure use tools sourced from the npm software registry.

Previously, the version of the tools used was not controlled. This was problematic because:

  • A different version of the tool may be used on the contributor's machine than on the CI runner, resulting in confusing failures.
  • The project is immediately subject to disruption or breakage resulting from a release of the tool.

An example of the latter is seen here, where a new release of a transitive dependency cause spurious failure of the "Check Markdown" workflow:

https://github.com/arduino/serial-monitor/runs/8159584293?check_suite_focus=true


These tools were installed via either of the following methods:

npx <pkg>

This approach has the following behaviors of interest:

https://docs.npmjs.com/cli/v8/commands/npx#description

If any requested packages are not present in the local project dependencies, then they are installed to a folder in the npm cache, which is added to the PATH environment variable in the executed process.

Package names provided without a specifier will be matched with whatever version exists in the local project. Package names with a specifier will only be considered a match if they have the exact same name and version as the local dependency.

This means that the version used was:

  1. Whatever happens to be present in the local npm cache
  2. The latest available version if it is not already present

npm install --global <pkg>

The latest available version of the package is used.


The new approach is to specify the version of the tools via the standard npm metadata files (package.json + package-lock.json). This approach was chosen over the npx <pkg>@<version> alternative for the following reasons:

The GitHub Actions workflows have tool dependencies sourced from npm.

Previously, the workflows were not configured to set up a specific version of Node.js, which caused the default version
from the GitHub Actions runner to be used. That version might be changed at any time, which left the workflows
vulnerable to breakage.

Setting a specific version of Node.js provides stability. Updates to the version can be made in a controlled manner,
with validation before the workflow goes into production use.

The version is set via an environment variable at the top of the workflow in order to facilitate updates.
Some of the project infrastructure uses tools sourced from the npm software registry.

Previously, the version of the tools used was not controlled. This was problematic because:

- A different version of the tool may be used on the contributor's machine than on the CI runner, resulting in confusing
  failures.
- The project is immediately subject to disruption or breakage resulting from a release of the tool.

---

These tools were installed via either of the following methods:

`npx <pkg>`

This approach has the following behaviors of interest:

https://docs.npmjs.com/cli/v8/commands/npx#description

> If any requested packages are not present in the local project dependencies, then they are installed to a folder in
the npm cache, which is added to the PATH environment variable in the executed process.

> Package names provided without a specifier will be matched with whatever version exists in the local project. Package
names with a specifier will only be considered a match if they have the exact same name and version as the local
dependency.

This means that the version used was:

1. Whatever happens to be present in the local cache
2. The latest available version if it is not already present

`npm install --global <pkg>`

The latest available version of the package is used.

---
`
The new approach is to specify the version of the tools via the standard npm metadata files (package.json +
package-lock.json). This approach was chosen over the `npx <pkg>@<version>` alternative for the following reasons:

- Enables automated updates via Dependabot PRs
- Enables automated vulnerability alerts
- Separates dependency management from the asset contents (i.e., no need to mess with the taskfile or workflow on every
  update)
- Matches how we are already managing Python dependencies (pyproject.toml + poetry.lock)
@per1234 per1234 added type: enhancement Proposed improvement topic: infrastructure Related to project infrastructure labels Sep 2, 2022
@per1234 per1234 self-assigned this Sep 2, 2022
@cmaglie cmaglie merged commit 1e3021a into arduino:main Sep 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: infrastructure Related to project infrastructure type: enhancement Proposed improvement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants