Skip to content

Commit b831ab2

Browse files
tchardingtcharding
committed
Add CI shell scripts
We would like to put all the CI scripts in a single place instead of copied to each repository. Add a `ci/` directory and in it a `run_task.sh` script as well as auxilary scripts required. Include a README to document the directory.
1 parent 7ed1438 commit b831ab2

File tree

2 files changed

+532
-0
lines changed

2 files changed

+532
-0
lines changed

‎ci/README.md

Lines changed: 212 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,212 @@
1+
# Continuous Integration
2+
3+
This directory contains tools used by crates in the `rust-bitcoin` org to implement Continuous
4+
Integration. Currently this is just a script `run_task.sh` that can be called from a GitHub workflow
5+
job to run a specific task.
6+
7+
TL;DR `./run_task.sh --help`
8+
9+
#### Table Of Contents
10+
11+
- [Usage](#usage)
12+
- [Lock file](#lock-file)
13+
- [Crates](#crates)
14+
* [Per crate environment variables](#per-crate-environment-variables)
15+
* [Additional crate specific tests](#additional-crate-specific-tests)
16+
- [Fuzzing](#fuzzing)
17+
- [Example workflows](#example-workflows)
18+
* [A job using a stable toolchain](#a-job-using-a-stable-toolchain)
19+
* [A job using a specific nightly toolchain](#a-job-using-a-specific-nightly-toolchain)
20+
21+
## Usage
22+
23+
The `run_task.sh` script expects a few things to be present when it runs:
24+
25+
In the repository root:
26+
27+
- A lock file `Cargo.lock`
28+
- A script that defines the crates: `REPO_DIR/contrib/crates.sh`
29+
30+
And for each crate there should exist a directory `REPO_DIR/CRATE/contrib/` containing:
31+
32+
- `test_vars.sh`: Defines environment variables
33+
- Optional: `extra_tests.sh`: Additional test script.
34+
35+
If the repository is not a workspace then per crate files go directly in `REPO_ROOT/contrib/`.
36+
37+
(See [Crates`](#crates) below.)
38+
39+
## Lock file
40+
41+
Repositories MUST contain a `Cargo.lock` file before running `run_task.sh`. `cargo` is typically
42+
called with `--locked`. If you don't care about dependency versions just run `cargo update` in your
43+
CI job (to create a lock file) before calling `run_task.sh`.
44+
45+
If you do care about versions consider adding:
46+
47+
- `Cargo-recent.lock`: A manifest with some recent versions numbers that pass CI.
48+
- `Cargo-minimal.lock`: A manifest with some minimal version numbers that pass CI.
49+
50+
Then you can use, for example:
51+
52+
```yaml
53+
strategy:
54+
matrix:
55+
dep: [minimal, recent]
56+
steps:
57+
58+
<!-- other stuff elided -->
59+
60+
- name: "Copy lock file"
61+
run: cp Cargo-${{ matrix.dep }}.lock Cargo.lock
62+
63+
```
64+
65+
(Tip: Create minimal lock file with`cargo +nightly build -- -Z minimal-versions`.)
66+
67+
## Crates
68+
69+
All repositories MUST include a `REPO_DIR/contrib/crates.sh` script:
70+
71+
```bash
72+
#!/usr/bin/env bash
73+
74+
# Crates in this workspace to test (note "fuzz" is only built not tested).
75+
CRATES=("base58" "bitcoin" "fuzz" "hashes" "internals" "io" "units")
76+
```
77+
78+
`CRATES` MUST be an array. If repository is not a workspace use `CRATES=(".")`).
79+
80+
### Per crate environment variables
81+
82+
All crates MUST include a file `REPO_DIR/CRATE/contrib/test_vars.sh`
83+
84+
```bash
85+
#!/usr/bin/env bash
86+
87+
# Test all these features with "std" enabled.
88+
#
89+
# Ignore this if crate does not have "std" feature.
90+
FEATURES_WITH_STD=""
91+
92+
# Test all these features without "std" enabled.
93+
#
94+
# Use this even if crate does not have "std" feature.
95+
FEATURES_WITHOUT_STD=""
96+
97+
# Run these examples.
98+
EXAMPLES=""
99+
```
100+
101+
#### The `EXAMPLES` variable
102+
103+
```bash
104+
EXAPMLES="example:feature"
105+
```
106+
107+
```bash
108+
EXAPMLES="example:feature1,feature2"
109+
```
110+
111+
```bash
112+
EXAPMLES="example_a:feature1,feature2 example_b:feature1"
113+
```
114+
115+
116+
Tip: if your example does not require any features consider using "default".
117+
118+
```bash
119+
EXAPMLES="example_a:default"
120+
```
121+
122+
### Additional crate specific tests
123+
124+
Additional tests can be put in an optional `contrib/extra_tests.sh` script. This script will be run
125+
as part of the `stable`, `nightly`, and `msrv` jobs after running unit tests.
126+
127+
## Fuzzing
128+
129+
Fuzz tests are expected to be in a crate called `REPO_DIR/fuzz/`. The `run_task.sh` script just
130+
builds the fuzz crate as a sanity check.
131+
132+
## Example workflows
133+
134+
### A job using a stable toolchain
135+
136+
To use the `run_task.sh` script you'll want to do something like this:
137+
138+
```yaml
139+
jobs:
140+
Stable: # 2 jobs, one per manifest.
141+
name: Test - stable toolchain
142+
runs-on: ubuntu-latest
143+
strategy:
144+
fail-fast: false
145+
matrix:
146+
dep: [minimal, recent]
147+
steps:
148+
- name: "Checkout repo"
149+
uses: actions/checkout@v4
150+
- name: "Checkout maintainer tools"
151+
uses: actions/checkout@v4
152+
with:
153+
repository: rust-bitcoin/rust-bitcoin-maintainer-tools
154+
path: maintainer-tools
155+
- name: "Select toolchain"
156+
uses: dtolnay/rust-toolchain@stable
157+
- name: "Copy lock file"
158+
run: cp Cargo-${{ matrix.dep }}.lock Cargo.lock
159+
- name: "Run test script"
160+
run: ./maintainer-tools/ci/run_task.sh stable
161+
```
162+
163+
### A job using a specific nightly toolchain
164+
165+
Have a file in the repository root with the nightly toolchain version to use.
166+
167+
```bash
168+
$ cat nightly_version
169+
nightly-2024-04-30
170+
```
171+
172+
And use a `Prepare` job to a set an environment variable using the file.
173+
174+
```yaml
175+
jobs:
176+
Prepare:
177+
runs-on: ubuntu-latest
178+
outputs:
179+
nightly_version: ${{ steps.read_toolchain.outputs.nightly_version }}
180+
steps:
181+
- name: Checkout Crate
182+
uses: actions/checkout@v4
183+
- name: Read nightly version
184+
id: read_toolchain
185+
run: echo "nightly_version=$(cat nightly-version)" >> $GITHUB_OUTPUT
186+
187+
Nightly: # 2 jobs, one per manifest.
188+
name: Test - nightly toolchain
189+
needs: Prepare
190+
runs-on: ubuntu-latest
191+
strategy:
192+
fail-fast: false
193+
matrix:
194+
dep: [minimal, recent]
195+
steps:
196+
- name: "Checkout repo"
197+
uses: actions/checkout@v4
198+
- name: "Checkout maintainer tools"
199+
uses: actions/checkout@v4
200+
with:
201+
repository: tcharding/rust-bitcoin-maintainer-tools
202+
ref: 05-02-ci
203+
path: maintainer-tools
204+
- name: "Select toolchain"
205+
uses: dtolnay/rust-toolchain@v1
206+
with:
207+
toolchain: ${{ needs.Prepare.outputs.nightly_version }}
208+
- name: "Copy lock file"
209+
run: cp Cargo-${{ matrix.dep }}.lock Cargo.lock
210+
- name: "Run test script"
211+
run: ./maintainer-tools/ci/run_task.sh nightly
212+
```

0 commit comments

Comments
 (0)