Skip to content

Commit 3d9ebbe

Browse files
tchardingtcharding
committed
Add CI shell scripts
We would like to put all the CI scripts in a single place instead of copied to each repository. Add a `ci/` directory and in it a `run_task.sh` script as well as auxilary scripts required. Include a README to document the directory.
1 parent 7ed1438 commit 3d9ebbe

File tree

2 files changed

+536
-0
lines changed

2 files changed

+536
-0
lines changed

‎ci/README.md

Lines changed: 216 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,216 @@
1+
# Continuous Integration
2+
3+
This directory contains tools used by crates in the `rust-bitcoin` org to implement Continuous
4+
Integration. Currently this is just a script `run_task.sh` that can be called from a GitHub workflow
5+
job to run a specific task.
6+
7+
TL;DR `./run_task.sh --help`
8+
9+
#### Table Of Contents
10+
11+
- [Usage](#usage)
12+
- [Lock file](#lock-file)
13+
- [Crates](#crates)
14+
* [Per crate environment variables](#per-crate-environment-variables)
15+
* [Additional crate specific tests](#additional-crate-specific-tests)
16+
- [Fuzzing](#fuzzing)
17+
- [Example workflows](#example-workflows)
18+
* [A job using a stable toolchain](#a-job-using-a-stable-toolchain)
19+
* [A job using a specific nightly toolchain](#a-job-using-a-specific-nightly-toolchain)
20+
21+
## Usage
22+
23+
The `run_task.sh` script expects a few things to be present when it runs:
24+
25+
In the repository root:
26+
27+
- A lock file `Cargo.lock`
28+
- A script that defines the crates: `contrib/crates.sh`
29+
30+
And for each crate:
31+
32+
- `test_vars.sh`
33+
- Optional: `extra_tests.sh`
34+
35+
(See [Crates`](#crates) below.)
36+
37+
## Lock file
38+
39+
Repositories MUST contain a `Cargo.lock` file before running `run_task.sh`. `cargo` is typically
40+
called with `--locked`. If you don't care about dependency versions just run `cargo update` in your
41+
CI job (to create a lock file) before calling `run_task.sh`.
42+
43+
If you do care about versions consider adding:
44+
45+
- `Cargo-recent.lock`: A manifest with some recent versions numbers that pass CI.
46+
- `Cargo-minimal.lock`: A manifest with some minimal version numbers that pass CI.
47+
48+
Then you can use, for example:
49+
50+
```yaml
51+
strategy:
52+
matrix:
53+
dep: [minimal, recent]
54+
steps:
55+
56+
<!-- other stuff elided -->
57+
58+
- name: "Copy lock file"
59+
run: cp Cargo-${{ matrix.dep }}.lock Cargo.lock
60+
61+
```
62+
63+
(Tip: Create minimal lock file with`cargo +nightly build -- -Z minimal-versions`.)
64+
65+
## Crates
66+
67+
All repositories MUST include a `contrib/crates.sh` script that declares the crates to be tested.
68+
69+
This is a `bash` array (if repository is not a workspace just use `CRATES=(".")`).
70+
71+
```bash
72+
#!/usr/bin/env bash
73+
74+
# Crates in this workspace to test (note "fuzz" is only built not tested).
75+
CRATES=("base58" "bitcoin" "fuzz" "hashes" "internals" "io" "units")
76+
```
77+
78+
### Per crate environment variables
79+
80+
All crates MUST include a file `contrib/test_vars.sh`
81+
82+
```bash
83+
#!/usr/bin/env bash
84+
85+
# Test all these features with "std" enabled.
86+
#
87+
# Ignore this if crate does not have "std" feature.
88+
FEATURES_WITH_STD=""
89+
90+
# Test all these features without "std" enabled.
91+
#
92+
# Use this even if crate does not have "std" feature.
93+
FEATURES_WITHOUT_STD=""
94+
95+
# Run these examples.
96+
EXAMPLES=""
97+
```
98+
99+
Tip: For non-worspace repositories this file should be in `contrib/` in the repository root.
100+
101+
#### The `EXAMPLES` variable
102+
103+
```bash
104+
EXAPMLES="example:feature"
105+
```
106+
107+
```bash
108+
EXAPMLES="example:feature1,feature2"
109+
```
110+
111+
```bash
112+
EXAPMLES="example_a:feature1,feature2 example_b:feature1"
113+
```
114+
115+
116+
Tip: if your example does not require any features consider using "default".
117+
118+
```bash
119+
EXAPMLES="example_a:default"
120+
```
121+
122+
### Additional crate specific tests
123+
124+
Additional tests can be put in an optional `contrib/extra_tests.sh` script. This script will be run
125+
as part of the `stable`, `nightly`, and `msrv` jobs after running unit tests.
126+
127+
As for other per-crate files, put it in either the `REPO/CRATE/contrib/` directory or
128+
`REPO_DIR/contrib/` directory depending on whether this is a workspace or not.
129+
130+
## Fuzzing
131+
132+
Fuzz tests are expected to be in a crate called `fuzz/`. The `run_task.sh` script just builds
133+
the fuzz crate as a sanity check.
134+
135+
## Example workflows
136+
137+
### A job using a stable toolchain
138+
139+
To use the `run_task.sh` script you'll want to do something like this:
140+
141+
142+
```yaml
143+
jobs:
144+
Stable: # 2 jobs, one per manifest.
145+
name: Test - stable toolchain
146+
runs-on: ubuntu-latest
147+
strategy:
148+
fail-fast: false
149+
matrix:
150+
dep: [minimal, recent]
151+
steps:
152+
- name: "Checkout repo"
153+
uses: actions/checkout@v4
154+
- name: "Checkout maintainer tools"
155+
uses: actions/checkout@v4
156+
with:
157+
repository: rust-bitcoin/rust-bitcoin-maintainer-tools
158+
path: maintainer-tools
159+
- name: "Select toolchain"
160+
uses: dtolnay/rust-toolchain@stable
161+
- name: "Copy lock file"
162+
run: cp Cargo-${{ matrix.dep }}.lock Cargo.lock
163+
- name: "Run test script"
164+
run: ./maintainer-tools/ci/run_task.sh stable
165+
```
166+
167+
### A job using a specific nightly toolchain
168+
169+
Have a file in the repository root with the nightly toolchain version to use.
170+
171+
```bash
172+
$ cat nightly_version
173+
nightly-2024-04-30
174+
```
175+
176+
And use a `Prepare` job to a set an environment variable using the file.
177+
178+
```yaml
179+
jobs:
180+
Prepare:
181+
runs-on: ubuntu-latest
182+
outputs:
183+
nightly_version: ${{ steps.read_toolchain.outputs.nightly_version }}
184+
steps:
185+
- name: Checkout Crate
186+
uses: actions/checkout@v4
187+
- name: Read nightly version
188+
id: read_toolchain
189+
run: echo "nightly_version=$(cat nightly-version)" >> $GITHUB_OUTPUT
190+
191+
Nightly: # 2 jobs, one per manifest.
192+
name: Test - nightly toolchain
193+
needs: Prepare
194+
runs-on: ubuntu-latest
195+
strategy:
196+
fail-fast: false
197+
matrix:
198+
dep: [minimal, recent]
199+
steps:
200+
- name: "Checkout repo"
201+
uses: actions/checkout@v4
202+
- name: "Checkout maintainer tools"
203+
uses: actions/checkout@v4
204+
with:
205+
repository: tcharding/rust-bitcoin-maintainer-tools
206+
ref: 05-02-ci
207+
path: maintainer-tools
208+
- name: "Select toolchain"
209+
uses: dtolnay/rust-toolchain@v1
210+
with:
211+
toolchain: ${{ needs.Prepare.outputs.nightly_version }}
212+
- name: "Copy lock file"
213+
run: cp Cargo-${{ matrix.dep }}.lock Cargo.lock
214+
- name: "Run test script"
215+
run: ./maintainer-tools/ci/run_task.sh nightly
216+
```

0 commit comments

Comments
 (0)