Skip to content

Commit d5b8f8a

Browse files
seismanweiji14
seisman
and
weiji14
authored
Separate workflows for running tests and building documentation (#1033)
Separates the current "Tests" workflow into two workflows to speedup our CI jobs. - ci_test.yaml: runs the full tests - ci_docs.yml: build the documentation and deploy it on master branch Co-authored-by: Wei Ji <[email protected]>
1 parent 1fb9e2e commit d5b8f8a

File tree

3 files changed

+186
-76
lines changed

3 files changed

+186
-76
lines changed

‎.github/workflows/ci_docs.yml

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
# This workflow installs PyGMT, builds and deploys documentation
2+
3+
name: Docs
4+
5+
on:
6+
push:
7+
branches: [ master ]
8+
pull_request:
9+
types: [opened, reopened, synchronize, ready_for_review]
10+
paths-ignore:
11+
- 'pygmt/tests/**'
12+
- '*.md'
13+
- '*.json'
14+
- 'LICENSE.txt'
15+
- '.gitignore'
16+
- '.pylintrc'
17+
release:
18+
types:
19+
- published
20+
21+
jobs:
22+
docs:
23+
name: ${{ matrix.os }} - Python ${{ matrix.python-version }}
24+
runs-on: ${{ matrix.os }}
25+
strategy:
26+
fail-fast: false
27+
matrix:
28+
python-version: [3.9]
29+
os: [ubuntu-latest, macOS-latest, windows-latest]
30+
# Is it a draft Pull Request (true or false)?
31+
isDraft:
32+
- ${{ github.event.pull_request.draft }}
33+
# Only run one job (Ubuntu + Python 3.9) for draft PRs
34+
exclude:
35+
- os: macOS-latest
36+
isDraft: true
37+
- os: windows-latest
38+
isDraft: true
39+
defaults:
40+
run:
41+
shell: bash -l {0}
42+
43+
steps:
44+
# Cancel previous runs that are not completed
45+
- name: Cancel Previous Runs
46+
uses: styfle/[email protected]
47+
with:
48+
access_token: ${{ github.token }}
49+
50+
# Checkout current git repository
51+
- name: Checkout
52+
uses: actions/[email protected]
53+
with:
54+
# fecth all history so that setuptools-scm works
55+
fetch-depth: 0
56+
57+
# Setup Miniconda
58+
- name: Setup Miniconda
59+
uses: conda-incubator/[email protected]
60+
with:
61+
activate-environment: pygmt
62+
python-version: ${{ matrix.python-version }}
63+
channels: conda-forge
64+
miniconda-version: "latest"
65+
66+
# Install GMT and other required dependencies from conda-forge
67+
- name: Install dependencies
68+
run: |
69+
conda install gmt=6.1.1 numpy pandas xarray netCDF4 packaging \
70+
ipython make myst-parser \
71+
sphinx sphinx-copybutton sphinx-gallery sphinx_rtd_theme=0.4.3
72+
73+
# Show installed pkg information for postmortem diagnostic
74+
- name: List installed packages
75+
run: conda list
76+
77+
# Download cached remote files (artifacts) from GitHub
78+
- name: Download remote data from GitHub
79+
uses: dawidd6/[email protected]
80+
with:
81+
workflow: cache_data.yaml
82+
workflow_conclusion: success
83+
name: gmt-cache
84+
path: .gmt
85+
86+
# Move downloaded files to ~/.gmt directory and list them
87+
- name: Move and list downloaded remote files
88+
run: |
89+
mkdir -p ~/.gmt
90+
mv .gmt/* ~/.gmt
91+
# Change modification times of the two files, so GMT won't refresh it
92+
touch ~/.gmt/server/gmt_data_server.txt ~/.gmt/server/gmt_hash_server.txt
93+
ls -lhR ~/.gmt
94+
95+
# Install the package that we want to test
96+
- name: Install the package
97+
run: |
98+
python setup.py sdist --formats=zip
99+
pip install dist/*
100+
101+
# Build the documentation
102+
- name: Build the documentation
103+
run: make -C doc clean all
104+
105+
- name: Checkout the gh-pages branch
106+
uses: actions/checkout@28c7f3d2b5162b5ddd3dfd9a45aa55eaf396478b
107+
with:
108+
ref: gh-pages
109+
# Checkout to this folder instead of the current one
110+
path: deploy
111+
# Download the entire history
112+
fetch-depth: 0
113+
if: (github.event_name == 'release' || github.event_name == 'push') && (matrix.os == 'ubuntu-latest')
114+
115+
- name: Push the built HTML to gh-pages
116+
run: |
117+
# Detect if this is a release or from the master branch
118+
if [[ "${GITHUB_EVENT_NAME}" == "release" ]]; then
119+
# Get the tag name without the "refs/tags/" part
120+
version="${GITHUB_REF#refs/*/}"
121+
else
122+
version=dev
123+
fi
124+
echo "Deploying version: $version"
125+
# Make the new commit message. Needs to happen before cd into deploy
126+
# to get the right commit hash.
127+
message="Deploy $version from $(git rev-parse --short HEAD)"
128+
cd deploy
129+
# Need to have this file so that Github doesn't try to run Jekyll
130+
touch .nojekyll
131+
# Delete all the files and replace with our new set
132+
echo -e "\nRemoving old files from previous builds of ${version}:"
133+
rm -rvf ${version}
134+
echo -e "\nCopying HTML files to ${version}:"
135+
cp -Rvf ../doc/_build/html/ ${version}/
136+
# If this is a new release, update the link from /latest to it
137+
if [[ "${version}" != "dev" ]]; then
138+
echo -e "\nSetup link from ${version} to 'latest'."
139+
rm -f latest
140+
ln -sf ${version} latest
141+
fi
142+
# Stage the commit
143+
git add -A .
144+
echo -e "\nChanges to be applied:"
145+
git status
146+
# Configure git to be the GitHub Actions account
147+
git config user.email "github-actions[bot]@users.noreply.github.com"
148+
git config user.name "github-actions[bot]"
149+
# If this is a dev build and the last commit was from a dev build
150+
# (detect if "dev" was in the previous commit message), reuse the
151+
# same commit
152+
if [[ "${version}" == "dev" && `git log -1 --format='%s'` == *"dev"* ]]; then
153+
echo -e "\nAmending last commit:"
154+
git commit --amend --reset-author -m "$message"
155+
else
156+
echo -e "\nMaking a new commit:"
157+
git commit -m "$message"
158+
fi
159+
# Make the push quiet just in case there is anything that could leak
160+
# sensitive information.
161+
echo -e "\nPushing changes to gh-pages."
162+
git push -fq origin gh-pages 2>&1 >/dev/null
163+
echo -e "\nFinished uploading generated files."
164+
if: (github.event_name == 'release' || github.event_name == 'push') && (matrix.os == 'ubuntu-latest')

‎.github/workflows/ci_tests.yaml

Lines changed: 8 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
1-
# This workflow installs PyGMT dependencies, build documentation and run tests
2-
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
1+
# This workflow installs PyGMT and runs tests
32

43
name: Tests
54

@@ -10,10 +9,13 @@ on:
109
types: [opened, reopened, synchronize, ready_for_review]
1110
paths-ignore:
1211
- 'doc/**'
12+
- 'examples/**'
1313
- '*.md'
1414
- '*.json'
1515
- 'README.rst'
1616
- 'LICENSE.txt'
17+
- '.gitignore'
18+
- '.pylintrc'
1719
release:
1820
types:
1921
- published
@@ -79,7 +81,10 @@ jobs:
7981

8082
# Install GMT and other required dependencies from conda-forge
8183
- name: Install dependencies
82-
run: conda env update --file environment.yml
84+
run: |
85+
conda install gmt=6.1.1 numpy pandas xarray netCDF4 packaging \
86+
codecov coverage[toml] ipython make \
87+
pytest-cov pytest-mpl pytest>=6.0
8388
8489
# Show installed pkg information for postmortem diagnostic
8590
- name: List installed packages
@@ -121,75 +126,10 @@ jobs:
121126
name: artifact-${{ runner.os }}-${{ matrix.python-version }}
122127
path: tmp-test-dir-with-unique-name
123128

124-
# Build the documentation
125-
- name: Build the documentation
126-
run: make -C doc clean all
127-
128129
# Upload coverage to Codecov
129130
- name: Upload coverage to Codecov
130131
uses: codecov/[email protected]
131132
with:
132133
file: ./coverage.xml # optional
133134
env_vars: OS,PYTHON
134135
fail_ci_if_error: false
135-
136-
- name: Checkout the gh-pages branch
137-
uses: actions/checkout@28c7f3d2b5162b5ddd3dfd9a45aa55eaf396478b
138-
with:
139-
ref: gh-pages
140-
# Checkout to this folder instead of the current one
141-
path: deploy
142-
# Download the entire history
143-
fetch-depth: 0
144-
if: (github.event_name == 'release' || github.event_name == 'push') && (matrix.os == 'ubuntu-latest') && (matrix.python-version == '3.9')
145-
146-
- name: Push the built HTML to gh-pages
147-
run: |
148-
# Detect if this is a release or from the master branch
149-
if [[ "${GITHUB_EVENT_NAME}" == "release" ]]; then
150-
# Get the tag name without the "refs/tags/" part
151-
version="${GITHUB_REF#refs/*/}"
152-
else
153-
version=dev
154-
fi
155-
echo "Deploying version: $version"
156-
# Make the new commit message. Needs to happen before cd into deploy
157-
# to get the right commit hash.
158-
message="Deploy $version from $(git rev-parse --short HEAD)"
159-
cd deploy
160-
# Need to have this file so that Github doesn't try to run Jekyll
161-
touch .nojekyll
162-
# Delete all the files and replace with our new set
163-
echo -e "\nRemoving old files from previous builds of ${version}:"
164-
rm -rvf ${version}
165-
echo -e "\nCopying HTML files to ${version}:"
166-
cp -Rvf ../doc/_build/html/ ${version}/
167-
# If this is a new release, update the link from /latest to it
168-
if [[ "${version}" != "dev" ]]; then
169-
echo -e "\nSetup link from ${version} to 'latest'."
170-
rm -f latest
171-
ln -sf ${version} latest
172-
fi
173-
# Stage the commit
174-
git add -A .
175-
echo -e "\nChanges to be applied:"
176-
git status
177-
# Configure git to be the GitHub Actions account
178-
git config user.email "github-actions[bot]@users.noreply.github.com"
179-
git config user.name "github-actions[bot]"
180-
# If this is a dev build and the last commit was from a dev build
181-
# (detect if "dev" was in the previous commit message), reuse the
182-
# same commit
183-
if [[ "${version}" == "dev" && `git log -1 --format='%s'` == *"dev"* ]]; then
184-
echo -e "\nAmending last commit:"
185-
git commit --amend --reset-author -m "$message"
186-
else
187-
echo -e "\nMaking a new commit:"
188-
git commit -m "$message"
189-
fi
190-
# Make the push quiet just in case there is anything that could leak
191-
# sensitive information.
192-
echo -e "\nPushing changes to gh-pages."
193-
git push -fq origin gh-pages 2>&1 >/dev/null
194-
echo -e "\nFinished uploading generated files."
195-
if: (github.event_name == 'release' || github.event_name == 'push') && (matrix.os == 'ubuntu-latest') && (matrix.python-version == '3.9')

‎MAINTENANCE.md

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ build and test the project on Linux, macOS and Windows.
6565
They rely on the `environment.yml` file to install required dependencies using
6666
conda and the `Makefile` to run the tests and checks.
6767

68-
There are 8 configuration files located in `.github/workflows`:
68+
There are 9 configuration files located in `.github/workflows`:
6969

7070
1. `style_checks.yaml` (Code lint and style checks)
7171

@@ -76,45 +76,51 @@ There are 8 configuration files located in `.github/workflows`:
7676

7777
This is run on every commit to the *master* and Pull Request branches.
7878
It is also scheduled to run daily on the *master* branch.
79-
In draft Pull Requests, only one job (Ubuntu + Python latest)
79+
In draft Pull Requests, only one job (Linux + Python latest)
8080
is triggered to save on Continuous Integration resources.
8181

82+
3. `ci_docs.yml` (Build documentation on Linux/macOS/Windows)
83+
84+
This is run on every commit to the *master* and Pull Request branches.
85+
In draft Pull Requests, only the job on Linux is triggered to save on
86+
Continuous Integration resources.
87+
8288
On the *master* branch, the workflow also handles the documentation
8389
deployment:
8490

8591
* Updating the development documentation by pushing the built HTML pages
8692
from the *master* branch onto the `dev` folder of the *gh-pages* branch.
8793
* Updating the `latest` documentation link to the new release.
8894

89-
3. `ci_tests_dev.yaml` (GMT Dev Tests on Linux/macOS/Windows).
95+
4. `ci_tests_dev.yaml` (GMT Dev Tests on Linux/macOS/Windows).
9096

9197
This is triggered when a PR is marked as "ready for review", or using the
9298
slash command `/test-gmt-dev`. It is also scheduled to run daily on the
9399
*master* branch.
94100

95-
4. `cache_data.yaml` (Caches GMT remote data files needed for GitHub Actions CI)
101+
5. `cache_data.yaml` (Caches GMT remote data files needed for GitHub Actions CI)
96102

97103
This is scheduled to run every Sunday at 12:00 (UTC).
98104
If new remote files are needed urgently, maintainers can manually uncomment
99105
the 'pull_request:' line in that `cache_data.yaml` file to refresh the cache.
100106

101-
5. `publish-to-pypi.yml` (Publish wheels to PyPI and TestPyPI)
107+
6. `publish-to-pypi.yml` (Publish wheels to PyPI and TestPyPI)
102108

103109
This workflow is run to publish wheels to PyPI and TestPyPI (for testing only).
104110
Archives will be pushed to TestPyPI on every commit to the *master* branch
105111
and tagged releases, and to PyPI for tagged releases only.
106112

107-
6. `release-drafter.yml` (Drafts the next release notes)
113+
7. `release-drafter.yml` (Drafts the next release notes)
108114

109115
This workflow is run to update the next releases notes as pull requests are
110116
merged into master.
111117

112-
7. `check-links.yml` (Check links in the repository and website)
118+
8. `check-links.yml` (Check links in the repository and website)
113119

114120
This workflow is run weekly to check all external links in plaintext and
115121
HTML files. It will create an issue if broken links are found.
116122

117-
8. `format-command.yml` (Format the codes using slash command)
123+
9. `format-command.yml` (Format the codes using slash command)
118124

119125
This workflow is triggered in a PR if the slash command `/format` is used.
120126

0 commit comments

Comments
 (0)