Create Github Action workflow for reporting DVC image diffs (#1104)

To make reviewing new baseline test images (*.png) easier.
the dvc-diff workflow checks what images have been added or
modified in a Pull Request. The changes are published in a
table and as a series of images by a bot-generated GitHub
comment.

* Refactor test_legend_entries to use mpl_image_compare
* Let actions/checkout fetch all history so that dvc diff works
* Use peter-evans/create-or-update-comment to publish image diff report
* Add bullet points with names for each of the images that have changed
* Collapsible image diff section and use correct git SHA in the report
* List dvc-diff.yml under MAINTENANCE.md
* Use awk 'NR>=7' instead of tail and add some whitespace indentation

Co-authored-by: Dongdong Tian <[email protected]>

Author: weiji14

.github/workflows/dvc-diff.yml

@@ -0,0 +1,95 @@
+# Checks for image diffs in a Pull Request and adds a GitHub comment showing the diff
+name: DVC image diff
+
+on:
+  pull_request:
+    paths:
+      - 'pygmt/tests/baseline/*.png.dvc'
+
+jobs:
+  dvc-diff:
+    name: DVC image diff
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout
+      uses: actions/[email protected]
+      with:
+        # fetch all history so that dvc diff works
+        fetch-depth: 0
+
+    - name: Setup data version control (DVC)
+      uses: iterative/[email protected]
+
+    - name: Setup continuous machine learning (CML)
+      uses: iterative/[email protected]
+
+    # Produce the markdown diff report, which should look like:
+    # ## Summary of changed images
+    #
+    # This is an auto-generated report of images that have changed on the DVC remote
+    #
+    # | Status   | Path                                |
+    # |----------|-------------------------------------|
+    # | added    | pygmt/tests/baseline/test_image.png |
+    - name: Put list of images that were added or changed into report
+      run: |
+        echo -e "## Summary of changed images\n" > report.md
+        echo -e "This is an auto-generated report of images that have changed on the DVC remote\n" >> report.md
+        dvc diff --show-md master HEAD >> report.md
+        cat report.md
+
+    - name: Pull image data from cloud storage
+      run: dvc pull --remote upstream
+
+    - name: Put image diff(s) into report
+      env:
+        repo_token: ${{ secrets.GITHUB_TOKEN }}
+      id: image-diff
+      run: |
+        # Get just the filename of the changed image from the report
+        awk 'NR>=7 {print $4}' report.md > diff_files.txt
+
+        # Append each image to the markdown report
+        echo -e "## Image diff(s)\n" >> report.md
+        echo -e "<details>\n" >> report.md
+
+        while IFS= read -r line; do
+          echo -e "- $line \n" >> report.md
+          cml-publish --title $line --md "$line" >> report.md
+        done < diff_files.txt
+
+        echo -e "</details>\n" >> report.md
+
+        # Mention git commit SHA in the report
+        echo -e "Report last updated at commit ${{ github.event.pull_request.head.sha }}" >> report.md
+
+        # Format report to escape newlines before publishing as GitHub comment
+        report=$(cat report.md)
+        report="${report//'%'/'%25'}"
+        report="${report//$'\n'/'%0A'}"
+        report="${report//$'\r'/'%0D'}"
+        echo ::set-output name=report::$report
+
+    - name: Find comment with image diff report
+      uses: peter-evans/[email protected]
+      id: fc
+      with:
+        issue-number: ${{ github.event.pull_request.number }}
+        comment-author: 'github-actions[bot]'
+        body-includes: 'This is an auto-generated report of images that have changed on the DVC remote'
+
+    - name: Create comment with image diff report
+      if: steps.fc.outputs.comment-id == ''
+      uses: peter-evans/[email protected]
+      with:
+        issue-number: ${{ github.event.pull_request.number }}
+        body: ${{ steps.image-diff.outputs.report }}
+
+    - name: Update comment with new image diff report
+      if: steps.fc.outputs.comment-id != ''
+      uses: peter-evans/[email protected]
+      with:
+        comment-id: ${{ steps.fc.outputs.comment-id }}
+        body: ${{ steps.image-diff.outputs.report }}
+        edit-mode: replace

MAINTENANCE.md

@@ -127,6 +127,12 @@ There are 9 configuration files located in `.github/workflows`:
 
    This workflow is triggered in a PR if the slash command `/format` is used.
 
+10. `dvc-diff.yml` (Report changes to test images on dvc remote)
+
+    This workflow is triggered in a PR when any *.png.dvc files have been added,
+    modified, or deleted. A GitHub comment will be published that contains a summary
+    table of the images that have changed along with a visual report.
+
 ## Continuous Documentation
 
 We use the [Vercel for GitHub](https://github.com/apps/vercel) App to preview changes

pygmt/tests/baseline/test_legend_entries.png

@@ -0,0 +1,4 @@
+outs:
+- md5: 4d1ed417fd2625d0b2fa5c17dafd32fd
+  size: 73728
+  path: test_legend_entries.png

pygmt/tests/baseline/test_legend_entries.png.dvc

@@ -5,7 +5,6 @@
 from pygmt import Figure
 from pygmt.exceptions import GMTInvalidInput
 from pygmt.helpers import GMTTempFile
-from pygmt.helpers.testing import check_figures_equal
 
 
 @pytest.mark.mpl_image_compare
@@ -44,42 +43,25 @@ def test_legend_default_position():
     return fig
 
 
-@check_figures_equal()
+@pytest.mark.mpl_image_compare
 def test_legend_entries():
     """
     Test different marker types/shapes.
     """
-    fig_ref, fig_test = Figure(), Figure()
-
-    # Use single-character arguments for the reference image
-    fig_ref = Figure()
-    fig_ref.basemap(J="x1i", R="0/7/3/7", B="")
-    fig_ref.plot(
-        data="@Table_5_11.txt",
-        S="c0.15i",
-        G="lightgreen",
-        W="faint",
-        l="Apples",
-    )
-    fig_ref.plot(data="@Table_5_11.txt", W="1.5p,gray", l='"My lines"')
-    fig_ref.plot(data="@Table_5_11.txt", S="t0.15i", G="orange", l="Oranges")
-    fig_ref.legend(D="JTR+jTR")
-
-    fig_test.basemap(projection="x1i", region=[0, 7, 3, 7], frame=True)
-    fig_test.plot(
+    fig = Figure()
+    fig.basemap(projection="x1i", region=[0, 7, 3, 7], frame=True)
+    fig.plot(
         data="@Table_5_11.txt",
         style="c0.15i",
         color="lightgreen",
         pen="faint",
         label="Apples",
     )
-    fig_test.plot(data="@Table_5_11.txt", pen="1.5p,gray", label='"My lines"')
-    fig_test.plot(
-        data="@Table_5_11.txt", style="t0.15i", color="orange", label="Oranges"
-    )
-    fig_test.legend(position="JTR+jTR")
+    fig.plot(data="@Table_5_11.txt", pen="1.5p,gray", label='"My lines"')
+    fig.plot(data="@Table_5_11.txt", style="t0.15i", color="orange", label="Oranges")
+    fig.legend(position="JTR+jTR")
 
-    return fig_ref, fig_test
+    return fig
 
 
 @pytest.mark.mpl_image_compare