Add a GitHub action to automatically update projects

[ezio-melotti]

This is a meta issue about a newly added GitHub action that automatically updates projects:

• Add a workflow to add issues/PRs to projects. cpython#94447
  • backport to 3.11: [3.11] Add a workflow to add issues/PRs to projects. (GH-94447) cpython#94477
  • backport to 3.10: [3.10] Add a workflow to add issues/PRs to projects. (GH-94447) cpython#94475
• Remove pull_request trigger from the project-updater GHA. cpython#94483
• Add more jobs for other labels/projects
  • Automatically update more GitHub projects. cpython#94921
  • Add sprint issues to the Sprint 2022 project. cpython#97788
• Backport to 3.11/3.10

---

python/cpython#94447 added a first version of the workflow that updates the Release and Deferred blockers project, and:

• I tested that the issue is added to the project when:
  • the release-blocker label is applied
  • the deferred-blocker label is applied
• removing the label doesn't remove the issue from the project
• there doesn't seem to be a way to set the type automatically from the action
  • it can be set manually either in the issue or in the project
• it takes about 10-15s before the issue (dis)appears in the project
• a manual refresh on the issue is needed before the project (dis)appears in the sidebar

After attempting backports in python/cpython#94477 and python/cpython#94475, the check failed because it couldn't access the secret. After some investigation it turns out that PRs created from forks can't access secrets. In addition, this action would add a check for each job in the PR (currently only one, but I'm planning to add more for the other labels/projects):

Click to see screenshot of the job in the PR checks list

Therefore, I submitted python/cpython#94483 to remove the pull_request trigger and only run the action for issues, solving the issue with the secret and the extra checks in PRs.

[ezio-melotti]

• See also Projects: Auto-add item to project automation  github/roadmap#576

[auscompgeek]

pull_request trigger

Would using the pull_request_target trigger work instead?

[ezio-melotti]

Even if it does, I'm not sure we want to add PRs to the projects anymore.

[ezio-melotti]

Projects now support a workflow to automatically add issues/PRs to projects, even though it seems to have some limitations (e.g. can't set custom fields based on a label, and also limitations based on the account type).
For now I enabled it as a test to the Release and deferred blockers project -- if it works well we could enable it on more projects and get rid of the workflow added in python/cpython#94447.

See:

• https://github.blog/changelog/2023-03-09-github-issues-projects-march-9th-update/#robot-automatically-add-and-archive-project-items
• https://docs.github.com/en/issues/planning-and-tracking-with-projects/automating-your-project/adding-items-automatically

[CAM-Gerlach]

Also, Triagers can now add issues/PRs to projects directly due to the changes in #460 , as far as I'm aware, so anyone who can add a label can also add directly to the project.

[StanFromIreland]

It would be handy for the action to remove them too, e.g. at the moment the blockers project is littered with issues that should no longer be there.

Proposed (draft) patch I made

Subject: [PATCH] Workflow
---
Index: .github/workflows/project-updater.yml
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/.github/workflows/project-updater.yml b/.github/workflows/project-updater.yml
--- a/.github/workflows/project-updater.yml	(revision 17718b0503e5d1c987253641893cab98e01f4535)
+++ b/.github/workflows/project-updater.yml	(date 1745611241929)
@@ -5,6 +5,7 @@
     types:
       - opened
       - labeled
+       - unlabeled
 
 permissions:
   contents: read
@@ -29,3 +30,35 @@
           project-url: https://github.com/orgs/python/projects/${{ matrix.project }}
           github-token: ${{ secrets.ADD_TO_PROJECT_PAT }}
           labeled: ${{ matrix.label }}
+
+  remove-from-project:
+    name: Remove issues from projects
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    if: ${{ github.event.action == 'unlabeled' }}
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          # if one of these labels is removed from an issue, it will be removed
+          # from its (past) project
+          - { project: 2, label: "release-blocker,deferred-blocker" }
+
+    steps:
+      - id: check-label
+        run: |
+          removed_label="${{ github.event.label.name }}"
+
+          IFS=',' read -ra labels <<< "${{ matrix.label }}"
+          if [[ " ${labels[@]} " =~ " ${removed_label} " ]]; then
+            echo "removed_relevant=true" >> "$GITHUB_OUTPUT"
+          else
+            echo "removed_relevant=false" >> "$GITHUB_OUTPUT"
+          fi
+
+      - if: steps.check-label.outputs.removed_relevant == 'true'
+        uses: joshmgross/remove-from-project-classic@main
+        with:
+          project-number: https://github.com/orgs/python/projects/${{ matrix.project }}
+          issue-number: ${{ github.event.number }}
+          token: ${{ secrets.PROJECT_TOKEN }}

Should I submit it as a pr?

(Note: remove-from-projects should be fine, it is written by a gh employee.)

[ezio-melotti]

Last time I checked, projects supported a few more features that allowed automatic addition (and possibly removal too) directly from the project settings. If these features are now advanced enough to support our use cases we might even be able to get rid of the workflow.

Would you be interested in looking into this?

[StanFromIreland]

I presume you mean using-the-built-in-automations ? These appear to be mini workflows that you configure in some way. They support adding items and archiving. They are fine for our use cases.

Unfortunately I cannot help further with this as I don't have permissions to use the button and I cannot open a pr :-(

[ezio-melotti]

I now updated the sqlite3 project with the Auto-add to project project workflow and verified that an issue was automatically added to the project when I added the topic-sqlite3 label to it:

As mentioned above, we could go through the existing projects and enable their Auto-add to project workflows (assuming it is a suitable replacement to the current project-updater workflow) and then remove the project-updater workflow (and the github-token that it uses).

Unfortunately there is no project workflow to automatically remove the issue when the label is removed. The project should be listed under the labels on the issue page, so it should be relatively easy for triagers to remove it from there too. Unless this is a common use-case, I'd say it's not worth adding a new workflow, especially if we remove the project-updater.

[StanFromIreland]

As mentioned above, we could go through the existing projects and enable their Auto-add to project workflows (assuming it is a suitable replacement to the current project-updater workflow) and then remove the project-updater workflow (and the github-token that it uses).

It seems to work fine, I am +1 on enabling it for all projects and removing the workflow (I can help with that one :-).

Unfortunately there is no project workflow to automatically remove the issue when the label is removed.

I see, we can come back to this in the future if gh adds this functionality.