CI: Add script and automated workflow to clean up stale PR preview documentation folders

[Copilot]

• Analyze the issue: Found 44 PR preview folders in gh-pages vs 17 open PRs
• Understand the current doc_preview action logic
• Create bash script to check PR status and clean up closed/merged preview folders
• Test script manually to verify it works correctly
• Place script in ci/ folder as requested
• Ensure script is executable and has proper error handling
• Add documentation/comments explaining script functionality
• Apply shellcheck improvements for better code quality
• Run security validation with CodeQL (0 issues found)
• Remove test file and separate documentation as requested
• Refactor script based on detailed code review feedback:
  • Made repository non-configurable (hardcoded to NVIDIA/cuda-python)
  • Added support for both -n and --dry-run flags like git clean
  • Removed redundant checks for jq and git tools
  • Removed hardcoded git config (user responsible for their own identity)
  • Made commit/push optional with --push flag (default: false, requires manual push)
  • Updated help message and examples to reflect new interface
• Address additional feedback:
  • Re-added jq dependency check (was removed by mistake)
  • Replaced git clone with git worktree for more efficient operation
  • Enhanced cleanup to properly remove worktree
• Fix GitHub API call:
  • Improved error handling for API failures
  • Better parsing of API response to handle error cases
  • Added descriptive error messages for common failure scenarios
  • Fixed issue where 404 errors were not properly handled
• Fix worktree cleanup logic:
  • Made cleanup conditional based on whether changes were pushed
  • Preserve worktree when changes haven't been pushed to prevent data loss
  • Added informative messages about manual cleanup when needed
  • Enhanced safety by tracking push status
• Implement safer worktree initialization:
  • Properly handle existing worktrees before creating new ones
  • Use git worktree remove before rm -rf to clean up git registrations
  • Added retry logic with cleanup for failed worktree creation
  • Enhanced error detection and recovery for worktree conflicts
• Add automated CI workflow:
  • Created .github/workflows/cleanup-pr-previews.yml
  • Scheduled to run nightly at 11pm EST (4am UTC)
  • Supports manual dispatch with optional dry-run mode
  • Automatically configures git identity and installs required tools
  • Pushes changes to gh-pages branch automatically

Script Interface

The cleanup script can be run manually with the following options:

./ci/cleanup-pr-previews -n          # Preview cleanup (recommended first)
./ci/cleanup-pr-previews             # Cleanup locally (no push, worktree preserved)  
./ci/cleanup-pr-previews --push      # Cleanup and push to gh-pages (worktree cleaned)

Automated Workflow

The workflow runs automatically every night at 11pm EST and can also be manually triggered from the GitHub Actions tab:

• Scheduled: Runs nightly and automatically pushes cleanup changes to gh-pages
• Manual Dispatch: Can be triggered on-demand with an optional dry-run mode for testing
• Requirements: Automatically installs jq, uses GitHub CLI, and configures git identity

The script safely handles existing worktrees and git registrations, preventing issues when re-running with the same temp directory name.

Fixes #628

Original prompt

---

This section details on the original issue you should resolve

<issue_title>CI: Per-PR doc previews are not always cleaned up</issue_title>
<issue_description>Compare the number of folders in the gh-pages branch
https://github.com/NVIDIA/cuda-python/tree/gh-pages/docs/pr-preview
with the number of open PRs that we have:
https://github.com/NVIDIA/cuda-python/pulls
I think the numbers mismatch.

Some logic was oversimplified in the doc_preview action, for example:

cuda-python/.github/actions/doc_preview/action.yml

Lines 47 to 69 in 99ffe14

	# The steps below are executed only when building on main.
	- name: Remove doc preview
	if: ${{ github.ref_name == 'main' }}
	uses: JamesIves/github-pages-deploy-action@v4
	with:
	git-config-name: cuda-python-bot
	git-config-email: [email protected]
	folder: ${{ inputs.source-folder }}
	target-folder: docs/pr-preview/pr-${{ inputs.pr-number }}/
	commit-message: "Clean up doc preview for PR ${{ inputs.pr-number }} (${{ github.sha }})"
	- name: Leave a comment after removal
	if: ${{ github.ref_name == 'main' }}
	uses: marocchino/sticky-pull-request-comment@v2
	with:
	header: pr-preview
	number: ${{ inputs.pr-number }}
	hide_and_recreate: true
	hide_classify: "OUTDATED"
	message: |
	Doc Preview CI
	:---:
	Preview removed because the pull request was closed or merged.

These steps are only run when the target branch is main. So one class of cleanup failures came from the previous path finder work, where PRs targeted a feature branch instead of the main branch. However, this bug alone does not seem to account for the discrepancy fully; there seem to exist other issues that I could not tell immediately. </issue_description>

<agent_instructions>Please read through all comments in the issue. The problem context and instructions are all there.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@leofang Another class of errors: The doc job is canceled (due to multiple merges), e.g. https://github.com/NVIDIA/cuda-python/commit/b9427f180505862c0802dcf3ec5dfe7c3288e680 @leofang In the `gh-pages` branch, under the folder `docs/pr-preview`, there are subfolders `pr-XXXXX` created to host doc preview for the corresponding PR. We need a bash script placed in the `main` branch, under the `ci` folder, that runs through all preview `pr-XXXXX` folders, check if the corresponding PR `XXXXX` is already merged or closed, and if so remove the `pr-XXXXX` folder. Keep the bash script runnable manually for now. We can consider adding a CI workflow to run it periodically later.

Fixes #628

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[leofang]

@copilot help me reduce my review burden, see my comments below.

[leofang]

@copilot

[leofang]

@copilot

[leofang]

pre-commit.ci autofix

[leofang]

@copilot

[cpcloud]

What's the status of this PR? It seems like it's been languishing for a while. Let's make a decision and either merge it or close it out.

[leofang]

@copilot Please add a CI workflow that runs this script every night at 11pm EST. Also make it manually dispatchable.

[Copilot]

@copilot Please add a CI workflow that runs this script every night at 11pm EST. Also make it manually dispatchable.

Added in commit f4d0dfa. Created .github/workflows/cleanup-pr-previews.yml that:

• Runs automatically every night at 11pm EST (4am UTC)
• Can be manually dispatched from the Actions tab
• Includes an optional dry-run mode for manual runs
• Automatically configures git identity and installs required tools (jq, gh)
• Runs the cleanup script with --push flag to automatically commit and push changes to gh-pages

[leofang]

pre-commit.ci autofix

[leofang]

Let's merge and I'll manually trigger the workflow to validate if it's functional.

[leofang]

dry-run works https://github.com/NVIDIA/cuda-python/actions/runs/19238249597/job/54993913798

[github-actions]

Doc Preview CI
Preview removed because the pull request was closed or merged.

[leofang]

actual run works:

• CI run: https://github.com/NVIDIA/cuda-python/actions/runs/19238402499/job/54994456114
• commit checked in: 6fc9084
• the preview folder is a lot cleaner: https://github.com/NVIDIA/cuda-python/tree/6fc90842669afea5af188941915ca3f325fa2d79/docs/pr-preview