feat: add test coverage reporting with GitHub Pages badge #62
Conversation
Configure pytest-cov with: - Branch coverage enabled for stackone_ai source - Exclusions for __init__.py and py.typed files - Standard exclusion patterns (TYPE_CHECKING, NotImplementedError, etc.) - JSON output to coverage/coverage.json for CI badge generation - HTML output to coverage/html for detailed reports
Add `just coverage` command that runs pytest with: - Terminal coverage report for quick feedback - JSON report for CI badge generation - HTML report for detailed analysis
Add .coverage and coverage/ to .gitignore to prevent committing locally generated coverage reports.
Add coverage job that runs on main branch pushes: - Run tests with pytest-cov to generate coverage reports - Generate coverage badge using coverage-badges-cli - Upload coverage artifacts to GitHub Pages Add deploy-coverage job to publish reports to GitHub Pages, making coverage badge and HTML report publicly accessible at https://stackonehq.github.io/stackone-ai-python/ Matches the coverage setup in stackone-ai-node repository.
Display test coverage percentage badge that links to the detailed HTML coverage report on GitHub Pages.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
1 issue found across 5 files
Prompt for AI agents (all 1 issue)
Check if these issues are valid — if so, understand the root cause of each and fix them.
<file name=".github/workflows/ci.yaml">
<violation number="1" location=".github/workflows/ci.yaml:15">
P2: Permissions `pages: write` and `id-token: write` are set at workflow level, granting elevated permissions to all jobs including `typos` and `ci` which don't need them. Consider moving these permissions to job-level for only the `coverage` and `deploy-coverage` jobs to follow the principle of least privilege.</violation>
</file>
Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR
|
|
||
| permissions: | ||
| contents: read | ||
| pages: write |
There was a problem hiding this comment.
P2: Permissions pages: write and id-token: write are set at workflow level, granting elevated permissions to all jobs including typos and ci which don't need them. Consider moving these permissions to job-level for only the coverage and deploy-coverage jobs to follow the principle of least privilege.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/ci.yaml, line 15:
<comment>Permissions `pages: write` and `id-token: write` are set at workflow level, granting elevated permissions to all jobs including `typos` and `ci` which don't need them. Consider moving these permissions to job-level for only the `coverage` and `deploy-coverage` jobs to follow the principle of least privilege.</comment>
<file context>
@@ -12,6 +12,8 @@ concurrency:
permissions:
contents: read
+ pages: write
+ id-token: write
</file context>
There was a problem hiding this comment.
Pull request overview
This PR adds comprehensive test coverage reporting infrastructure to the Python SDK, mirroring the Node.js SDK's setup. It configures pytest-cov for branch coverage tracking, adds a local coverage command, implements CI workflow jobs to generate and deploy coverage reports to GitHub Pages, and displays a coverage badge in the README.
Key Changes:
- Configures pytest-cov with branch coverage and appropriate exclusions in pyproject.toml
- Adds GitHub Actions jobs to generate coverage reports and deploy to GitHub Pages on main branch pushes
- Adds coverage badge to README linking to hosted HTML reports
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| pyproject.toml | Configures pytest-cov with branch coverage, source directory, omit patterns, and output paths for JSON and HTML reports |
| justfile | Adds coverage command to run tests with coverage reporting in terminal, JSON, and HTML formats |
| README.md | Adds coverage badge linking to GitHub Pages-hosted coverage reports |
| .gitignore | Excludes coverage artifacts (.coverage file and coverage/ directory) from version control |
| .github/workflows/ci.yaml | Adds coverage and deploy-coverage jobs that run on main branch to generate coverage reports and deploy to GitHub Pages |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| uses: ./.github/actions/setup-nix | ||
|
|
||
| - name: Install dependencies | ||
| run: nix develop --command just install --all-extras |
There was a problem hiding this comment.
The coverage job installs dependencies with --all-extras, which includes optional extras like 'examples' that may not be necessary for coverage calculation. This increases build time and could affect coverage measurements if optional dependencies have side effects. Consider using the same dependency installation approach as the ci job's matrix strategy to ensure consistency.
| run: nix develop --command just install --all-extras | |
| run: nix develop --command just install |
| branch = true | ||
| omit = [ | ||
| "stackone_ai/__init__.py", | ||
| "**/py.typed", |
There was a problem hiding this comment.
The omit pattern "**/py.typed" will match any py.typed file in any directory at any level, which could be overly broad if there are multiple packages or nested structures. Since stackone_ai/py.typed is already covered by the directory-specific omit pattern "stackone_ai/init.py", consider using a more specific pattern like "stackone_ai/py.typed" or removing this pattern if it's redundant.
| "**/py.typed", | |
| "stackone_ai/py.typed", |
| permissions: | ||
| contents: read | ||
| pages: write | ||
| id-token: write |
There was a problem hiding this comment.
The GitHub Pages permissions are added at the workflow level, which means they apply to all jobs including typos and ci jobs that don't need them. This violates the principle of least privilege. Consider moving these permissions to only the jobs that need them (coverage and deploy-coverage) using job-level permissions.
| deploy-coverage: | ||
| needs: coverage | ||
| runs-on: ubuntu-latest | ||
| if: github.ref == 'refs/heads/main' | ||
| environment: | ||
| name: github-pages | ||
| url: ${{ steps.deployment.outputs.page_url }} | ||
| steps: | ||
| - name: Deploy to GitHub Pages | ||
| id: deployment | ||
| uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4.0.5 |
There was a problem hiding this comment.
The deploy-coverage job could have concurrency issues. If multiple pushes to main happen in quick succession, the workflow cancellation strategy (cancel-in-progress: true) combined with the job dependency could result in incomplete deployments or race conditions. Consider adding a concurrency group specifically for the deployment jobs to ensure only one deployment runs at a time.
Summary
Add test coverage calculation and badge generation to match the setup in stackone-ai-node repository.
What Changed
pyproject.tomljust coveragecommand for local coverage testingWhy
Provides visibility into test coverage percentage directly in the README, matching the Node.js SDK's approach. The badge links to detailed HTML coverage reports hosted on GitHub Pages.
Setup Required
After merging, GitHub Pages needs to be enabled for this repository:
The coverage will then be available at https://stackonehq.github.io/stackone-ai-python/
Summary by cubic
Add test coverage reporting with pytest-cov and publish a badge and HTML report to GitHub Pages. Coverage shows in the README and updates on main.
New Features
Migration
Written for commit c71a478. Summary will update automatically on new commits.