Run codgen build step

[strawgate]

We should run make codegen lint as part of the build and if anything changes fail the build

[claude]

Claude finished @strawgate's task —— View job

---

Investigation Complete ✓

tl;dr

Add a CI check that runs make codegen lint and fails if any files are modified. This ensures the sync library stays in sync with async code changes and prevents merge conflicts like those seen in several recent PRs.

---

Recommendation

Implement a new GitHub Actions workflow step in .github/workflows/test.yml that:

1. Runs make codegen lint on Python 3.10 (required for consistent codegen output)
2. Checks if any files were modified using git diff --exit-code
3. Fails the build if changes are detected, prompting developers to run make codegen locally

This check should run early in the CI pipeline (after dependency installation, before tests) to provide fast feedback. It will catch forgotten codegen runs before code review, saving time and preventing sync library drift.

---

Detailed Action Plan

Implementation Steps

1. Add new job to .github/workflows/test.yml:

   codegen_check:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout repository
         uses: actions/checkout@v4
       
       - name: Set up Python 3.10
         uses: actions/setup-python@v5
         with:
           python-version: '3.10'
       
       - name: Install uv
         uses: astral-sh/setup-uv@v6
       
       - name: Install dependencies
         run: make sync
       
       - name: Run codegen and lint
         run: make codegen lint
       
       - name: Check for uncommitted changes
         run: |
           if ! git diff --exit-code; then
             echo "❌ Error: Codegen produced changes. Please run 'make codegen' locally and commit the results."
             echo ""
             echo "Files changed:"
             git diff --name-only
             exit 1
           fi
           echo "✅ Codegen check passed - no changes detected"

2. Add job dependency: Make other jobs depend on codegen_check to prevent wasted CI time on invalid PRs:

   static_analysis:
     needs: codegen_check
     # ... existing config

3. Update documentation:

   • Add codegen check to DEVELOPING.md pre-commit section
   • Mention in AGENTS.md that sync library must always be regenerated after async changes
   • Update Makefile help text to emphasize codegen importance

Alternative Approach (If preferred)

Add the check to the existing static_analysis job instead of creating a new job:

static_analysis:
  # ... existing setup steps
  
  - name: "Check codegen is up to date"
    if: matrix.project == 'key-value/key-value-aio'
    run: |
      make sync  # Install all deps for codegen
      make codegen lint
      git diff --exit-code || (echo "❌ Run 'make codegen' locally" && exit 1)
  
  - name: "Lint"
    run: # ... existing lint step

This is more efficient (reuses existing job) but provides less clear separation of concerns.

Findings

Current State

1. Codegen Process:

   • make codegen runs scripts/build_sync_library.py
   • Converts async code (key-value-aio) to sync code (key-value-sync) using AST transformation
   • Requires Python 3.10 for consistent output (line 36 in build_sync_library.py)
   • Automatically lints generated code

2. Existing CI Checks:

   • .github/workflows/test.yml runs linting, type checking, and tests
   • Does NOT verify codegen is up to date
   • Tests run on both key-value-aio and key-value-sync packages

3. Current Workflow:

   • make precommit (line 122 in Makefile) runs: lint codegen lint typecheck
   • This runs codegen but doesn't check if it produces changes
   • Developers can forget to commit generated files

Evidence from Repository History

Multiple PRs mention codegen issues:

• PR Add Python 3.10 to Claude workflows #152: "Claude workflows were running with Ubuntu's default Python 3.12, causing codegen script failures since it requires Python 3.10"

  • Shows codegen version sensitivity is a real problem

• PR feat: add native storage mode for Elasticsearch #149: Note says "Sync library needs to be regenerated with make codegen using Python 3.10"

  • Developer had to manually note this in PR description

• PR feat: require single TTL for all items in put_many #121: "Post-Merge Action Required: After merging, run make codegen"

  • Again, manual reminder needed

• Issue Review Previous Pull Request reviews #142: "Review Previous Pull Request reviews" found common pattern: "The most common issue by far was forgetting to run make codegen after modifying async code"

  • This is a systemic problem that needs automated enforcement

• Issue Change Claude Workflow to use Python 3.10 #151: "Change Claude Workflow to use Python 3.10" - specifically about codegen failures

Why Current make precommit Isn't Sufficient

The Makefile line 122 shows:

precommit: lint codegen lint typecheck

This runs codegen but doesn't verify it doesn't change anything. Developers can:

1. Run make precommit
2. Codegen updates sync files
3. Forget to git add the changes
4. Commit only their async changes
5. CI passes (doesn't check for codegen drift)

Testing Performed

Ran make codegen lint locally on the main branch:

• ✅ Codegen completed successfully
• ✅ Linting passed
• ✅ git status showed "working tree clean" (no changes)
• This confirms codegen is currently in sync, but there's no CI check preventing future drift

Related Items

Related Issues

Repository	Issue	Relevance
strawgate/py-key-value	#151 - Change Claude Workflow to use Python 3.10	Claude workflows needed Python 3.10 specifically for codegen, showing version sensitivity
strawgate/py-key-value	#142 - Review Previous Pull Request reviews	Found forgetting make codegen was the most common issue across PRs
strawgate/py-key-value	#112 - Update Claude prompt for using make	Related to enabling make commands in CI workflows
strawgate/py-key-value	#108 - Claude can not run make commands	Background on enabling make in CI

Related Pull Requests

Repository	PR	Relevance
strawgate/py-key-value	#152 - Add Python 3.10 to Claude workflows	Fixed codegen failures by ensuring Python 3.10 availability
strawgate/py-key-value	#149 - feat: add native storage mode for Elasticsearch	PR note: "Sync library needs to be regenerated with make codegen"
strawgate/py-key-value	#121 - feat: require single TTL for all items in put_many	PR note: "After merging, run make codegen" - manual reminder needed
strawgate/py-key-value	#109 - Enable make commands for Claude Code workflows	Added Bash(make:*) to allowed tools in CI

Related Files

Repository	File	Relevance	Key Lines
strawgate/py-key-value	Makefile	Defines codegen and lint targets	44-46 (codegen), 122 (precommit)
strawgate/py-key-value	scripts/build_sync_library.py	The codegen script that transforms async to sync	36 (Python 3.10 requirement), 68-101 (main function)
strawgate/py-key-value	.github/workflows/test.yml	Current CI test workflow	33-65 (static_analysis job where check should go)
strawgate/py-key-value	DEVELOPING.md	Developer guide mentioning codegen	78-91 (codegen section)
strawgate/py-key-value	AGENTS.md	AI agent guidelines (if exists)	Mentions async-first development and codegen requirements

---

Problems Encountered During Investigation

None - all investigation tools and resources were accessible.

---

[strawgate]

@claude please make a PR

[claude]

Claude finished @strawgate's task —— View job

---

Codegen CI Check - Permission Limitation

I've implemented the codegen check for your CI workflow, but encountered a GitHub App permission issue that prevents me from pushing workflow file changes. The implementation is complete and ready for you to apply manually.

---

What Was Implemented

A new codegen_check job in .github/workflows/test.yml that:

• ✅ Runs on Python 3.10 (required for consistent codegen output)
• ✅ Installs all dependencies (uv, markdownlint)
• ✅ Runs make codegen lint
• ✅ Checks for uncommitted changes using git diff --exit-code
• ✅ Fails the build if codegen produces any modifications
• ✅ Makes static_analysis job depend on this check for fast feedback

This addresses issue #153 and prevents the common problem of forgetting to run make codegen after modifying async code.

---

The Permission Issue

When attempting to push, GitHub rejected the changes:

refusing to allow a GitHub App to create or update workflow 
`.github/workflows/test.yml` without `workflows` permission

This is a GitHub security feature that prevents apps from modifying CI workflows without explicit permission.

---

How to Apply the Changes

Option 1: Manual Application (Quick - 2 minutes)

Apply these changes to .github/workflows/test.yml:

📋 Click to see the complete changes

Step 1: Add the new codegen_check job after the markdown_lint job (around line 32):

  codegen_check:
    runs-on: ubuntu-latest
    permissions:
      contents: read

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Set up Python 3.10
        uses: actions/setup-python@v5
        with:
          python-version: '3.10'

      - name: Install uv
        uses: astral-sh/setup-uv@v6

      - name: Install dependencies
        run: uv sync --all-packages --group dev

      - name: Install Node.js for markdown linting
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install markdownlint
        run: npm install -g markdownlint-cli

      - name: Run codegen and lint
        run: make codegen lint

      - name: Check for uncommitted changes
        run: |
          if ! git diff --exit-code; then
            echo "❌ Error: Codegen produced changes. Please run 'make codegen' locally and commit the results."
            echo ""
            echo "Files changed:"
            git diff --name-only
            exit 1
          fi
          echo "✅ Codegen check passed - no changes detected"

Step 2: Add the dependency to the static_analysis job (around line 75):

Change:

  static_analysis:
    runs-on: ubuntu-latest

To:

  static_analysis:
    needs: codegen_check
    runs-on: ubuntu-latest

Option 2: Grant Workflow Permissions (For future automation)

If you want me to be able to modify workflows in the future:

1. Go to your GitHub App settings
2. Add workflows: write permission to the Claude Code app
3. Reinstall the app with the new permissions
4. Ask me to create the PR again

See GitHub's documentation on App permissions for details.

---

Testing the Implementation

Once applied, the new check will:

1. Run early in the CI pipeline (before tests)
2. Catch forgotten make codegen runs immediately
3. Display a clear error message with the list of changed files
4. Prevent wasted CI time on other jobs if codegen is out of sync

You can test it by:

1. Making a change to async code
2. Not running make codegen
3. Creating a PR - the CI should fail with a helpful message

---

Problems Encountered

GitHub App Workflow Permissions: The Claude Code GitHub App doesn't have the workflows permission, which is required to modify files in .github/workflows/. This is a security feature that prevents automated changes to CI pipelines without explicit authorization.

---