Skip to content

Conversation

aaronsteers
Copy link
Contributor

feat: Replace MyPy with pyrefly type checker (do not merge)

Summary

This PR replaces MyPy with Meta's pyrefly type checker as a test to evaluate pyrefly as a potential MyPy alternative for the airbyte-python-cdk repository. The integration uses pyrefly's --suppress-errors auto-noqa feature to automatically add suppression comments for existing type violations, allowing us to focus on evaluating the tool's performance and usability rather than fixing legacy type issues.

Key Changes:

  • Replaced mypy dependency with pyrefly ^0.26.0
  • Updated CI workflow from mypy-check to pyrefly-check
  • Updated type-check Poe task to use pyrefly check airbyte_cdk
  • Added pyrefly configuration excluding bin, docs, and unit_tests directories
  • Auto-generated suppression comments in 84 files for 530 type violations

Results: Local testing shows 0 errors (530 ignored) - pyrefly successfully runs with auto-noqa suppression.

Review & Testing Checklist for Human

  • Verify CI passes: Ensure the new pyrefly-check job completes successfully and doesn't timeout
  • Spot-check suppression comments: Review a sample of auto-generated # pyrefly: ignore comments to ensure they're reasonable and not masking critical type safety issues
  • Test local development: Run poetry run poe type-check locally to confirm the workflow works for developers
  • Performance comparison: Compare pyrefly runtime vs mypy (if data available) to assess performance implications

Diagram

%%{ init : { "theme" : "default" }}%%
graph TD
    pyproject["pyproject.toml<br/>Dependency & Config"]:::major-edit
    workflow[".github/workflows/<br/>python_lint.yml<br/>CI Pipeline"]:::major-edit
    poetry_lock["poetry.lock<br/>Lock File"]:::minor-edit
    airbyte_cdk["airbyte_cdk/<br/>84 Python Files<br/>with Suppressions"]:::minor-edit
    
    pyproject -->|"defines pyrefly ^0.26.0"| poetry_lock
    pyproject -->|"configures tool.pyrefly"| workflow
    workflow -->|"runs pyrefly check"| airbyte_cdk
    pyproject -->|"poe type-check task"| airbyte_cdk
    
    subgraph Legend
        L1[Major Edit]:::major-edit
        L2[Minor Edit]:::minor-edit
        L3[Context/No Edit]:::context
    end

classDef major-edit fill:#90EE90
classDef minor-edit fill:#87CEEB  
classDef context fill:#FFFFFF
Loading

Notes

  • Testing Purpose: This PR is for evaluation only - marked as "do not merge"
  • Auto-noqa Strategy: Used --suppress-errors to focus on tool evaluation rather than fixing existing type issues
  • Requested by: @aaronsteers for testing Rust-based Python type checkers as MyPy alternatives
  • Session: https://app.devin.ai/sessions/89f33f9cbd9f48bda7165514bc3c1005
  • Scope: airbyte-python-cdk only (PyAirbyte already has required type checking)

The large number of auto-generated suppressions (530) indicates significant type inconsistencies in the legacy codebase that pyrefly detected. This provides valuable insight into the codebase's type safety status while allowing evaluation of pyrefly's capabilities.

- Replace mypy dependency with pyrefly ^0.26.0
- Update CI workflow from mypy-check to pyrefly-check
- Update type-check Poe task to use pyrefly
- Add pyrefly configuration with python_version and project_excludes
- Use pyrefly --suppress-errors to auto-add noqa comments for legacy code
- Test integration shows 0 errors with 530 ignored violations

Co-Authored-By: AJ Steers <[email protected]>
Copy link
Contributor

Original prompt from AJ Steers:

@Devin - I'm wanting to put the two new rust Python typing libraries to the test. Create one PR to use ty and one PR to use pyre-something from meta.

Copy link
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@github-actions github-actions bot added the enhancement New feature or request label Jul 30, 2025
Copy link

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1753838067-test-pyrefly-type-checker#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1753838067-test-pyrefly-type-checker

Helpful Resources

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /autofix - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test - Runs connector tests with the updated CDK
  • /poe <command> - Runs any poe command in the CDK environment

📝 Edit this welcome message.

Copy link

github-actions bot commented Jul 30, 2025

PyTest Results (Fast)

3 689 tests  ±0   3 678 ✅ ±0   6m 34s ⏱️ ±0s
    1 suites ±0      11 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit 1e5f8d2. ± Comparison against base commit e4cbaaf.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Jul 30, 2025

PyTest Results (Full)

3 692 tests  ±0   3 681 ✅ ±0   11m 29s ⏱️ -16s
    1 suites ±0      11 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit 1e5f8d2. ± Comparison against base commit e4cbaaf.

♻️ This comment has been updated with latest results.

devin-ai-integration bot and others added 2 commits July 30, 2025 01:43
- Fix pyrefly suppression comments displaced by ruff formatting
- Move suppression comments to correct lines for type errors
- Apply ruff format fixes to maintain code style consistency
- Verify pyrefly still works: 0 errors (528 ignored)

Co-Authored-By: AJ Steers <[email protected]>
…ment

- Move pyrefly suppression comment to correct line after ruff formatting
- Verify both ruff format --check and pyrefly type-check pass locally
- Final fix for Ruff Format Check CI failure

Co-Authored-By: AJ Steers <[email protected]>
@aaronsteers aaronsteers changed the title feat: Replace MyPy with pyrefly type checker (do not merge) chore: Replace MyPy with pyrefly type checker (do not merge) Jul 30, 2025
@github-actions github-actions bot added the chore label Jul 31, 2025
@aaronsteers aaronsteers changed the title chore: Replace MyPy with pyrefly type checker (do not merge) chore: Replacempyp with pyrefly type checker (do not merge) Jul 31, 2025
@aaronsteers aaronsteers changed the title chore: Replacempyp with pyrefly type checker (do not merge) chore: Replacemypy with pyrefly type checker (do not merge) Jul 31, 2025
aaronsteers and others added 2 commits July 30, 2025 19:00
…in permissions

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
- Run pyrefly init to migrate mypy.ini to pyrefly.toml
- Auto-suppress 2 errors in bin/ and docs/ files
- Remove redundant [tool.pyrefly] section from pyproject.toml
- Remove obsolete mypy.ini file
- Pyrefly check now shows 0 errors (526 ignored)

Co-Authored-By: AJ Steers <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chore enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant