Skip to content

Add selective build support for prim ops #14332

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add selective build support for prim ops #14332

wants to merge 1 commit into from

Conversation

kimishpatel
Copy link
Contributor

Summary:
This diff implements selective build functionality for primitive operations (prim ops) in ExecutorTorch, allowing users to include only specific prim ops in their builds to reduce binary size and compilation time.

Key Changes:

  1. Conditional compilation in register_prim_ops.cpp: Wrapped each of the prim op registrations with conditional compilation macros that check both selective build enablement (EXECUTORCH_ENABLE_PRIM_OPS_SELECTIVE_BUILD) and individual op selection (e.g., INCLUDE_EXECUTORCH_PRIM_ET_VIEW_DEFAULT).

  2. Code generation tool: Added gen_selected_prim_ops.py that takes comma-separated prim op names and generates a header file (selected_prim_ops.h) containing appropriate #define statements for selected ops. The tool normalizes op names to macro-safe format (e.g., executorch_prim::et_view.default → INCLUDE_EXECUTORCH_PRIM_ET_VIEW_DEFAULT).

  3. Build system integration:
    In order to make et_operator_library also handle prim of selective build we
    make a few changes.

  4. Extract prim ops in et_operator_library

  5. Similar to gen_op_list, we invoke script that geneates selected_prim_ops.h
    file per et_operator_library target. Thus et_operator_library now generates
    selected_operators.yaml and selected_prim_ops.h.
    Note that in order to make these work we have to allow et_operator_libray to
    handle the following cases.

    1. All ops are aten ops
    2. All ops are prim ops
    3. Mix
      To do this we must make sure that the genrule continues to produce the file
      it says it will produce. In the case of 1 we have to produce empty
      selected_prim_opsh. and in case 2 we have to produce emtpy
      selected_operators.yaml

  6. In gen_all_oplist we allow for empty selected_operators.yaml and skip the
    file.

  7. Similar to gen_all_oplist we introduce another binary that combines all
    selected_prim_ops.h.

  8. Then in executorch_generated_lib we query targets from 4 that have
    selected_prim_ops and use those to compile register_prim_ops.cpp.
    In executorch_generate_lib we introduce include_all_prim_ops which by default
    is True. Hence if one wants to enable selective build for prim ops one must
    turn off that flag

Usage:

Users can now specify prim ops like:

et_operator_library(name="my_aten_prim_ops", ops=["aten::mul.out", "executorch_prim::et_view.default", "aten::sym_size.int"])
executorch_generated_lib(name="my_lib", deps=[":my_aten_prim_ops"] +
other_deps, include_all_prim_ops=False)

Reviewed By: ivayloen, larryliu0820

Differential Revision: D81648030

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Sep 16, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14332

Note: Links to docs will display an error until the docs builds have been completed.

❌ 12 New Failures, 1 Cancelled Job

As of commit 7178438 with merge base 2f21092 (image):

NEW FAILURES - The following jobs have failed:

CANCELLED JOB - The following job was cancelled. Please retry:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 16, 2025
@facebook-github-bot
Copy link
Contributor

@kimishpatel has exported this pull request. If you are a Meta employee, you can view the originating diff in D81648030.

larryliu0820
larryliu0820 approved these changes Sep 16, 2025
View reviewed changes
Copy link
Contributor

@larryliu0820 larryliu0820 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review automatically exported from Phabricator review in Meta.

@github-actions GitHub Actions
Copy link

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

kimishpatel added a commit to kimishpatel/executorch-1 that referenced this pull request Sep 16, 2025
@kimishpatel @facebook-github-bot
Add selective build support for prim ops (pytorch#14332)
e64e3f1 
Summary:

This diff implements selective build functionality for primitive operations (prim ops) in ExecutorTorch, allowing users to include only specific prim ops in their builds to reduce binary size and compilation time.

## Key Changes:

1. **Conditional compilation in register_prim_ops.cpp**: Wrapped each of the prim op registrations with conditional compilation macros that check both selective build enablement (`EXECUTORCH_ENABLE_PRIM_OPS_SELECTIVE_BUILD`) and individual op selection (e.g., `INCLUDE_EXECUTORCH_PRIM_ET_VIEW_DEFAULT`).

2. **Code generation tool**: Added `gen_selected_prim_ops.py` that takes comma-separated prim op names and generates a header file (`selected_prim_ops.h`) containing appropriate `#define` statements for selected ops. The tool normalizes op names to macro-safe format (e.g., `executorch_prim::et_view.default` → `INCLUDE_EXECUTORCH_PRIM_ET_VIEW_DEFAULT`).

3. **Build system integration**:
  In order to make et_operator_library also handle prim of selective build we
  make a few changes.
  1. Extract prim ops in et_operator_library
  2. Similar to gen_op_list, we invoke script that geneates selected_prim_ops.h
  file per et_operator_library target. Thus et_operator_library now generates
  selected_operators.yaml and selected_prim_ops.h.
     Note that in order to make these work we have to allow et_operator_libray to
     handle the following cases.
     1. All ops are aten ops
     2. All ops are prim ops
     3. Mix
     To do this we must make sure that the genrule continues to produce the file
     it says it will produce. In the case of 1 we have to produce empty
     selected_prim_opsh. and in case 2 we have to produce emtpy
     selected_operators.yaml
  3. In gen_all_oplist we allow for empty selected_operators.yaml and skip the
  file.
  4. Similar to gen_all_oplist we introduce another binary that combines all
  selected_prim_ops.h.
  5. Then in executorch_generated_lib we query targets from 4 that have
  selected_prim_ops and use those to compile register_prim_ops.cpp.
  In executorch_generate_lib we introduce include_all_prim_ops which by default
  is True. Hence if one wants to enable selective build for prim ops one must
  turn off that flag

## Usage:

Users can now specify prim ops like:
```
et_operator_library(name="my_aten_prim_ops", ops=["aten::mul.out", "executorch_prim::et_view.default", "aten::sym_size.int"])
executorch_generated_lib(name="my_lib", deps=[":my_aten_prim_ops"] +
other_deps, include_all_prim_ops=False)
```

Reviewed By: ivayloen, larryliu0820

Differential Revision: D81648030
@facebook-github-bot
Copy link
Contributor

@kimishpatel has exported this pull request. If you are a Meta employee, you can view the originating diff in D81648030.

@kimishpatel @facebook-github-bot
Add selective build support for prim ops (pytorch#14332)
7178438 
Summary:

This diff implements selective build functionality for primitive operations (prim ops) in ExecutorTorch, allowing users to include only specific prim ops in their builds to reduce binary size and compilation time.

## Key Changes:

1. **Conditional compilation in register_prim_ops.cpp**: Wrapped each of the prim op registrations with conditional compilation macros that check both selective build enablement (`EXECUTORCH_ENABLE_PRIM_OPS_SELECTIVE_BUILD`) and individual op selection (e.g., `INCLUDE_EXECUTORCH_PRIM_ET_VIEW_DEFAULT`).

2. **Code generation tool**: Added `gen_selected_prim_ops.py` that takes comma-separated prim op names and generates a header file (`selected_prim_ops.h`) containing appropriate `#define` statements for selected ops. The tool normalizes op names to macro-safe format (e.g., `executorch_prim::et_view.default` → `INCLUDE_EXECUTORCH_PRIM_ET_VIEW_DEFAULT`).

3. **Build system integration**:
  In order to make et_operator_library also handle prim of selective build we
  make a few changes.
  1. Extract prim ops in et_operator_library
  2. Similar to gen_op_list, we invoke script that geneates selected_prim_ops.h
  file per et_operator_library target. Thus et_operator_library now generates
  selected_operators.yaml and selected_prim_ops.h.
     Note that in order to make these work we have to allow et_operator_libray to
     handle the following cases.
     1. All ops are aten ops
     2. All ops are prim ops
     3. Mix
     To do this we must make sure that the genrule continues to produce the file
     it says it will produce. In the case of 1 we have to produce empty
     selected_prim_opsh. and in case 2 we have to produce emtpy
     selected_operators.yaml
  3. In gen_all_oplist we allow for empty selected_operators.yaml and skip the
  file.
  4. Similar to gen_all_oplist we introduce another binary that combines all
  selected_prim_ops.h.
  5. Then in executorch_generated_lib we query targets from 4 that have
  selected_prim_ops and use those to compile register_prim_ops.cpp.
  In executorch_generate_lib we introduce include_all_prim_ops which by default
  is True. Hence if one wants to enable selective build for prim ops one must
  turn off that flag

## Usage:

Users can now specify prim ops like:
```
et_operator_library(name="my_aten_prim_ops", ops=["aten::mul.out", "executorch_prim::et_view.default", "aten::sym_size.int"])
executorch_generated_lib(name="my_lib", deps=[":my_aten_prim_ops"] +
other_deps, include_all_prim_ops=False)
```

Reviewed By: ivayloen, larryliu0820

Differential Revision: D81648030
@facebook-github-bot
Copy link
Contributor

@kimishpatel has exported this pull request. If you are a Meta employee, you can view the originating diff in D81648030.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@larryliu0820 larryliu0820 larryliu0820 approved these changes

@lucylq lucylq Awaiting requested review from lucylq lucylq is a code owner

@manuelcandales manuelcandales Awaiting requested review from manuelcandales manuelcandales is a code owner

@swolchok swolchok Awaiting requested review from swolchok swolchok is a code owner

@JacobSzwejbka JacobSzwejbka Awaiting requested review from JacobSzwejbka JacobSzwejbka is a code owner

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported meta-exported
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@kimishpatel @facebook-github-bot @larryliu0820