Use CMake EXPORT #8954

swolchok · 2025-03-05T01:40:22Z

The CMake EXPORT feature makes target_compile_options/target_link_libraries etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. We are currently working around our lack of EXPORT by (incompletely and therefore incorrectly!) repeating our configuration in tools/cmake/executorch-config.cmake.

I was previously stalled here by XNNPACK's lack of use of EXPORT, but I discovered that I simply wasn't being thorough enough with installing/exporting the entire target graph.

Note that this diff raises the size test thresholds. This is because previously the size test wasn't including prim op registration, and this diff effectively fixes builds using find_package to always include that.

[ghstack-poisoned]

swolchok · 2025-03-05T01:40:23Z

Stack from ghstack (oldest at bottom):

pytorch-bot · 2025-03-05T01:40:26Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/8954

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 17 New Failures, 17 Unrelated Failures

As of commit 1029ce3 with merge base cfefd2c ():

NEW FAILURES - The following jobs have failed:

pull / unittest / linux / linux-job (gh)
RuntimeError: Command docker exec -t cc29aac1f152aa811452fa95a0043f4ebf1a4110ceecc373aae56d3397a37b91 /exec failed with exit code 5
pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh)
RuntimeError: Command docker exec -t cb10fd1fa7db5e1b926c9a3c55e3834913b70b504ad9c25ed57761db745091a0 /exec failed with exit code 128
pull / unittest-arm-backend-with-no-fvp (test_pytest_ops) / linux-job (gh)
RuntimeError: Command docker exec -t 4e9ec0537001c5d20837afffb4d8767564e8278f3d26c1d1a3e6e08757efd03b /exec failed with exit code 128
pull / unittest-editable / linux / linux-job (gh)
RuntimeError: Command docker exec -t 461f371d2a7eaa3ccd77c07dd6168adf62a1dcfcd45942476d7e465ac3dbd899 /exec failed with exit code 5
trunk / test-arm-backend (test_models_ethos-u55) / linux-job (gh)
RuntimeError: Command docker exec -t 8e0d70b74365ebad545de56916946f185c78522452c6120d51a9c200bb24e285 /exec failed with exit code 128
trunk / test-arm-backend (test_models_ethos-u85) / linux-job (gh)
RuntimeError: Command docker exec -t 8d391f4189e572c712ef1a02873eae577e634faebb61b506fa6ab14e7bf803bb /exec failed with exit code 128
trunk / test-arm-backend (test_models_tosa) / linux-job (gh)
RuntimeError: Command docker exec -t f6b4ae5be308357e86ff5801e5b0732c9e343c522a41a67ffad5c589231e2e5e /exec failed with exit code 128
trunk / test-arm-backend (test_pytest_models_ethosu_fvp) / linux-job (gh)
RuntimeError: Command docker exec -t be024065f241dc1ad86ba0c0dc6197cfee3a267bc4b3300686d284aea15d5a41 /exec failed with exit code 128
trunk / test-arm-backend (test_pytest_ops_ethosu_fvp) / linux-job (gh)
RuntimeError: Command docker exec -t b57d0f9f2890b76066f3f78fbd23ea53995e9a9d09ecaf8d405fb4a7e807983a /exec failed with exit code 128
trunk / test-arm-backend (test_run_ethosu_fvp) / linux-job (gh)
RuntimeError: Command docker exec -t 7f031347668c795ae2064e0b4550328e02ecec35eb9498dbee737dfc2ff8cba0 /exec failed with exit code 128
trunk / test-arm-cortex-m-size-test (bare_metal) / linux-job (gh)
RuntimeError: Command docker exec -t c1ffe168fa22510a00d5117ea8bf5618c3b02e4a4113fdaec65e5415803d0f2b /exec failed with exit code 128
trunk / test-arm-cortex-m-size-test (zephyr-preset) / linux-job (gh)
RuntimeError: Command docker exec -t fa1cab46200e929e15cd735c50baed1644024eb05ecc3ec027acff8481deb829 /exec failed with exit code 128
trunk / test-arm-ootb-linux / linux-job (gh)
RuntimeError: Command docker exec -t 2d67675f3d3a870f2b059734cb78a8e86261920d652545af67ba597f35d4f3be /exec failed with exit code 128
trunk / test-llama-runner-mac (fp32, coreml) / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
trunk / test-llama-runner-mac (fp32, mps) / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
trunk / test-llama-runner-mac (fp32, xnnpack+custom+quantize_kv) / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 2
trunk / unittest-release / linux / linux-job (gh)
RuntimeError: Command docker exec -t ea376f3d96188bb76522e477cc58202f6077498a6627380fe0bc0980a1c2bb20 /exec failed with exit code 5

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

pull / unittest / macos / macos-job (gh) (similar failure)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 5
pull / unittest-editable / macos / macos-job (gh) (similar failure)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 5
trunk / test-llama-runner-linux (bf16, custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
trunk / test-llama-runner-linux (bf16, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
trunk / test-llama-runner-linux (fp32, portable, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
trunk / test-llama-runner-linux (fp32, xnnpack+custom, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11... / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
trunk / test-models-linux-aarch64 (add_mul, portable, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
trunk / test-models-linux-aarch64 (add_mul, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
trunk / test-models-linux-aarch64 (emformer_transcribe, portable, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
trunk / test-models-linux-aarch64 (emformer_transcribe, xnnpack-quantization-delegation, linux.arm64.2xla... / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
trunk / test-models-linux-aarch64 (ic3, portable, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
trunk / test-models-linux-aarch64 (llama3_2_vision_encoder, portable, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
trunk / test-models-linux-aarch64 (mobilebert, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
trunk / test-models-linux-aarch64 (mv3, portable, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
trunk / test-models-linux-aarch64 (softmax, portable, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1
trunk / test-models-linux-aarch64 (vit, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
The process '/usr/bin/git' failed with exit code 1

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

trunk / unittest-release / macos / macos-job (gh) (trunk failure)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 5

This comment was automatically generated by Dr. CI and updates every 15 minutes.

The CMake EXPORT feature seems like it's supposed to make target_compile_options etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. Using EXPORT would make it easy to have similar behavior to Buck's exported_preprocessor_flags, for example. Unfortunately, I am currently stalled here by XNNPACK's lack of use of EXPORT. I tried working around dependencies that don't use EXPORT by re-installing them with EXPORT ourselves, but XNNPACK has rather a lot so I'm sending this out to highlight the problem and ask if there's another way. I found some similar issues in TFLite: tensorflow/tensorflow#57658 tensorflow/tensorflow#87172 ghstack-source-id: e3e5b36 ghstack-comment-id: 2699501745 Pull Request resolved: #8954

github-actions · 2025-03-05T01:41:02Z

This PR needs a `release notes:` label

If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

swolchok · 2025-07-14T21:20:37Z

have an update cooking, need to make sure check whether warnings I'm seeing about macOS version mismatch during install_executorch.sh are new and if so problematic

[ghstack-poisoned]

The CMake EXPORT feature seems like it's supposed to make target_compile_options etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. Using EXPORT would make it easy to have similar behavior to Buck's exported_preprocessor_flags, for example. Unfortunately, I am currently stalled here by XNNPACK's lack of use of EXPORT. I tried working around dependencies that don't use EXPORT by re-installing them with EXPORT ourselves, but XNNPACK has rather a lot so I'm sending this out to highlight the problem and ask if there's another way. I found some similar issues in TFLite: tensorflow/tensorflow#57658 tensorflow/tensorflow#87172 ghstack-source-id: af3cc8d ghstack-comment-id: 2699501745 Pull Request resolved: #8954

[ghstack-poisoned]

The CMake EXPORT feature seems like it's supposed to make target_compile_options etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. Using EXPORT would make it easy to have similar behavior to Buck's exported_preprocessor_flags, for example. Unfortunately, I am currently stalled here by XNNPACK's lack of use of EXPORT. I tried working around dependencies that don't use EXPORT by re-installing them with EXPORT ourselves, but XNNPACK has rather a lot so I'm sending this out to highlight the problem and ask if there's another way. I found some similar issues in TFLite: tensorflow/tensorflow#57658 tensorflow/tensorflow#87172 ghstack-source-id: 8b7ec9a ghstack-comment-id: 2699501745 Pull Request resolved: #8954

[ghstack-poisoned]

The CMake EXPORT feature seems like it's supposed to make target_compile_options etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. Using EXPORT would make it easy to have similar behavior to Buck's exported_preprocessor_flags, for example. Unfortunately, I am currently stalled here by XNNPACK's lack of use of EXPORT. I tried working around dependencies that don't use EXPORT by re-installing them with EXPORT ourselves, but XNNPACK has rather a lot so I'm sending this out to highlight the problem and ask if there's another way. I found some similar issues in TFLite: tensorflow/tensorflow#57658 tensorflow/tensorflow#87172 ghstack-source-id: da025ca ghstack-comment-id: 2699501745 Pull Request resolved: #8954

[ghstack-poisoned]

The CMake EXPORT feature seems like it's supposed to make target_compile_options etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. Using EXPORT would make it easy to have similar behavior to Buck's exported_preprocessor_flags, for example. Unfortunately, I am currently stalled here by XNNPACK's lack of use of EXPORT. I tried working around dependencies that don't use EXPORT by re-installing them with EXPORT ourselves, but XNNPACK has rather a lot so I'm sending this out to highlight the problem and ask if there's another way. I found some similar issues in TFLite: tensorflow/tensorflow#57658 tensorflow/tensorflow#87172 ghstack-source-id: 5823280 ghstack-comment-id: 2699501745 Pull Request resolved: #8954

digantdesai · 2025-07-23T15:12:26Z

.github/workflows/pull.yml

@@ -371,7 +371,7 @@ jobs:
        size=${arr[4]}
        # threshold=48120 on devserver with gcc11.4
        # todo(lfq): update once binary size is below 50kb.
-        threshold="55584"
+        threshold="63776"


This is a bigger jump, do you know what is getting pulled in the build with these changes esp just with gcc and not with clang?

nvm saw this - #8954 (comment)

why just for gcc though?

it's not just for gcc. they both had a significant size increase; I can't speak to the difference between them or the difference in overage in the thresholds previously.

tools/cmake/executorch-config.cmake

[ghstack-poisoned]

The CMake EXPORT feature seems like it's supposed to make target_compile_options etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. Using EXPORT would make it easy to have similar behavior to Buck's exported_preprocessor_flags, for example. Unfortunately, I am currently stalled here by XNNPACK's lack of use of EXPORT. I tried working around dependencies that don't use EXPORT by re-installing them with EXPORT ourselves, but XNNPACK has rather a lot so I'm sending this out to highlight the problem and ask if there's another way. I found some similar issues in TFLite: tensorflow/tensorflow#57658 tensorflow/tensorflow#87172 ghstack-source-id: 5e2ac91 ghstack-comment-id: 2699501745 Pull Request resolved: #8954

[ghstack-poisoned]

The CMake EXPORT feature seems like it's supposed to make target_compile_options etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. Using EXPORT would make it easy to have similar behavior to Buck's exported_preprocessor_flags, for example. Unfortunately, I am currently stalled here by XNNPACK's lack of use of EXPORT. I tried working around dependencies that don't use EXPORT by re-installing them with EXPORT ourselves, but XNNPACK has rather a lot so I'm sending this out to highlight the problem and ask if there's another way. I found some similar issues in TFLite: tensorflow/tensorflow#57658 tensorflow/tensorflow#87172 ghstack-source-id: 4368718 ghstack-comment-id: 2699501745 Pull Request resolved: #8954

[ghstack-poisoned]

The CMake EXPORT feature seems like it's supposed to make target_compile_options etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. Using EXPORT would make it easy to have similar behavior to Buck's exported_preprocessor_flags, for example. Unfortunately, I am currently stalled here by XNNPACK's lack of use of EXPORT. I tried working around dependencies that don't use EXPORT by re-installing them with EXPORT ourselves, but XNNPACK has rather a lot so I'm sending this out to highlight the problem and ask if there's another way. I found some similar issues in TFLite: tensorflow/tensorflow#57658 tensorflow/tensorflow#87172 ghstack-source-id: 8c35528 ghstack-comment-id: 2699501745 Pull Request resolved: #8954

swolchok · 2025-07-23T21:33:28Z

looks like we might finally have green CI (given the upstream repo issue) except that the arm baremetal size tests need a threshold bump!

[ghstack-poisoned]

The CMake EXPORT feature seems like it's supposed to make target_compile_options etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. Using EXPORT would make it easy to have similar behavior to Buck's exported_preprocessor_flags, for example. Unfortunately, I am currently stalled here by XNNPACK's lack of use of EXPORT. I tried working around dependencies that don't use EXPORT by re-installing them with EXPORT ourselves, but XNNPACK has rather a lot so I'm sending this out to highlight the problem and ask if there's another way. I found some similar issues in TFLite: tensorflow/tensorflow#57658 tensorflow/tensorflow#87172 ghstack-source-id: f1615a2 ghstack-comment-id: 2699501745 Pull Request resolved: #8954

CMakeLists.txt

backends/apple/coreml/CMakeLists.txt

extension/llm/runner/CMakeLists.txt

kernels/portable/cpu/util/CMakeLists.txt

schema/CMakeLists.txt

tools/cmake/Utils.cmake

tools/cmake/executorch-config.cmake

…t failure for the EXPORT PR [ghstack-poisoned]

The CMake EXPORT feature seems like it's supposed to make target_compile_options etc. work nicely for both in-project builds and also projects that consume your project by installing it and finding it with find_package. Using EXPORT would make it easy to have similar behavior to Buck's exported_preprocessor_flags, for example. Unfortunately, I am currently stalled here by XNNPACK's lack of use of EXPORT. I tried working around dependencies that don't use EXPORT by re-installing them with EXPORT ourselves, but XNNPACK has rather a lot so I'm sending this out to highlight the problem and ask if there's another way. I found some similar issues in TFLite: tensorflow/tensorflow#57658 tensorflow/tensorflow#87172 ghstack-source-id: efb6f61 ghstack-comment-id: 2699501745 Pull Request resolved: #8954

Update

6d51bff

[ghstack-poisoned]

swolchok requested review from dbort, JacobSzwejbka, lucylq, manuelcandales, kimishpatel, shoumikhin, jackzhxng, iseeyuan, larryliu0820, digantdesai, mcr229, SS-JIA and cccclai as code owners March 5, 2025 01:40

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 5, 2025

swolchok marked this pull request as draft March 5, 2025 01:40

github-actions bot mentioned this pull request Mar 10, 2025

Weekly pr metrics report - 2025-03-01..2025-03-07 wdvr/pytorch#16

Open

This was referenced Mar 17, 2025

Weekly pr metrics report - 2025-03-01..2025-03-07 wdvr/pytorch#18

Open

Weekly pr metrics report - 2025-03-01..2025-03-07 wdvr/pytorch#20

Open

github-actions bot mentioned this pull request Mar 31, 2025

Weekly pr metrics report - 2025-03-01..2025-03-07 wdvr/pytorch#22

Open

github-actions bot mentioned this pull request Apr 7, 2025

Weekly pr metrics report - 2025-03-01..2025-03-07 wdvr/pytorch#26

Open

Update

b87d2fb

[ghstack-poisoned]

This was referenced Jul 14, 2025

[easy] cmake-format CMakeLists.txt #12450

Merged

[easy] cmake-format extension/training/CMakeLists.txt #12451

Merged

Update

2c62f9b

[ghstack-poisoned]

Update

098f4ad

[ghstack-poisoned]

Update

5eaa349

[ghstack-poisoned]

swolchok mentioned this pull request Jul 23, 2025

Don't build MmapDataLoader if sys/mman.h isn't available and we're not on Windows #12743

Open

swolchok changed the base branch from gh/swolchok/525/head to gh/swolchok/528/head July 23, 2025 04:23

swolchok mentioned this pull request Jul 23, 2025

Fix missing cmake_deps.toml dep from size_test to extension_data_loader #12744

Open

digantdesai reviewed Jul 23, 2025

View reviewed changes

Update

313ccaf

[ghstack-poisoned]

Update

029ce6b

[ghstack-poisoned]

forgot to restore tokenizers bump. format a CMake file

ef42dd0

[ghstack-poisoned]

Update

5596d52

[ghstack-poisoned]

swolchok commented Jul 23, 2025

View reviewed changes

fix zephyr size test to build data_loader extension. might be the las…

1029ce3

…t failure for the EXPORT PR [ghstack-poisoned]

swolchok mentioned this pull request Jul 23, 2025

Sweep major CMake files for use of include/lib instead of CMAKE_INSTALL_INCLUDE/LIBDIR #12792

Open

swolchok requested review from jathu and removed request for shoumikhin, dbort, iseeyuan, jackzhxng and mcr229 July 24, 2025 00:05

GregoryComer mentioned this pull request Jul 24, 2025

Add top-level CMake extensions target #12696

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Use CMake EXPORT #8954

Use CMake EXPORT #8954

Uh oh!

swolchok commented Mar 5, 2025 •

edited

Loading

Uh oh!

swolchok commented Mar 5, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Mar 5, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Mar 5, 2025

Uh oh!

swolchok commented Jul 14, 2025

Uh oh!

digantdesai Jul 23, 2025

Uh oh!

digantdesai Jul 23, 2025

Uh oh!

digantdesai Jul 23, 2025

Uh oh!

swolchok Jul 23, 2025

Uh oh!

Uh oh!

swolchok commented Jul 23, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Use CMake EXPORT #8954

Are you sure you want to change the base?

Use CMake EXPORT #8954

Uh oh!

Conversation

swolchok commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

swolchok commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/8954

❌ 17 New Failures, 17 Unrelated Failures

Uh oh!

github-actions bot commented Mar 5, 2025

This PR needs a release notes: label

Uh oh!

swolchok commented Jul 14, 2025

Uh oh!

digantdesai Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

digantdesai Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

digantdesai Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

swolchok Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

swolchok commented Jul 23, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

swolchok commented Mar 5, 2025 •

edited

Loading

swolchok commented Mar 5, 2025 •

edited

Loading

pytorch-bot bot commented Mar 5, 2025 •

edited

Loading

This PR needs a `release notes:` label