Skip to content

Conversation

MaxDesiatov
Copy link
Contributor

Currently test/CMakeLists.txt can only set SWIFT_LIT_ARGS for all tests uniformly. This means that we can't have tests for Embedded Swift with a different set of params.

Let's refactor computation of lit.py options and arguments into a separate function, which means these params can set separately for different tests.

In this change we're only using it once, but in the future we anticipate another use of setup_lit_args specifically for Embedded Swift testing.

Currently `test/CMakeLists.txt` can only set `SWIFT_LIT_ARGS` for all tests uniformly. This means that we can't have tests for Embedded Swift with a different set of params.

Let's refactor computation of `lit.py` options and arguments into a separate function, which means these params can set separately for different tests.

In this change we're only using it once, but in the future we anticipate another use of `setup_lit_args` specifically for Embedded Swift testing.
@MaxDesiatov MaxDesiatov added test suite Area: test suite embedded Embedded Swift labels Jul 8, 2025
@MaxDesiatov
Copy link
Contributor Author

@swift-ci test

@MaxDesiatov MaxDesiatov requested a review from etcwilde July 8, 2025 16:10
@MaxDesiatov
Copy link
Contributor Author

@swift-ci smoke test linux

@MaxDesiatov
Copy link
Contributor Author

@swift-ci test linux

@MaxDesiatov
Copy link
Contributor Author

MaxDesiatov commented Jul 9, 2025

Verified manually that the total number of discovered tests hasn't significantly changed with this PR when comparing to other CI runs on the same day. E.g. on Linux:

Before:

[2025-07-09T08:14:16.731Z] Total Discovered Tests: 20150
[2025-07-09T08:14:16.731Z]   Unsupported      :  2603 (12.92%)
[2025-07-09T08:14:16.731Z]   Passed           : 17502 (86.86%)
[2025-07-09T08:14:16.731Z]   Expectedly Failed:    45 (0.22%)

After:

[2025-07-09T12:19:14.784Z] Total Discovered Tests: 20150
[2025-07-09T12:19:14.784Z]   Unsupported      :  2604 (12.92%)
[2025-07-09T12:19:14.784Z]   Passed           : 17501 (86.85%)
[2025-07-09T12:19:14.784Z]   Expectedly Failed:    45 (0.22%)

@MaxDesiatov MaxDesiatov merged commit e34eb33 into swiftlang:main Jul 10, 2025
5 checks passed
@MaxDesiatov MaxDesiatov deleted the maxd/setup-lit-args branch July 10, 2025 08:02
MaxDesiatov added a commit to MaxDesiatov/swift that referenced this pull request Jul 10, 2025
After swiftlang#82878 `SWIFT_LIT_ARGS` wasn't merged into correct returned variable in `setup_lit_args`, let's fix that to make sure that options like `-v --debug` are passed correctly from CLI `build-script` invocations or from presets that have these options predefined.
MaxDesiatov added a commit that referenced this pull request Jul 10, 2025
…82953)

After #82878 `SWIFT_LIT_ARGS` wasn't merged into correct returned variable in `setup_lit_args`, let's fix that to make sure that options like `-v --debug` are passed correctly from CLI `build-script` invocations or from presets that have these options predefined.
susmonteiro pushed a commit to susmonteiro/swift that referenced this pull request Jul 17, 2025
…wiftlang#82953)

After swiftlang#82878 `SWIFT_LIT_ARGS` wasn't merged into correct returned variable in `setup_lit_args`, let's fix that to make sure that options like `-v --debug` are passed correctly from CLI `build-script` invocations or from presets that have these options predefined.
@MaxDesiatov MaxDesiatov restored the maxd/setup-lit-args branch July 24, 2025 08:32
@MaxDesiatov MaxDesiatov deleted the maxd/setup-lit-args branch July 24, 2025 08:34
MaxDesiatov added a commit to MaxDesiatov/swift that referenced this pull request Jul 24, 2025
Currently `test/CMakeLists.txt` can only set `SWIFT_LIT_ARGS` for all tests uniformly. This means that we can't have tests for Embedded Swift with a different set of params.

Let's refactor computation of `lit.py` options and arguments into a separate function, which means these params can set separately for different tests.

In this change we're only using it once, but in the future we anticipate another use of `setup_lit_args` specifically for Embedded Swift testing.
# Conflicts:
#	test/CMakeLists.txt
MaxDesiatov added a commit to MaxDesiatov/swift that referenced this pull request Jul 24, 2025
…wiftlang#82953)

After swiftlang#82878 `SWIFT_LIT_ARGS` wasn't merged into correct returned variable in `setup_lit_args`, let's fix that to make sure that options like `-v --debug` are passed correctly from CLI `build-script` invocations or from presets that have these options predefined.
MaxDesiatov added a commit to MaxDesiatov/swift that referenced this pull request Jul 24, 2025
Currently `test/CMakeLists.txt` can only set `SWIFT_LIT_ARGS` for all tests uniformly. This means that we can't have tests for Embedded Swift with a different set of params.

Let's refactor computation of `lit.py` options and arguments into a separate function, which means these params can set separately for different tests.

In this change we're only using it once, but in the future we anticipate another use of `setup_lit_args` specifically for Embedded Swift testing.
# Conflicts:
#	test/CMakeLists.txt
MaxDesiatov added a commit to MaxDesiatov/swift that referenced this pull request Jul 24, 2025
…wiftlang#82953)

After swiftlang#82878 `SWIFT_LIT_ARGS` wasn't merged into correct returned variable in `setup_lit_args`, let's fix that to make sure that options like `-v --debug` are passed correctly from CLI `build-script` invocations or from presets that have these options predefined.
MaxDesiatov added a commit that referenced this pull request Jul 26, 2025
Cherry-pick of #83128, #82399, and #82878, merged as ea6ca2b, 0c4e561, and e34eb33 respectively.

**Explanation**: Currently `test/CMakeLists.txt` can only set `SWIFT_LIT_ARGS` for all tests uniformly. This means that we can't have tests for Embedded Swift with a different set of `lit.py` arguments.

Also, create new `check-swift-embedded-wasi` target from `test/CMakeLists.txt`, tweak `lit.cfg` to support WASI Clang resource dir, exclude unsupported tests based on `CPU=wasm32` instead of `OS=wasi`.

**Scope**: Limited to Embedded Swift test suite.
**Risk**: Low, due to limited scope.
**Testing**: #82878 was incubated on `main` for 2 weeks, #82399 for 3 weeks with no disruption, #83128 merged this week, but enables all these tests on CI, which are consistently passing.
**Issue**: rdar://156585717
**Reviewer**: @bnbarham
MaxDesiatov added a commit to MaxDesiatov/swift that referenced this pull request Jul 28, 2025
Currently `test/CMakeLists.txt` can only set `SWIFT_LIT_ARGS` for all tests uniformly. This means that we can't have tests for Embedded Swift with a different set of params.

Let's refactor computation of `lit.py` options and arguments into a separate function, which means these params can set separately for different tests.

In this change we're only using it once, but in the future we anticipate another use of `setup_lit_args` specifically for Embedded Swift testing.
# Conflicts:
#	test/CMakeLists.txt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cmake embedded Embedded Swift test suite Area: test suite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant