Skip to content

[StaticWebAssets] Improve globbing match performance #44159

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Nov 22, 2024
Merged

[StaticWebAssets] Improve globbing match performance #44159

merged 1 commit into from
Nov 22, 2024

Conversation

@javiercn
Copy link
Member

@javiercn javiercn commented Oct 14, 2024

~.6s savings

Before

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:07.31

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:07.19

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:07.17

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:07.14

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:07.16

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:07.18

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:07.13

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:07.22

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:07.20

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:07.21

After

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:06.62

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:06.57

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:05.87

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:06.57

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:06.54

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:06.64

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:06.61

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:06.59

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:06.58

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:06.53

@ghost ghost added Area-AspNetCore RazorSDK, BlazorWebAssemblySDK, dotnet-watch untriaged Request triage from a team member labels Oct 14, 2024
@dotnet-policy-service
Copy link
Contributor

dotnet-policy-service bot commented Oct 14, 2024

Thanks for your PR, @javiercn.
To learn about the PR process and branching schedule of this repo, please take a look at the SDK PR Guide.

@javiercn javiercn force-pushed the javiercn/replace-swa-globbing branch 2 times, most recently from d5e70cd to 2989be9 Compare October 17, 2024 19:34
@build-analysis build-analysis bot mentioned this pull request Oct 17, 2024
Closed
@javiercn javiercn force-pushed the javiercn/replace-swa-globbing branch from 2989be9 to 63d46de Compare October 18, 2024 12:19
@build-analysis build-analysis bot mentioned this pull request Oct 18, 2024
Open
3 tasks
@javiercn javiercn force-pushed the javiercn/replace-swa-globbing branch 3 times, most recently from 4ab1292 to f9901f5 Compare October 20, 2024 12:22
This was referenced Oct 20, 2024
Open
Closed
@javiercn javiercn force-pushed the javiercn/replace-swa-globbing branch 3 times, most recently from 18667d3 to 2ecffdf Compare October 24, 2024 18:06
@build-analysis build-analysis bot mentioned this pull request Oct 24, 2024
Open
@javiercn javiercn force-pushed the javiercn/replace-swa-globbing branch 2 times, most recently from cc17e82 to cbef77d Compare October 25, 2024 09:42
@javiercn javiercn force-pushed the javiercn/replace-swa-globbing branch 3 times, most recently from 968cdac to 5f24b5c Compare November 19, 2024 21:42
javiercn
javiercn commented Nov 20, 2024
View reviewed changes
src/StaticWebAssetsSdk/Tasks/Data/StaticWebAssetPathPattern.cs
{
// Token was embedded, so add it to the dictionary.
dictionary[part.Name] = part.Value;
dictionary[part.Name.ToString()] = part.Value.ToString();
Copy link
Member Author

@javiercn javiercn Nov 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are going to end up as strings, so it's ok to stringify here.

javiercn
javiercn commented Nov 20, 2024
View reviewed changes
src/StaticWebAssetsSdk/Tasks/DefineStaticWebAssetEndpoints.cs
Comment on lines +48 to +62
Parallel.For(
0,
CandidateAssets.Length,
() => new ParallelWorker(
endpoints,
new List<StaticWebAssetEndpoint>(),
CandidateAssets,
existingEndpointsByAssetFile,
Log,
contentTypeProvider,
_assetFileDetails,
TestLengthResolver,
TestLastWriteResolver),
static (i, loop, state) => state.Process(i, loop),
static worker => worker.Finally());
Copy link
Member Author

@javiercn javiercn Nov 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is cleaning up the implementation to avoid excessive locking by storing the intermediate results into a local list for each worker and saving them to the final collection at the end of processing.

@javiercn javiercn force-pushed the javiercn/replace-swa-globbing branch from b5a6c66 to 377bdd9 Compare November 21, 2024 12:41
@javiercn
[StaticWebAssets] Replace globbing logic with a new implementation
199c9bf 
* Removes the usage of Microsoft.Extensions.FileSystemGlobbing in favor
  of a custom implementation optimized for our scenarios.
* Improves the parallelism in DefineStaticWebAssetEndpoints.
* Spanifies the string manipulation logic.
@javiercn javiercn force-pushed the javiercn/replace-swa-globbing branch from 377bdd9 to 199c9bf Compare November 21, 2024 12:42
@javiercn javiercn changed the title Javiercn/replace swa globbing [StaticWebAssets] Improve globbing match performance Nov 21, 2024
@javiercn javiercn marked this pull request as ready for review November 21, 2024 13:29
@javiercn javiercn requested a review from a team as a code owner November 21, 2024 13:29
MackinnonBuck
MackinnonBuck approved these changes Nov 21, 2024
View reviewed changes
Copy link
Member

@MackinnonBuck MackinnonBuck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!

@javiercn javiercn merged commit c371ad2 into main Nov 22, 2024
38 checks passed
@javiercn javiercn deleted the javiercn/replace-swa-globbing branch November 22, 2024 09:29
javiercn added a commit that referenced this pull request Mar 26, 2025
@javiercn
[StaticWebAssets] Improve globbing match performance (#44159)
c4d709c 
* Removes the usage of Microsoft.Extensions.FileSystemGlobbing in favor
  of a custom implementation optimized for our scenarios.
* Improves the parallelism in DefineStaticWebAssetEndpoints.
* Spanifies the string manipulation logic.
@javiercn javiercn mentioned this pull request Mar 26, 2025
Merged
javiercn added a commit that referenced this pull request Apr 21, 2025
@javiercn
[StaticWebAssets] Improve globbing match performance (#44159)
3287d34 
* Removes the usage of Microsoft.Extensions.FileSystemGlobbing in favor
  of a custom implementation optimized for our scenarios.
* Improves the parallelism in DefineStaticWebAssetEndpoints.
* Spanifies the string manipulation logic.
javiercn added a commit that referenced this pull request Apr 21, 2025
@javiercn
[StaticWebAssets] Backport perf fixes (#47913)
d79ec02 
Backports
* [StaticWebAssets] Process endpoint definitions in parallel #43736
* [StaticWebAssets] Improve globbing match performance #44159
* [StaticWebAssets] Adds incrementalism to DefineStaticWebAssets #47701
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@MackinnonBuck MackinnonBuck MackinnonBuck approved these changes

Assignees

No one assigned

Labels

Area-AspNetCore RazorSDK, BlazorWebAssemblySDK, dotnet-watch untriaged Request triage from a team member

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@javiercn @MackinnonBuck