Add initial support for modeling deployment pipelines in Aspire #11953
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| FROM mcr.microsoft.com/dotnet/runtime:9.0 | ||
|
|
||
| # Create a directory for the bind mount | ||
| RUN mkdir -p /data | ||
|
|
||
| # Set the working directory | ||
| WORKDIR /app | ||
|
|
||
| # Create a simple script that reads and returns the file size | ||
| RUN printf '#!/bin/sh\n\ | ||
| if [ -f /data/sample.txt ]; then\n\ | ||
| SIZE=$(stat -f%%z /data/sample.txt 2>/dev/null || stat -c%%s /data/sample.txt 2>/dev/null)\n\ | ||
| echo "File size of sample.txt: $SIZE bytes"\n\ | ||
| else\n\ | ||
| echo "sample.txt not found at /data/sample.txt"\n\ | ||
| exit 1\n\ | ||
| fi\n' > /app/check-file-size.sh && chmod +x /app/check-file-size.sh | ||
|
|
||
| # Run the script when container starts | ||
| ENTRYPOINT ["/app/check-file-size.sh"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
|
||
| <PropertyGroup> | ||
| <OutputType>Exe</OutputType> | ||
| <TargetFramework>$(DefaultTargetFramework)</TargetFramework> | ||
| <ImplicitUsings>enable</ImplicitUsings> | ||
| <Nullable>enable</Nullable> | ||
| <IsAspireHost>true</IsAspireHost> | ||
| <UserSecretsId>f6688234-abcb-49bc-91cc-04a541ddee18</UserSecretsId> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <Compile Include="..\..\KnownResourceNames.cs" Link="KnownResourceNames.cs" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <AspireProjectOrPackageReference Include="Aspire.Hosting.AppHost" /> | ||
| <AspireProjectOrPackageReference Include="Aspire.Hosting.Storage" /> | ||
| <AspireProjectOrPackageReference Include="Aspire.Hosting.Azure.AppContainers" /> | ||
| <AspireProjectOrPackageReference Include="Aspire.Hosting.Azure.AppService" /> | ||
| <PackageReference Include="Azure.Storage.Files.Shares" /> | ||
| <PackageReference Include="Azure.Identity" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="..\..\publishers\Publishers.ApiService\Publishers.ApiService.csproj" /> | ||
| <ProjectReference Include="..\Pipelines.Library\Pipelines.Library.csproj" IsAspireProjectResource="false" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| { | ||
| "$schema": "https://json.schemastore.org/launchsettings.json", | ||
| "profiles": { | ||
| "https": { | ||
| "commandName": "Project", | ||
| "dotnetRunMessages": true, | ||
| "launchBrowser": true, | ||
| "applicationUrl": "https://localhost:17266;http://localhost:15173", | ||
| "environmentVariables": { | ||
| "ASPNETCORE_ENVIRONMENT": "Development", | ||
| "DOTNET_ENVIRONMENT": "Development", | ||
| "ASPIRE_DASHBOARD_OTLP_ENDPOINT_URL": "https://localhost:21055", | ||
| "ASPIRE_RESOURCE_SERVICE_ENDPOINT_URL": "https://localhost:22065" | ||
| } | ||
| }, | ||
| "http": { | ||
| "commandName": "Project", | ||
| "dotnetRunMessages": true, | ||
| "launchBrowser": true, | ||
| "applicationUrl": "http://localhost:15173", | ||
| "environmentVariables": { | ||
| "ASPNETCORE_ENVIRONMENT": "Development", | ||
| "DOTNET_ENVIRONMENT": "Development", | ||
| "ASPIRE_DASHBOARD_OTLP_ENDPOINT_URL": "http://localhost:19201", | ||
| "ASPIRE_RESOURCE_SERVICE_ENDPOINT_URL": "http://localhost:20169" | ||
| } | ||
| }, | ||
| "deploy": { | ||
| "commandName": "Project", | ||
| "commandLineArgs": "--publisher default --deploy true --output-path ./deploy-output", | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| { | ||
| "Logging": { | ||
| "LogLevel": { | ||
| "Default": "Information", | ||
| "Microsoft.AspNetCore": "Warning" | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| { | ||
| "Logging": { | ||
| "LogLevel": { | ||
| "Default": "Information", | ||
| "Microsoft.AspNetCore": "Warning", | ||
| "Aspire.Hosting.Dcp": "Warning" | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,206 @@ | ||
| #pragma warning disable ASPIREPUBLISHERS001 | ||
| #pragma warning disable ASPIRECOMPUTE001 | ||
| #pragma warning disable ASPIREAZURE001 | ||
| #pragma warning disable ASPIREPIPELINES001 | ||
|
|
||
| using System.Diagnostics; | ||
| using System.IO.Compression; | ||
| using System.Net.Http.Headers; | ||
| using Aspire.Hosting.ApplicationModel; | ||
| using Aspire.Hosting.Azure; | ||
| using Aspire.Hosting.Pipelines; | ||
| using Aspire.Hosting.Publishing; | ||
| using Azure.Core; | ||
| using Azure.Identity; | ||
| using Azure.Provisioning; | ||
|
|
||
| namespace Pipelines.Library; | ||
|
|
||
| public static class DistributedApplicationPipelineExtensions | ||
| { | ||
| public static IDistributedApplicationPipeline AddAppServiceZipDeploy(this IDistributedApplicationPipeline pipeline) | ||
| { | ||
| pipeline.AddStep("app-service-zip-deploy", async context => | ||
| { | ||
| var appServiceEnvironments = context.Model.Resources.OfType<AzureAppServiceEnvironmentResource>(); | ||
| if (!appServiceEnvironments.Any()) | ||
| { | ||
| return; | ||
| } | ||
|
|
||
| foreach (var appServiceEnvironment in appServiceEnvironments) | ||
| { | ||
| foreach (var resource in context.Model.GetComputeResources()) | ||
| { | ||
| var annotation = resource.GetDeploymentTargetAnnotation(); | ||
| if (annotation != null && | ||
| annotation.ComputeEnvironment == appServiceEnvironment && | ||
| annotation.DeploymentTarget is AzureAppServiceWebSiteResource websiteResource) | ||
| { | ||
| if (resource is not ProjectResource projectResource) | ||
| { | ||
| continue; | ||
| } | ||
|
|
||
| await DeployProjectToAppServiceAsync( | ||
| context, | ||
| projectResource, | ||
| websiteResource, | ||
| appServiceEnvironment, | ||
| context.CancellationToken).ConfigureAwait(false); | ||
| } | ||
| } | ||
| } | ||
| }, dependsOn: WellKnownPipelineSteps.DeployCompute); | ||
|
|
||
| return pipeline; | ||
| } | ||
|
|
||
| private static async Task DeployProjectToAppServiceAsync( | ||
| DeployingContext context, | ||
| ProjectResource projectResource, | ||
| AzureAppServiceWebSiteResource websiteResource, | ||
| AzureAppServiceEnvironmentResource appServiceEnvironment, | ||
| CancellationToken cancellationToken) | ||
| { | ||
| var stepName = $"deploy-{projectResource.Name}"; | ||
| var step = await context.ActivityReporter.CreateStepAsync(stepName, cancellationToken).ConfigureAwait(false); | ||
|
|
||
| await using (step.ConfigureAwait(false)) | ||
| { | ||
| var projectMetadata = projectResource.GetProjectMetadata(); | ||
| var projectPath = projectMetadata.ProjectPath; | ||
|
|
||
| var publishTask = await step.CreateTaskAsync($"Publishing {projectResource.Name}", cancellationToken).ConfigureAwait(false); | ||
| await using (publishTask.ConfigureAwait(false)) | ||
| { | ||
| var publishDir = Path.Combine(Path.GetTempPath(), $"aspire-publish-{Guid.NewGuid()}"); | ||
| Directory.CreateDirectory(publishDir); | ||
|
|
||
| try | ||
| { | ||
| var publishProcess = Process.Start(new ProcessStartInfo | ||
| { | ||
| FileName = "dotnet", | ||
| Arguments = $"publish \"{projectPath}\" -c Release -o \"{publishDir}\"", | ||
| RedirectStandardOutput = true, | ||
| RedirectStandardError = true, | ||
| UseShellExecute = false, | ||
| CreateNoWindow = true | ||
| }); | ||
|
|
||
| if (publishProcess == null) | ||
| { | ||
| await publishTask.CompleteAsync( | ||
| "Failed to start dotnet publish", | ||
| CompletionState.CompletedWithError, | ||
| cancellationToken).ConfigureAwait(false); | ||
| return; | ||
| } | ||
|
|
||
| await publishProcess.WaitForExitAsync(cancellationToken).ConfigureAwait(false); | ||
|
|
||
| if (publishProcess.ExitCode != 0) | ||
| { | ||
| var error = await publishProcess.StandardError.ReadToEndAsync(cancellationToken).ConfigureAwait(false); | ||
| await publishTask.CompleteAsync( | ||
| $"Publish failed: {error}", | ||
| CompletionState.CompletedWithError, | ||
| cancellationToken).ConfigureAwait(false); | ||
| return; | ||
| } | ||
|
|
||
| await publishTask.CompleteAsync( | ||
| "Publish completed", | ||
| CompletionState.Completed, | ||
| cancellationToken).ConfigureAwait(false); | ||
|
|
||
| var zipTask = await step.CreateTaskAsync($"Creating deployment package", cancellationToken).ConfigureAwait(false); | ||
| await using (zipTask.ConfigureAwait(false)) | ||
| { | ||
| var zipPath = Path.Combine(Path.GetTempPath(), $"aspire-deploy-{Guid.NewGuid()}.zip"); | ||
|
|
||
| ZipFile.CreateFromDirectory(publishDir, zipPath); | ||
|
|
||
| await zipTask.CompleteAsync( | ||
| "Deployment package created", | ||
| CompletionState.Completed, | ||
| cancellationToken).ConfigureAwait(false); | ||
|
|
||
| var uploadTask = await step.CreateTaskAsync($"Uploading to {projectResource.Name}", cancellationToken).ConfigureAwait(false); | ||
| await using (uploadTask.ConfigureAwait(false)) | ||
| { | ||
| try | ||
| { | ||
| var siteName = websiteResource.Outputs[$"{Infrastructure.NormalizeBicepIdentifier(websiteResource.Name)}_name"]?.ToString(); | ||
| if (string.IsNullOrEmpty(siteName)) | ||
| { | ||
| siteName = appServiceEnvironment.Outputs["name"]?.ToString(); | ||
| } | ||
|
|
||
| if (string.IsNullOrEmpty(siteName)) | ||
| { | ||
| await uploadTask.CompleteAsync( | ||
| "Could not determine website name", | ||
| CompletionState.CompletedWithError, | ||
| cancellationToken).ConfigureAwait(false); | ||
| return; | ||
| } | ||
|
|
||
| var credential = new AzureCliCredential(); | ||
| var tokenRequestContext = new TokenRequestContext(["https://management.azure.com/.default"]); | ||
| var accessToken = await credential.GetTokenAsync(tokenRequestContext, cancellationToken).ConfigureAwait(false); | ||
|
|
||
| var kuduUrl = $"https://{siteName}.scm.azurewebsites.net/api/zipdeploy"; | ||
|
|
||
| using var httpClient = new HttpClient(); | ||
| httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", accessToken.Token); | ||
| httpClient.Timeout = TimeSpan.FromMinutes(30); | ||
|
|
||
| await using var zipStream = File.OpenRead(zipPath); | ||
| using var content = new StreamContent(zipStream); | ||
| content.Headers.ContentType = new MediaTypeHeaderValue("application/zip"); | ||
|
|
||
| var response = await httpClient.PostAsync(kuduUrl, content, cancellationToken).ConfigureAwait(false); | ||
|
There was a problem hiding this comment. No upload progress!?!? 😄 Biggest downside of the new ux. There was a problem hiding this comment. FWIW, we lacked the ability to do percentage-based progress in the old UX too. Assuming that's what you mean about upload progress. We'd likely need some deltas in the PublishingActivityReporter and backchannel types to support percentage-based progress. |
||
|
|
||
| if (!response.IsSuccessStatusCode) | ||
| { | ||
| var errorContent = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false); | ||
| await uploadTask.CompleteAsync( | ||
| $"Upload failed: {response.StatusCode} - {errorContent}", | ||
| CompletionState.CompletedWithError, | ||
| cancellationToken).ConfigureAwait(false); | ||
| return; | ||
| } | ||
|
|
||
| await uploadTask.CompleteAsync( | ||
| "Upload completed successfully", | ||
| CompletionState.Completed, | ||
| cancellationToken).ConfigureAwait(false); | ||
| } | ||
| finally | ||
| { | ||
| if (File.Exists(zipPath)) | ||
| { | ||
| File.Delete(zipPath); | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| finally | ||
| { | ||
| if (Directory.Exists(publishDir)) | ||
| { | ||
| Directory.Delete(publishDir, recursive: true); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| await step.CompleteAsync( | ||
| "Deployment completed", | ||
| CompletionState.Completed, | ||
| cancellationToken).ConfigureAwait(false); | ||
| } | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
|
||
| <PropertyGroup> | ||
| <TargetFramework>$(DefaultTargetFramework)</TargetFramework> | ||
| <ImplicitUsings>enable</ImplicitUsings> | ||
| <Nullable>enable</Nullable> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <AspireProjectOrPackageReference Include="Aspire.Hosting" /> | ||
| <AspireProjectOrPackageReference Include="Aspire.Hosting.Azure.AppService" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| # Sample Data for Pipeline Processing | ||
|
|
||
| ## User Records | ||
| ID,Name,Email,Department,Salary,JoinDate | ||
| 1,John Smith,[email protected],Engineering,75000,2023-01-15 | ||
| 2,Sarah Johnson,[email protected],Marketing,65000,2023-02-20 | ||
| 3,Mike Davis,[email protected],Engineering,80000,2022-11-10 | ||
| 4,Lisa Wilson,[email protected],Sales,70000,2023-03-05 | ||
| 5,David Brown,[email protected],HR,60000,2023-01-30 | ||
|
|
||
| ## Application Logs | ||
| 2024-01-15T10:30:15.123Z INFO Starting application service | ||
| 2024-01-15T10:30:16.456Z INFO Database connection established | ||
| 2024-01-15T10:30:17.789Z WARN Slow query detected: SELECT * FROM users WHERE department = 'Engineering' | ||
| 2024-01-15T10:30:18.012Z INFO Processing 150 user records | ||
| 2024-01-15T10:30:19.345Z ERROR Failed to process record ID: 42 - Invalid email format | ||
| 2024-01-15T10:30:20.678Z INFO Successfully processed 149 records | ||
| 2024-01-15T10:30:21.901Z INFO Application service stopped gracefully | ||
|
|
||
| ## Configuration Settings | ||
| { | ||
| "database": { | ||
| "connectionString": "Server=localhost;Database=AspireDemo;Trusted_Connection=true;", | ||
| "timeout": 30 | ||
| }, | ||
| "messaging": { | ||
| "rabbitMQ": { | ||
| "host": "localhost", | ||
| "port": 5672, | ||
| "virtualHost": "/" | ||
| } | ||
| }, | ||
| "redis": { | ||
| "connectionString": "localhost:6379" | ||
| } | ||
| } | ||
|
|
||
| ## Sample Messages | ||
| MSG001: User registration completed for user ID 1001 | ||
| MSG002: Password reset requested for user [email protected] | ||
| MSG003: Order #12345 shipped to customer address | ||
| MSG004: Payment processed successfully for amount $299.99 | ||
| MSG005: Inventory updated: Product SKU-ABC123 quantity changed from 100 to 85 | ||
|
|
||
| ## Performance Metrics | ||
| Timestamp,CPU_Usage,Memory_MB,Requests_Per_Second,Response_Time_MS | ||
| 2024-01-15T10:00:00Z,25.4,512,150,45 | ||
| 2024-01-15T10:01:00Z,32.1,548,175,52 | ||
| 2024-01-15T10:02:00Z,28.7,534,165,48 | ||
| 2024-01-15T10:03:00Z,45.2,612,220,67 | ||
| 2024-01-15T10:04:00Z,38.9,589,198,58 |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had made a change to the AppServiceContext object to support setting this as an output but I reverted it in favor of #11931 so I don't have to deal with all the deltas to the Bicep snapshot tests in this PR. :D
This does technically mean this won't work as is.