Parallel build fails for .Net Core project #211
Description
Description:
Attempting to build a largely dotnet codebase with sam build --parallel causes the build to fail, despite sam build working
Steps to reproduce:
- Setup project with multiple dotnet lambdas
- Run
sam build --parallel
Observed result:
The build fails, seemingly while attempting to install the amazon.lambda.tools for dotnet.
2020-11-20 20:20:48,873 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2020-11-20 20:20:48,875 | 'build' command is called
2020-11-20 20:20:48,879 | Collected default values for parameters: {}
2020-11-20 20:20:48,895 | 2 resources found in the template
2020-11-20 20:20:48,895 | Found Serverless function with name='Function' and CodeUri='src/Function'
2020-11-20 20:20:48,895 | Found Serverless function with name='Function2' and CodeUri='src/Function2'
2020-11-20 20:20:48,895 | Collected default values for parameters: {}
2020-11-20 20:20:48,910 | Instantiating build definitions
2020-11-20 20:20:48,912 | Same function build definition found, adding function (Previous: BuildDefinition(dotnetcore3.1, src/Function, , df11c1a0-4caa-4de5-9ac2-f807567cb7b8, {}, []), Current: BuildDefinition(dotnetcore3.1, src/Function, , 63f2774b-d8b5-4491-9a63-83e22722f684, {}, []), Function: Function(name='Function', functionname='local-Function', runtime='dotnetcore3.1', memory=3008, timeout=30, handler='Function::Function.Function::FunctionHandler', codeuri='src/Function', environment=None, rolearn=None, layers=[], events=None, metadata=None))
2020-11-20 20:20:48,912 | Same function build definition found, adding function (Previous: BuildDefinition(dotnetcore3.1, src/Function2, , 60574124-e2c3-4140-ab5d-0b47bb2e9a59, {}, []), Current: BuildDefinition(dotnetcore3.1, src/Function2, , c5e292d2-ddfd-41ad-8c08-94317ff0db37, {}, []), Function: Function(name='Function2', functionname='local-Function2', runtime='dotnetcore3.1', memory=3008, timeout=30, handler='Function::Function.Function::FunctionHandler', codeuri='src/Function2', environment=None, rolearn=None, layers=[], events=None, metadata=None))
2020-11-20 20:20:48,913 | Async execution started
2020-11-20 20:20:48,913 | Invoking function functools.partial(<bound method DefaultBuildStrategy.build_single_function_definition of <samcli.lib.build.build_strategy.DefaultBuildStrategy object at 0x7faa6c91c9a0>>, <samcli.lib.build.build_graph.FunctionBuildDefinition object at 0x7faa6c91ccd0>)
2020-11-20 20:20:48,914 | Building codeuri: src/Function runtime: dotnetcore3.1 metadata: {} functions: ['Function']
2020-11-20 20:20:48,914 | Building to following folder /home/nikko/dev/dntest/.aws-sam/build/Function
2020-11-20 20:20:48,914 | Invoking function functools.partial(<bound method DefaultBuildStrategy.build_single_function_definition of <samcli.lib.build.build_strategy.DefaultBuildStrategy object at 0x7faa6c91c9a0>>, <samcli.lib.build.build_graph.FunctionBuildDefinition object at 0x7faa6c8fb670>)
2020-11-20 20:20:48,914 | Building codeuri: src/Function2 runtime: dotnetcore3.1 metadata: {} functions: ['Function2']
2020-11-20 20:20:48,914 | Building to following folder /home/nikko/dev/dntest/.aws-sam/build/Function2
2020-11-20 20:20:48,915 | Waiting for async results
2020-11-20 20:20:48,915 | Loading workflow module 'aws_lambda_builders.workflows'
2020-11-20 20:20:48,915 | Loading workflow module 'aws_lambda_builders.workflows'
2020-11-20 20:20:48,917 | Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
2020-11-20 20:20:48,918 | Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
2020-11-20 20:20:48,919 | Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
2020-11-20 20:20:48,920 | Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
2020-11-20 20:20:48,930 | Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
2020-11-20 20:20:48,931 | Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
2020-11-20 20:20:48,932 | Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
2020-11-20 20:20:48,933 | Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
2020-11-20 20:20:48,934 | Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)'
2020-11-20 20:20:48,934 | Found workflow 'DotnetCliPackageBuilder' to support capabilities 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
2020-11-20 20:20:48,935 | Running workflow 'DotnetCliPackageBuilder'
2020-11-20 20:20:48,935 | Running DotnetCliPackageBuilder:GlobalToolInstall
2020-11-20 20:20:48,935 | Installing Amazon.Lambda.Tools Global Tool
2020-11-20 20:20:48,934 | Found workflow 'DotnetCliPackageBuilder' to support capabilities 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
2020-11-20 20:20:48,935 | executing dotnet: ['dotnet', 'tool', 'install', '-g', 'Amazon.Lambda.Tools', '--ignore-failed-sources']
2020-11-20 20:20:48,936 | Running workflow 'DotnetCliPackageBuilder'
2020-11-20 20:20:48,937 | Running DotnetCliPackageBuilder:GlobalToolInstall
2020-11-20 20:20:48,937 | Installing Amazon.Lambda.Tools Global Tool
2020-11-20 20:20:48,937 | executing dotnet: ['dotnet', 'tool', 'install', '-g', 'Amazon.Lambda.Tools', '--ignore-failed-sources']
2020-11-20 20:20:49,148 |
2020-11-20 20:20:49,148 | Error installing probably due to already installed. Attempt to update to latest version.
2020-11-20 20:20:49,148 | executing dotnet: ['dotnet', 'tool', 'update', '-g', 'Amazon.Lambda.Tools', '--ignore-failed-sources']
2020-11-20 20:20:49,162 |
2020-11-20 20:20:49,162 | Error installing probably due to already installed. Attempt to update to latest version.
2020-11-20 20:20:49,162 | executing dotnet: ['dotnet', 'tool', 'update', '-g', 'Amazon.Lambda.Tools', '--ignore-failed-sources']
2020-11-20 20:20:49,447 |
2020-11-20 20:20:49,447 | DotnetCliPackageBuilder:GlobalToolInstall failed
Traceback (most recent call last):
File "/home/linuxbrew/.linuxbrew/Cellar/aws-sam-cli/1.11.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflows/dotnet_clipackage/actions.py", line 33, in execute
self.subprocess_dotnet.run(["tool", "install", "-g", "Amazon.Lambda.Tools", "--ignore-failed-sources"])
File "/home/linuxbrew/.linuxbrew/Cellar/aws-sam-cli/1.11.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflows/dotnet_clipackage/dotnetcli.py", line 67, in run
raise DotnetCLIExecutionError(message=err.decode(encoding).strip())
aws_lambda_builders.workflows.dotnet_clipackage.dotnetcli.DotnetCLIExecutionError: Dotnet CLI Failed: Tool 'amazon.lambda.tools' is already installed.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/linuxbrew/.linuxbrew/Cellar/aws-sam-cli/1.11.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflows/dotnet_clipackage/actions.py", line 37, in execute
self.subprocess_dotnet.run(["tool", "update", "-g", "Amazon.Lambda.Tools", "--ignore-failed-sources"])
File "/home/linuxbrew/.linuxbrew/Cellar/aws-sam-cli/1.11.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflows/dotnet_clipackage/dotnetcli.py", line 67, in run
raise DotnetCLIExecutionError(message=err.decode(encoding).strip())
aws_lambda_builders.workflows.dotnet_clipackage.dotnetcli.DotnetCLIExecutionError: Dotnet CLI Failed: Tool 'amazon.lambda.tools' failed to update due to the following:
Failed to uninstall tool 'amazon.lambda.tools': Failed to retrieve tool configuration: Could not find a part of the path '/home/nikko/.dotnet/tools/.store/amazon.lambda.tools/4.3.0/amazon.lambda.tools/4.3.0/tools/netcoreapp2.1/any/DotnetToolSettings.xml'.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/linuxbrew/.linuxbrew/Cellar/aws-sam-cli/1.11.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflow.py", line 269, in run
action.execute()
File "/home/linuxbrew/.linuxbrew/Cellar/aws-sam-cli/1.11.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflows/dotnet_clipackage/actions.py", line 39, in execute
raise ActionFailedError("Error configuring the Amazon.Lambda.Tools .NET Core Global Tool: " + str(ex))
aws_lambda_builders.actions.ActionFailedError: Error configuring the Amazon.Lambda.Tools .NET Core Global Tool: Dotnet CLI Failed: Tool 'amazon.lambda.tools' failed to update due to the following:
Failed to uninstall tool 'amazon.lambda.tools': Failed to retrieve tool configuration: Could not find a part of the path '/home/nikko/.dotnet/tools/.store/amazon.lambda.tools/4.3.0/amazon.lambda.tools/4.3.0/tools/netcoreapp2.1/any/DotnetToolSettings.xml'.
2020-11-20 20:20:50,594 | Tool 'amazon.lambda.tools' was reinstalled with the latest stable version (version '4.3.0').
2020-11-20 20:20:50,594 | DotnetCliPackageBuilder:GlobalToolInstall succeeded
2020-11-20 20:20:50,594 | Running DotnetCliPackageBuilder:RunPackageAction
2020-11-20 20:20:50,594 | Running `dotnet lambda package` in /home/nikko/dev/dntest/src/Function2
2020-11-20 20:20:50,594 | executing dotnet: ['dotnet', 'lambda', 'package', '--output-package', '/home/nikko/dev/dntest/.aws-sam/build/Function2/Function2.zip']
2020-11-20 20:20:52,369 | Amazon Lambda Tools for .NET Core applications (4.3.0)
Project Home: https://github.com/aws/aws-extensions-for-dotnet-cli, https://github.com/aws/aws-lambda-dotnet
Executing publish command
Deleted previous publish folder
... invoking 'dotnet publish', working folder '/home/nikko/dev/dntest/src/Function2/bin/Release/netcoreapp3.1/publish'
... dotnet publish --output "/home/nikko/dev/dntest/src/Function2/bin/Release/netcoreapp3.1/publish" --configuration "Release" --framework "netcoreapp3.1" /p:GenerateRuntimeConfigurationFiles=true --runtime linux-x64 --self-contained false
... publish: Microsoft (R) Build Engine version 16.7.1+52cd83677 for .NET
... publish: Copyright (C) Microsoft Corporation. All rights reserved.
... publish: Determining projects to restore...
... publish: All projects are up-to-date for restore.
... publish: Function -> /home/nikko/dev/dntest/src/Function2/bin/Release/netcoreapp3.1/linux-x64/Function.dll
... publish: Function -> /home/nikko/dev/dntest/src/Function2/bin/Release/netcoreapp3.1/publish/
Changed permissions on published file (chmod +rx Function.pdb).
Changed permissions on published file (chmod +rx Amazon.Lambda.Core.dll).
Changed permissions on published file (chmod +rx Amazon.Lambda.Serialization.SystemTextJson.dll).
Changed permissions on published file (chmod +rx Function.deps.json).
Changed permissions on published file (chmod +rx Function.dll).
Changed permissions on published file (chmod +rx Function.runtimeconfig.json).
Zipping publish folder /home/nikko/dev/dntest/src/Function2/bin/Release/netcoreapp3.1/publish to /home/nikko/dev/dntest/.aws-sam/build/Function2/Function2.zip
Creating directory /home/nikko/dev/dntest/.aws-sam/build/Function2
... zipping: adding: Function.pdb (deflated 41%)
... zipping: adding: Amazon.Lambda.Core.dll (deflated 47%)
... zipping: adding: Amazon.Lambda.Serialization.SystemTextJson.dll (deflated 45%)
... zipping: adding: Function.deps.json (deflated 68%)
... zipping: adding: Function.dll (deflated 53%)
... zipping: adding: Function.runtimeconfig.json (deflated 23%)
Created publish archive (/home/nikko/dev/dntest/.aws-sam/build/Function2/Function2.zip).
Lambda project successfully packaged: /home/nikko/dev/dntest/.aws-sam/build/Function2/Function2.zip
2020-11-20 20:20:52,370 | DotnetCliPackageBuilder:RunPackageAction succeeded
2020-11-20 20:20:52,371 | Exception raised during the execution
Build Failed
2020-11-20 20:20:52,371 | Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'duration': 3498, 'exitReason': 'WorkflowFailedError', 'exitCode': 1, 'requestId': '90433acc-0257-4c44-875e-77b55f7a9d79', 'installationId': 'c6eb94da-0115-444c-9900-00305e965c94', 'sessionId': 'fb99530b-a145-4a34-b0ff-2799ceb50623', 'executionEnvironment': 'CLI', 'pyversion': '3.8.6', 'samcliVersion': '1.11.0'}}]}
2020-11-20 20:20:52,788 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
Error: DotnetCliPackageBuilder:GlobalToolInstall - Error configuring the Amazon.Lambda.Tools .NET Core Global Tool: Dotnet CLI Failed: Tool 'amazon.lambda.tools' failed to update due to the following:
Failed to uninstall tool 'amazon.lambda.tools': Failed to retrieve tool configuration: Could not find a part of the path '/home/nikko/.dotnet/tools/.store/amazon.lambda.tools/4.3.0/amazon.lambda.tools/4.3.0/tools/netcoreapp2.1/any/DotnetToolSettings.xml'.
Expected result:
The build succeeds, similar to running sam build without parallelisation
Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
- OS: Ubuntu 20.04
sam --version: 1.11.0