Skip to content

Feature/dotnet6 lt #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 39 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
21c487c
fixing .net 8 minimal api
Nov 4, 2023
2cea28e
Merge pull request #2 from rventuri76/feature/aws-lambda-web-adapter
rventuri76 Nov 5, 2023
bab15e6
.
Nov 5, 2023
6a1ec03
Merge pull request #3 from rventuri76/feature/aws-lambda-web-adapter
rventuri76 Nov 5, 2023
57e2976
fixing load test with codebuild
Nov 5, 2023
994cd7f
adding sdk 8.0 for code build compilation
Nov 5, 2023
b7d1f45
Merge pull request #4 from rventuri76/feature/aws-lambda-web-adapter
rventuri76 Nov 5, 2023
a97f739
.
Nov 5, 2023
74b5450
fixing typo
Nov 5, 2023
ae7ee98
Merge pull request #6 from rventuri76/feature/aws-lambda-web-adapter
rventuri76 Nov 5, 2023
a8bd2d6
.
Nov 5, 2023
87cd54e
.
Nov 5, 2023
aa881ce
Merge pull request #7 from rventuri76/feature/aws-lambda-web-adapter
rventuri76 Nov 5, 2023
3a3b624
Merge pull request #8 from rventuri76/feature/aws-lambda-web-adapter
rventuri76 Nov 5, 2023
2d9db90
adding net8 minimila api stats after 10 min loadtest
Nov 5, 2023
130cdb5
updating NET8 project with load test script
Nov 10, 2023
e340cec
adding net8 load test in global script
Nov 10, 2023
d0a285a
saving NET8 load test report in artifacts
Nov 10, 2023
d9a422e
updatin net 8 projects
Nov 10, 2023
34efac0
testing net8native
Nov 11, 2023
197715b
.
Nov 11, 2023
ee76535
.
Nov 11, 2023
d7e6d1a
.
Nov 11, 2023
bb4c770
fixing NET8 Native
Nov 12, 2023
7784f4e
fixing net8 native minimal api
Nov 12, 2023
c664217
adding ability to choose .netsdk to be used in codebuild
Nov 12, 2023
13103c6
.
Nov 12, 2023
befff93
.
Nov 12, 2023
2ad45f1
refactoring NET6 projects adding load testing script
Nov 13, 2023
893391b
.
Nov 13, 2023
87ca066
Merge pull request #9 from rventuri76/feature/dotnet8-rc2
rventuri76 Nov 13, 2023
7b7e564
.
Nov 13, 2023
174bc3d
.
Nov 13, 2023
f07d301
adding loadtestcli to generate global html with all results to be exp…
Nov 13, 2023
b50f8b9
.
Nov 13, 2023
7eccd1a
moving loadtestcli to net6
Nov 13, 2023
d9aaed6
fixing net6 container load test
Nov 13, 2023
f0adb25
refactoring load test automation for .net6 projects and removing net6…
Nov 15, 2023
90650b5
.
Nov 16, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 27 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ This implementation uses the new features detailed in [this link](https://aws.am
- Source generation
- Executable assemblies

### Minimal API
### .NET6 Minimal API
There is a single project named ApiBootstrap that contains all the start-up code and API endpoint mapping. The SAM template still deploys a separate function per API endpoint to negate concurrency issues.

It uses the new minimal API hosting model as detailed [here](https://aws.amazon.com/blogs/compute/introducing-the-net-6-runtime-for-aws-lambda/).
Expand All @@ -59,12 +59,6 @@ It uses the new minimal API hosting model as detailed [here](https://aws.amazon.

Same as minimal API but instead of using Amazon.Lambda.AspNetCoreServer.Hosting/Amazon.Lambda.AspNetCoreServer it is based on [Aws Lambda Web Adapter](https://github.com/awslabs/aws-lambda-web-adapter)

### .NET 6 native AOT

The code is compiled natively for either Linux-x86_64 or Linux-ARM64 and then deployed manually to Lambda as a zip file. The SAM deploy can still be used to stand up the API Gateway endpoints and DynamoDb table, but won't be able to deploy native AOT .NET Lambda functions yet. Packages need to be published from Linux, since cross-OS native compilation is not supported yet.

Details for compiling .NET 6 native AOT can be found [here](https://github.com/dotnet/runtimelab/blob/feature/NativeAOT/docs/using-nativeaot/compiling.md)

### .NET 7 Custom Runtime

The code is compiled on a custom runtime and deployed to the provided.al2 Lambda runtime because Lambda doesn't have a .NET 7 runtime. The code is compiled as ReadyToRun and Self-Contained because there is not .NET runtime on provided.al2 to depend on. This type of deployment is expected to be slower than a fully supported Lambda runtime like .NET 6. This sample should be able to be tested with `sam build` and then `sam deploy --guided`.
Expand Down Expand Up @@ -295,28 +289,6 @@ filter @type="REPORT"
<td><b style="color: green">36.71</b></td>
<td>111.28</td>
</tr>
<tr>
<th>Native AOT on ARM64</th>
<td>1277.19</td>
<td>1326.64</td>
<td>1358.84</td>
<td>1367.49</td>
<td><b style="color: green">6.10</b></td>
<td><b style="color: green">9.37</b></td>
<td><b style="color: green">17.97</b></td>
<td>838.78</td>
</tr>
<tr>
<th>Native AOT on X86</th>
<td>466.81</td>
<td>542.86</td>
<td>700.45</td>
<td>730.51</td>
<td><b style="color: green">6.21</b></td>
<td><b style="color: green">11.34</b></td>
<td><b style="color: green">24.69</b></td>
<td>371.16</td>
</tr>
</table>

### .NET 7
Expand Down Expand Up @@ -557,6 +529,32 @@ The .NET 8 benchmarks include the number of cold and warm starts, alongside the
<td><b style="color: green">23.84</b></td>
<td>120.45</td>
</tr>
<tr>
<th>Minimal API on X86</th>
<td>195</td>
<td>1672.85</td>
<td>1737.61</td>
<td>1833.97</td>
<td>1889.15</td>
<td>136,006</td>
<td><b style="color: green">6.30</b></td>
<td><b style="color: green">10.98</b></td>
<td><b style="color: green">26.72</b></td>
<td>124.67</td>
</tr>
<tr>
<th>Minimal API on ARM64</th>
<td>242</td>
<td>1972.79</td>
<td>2049.16</td>
<td>2107.32</td>
<td>2124.55</td>
<td>136,816</td>
<td><b style="color: green">6.01</b></td>
<td><b style="color: green">9.37</b></td>
<td><b style="color: green">24.69</b></td>
<td>331.6</td>
</tr>
</table>

*The .NET 8 native AOT examples need to be compiled on Amazon Linux 2, this is a temporary solution as a pre-cursor to a SAM build image being available for .NET 8.
Expand Down
15 changes: 12 additions & 3 deletions loadtest/codebuild/load-test-buildspec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,23 @@ version: 0.2
phases:
install:
commands:
- npm i artillery -g
- cd loadtest/codebuild
- ./setup.sh
build:
commands:
- export temp1=$(pwd);echo $temp1
- cd loadtest/codebuild
- ./run-all-load-tests.sh
artifacts:
files:
- src/NET6/Report/*
- src/NET6Containers/Report/*
- src/NET6CustomRuntime/Report/*
- src/NET6MinimalAPI/Report/*
- src/NET6MinimalAPIWebAdapter/Report/*
- src/NET6TopLevelStatements/Report/*
- src/NET6WithPowerTools/Report/*
- src/NET8/Report/*
- src/NET8MinimalAPI/Report/*
- src/NET8Native/Report/*
- src/NET8NativeMinimalAPI/Report/*
- loadtestcli/Report/*
name: loadtest-$(date +%Y-%m-%H-%M)
32 changes: 32 additions & 0 deletions loadtest/codebuild/load-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
config:
target: "{{ $processEnvironment.API_URL }}"
http:
timeout : 60
processor: "../generator.js"
phases:
- duration: 60
arrivalRate: 100

scenarios:
- name: "Generate products"
weight: 8
flow:
- function: "generateProduct"
- put:
url: "/{{ Id }}"
headers:
Content-Type: "application/json"
json:
Id: "{{ Id }}"
Name: "{{ Name }}"
Price: "{{ Price }}"
- get:
url: "/{{ Id }}"
- think: 3
- delete:
url: "/{{ Id }}"
- name: "Get products"
weight: 2
flow:
- get:
url: "{{ $processEnvironment.API_URL }}"
115 changes: 111 additions & 4 deletions loadtest/codebuild/run-all-load-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ TEST_DURATIOMN_SEC=60
LOG_INTERVAL_MIN=20
LOG_DELETE=yes
DELETE_STACK=yes
ECR_URI=NotSet

if [ "x${LT_TEST_DURATIOMN_SEC}" != x ];
then
Expand All @@ -23,29 +24,135 @@ then
DELETE_STACK=$LT_DELETE_STACK
fi

if [ "x${LT_ECR_URI}" != x ];
then
ECR_URI=$LT_ECR_URI
fi


echo --------------------------------------------
echo TEST_DURATIOMN_SEC: $TEST_DURATIOMN_SEC
echo LOG_INTERVAL_MIN: $LOG_INTERVAL_MIN
echo LOG_DELETE: $LOG_DELETE
echo DELETE_STACK: $DELETE_STACK
echo ECR_URI: $ECR_URI
echo LT_SNS_TOPIC_ARN: $LT_SNS_TOPIC_ARN
echo --------------------------------------------

if [ "$LT_NET6" != yes ];
then
echo SKIPPING net6 - LT_NET6=$LT_NET6
else
echo "RUNNING load test for net6"
cd ../../src/NET6/
source ./deploy.sh $DELETE_STACK
source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN
fi

if [ "$LT_NET6_CONTAINERS" != yes ];
then
echo SKIPPING net6 containers - LT_NET6_CONTAINERS=$LT_NET6_CONTAINERS
else
echo "RUNNING load test for net6 containers"
cd ../../src/NET6Containers/
source ./deploy.sh $ECR_URI $DELETE_STACK
source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN
fi

if [ "$LT_NET6_CUSTOM" != yes ];
then
echo SKIPPING net6 custom runtime - LT_NET6_CUSTOM=$LT_NET6_CUSTOM
else
echo "RUNNING load test for net6 custom runtime"
cd ../../src/NET6CustomRuntime/
source ./deploy.sh $DELETE_STACK
source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN
fi

if [ "$LT_NET6_MINIMAL_API" != yes ];
then
echo SKIPPING net6 minimal api :$LT_NET6_MINIMAL_API
echo SKIPPING net6 minimal api - LT_NET6_MINIMAL_API=$LT_NET6_MINIMAL_API
else
echo "RUNNING load test for net6 minimal api"
cd ../../src/NET6MinimalAPI/
source ./deploy.sh $DELETE_STACK
source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE
source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN
fi

if [ "$LT_NET6_MINIMAL_API_WEB_ADAPTER" != yes ];
then
echo SKIPPING net6 minimal api web adapter :$LT_NET6_MINIMAL_API_WEB_ADAPTER
echo SKIPPING net6 minimal api web adapter - LT_NET6_MINIMAL_API_WEB_ADAPTER = $LT_NET6_MINIMAL_API_WEB_ADAPTER
else
echo "RUNNING load test for net6 minimal api web adapter"
cd ../../src/NET6MinimalAPIWebAdapter/
source ./deploy.sh $DELETE_STACK
source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE
source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN
fi

if [ "$LT_NET6_TOPLEVEL" != yes ];
then
echo SKIPPING net6 top level - LT_NET6_TOPLEVEL = $LT_NET6_TOPLEVEL
else
echo "RUNNING load test for net6 top level"
cd ../../src/NET6TopLevelStatements/
source ./deploy.sh $DELETE_STACK
source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN
fi

if [ "$LT_NET6_POWERTOOLS" != yes ];
then
echo SKIPPING net6 power tools - LT_NET6_POWERTOOLS = $LT_NET6_POWERTOOLS
else
echo "RUNNING load test for net6 power tools"
cd ../../src/NET6WithPowerTools/
source ./deploy.sh $DELETE_STACK
source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN
fi

if [ "$LT_NET8" != yes ];
then
echo SKIPPING net8 - LT_NET8=$LT_NET8
else
echo "RUNNING load test for net8"
cd ../../src/NET8/
source ./deploy.sh $DELETE_STACK
source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN
fi

if [ "$LT_NET8_MINIMAL_API" != yes ];
then
echo SKIPPING net8 minimal api - LT_NET8_MINIMAL_API=$LT_NET8_MINIMAL_API
else
echo "RUNNING load test for net8 minimal api"
cd ../../src/NET8MinimalAPI/
source ./deploy.sh $DELETE_STACK
source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN
fi

if [ "$LT_NET8_NATIVE" != yes ];
then
echo SKIPPING net8 native - LT_NET8_NATIVE=$LT_NET8_NATIVE
else
echo "RUNNING load test for net8 native"
cd ../../src/NET8Native/
source ./deploy.sh $DELETE_STACK
source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN
fi

if [ "$LT_NET8_NATIVE_MINIMAL_API" != yes ];
then
echo SKIPPING net8 native minimal api LT_NET8_NATIVE_MINIMAL_API=$LT_NET8_NATIVE_MINIMAL_API
else
echo "RUNNING load test for net8 native minimal api"
cd ../../src/NET8NativeMinimalAPI/
source ./deploy.sh $DELETE_STACK
source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN
fi

echo --------------------------------------------
echo GENERATING FULL REPORT
echo --------------------------------------------
cd ../../loadtestcli
dotnet run ../src $LT_SNS_TOPIC_ARN


16 changes: 16 additions & 0 deletions loadtest/codebuild/setup.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#LT_NET_SDK_CHANNEL
#Specifies the source channel for the installation. The possible values are:
# STS: The most recent Standard Term Support release.
# LTS: The most recent Long Term Support release.
# Two-part version in A.B format, representing a specific release (for example, 3.1 or 6.0).
# Three-part version in A.B.Cxx format, representing a specific SDK release (for example, 6.0.1xx or 6.0.2xx). Available since the 5.0 release.
echo --------------------------------------------
echo Installing dotnet runtime version: $LT_NET_SDK_CHANNEL
echo --------------------------------------------

curl -sSL https://dot.net/v1/dotnet-install.sh | bash /dev/stdin --channel $LT_NET_SDK_CHANNEL

echo --------------------------------------------
echo Installing artillery
echo --------------------------------------------
npm i artillery -g
15 changes: 15 additions & 0 deletions loadtestcli/LoadtestCli.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
<Project Sdk="Microsoft.NET.Sdk">

<ItemGroup>
<PackageReference Include="AWSSDK.Core" Version="3.7.300.2" />
<PackageReference Include="AWSSDK.SimpleNotificationService" Version="3.7.300.2" />
</ItemGroup>

<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net6.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>

</Project>
Loading