Start pooling Http2Streams #18601

jkotalik · 2020-01-27T17:02:37Z

Fixes #6192, however, I'd like to discuss the tradeoffs here with this change before adding tests and doing some cleanup work.

The perf generally seems to have improved a slight amount. I've run it multiple times and here are the results I'm seeing.
Before:

RequestsPerSecond:           476,895
Max CPU (%):                 72
WorkingSet (MB):             411
Avg. Latency (ms):           12
Startup (ms):                538
First Request (ms):          128.86
Latency (ms):                0.19
Total Requests:              7,153,420
Duration: (ms)               15,000
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             5,029
Published Size (KB):         105,302
SDK:                         5.0.100-alpha.1.20073.4
Runtime:                     5.0.0-alpha.1.20074.1
ASP.NET Core:                5.0.0-alpha.1.20076.1

Counters:
CPU Usage (%):               71
Working Set (MB):            430
GC Heap Size (MB):           177
Gen 0 GC (#/s):              9
Gen 1 GC (#/s):              4
Gen 2 GC (#/s):              0
Time in GC (%):              2
Gen 0 Size (B):              672
Gen 1 Size (B):              10,007,008
Gen 2 Size (B):              4,207,584
LOH Size (B):                1,294,856
Allocation Rate (B/sec):     2,760,162,909
# of Assemblies Loaded:      112
Exceptions (#/s):            2
ThreadPool Threads Count:    63
Lock Contention (#/s):       81,659
ThreadPool Queue Length:     342
ThreadPool Items (#/s):      481,982

After:

RequestsPerSecond:           487,742
Max CPU (%):                 71
WorkingSet (MB):             434
Avg. Latency (ms):           18.18
Startup (ms):                546
First Request (ms):          202.5
Latency (ms):                0.23
Total Requests:              7,316,135
Duration: (ms)               15,000
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             11,557
Published Size (KB):         105,302
SDK:                         5.0.100-alpha.1.20073.4
Runtime:                     5.0.0-alpha.1.20074.1
ASP.NET Core:                5.0.0-alpha.1.20076.1

Counters:
CPU Usage (%):               70
Working Set (MB):            456
GC Heap Size (MB):           197
Gen 0 GC (#/s):              6
Gen 1 GC (#/s):              3
Gen 2 GC (#/s):              0
Time in GC (%):              1
Gen 0 Size (B):              672
Gen 1 Size (B):              13,007,064
Gen 2 Size (B):              8,319,032
LOH Size (B):                901,472
Allocation Rate (B/sec):     1,862,939,990
# of Assemblies Loaded:      112
Exceptions (#/s):            2
ThreadPool Threads Count:    54
Lock Contention (#/s):       78,246

As you can see, the Allocation Rate has dropped by around 30%. However, there is a lot more idle memory per connection. The GC Generations aren't exactly indicative of the amount of memory, as GCs can happen at any time, however, there is a lot more memory in Gen 1 and Gen 2, presumably due to the pooled streams not being GC'd. cc @anurse @davidfowl @Tratcher @halter73 for your thoughts 😄

I'm curious to see what other people think about pooling strategies to reduce idle memory. Few things I have thought of:

Having a global pool; reduces idle memory but requires lock safe access across multiple connections
Pruning pool over time for long running connections - At that point, why not just let the objects get GC'd

There are also some other wins we could get by pooling other objects; I'll explore them as well.

Tratcher · 2020-01-27T17:14:41Z

As part of this can you share a profile of allocations in this Http2 scenario, before and after? I just want to make sure we're focusing on the largest objects.

src/Servers/Kestrel/Core/src/Internal/Http2/Http2Connection.cs

Tratcher · 2020-01-27T17:28:54Z

src/Servers/Kestrel/Core/src/Internal/Http2/Http2Connection.cs

@@ -67,6 +67,13 @@ internal partial class Http2Connection : IHttp2StreamLifetimeHandler, IHttpHeade
        private int _gracefulCloseInitiator;
        private int _isClosed;

+        private readonly Http2Stream[] _streamPool;


Why not use a Queue or Stack so you don't have to do your own bookkeeping? It could even be a concurrent one which have better locking.

ConcurrentQueueSegment<T> is the ideal one, removing everything from of it that's not needed for TryDequeue and TryEnqueue.

Size must be a power of two; however is ideal for object pooling.

Alas it never was turned into a BoundedConcurrentQueue api

src/Servers/Kestrel/Core/src/Internal/Http2/Http2Connection.cs

jkotalik · 2020-01-27T17:38:37Z

As part of this can you share a profile of allocations in this Http2 scenario, before and after? I just want to make sure we're focusing on the largest objects.

Can you clarify this more? Are you asking for a microbenchmark of a single request and the allocations before and after? Are you asking for a profile specifically on the benchmark server (I'm not sure how to get a profile there).

It seems fairly clear to me that the Http2Streams take up a large portion of allocations. By pooling them, we are saving around 30% on allocations per second (2,760,162,909 -> 1,862,939,990).

jkotalik · 2020-01-27T19:53:02Z

Each Http2Stream allocates 768 bytes per stream itself and retains 5352 bytes. The rest of the ~4500 bytes are coming from Http2OutputProducer, KestrelServerOptions, HttpRequestHeaders, HttpResponseHeaders, and the RequestBodyPipe.

Other areas where we could see improvement with stream allocations include pooling Http2OutputProducers. HttpRequestHeaders/HttpResponseHeaders are resettable, so we shouoldn't need to do anything there. KestrelServerOptions isn't a concern. RequestBodyPipe could be reset instead of allocated each request as well?

jkotalik · 2020-01-27T20:05:27Z

I just realized I had a mistake that was causing HttpRequestHeaders not to be pooled. Now they are; updated numbers:

RequestsPerSecond:           498,680
Max CPU (%):                 70
WorkingSet (MB):             438
Avg. Latency (ms):           15.04
Startup (ms):                544
First Request (ms):          191.65
Latency (ms):                0.22
Total Requests:              7,480,195
Duration: (ms)               15,000
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             5,512
Published Size (KB):         105,302
SDK:                         5.0.100-alpha.1.20073.4
Runtime:                     5.0.0-alpha.1.20074.1
ASP.NET Core:                5.0.0-alpha.1.20076.1

Counters:
CPU Usage (%):               71
Working Set (MB):            459
GC Heap Size (MB):           175
Gen 0 GC (#/s):              5
Gen 1 GC (#/s):              2
Gen 2 GC (#/s):              0
Time in GC (%):              1
Gen 0 Size (B):              672
Gen 1 Size (B):              9,878,856
Gen 2 Size (B):              9,825,528
LOH Size (B):                1,294,856
Allocation Rate (B/sec):     1,487,977,414
# of Assemblies Loaded:      112
Exceptions (#/s):            0
ThreadPool Threads Count:    54
Lock Contention (#/s):       80,881
ThreadPool Queue Length:     234
ThreadPool Items (#/s):      496,790

Continuing to play around with other things that can be pooled.

Tratcher · 2020-01-27T20:38:29Z

That's a lot of test failures. Any chance they're related?

jkotalik · 2020-01-27T22:00:23Z

There are two that are definitely related that I can repro locally. Jasmine ones I doubt are related.

I'm still researching more about which things we should be caching. I'll eventually update this PR with feedback.

src/Servers/Kestrel/Core/src/Internal/Http2/Http2Connection.cs

halter73 · 2020-01-28T23:46:02Z

src/Servers/Kestrel/Core/src/Internal/Http/HttpProtocol.cs

        {
            _context = context;

            ServerOptions = ServiceContext.ServerOptions;
-            HttpRequestHeaders = new HttpRequestHeaders(reuseHeaderValues: !ServerOptions.DisableStringReuse);
+            HttpRequestHeaders.ReuseHeaderValues = !ServerOptions.DisableStringReuse;


ServerOptions.DisableStringReuse can't change, right?

I wasn't certain, hence I did this. It comes from KestrelServerOptions which I believe can't change.

KestrelServerOptions can change, users have access to it, we just never told anybody what would happen if they did so.

Can update Endpoints and Certificates? Spicy 😉

You can try 😁. Let us know how that works out for you.

Incidentally, HttpSys does support live endpoint and cert changes. Kestrel could too but it would be a lot of work.

halter73 · 2020-01-29T01:03:01Z

@aspnet-hello benchmark http2

pr-benchmarks · 2020-01-29T01:03:32Z

Starting 'http2' pipelined plaintext benchmark with session ID 'a5af887470b54a629133311f92d90016'. This could take up to 30 minutes...

pr-benchmarks · 2020-01-29T01:07:59Z

Baseline

Starting baseline run on '2cc333ac3013218a3985ecd0809b0c541cbe1f91'...
RequestsPerSecond:           97,168
Max CPU (%):                 92
WorkingSet (MB):             166
Avg. Latency (ms):           3.23
Startup (ms):                412
First Request (ms):          157.28
Latency (ms):                1.27
Total Requests:              1,457,525
Duration: (ms)               15,000
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             6,009
Published Size (KB):         128,309
SDK:                         5.0.100-alpha.1.20063.3
Runtime:                     5.0.0-alpha.1.20075.5
ASP.NET Core:                5.0.0-alpha.1.20076.1

PR

Starting PR run on '96298771eed8a837c9e67377d30bbdbb4af22593'...
| Description |    RPS | CPU (%) | Memory (MB) | Avg. Latency (ms) | Startup (ms) | Build Time (ms) | Published Size (KB) | First Request (ms) | Latency (ms) | Errors | Ratio |
| ----------- | ------ | ------- | ----------- | ----------------- | ------------ | --------------- | ------------------- | ------------------ | ------------ | ------ | ----- |
|      Before | 97,168 |      92 |         166 |              3.23 |          412 |            6009 |              128309 |             157.28 |         1.27 |      0 |  1.00 |
|       After | 96,170 |      92 |         168 |              3.68 |          455 |            5502 |              128309 |              176.9 |         0.97 |      0 |  0.99 |

jkotalik · 2020-01-29T01:11:13Z

I think I regressed something with my latest change. Investigating locally 😄 .

I will continue to use this though!

halter73 · 2020-01-29T01:12:26Z

It's also certainly possible that we need to tune the parameters or increase the VM size for the benchmark machines. The relevant benchmark changes are in aspnet/Benchmarks@555dae1.

jkotalik · 2020-01-29T01:26:16Z

@aspnet-hello benchmark http2

jkotalik · 2020-01-29T01:32:25Z

@aspnet-hello benchmark http2

benaadams · 2020-01-29T01:39:03Z

@aspnet-hello benchmark http2

sebastienros · 2020-01-29T01:41:15Z

@aspnet-hello benchmark http2

jkotalik · 2020-01-29T01:57:40Z

@halter73 did you hard code your name 😄

jkotalik · 2020-01-29T02:09:59Z

I'm still seeing good perf numbers:
Before:

RequestsPerSecond:           467,633
Max CPU (%):                 74
WorkingSet (MB):             413
Avg. Latency (ms):           12.69
Startup (ms):                513
First Request (ms):          126.65
Latency (ms):                0.21
Total Requests:              7,014,496
Duration: (ms)               15,000
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             7,523
Published Size (KB):         105,302
SDK:                         5.0.100-alpha.1.20073.10
Runtime:                     5.0.0-alpha.1.20074.1
ASP.NET Core:                5.0.0-alpha.1.20076.1
Counters:
CPU Usage (%):               74
Working Set (MB):            432
GC Heap Size (MB):           203
Gen 0 GC (#/s):              9
Gen 1 GC (#/s):              4
Gen 2 GC (#/s):              0
Time in GC (%):              2
Gen 0 Size (B):              672
Gen 1 Size (B):              10,252,368
Gen 2 Size (B):              4,149,488
LOH Size (B):                1,032,600
Allocation Rate (B/sec):     2,766,103,350
# of Assemblies Loaded:      112
Exceptions (#/s):            4
ThreadPool Threads Count:    48
Lock Contention (#/s):       81,982
ThreadPool Queue Length:     463
ThreadPool Items (#/s):      483,318

After:

RequestsPerSecond:           486,982
Max CPU (%):                 67
WorkingSet (MB):             453
Avg. Latency (ms):           15.88
Startup (ms):                539
First Request (ms):          199.24
Latency (ms):                0.18
Total Requests:              7,304,734
Duration: (ms)               15,000
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             5,035
Published Size (KB):         105,302
SDK:                         5.0.100-alpha.1.20073.10
Runtime:                     5.0.0-alpha.1.20074.1
ASP.NET Core:                5.0.0-alpha.1.20076.1

Counters:
CPU Usage (%):               69
Working Set (MB):            474
GC Heap Size (MB):           200
Gen 0 GC (#/s):              5
Gen 1 GC (#/s):              2
Gen 2 GC (#/s):              0
Time in GC (%):              1
Gen 0 Size (B):              672
Gen 1 Size (B):              9,935,888
Gen 2 Size (B):              9,829,000
LOH Size (B):                901,472
Allocation Rate (B/sec):     1,500,760,879
# of Assemblies Loaded:      112
Exceptions (#/s):            3
ThreadPool Threads Count:    53
Lock Contention (#/s):       85,197
ThreadPool Queue Length:     232
ThreadPool Items (#/s):      500,449

pr-benchmarks · 2020-01-31T16:01:31Z

Starting 'grpc' pipelined plaintext benchmark with session ID '9b3eec27e6ef462eae7072dbf3bb0722'. This could take up to 30 minutes...

pr-benchmarks · 2020-01-31T16:03:51Z

Baseline

stdout: Starting baseline run on '2cc333ac3013218a3985ecd0809b0c541cbe1f91'...
Job named 'grpc' not found in the job definition file.


stderr: Baseline benchmark run on '2cc333ac3013218a3985ecd0809b0c541cbe1f91' failed.

PR

jkotalik · 2020-01-31T17:34:56Z

Holding this PR for a bit. For some reason, I'm seeing a ton of exceptions when running the grpc benchmarks.

JamesNK · 2020-02-01T00:49:51Z

There are a couple of gRPC benchmarks that would be useful to run:

Unary call
Server streaming
Ping pong streaming

Streaming benchmarks shouldn't see a difference. They start a connection stream and leave it open for the entire benchmark.

There are also some gRPC benchmarks that send 1MB messages, but the relatively small stream allocation savings wouldn't change those.

JamesNK · 2020-02-01T00:52:24Z

gRPC features to test still work:

Trailers
Stream cancellation. Could be initiated by the client or the server (reset feature on server can be used for this)

jkotalik · 2020-02-07T00:11:56Z

@JamesNK could you help me run these benchmarks on CI? I don't know what parameters I should be passing in for everything.

JamesNK · 2020-02-07T00:44:27Z

I don't know what is involved with running benchmarks on CI.

They are configured here in aspnet/benchmarks: https://github.com/aspnet/Benchmarks/blob/7cf6b39a6472df1108a95429834b25cfd5937e8e/build/scenarios-grpc.sh

Scenarios:

Unary: "-n GrpcUnary-h2load --webHost KestrelSockets $trend $grpcManagedJobs --connections 64 --warmup 5 -m h2c --headers None"
Server streaming: "-n GrpcServerStreaming-GrpcNetClient --webHost KestrelSockets $trend $grpcManagedJobs --connections 64 --warmup 5 -m h2c"
Ping pong streaming: "-n GrpcPingPongStreaming-GrpcNetClient --webHost KestrelSockets $trend $grpcManagedJobs --connections 64 --warmup 5 -m h2c"

Change h2c to h2 for TLS.

Does that help?

jkotalik · 2020-02-08T00:26:33Z

Perf results from gRPC:

GrpcUnary-h2load
Before:

RequestsPerSecond:           230,177
Max CPU (%):                 98
WorkingSet (MB):             453
Avg. Latency (ms):           58.97
Startup (ms):                302
First Request (ms):          243.31
Latency (ms):                0.31
Total Requests:              3,452,657
Duration: (ms)               15,000
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             9,503
Published Size (KB):         99,969
SDK:                         5.0.100-alpha.1.20073.4
Runtime:                     5.0.0-alpha.1.20074.1
ASP.NET Core:                5.0.0-alpha.1.20076.1

Counters:
CPU Usage (%):               98
Working Set (MB):            472
GC Heap Size (MB):           158
Gen 0 GC (#/s):              7
Gen 1 GC (#/s):              0
Gen 2 GC (#/s):              0
Time in GC (%):              0
Gen 0 Size (B):              672
Gen 1 Size (B):              6,938,584
Gen 2 Size (B):              7,334,056
LOH Size (B):                1,208,280
Allocation Rate (B/sec):     2,105,617,066
# of Assemblies Loaded:      119
Exceptions (#/s):            0
ThreadPool Threads Count:    28
Lock Contention (#/s):       84
ThreadPool Queue Length:     10
ThreadPool Items (#/s):      959,331

After:

RequestsPerSecond:           231,598
Max CPU (%):                 99
WorkingSet (MB):             441
Avg. Latency (ms):           40.08
Startup (ms):                351
First Request (ms):          217.59
Latency (ms):                0.24
Total Requests:              3,473,969
Duration: (ms)               15,000
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             4,502
Published Size (KB):         99,969
SDK:                         5.0.100-alpha.1.20073.4
Runtime:                     5.0.0-alpha.1.20074.1
ASP.NET Core:                5.0.0-alpha.1.20076.1

Counters:
CPU Usage (%):               98
Working Set (MB):            460
GC Heap Size (MB):           178
Gen 0 GC (#/s):              4
Gen 1 GC (#/s):              0
Gen 2 GC (#/s):              0
Time in GC (%):              0
Gen 0 Size (B):              672
Gen 1 Size (B):              5,570,024
Gen 2 Size (B):              7,658,080
LOH Size (B):                1,470,536
Allocation Rate (B/sec):     1,308,236,344
# of Assemblies Loaded:      119
Exceptions (#/s):            0
ThreadPool Threads Count:    28
Lock Contention (#/s):       173
ThreadPool Queue Length:     18
ThreadPool Items (#/s):      944,288

GrpcServerStreaming-GrpcNetClient

Before:

RequestsPerSecond:           1,369,132
Max CPU (%):                 100
WorkingSet (MB):             457
Avg. Latency (ms):           0.05
Startup (ms):                299
First Request (ms):          0
Latency (ms):                0
Total Requests:              20,550,671
Duration: (ms)               15,011
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             4,001
Published Size (KB):         99,969
SDK:                         5.0.100-alpha.1.20073.4
Runtime:                     5.0.0-alpha.1.20074.1
ASP.NET Core:                5.0.0-alpha.1.20076.1

Counters:
CPU Usage (%):               99
Working Set (MB):            478
GC Heap Size (MB):           173
Gen 0 GC (#/s):              1
Gen 1 GC (#/s):              0
Gen 2 GC (#/s):              0
Time in GC (%):              0
Gen 0 Size (B):              672
Gen 1 Size (B):              1,238,880
Gen 2 Size (B):              13,987,472
LOH Size (B):                5,404,376
Allocation Rate (B/sec):     346,570,019
# of Assemblies Loaded:      119
Exceptions (#/s):            0
ThreadPool Threads Count:    28
Lock Contention (#/s):       448
ThreadPool Queue Length:     114
ThreadPool Items (#/s):      11,521

After:

RequestsPerSecond:           1,373,063
Max CPU (%):                 100
WorkingSet (MB):             459
Avg. Latency (ms):           0.05
Startup (ms):                327
First Request (ms):          0
Latency (ms):                0
Total Requests:              20,602,810
Duration: (ms)               15,005
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             4,001
Published Size (KB):         99,969
SDK:                         5.0.100-alpha.1.20073.4
Runtime:                     5.0.0-alpha.1.20074.1
ASP.NET Core:                5.0.0-alpha.1.20076.1

Counters:
CPU Usage (%):               99
Working Set (MB):            480
GC Heap Size (MB):           184
Gen 0 GC (#/s):              1
Gen 1 GC (#/s):              0
Gen 2 GC (#/s):              0
Time in GC (%):              0
Gen 0 Size (B):              672
Gen 1 Size (B):              1,300,744
Gen 2 Size (B):              13,794,680
LOH Size (B):                8,158,008
Allocation Rate (B/sec):     354,504,283
# of Assemblies Loaded:      119
Exceptions (#/s):            0
ThreadPool Threads Count:    28
Lock Contention (#/s):       302
ThreadPool Queue Length:     113
ThreadPool Items (#/s):      6,410

Ping pong

Before:

RequestsPerSecond:           304,924
Max CPU (%):                 99
WorkingSet (MB):             458
Avg. Latency (ms):           0.21
Startup (ms):                293
First Request (ms):          0
Latency (ms):                0
Total Requests:              4,575,382
Duration: (ms)               15,006
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             4,001
Published Size (KB):         99,969
SDK:                         5.0.100-alpha.1.20073.4
Runtime:                     5.0.0-alpha.1.20074.1
ASP.NET Core:                5.0.0-alpha.1.20076.1

Counters:
CPU Usage (%):               99
Working Set (MB):            480
GC Heap Size (MB):           148
Gen 0 GC (#/s):              2
Gen 1 GC (#/s):              0
Gen 2 GC (#/s):              0
Time in GC (%):              0
Gen 0 Size (B):              672
Gen 1 Size (B):              1,507,888
Gen 2 Size (B):              13,097,696
LOH Size (B):                3,863,736
Allocation Rate (B/sec):     531,963,859
# of Assemblies Loaded:      119
Exceptions (#/s):            27
ThreadPool Threads Count:    28
Lock Contention (#/s):       280
ThreadPool Queue Length:     8
ThreadPool Items (#/s):      1,314,354

After:

RequestsPerSecond:           299,628
Max CPU (%):                 99
WorkingSet (MB):             453
Avg. Latency (ms):           0.21
Startup (ms):                325
First Request (ms):          0
Latency (ms):                0
Total Requests:              4,495,324
Duration: (ms)               15,003
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             4,001
Published Size (KB):         99,969
SDK:                         5.0.100-alpha.1.20073.4
Runtime:                     5.0.0-alpha.1.20074.1
ASP.NET Core:                5.0.0-alpha.1.20076.1

Counters:
CPU Usage (%):               99
Working Set (MB):            474
GC Heap Size (MB):           189
Gen 0 GC (#/s):              2
Gen 1 GC (#/s):              0
Gen 2 GC (#/s):              0
Time in GC (%):              0
Gen 0 Size (B):              672
Gen 1 Size (B):              1,276,384
Gen 2 Size (B):              14,006,424
LOH Size (B):                2,519,560
Allocation Rate (B/sec):     487,157,944
# of Assemblies Loaded:      119
Exceptions (#/s):            32
ThreadPool Threads Count:    28
Lock Contention (#/s):       282
ThreadPool Queue Length:     1
ThreadPool Items (#/s):      1,204,715

jkotalik · 2020-02-08T00:27:36Z

Overall, not a lot of changes, but the allocations/second have reduced significantly in GrpcUnary-h2load and Ping pong as expected.

jkotalik · 2020-02-08T00:27:59Z

@aspnet-hello benchmark http2

pr-benchmarks · 2020-02-08T00:28:28Z

Starting 'http2' pipelined plaintext benchmark with session ID 'acef10e275b0431c92cf668ce6b6755f'. This could take up to 30 minutes...

pr-benchmarks · 2020-02-08T00:33:41Z

Baseline

Starting baseline run on 'b49dd9869e48145ef6b18b2c4e6e550fb1023ba3'...
RequestsPerSecond:           98,911
Max CPU (%):                 91
WorkingSet (MB):             216
Avg. Latency (ms):           6.97
Startup (ms):                471
First Request (ms):          207.92
Latency (ms):                0.7
Total Requests:              1,483,670
Duration: (ms)               15,010
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             20,510
Published Size (KB):         118,464
SDK:                         5.0.100-preview.1.20106.1
Runtime:                     5.0.0-alpha.1.20103.10
ASP.NET Core:                5.0.0-preview.1.20107.2

PR

Starting PR run on '9f2c3dfad048904ea4f805a64555d08c44d4faf2'...
| Description |     RPS | CPU (%) | Memory (MB) | Avg. Latency (ms) | Startup (ms) | Build Time (ms) | Published Size (KB) | First Request (ms) | Latency (ms) | Errors | Ratio |
| ----------- | ------- | ------- | ----------- | ----------------- | ------------ | --------------- | ------------------- | ------------------ | ------------ | ------ | ----- |
|      Before |  98,911 |      91 |         216 |              6.97 |          471 |           20510 |              118464 |             207.92 |          0.7 |      0 |  1.00 |
|       After | 105,902 |      92 |         196 |               6.8 |          458 |            7003 |              118464 |             172.96 |         0.69 |      0 |  1.07 |

jkotalik · 2020-02-08T00:35:12Z

Overall, seems like the grpc scenarios won't significantly improve and/or stay the same, but the http2 scenario seems to improve a good amount.

Is this good to merge?

JamesNK · 2020-02-08T02:58:32Z

Overall, not a lot of changes, but the allocations/second have reduced significantly in GrpcUnary-h2load and Ping pong as expected.

I wouldn't have expected ping pong to change. It is using the same stream for the whole benchmark. Perhaps allocations/s is slightly lower just because RPS is slightly lower in this run.

I think you posted the same before and after for server streaming. The results are exactly the same.

jkotalik · 2020-02-08T19:20:01Z

I think you posted the same before and after for server streaming. The results are exactly the same.

Updated.

Start pooling http2streams

4deaf2f

jkotalik requested review from analogrelay, halter73 and Tratcher as code owners January 27, 2020 17:02

ghost added the area-servers label Jan 27, 2020

Tratcher reviewed Jan 27, 2020

View reviewed changes

analogrelay added this to the 5.0.0-preview1 milestone Jan 27, 2020

Updating pooling

6007e85

halter73 reviewed Jan 28, 2020

View reviewed changes

src/Servers/Kestrel/Core/src/Internal/Http2/Http2Connection.cs Outdated Show resolved Hide resolved

reset state

41e191e

halter73 reviewed Jan 28, 2020

View reviewed changes

Cleanup

9629877

ReadOnly structs are bad

507d8b3

jkotalik added 3 commits January 31, 2020 14:13

Only reset keep alive and abort in Http2

c5f4a18

Fix tests

8489cf8

Decrease size

ecfcc16

jkotalik force-pushed the jkotalik/poolHttp2Streams branch from 1ee4da5 to ecfcc16 Compare January 31, 2020 23:28

Update Http2StreamStack.cs

9f2c3df

dotnet deleted a comment from pr-benchmarks bot Feb 1, 2020

halter73 approved these changes Feb 8, 2020

View reviewed changes

jkotalik merged commit bc6fb44 into master Feb 9, 2020

jkotalik deleted the jkotalik/poolHttp2Streams branch February 9, 2020 03:28

halter73 mentioned this pull request May 26, 2020

Document supported runtimes dotnet/yarp#182

Merged

halter73 mentioned this pull request Aug 10, 2020

Increase HTTP/2 stream pool max size to 100 #24712

Merged

amcasey added area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions and removed area-runtime labels Jun 6, 2023

github-actions bot locked and limited conversation to collaborators Dec 8, 2023

Start pooling Http2Streams #18601

Start pooling Http2Streams #18601

Uh oh!

Conversation

jkotalik commented Jan 27, 2020

Uh oh!

Tratcher commented Jan 27, 2020

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jkotalik commented Jan 27, 2020

Uh oh!

jkotalik commented Jan 27, 2020

Uh oh!

jkotalik commented Jan 27, 2020

Uh oh!

Tratcher commented Jan 27, 2020

Uh oh!

jkotalik commented Jan 27, 2020

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

halter73 commented Jan 29, 2020

Uh oh!

pr-benchmarks bot commented Jan 29, 2020

Uh oh!

pr-benchmarks bot commented Jan 29, 2020

Baseline

PR

Uh oh!

jkotalik commented Jan 29, 2020

Uh oh!

halter73 commented Jan 29, 2020

Uh oh!

jkotalik commented Jan 29, 2020

Uh oh!

jkotalik commented Jan 29, 2020

Uh oh!

benaadams commented Jan 29, 2020

Uh oh!

sebastienros commented Jan 29, 2020

Uh oh!

jkotalik commented Jan 29, 2020

Uh oh!

jkotalik commented Jan 29, 2020

Uh oh!

pr-benchmarks bot commented Jan 31, 2020

Uh oh!

pr-benchmarks bot commented Jan 31, 2020

Baseline

PR

Uh oh!

jkotalik commented Jan 31, 2020

Uh oh!

JamesNK commented Feb 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JamesNK commented Feb 1, 2020

Uh oh!

jkotalik commented Feb 7, 2020

Uh oh!

JamesNK commented Feb 7, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

JamesNK commented Feb 1, 2020 •

edited

Loading

JamesNK commented Feb 7, 2020 •

edited

Loading

jkotalik commented Feb 8, 2020 •

edited

Loading