Precomputed header bytes #367

benaadams · 2015-11-11T19:12:38Z

Encode all the standard preamble once for all requests

benaadams · 2015-11-11T19:25:12Z

705,348.96 rps -> 1,030,679.42 rps

/cc @davidfowl @halter73

rynowak · 2015-11-11T19:29:35Z

That's some SRS RPS 👍

benaadams · 2015-11-11T19:30:00Z

1,069,312.35 on the 3rd wrk run (with Kestrel warmed up)

benaadams · 2015-11-11T19:38:52Z

If SocketOutput used MemoryPool2 internally and there was a Write(+Async) overload that took (MemoryPoolBlock2 block, int length, ...) and which took ownership of the pooled block (and Released it at write callback); then the copy step at the start of public Task WriteAsync could be skipped for the header preamble.

AspNetSmurfLab · 2015-11-11T21:08:55Z

New numbers from SmurfLab with this PR, up from ~497k previously:

Running 10s test @ http://10.0.0.100:5001/plaintext
  32 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    10.36ms   23.70ms 316.86ms   93.90%
    Req/Sec    18.86k     3.02k   35.23k    77.74%
  5912136 requests in 10.10s, 744.25MB read
  Socket errors: connect 0, read 0, write 337, timeout 0
Requests/sec: 585370.49
Transfer/sec:     73.69MB

benaadams · 2015-11-11T21:14:14Z

Try non pipelined? I'm also seeing an improvement there.

benaadams · 2015-11-11T21:31:31Z

Added precomputing Http Version bytes also

AspNetSmurfLab · 2015-11-11T22:47:14Z

Wow. With the addition of 371b2ad we break 600K:

Running 10s test @ http://10.0.0.100:5001/plaintext
  32 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    11.26ms   25.12ms 265.05ms   92.61%
    Req/Sec    19.57k     3.08k   43.89k    82.55%
  6115424 requests in 10.10s, 769.84MB read
  Socket errors: connect 0, read 0, write 300, timeout 0
Requests/sec: 605502.09
Transfer/sec:     76.22MB

AspNetSmurfLab · 2015-11-11T22:48:10Z

And non-pipelined results:

Running 10s test @ http://10.0.0.100:5001/plaintext
  32 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.52ms    1.35ms  56.63ms   96.06%
    Req/Sec     5.68k   667.38     8.41k    83.08%
  1778313 requests in 10.10s, 223.86MB read
  Socket errors: connect 0, read 0, write 552, timeout 0
Requests/sec: 176086.22
Transfer/sec:     22.17MB

benaadams · 2015-11-12T02:16:24Z

Have one last additional item for Server and Date; just need to sort out tests

AspNetSmurfLab · 2015-11-12T03:27:29Z

Latest results, getting higher:

Running 10s test @ http://10.0.0.100:5001/plaintext
  32 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    10.50ms   23.15ms 230.54ms   92.07%
    Req/Sec    19.83k     2.93k   35.72k    85.37%
  6204867 requests in 10.10s, 781.10MB read
  Socket errors: connect 0, read 0, write 587, timeout 0
Requests/sec: 614363.71
Transfer/sec:     77.34MB

benaadams · 2015-11-12T07:13:40Z

Added Server and Date precomputation; while still allowing swap out/clearing

davidfowl · 2015-11-20T01:59:14Z

src/Microsoft.AspNet.Server.Kestrel/Infrastructure/MemoryPoolIterator2.cs

+            }
+        }
+
+        public void CopyFromAscii(byte[] data)


Why is this called CopyFromAscii?

Thought that's what you said? CopyAsciiFrom?

Ahh; just the string method?

Add ProducingStart and ProducingComplete methods to ISocketOutput. These new methods can help prevent double buffering when encoding.

benaadams · 2015-11-20T19:31:29Z

Rebased on #410 doing integration now

/cc @halter73

lodejard · 2015-11-20T20:03:37Z

I just filed dotnet/extensions#64 we might want to consider.

It's related to this PR in that it would open a channel from app to server for pre-computed ascii byte[] response headers to be written.

davidfowl · 2015-11-20T21:32:18Z

There's some duplication here with the CopyFrom logic. This looks much cleaner with @halter73 's changes

davidfowl · 2015-11-20T21:34:07Z

src/Microsoft.AspNet.Server.Kestrel/Http/Frame.cs

@@ -29,6 +29,17 @@ public partial class Frame : FrameContext, IFrameControl
        private static readonly ArraySegment<byte> _emptyData = new ArraySegment<byte>(new byte[0]);
        private static readonly byte[] _hex = Encoding.ASCII.GetBytes("0123456789abcdef");

+        private static readonly byte[] _bytesConnectionClose = Encoding.ASCII.GetBytes("\r\nConnection: close");
+        private static readonly byte[] _bytesConnectionKeepAlive = Encoding.ASCII.GetBytes("\r\nConnection: keep-alive");


Should move these out of frame into a static class? Similar to reason phrases?

Less clutter?

benaadams · 2015-11-20T21:41:35Z

Seem to be spitting out the right length data but in lots of zero bytes; not sure I've integrated right.

halter73 · 2015-11-20T22:37:58Z

src/Microsoft.AspNet.Server.Kestrel/Http/Frame.cs

+            count += end.CopyFrom(_httpVersion == HttpVersionType.Http1_1 ? _bytesHttpVersion1_1 : _bytesHttpVersion1_0);
+            count += end.CopyFrom(statusBytes);
+            count += _responseHeaders.CopyTo(ref end);
+            count += end.CopyFrom(_bytesEndHeaders, 0, _bytesEndHeaders.Length);


This looks good. I was considering making CopyFrom mutate the iterator. I wonder if CopyTo should also mutate for consistency.

Needed to get the count for the count; though I have 27 failing tests I need to work out.

huh? absolutely CopyFrom should mutate the iterator, doesn't it? CopyTo can be changed to match.

ProducingComplete should only take (end), or (begin,end), but not the (count). The SocketOutput knows what to write based on the original tail's iterator returned, and the complete iterator passed in

the count is for bytesPrecompted rather than traversing the iterator to work it out.

The majority of the cases are where the entire response fits in one block, so having SocketOutput call begin.GetLength(end) will just subtract the index integers.

Yep. The count is only to simplify write-behind buffering calculation. We do traverse the blocks and count how many bytes we're actually writing before calling uv_write.

benaadams · 2015-11-21T00:28:55Z

Rebased on SocketOuput changes; fingers crossed for performance

benaadams · 2015-11-21T00:54:37Z

Looks like it might be a good one, a lot of the allocations drop out

Before

After

benaadams · 2015-11-21T12:48:01Z

RPS isn't great; but CPU is low, so I think there is high lock contention - making some minor adjusts to move work out of the locks.

benaadams · 2015-11-21T15:01:52Z

Reduced the in lock work; but also turned out I had receive side scaling turned off. When I turn that on... Oh my...

  32 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    11.67ms   34.29ms 657.95ms   90.26%
    Req/Sec    40.62k     4.81k   76.06k    77.62%
  77224953 requests in 1.00m, 9.49GB read
Requests/sec: 1284937.56
Transfer/sec:    161.75MB

Best I've seen in my environment yet

halter73 · 2015-11-23T23:09:41Z

Nice. I'm merging this in now. I plan to make a few more changes, like changing the signature of ProducingComplete to not take a count, but that can be done in another PR.

benaadams · 2015-11-23T23:19:49Z

🏁 time to rebase all the things...

dnfclas added the cla-already-signed label Nov 11, 2015

benaadams force-pushed the faster-headers branch from d9cf936 to 6c7684a Compare November 12, 2015 07:36

benaadams mentioned this pull request Nov 12, 2015

Store header values as byte[] #169

Closed

benaadams force-pushed the faster-headers branch 15 times, most recently from 1111def to e9f1a8b Compare November 12, 2015 11:16

benaadams mentioned this pull request Nov 19, 2015

Don't wait to consume the entire request body for Connection: close requests #406

Closed

davidfowl reviewed Nov 20, 2015
View reviewed changes

Use MemoryPool2 for SocketOutput Buffers

2572256

Add ProducingStart and ProducingComplete methods to ISocketOutput. These new methods can help prevent double buffering when encoding.

benaadams force-pushed the faster-headers branch from 93c7e0c to 5befd5c Compare November 20, 2015 19:29

Smarter unpinning of MemoryPoolBlock2 in SocketOutput

0dbf108

benaadams mentioned this pull request Nov 20, 2015

Use MemoryPool2 for SocketOutput Buffers #410

Merged

davidfowl reviewed Nov 20, 2015
View reviewed changes

halter73 reviewed Nov 20, 2015
View reviewed changes

Precomputed header bytes

a3173c4

benaadams force-pushed the faster-headers branch from 85de4fa to 438a707 Compare November 21, 2015 00:23

merge preamble-output, socket-output

feb4040

benaadams force-pushed the faster-headers branch from 438a707 to feb4040 Compare November 21, 2015 04:18

less work in locks

81dba39

benaadams mentioned this pull request Nov 22, 2015

loopback fast-path and zero-copy send (Windows) #415

Closed

halter73 merged commit 81dba39 into aspnet:dev Nov 23, 2015

benaadams deleted the faster-headers branch November 23, 2015 23:19

Precomputed header bytes #367

Precomputed header bytes #367

Uh oh!

Conversation

benaadams commented Nov 11, 2015

Uh oh!

benaadams commented Nov 11, 2015

Uh oh!

rynowak commented Nov 11, 2015

Uh oh!

benaadams commented Nov 11, 2015

Uh oh!

benaadams commented Nov 11, 2015

Uh oh!

AspNetSmurfLab commented Nov 11, 2015

Uh oh!

benaadams commented Nov 11, 2015

Uh oh!

benaadams commented Nov 11, 2015

Uh oh!

AspNetSmurfLab commented Nov 11, 2015

Uh oh!

AspNetSmurfLab commented Nov 11, 2015

Uh oh!

benaadams commented Nov 12, 2015

Uh oh!

AspNetSmurfLab commented Nov 12, 2015

Uh oh!

benaadams commented Nov 12, 2015

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

benaadams commented Nov 20, 2015

Uh oh!

lodejard commented Nov 20, 2015

Uh oh!

davidfowl commented Nov 20, 2015

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

benaadams commented Nov 20, 2015

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

benaadams commented Nov 21, 2015

Uh oh!

benaadams commented Nov 21, 2015

Uh oh!

benaadams commented Nov 21, 2015

Uh oh!

benaadams commented Nov 21, 2015

Uh oh!

halter73 commented Nov 23, 2015

Uh oh!

benaadams commented Nov 23, 2015

Uh oh!

Uh oh!