Meeting notes 3/20/2018

[divega]

Attendees: @anpete @ajcvickers @bgrainger @DamianEdwards @sebastienros @divega

(feel free to make edits or reply if you have any clarification to make)

Status

MySqlConnector

A few new optimization added to MySqlConnector between 0.32 and 0.37 (thanks Bradley for joining the call!):

• Fast path for single connection string in the pool
• Opt out from ping to keep idle connections alive
• Allocation reduction

Results are very encouraging:

MySqlConnector	Linux Fortunes best RPS
.NET Core 2.0 Libuv + 0.32.0	19,012
.NET Core 2.0 Libuv + 0.37.0	80,754
.NET Core 2.1 Sockets + 0.32.0	70,626
.NET Core 2.1 Sockets + 0.37.0	129,195

@bgrainger has also implemented a "micro-provider" (homologous to Peregrine) for MySQL that is useful for experimenting with new features and interpolating their potential gains.

Next planned steps:

• Lock free connection pool (~3% improvement expected)
• Prepared and auto-prepared statements (~15% improvement expected)

Npgsql

@roji released 4.0.0 preview1 to NuGet and @sebastienros took it for a spin. These are the results:

Npgsql	Linux Fortunes best RPS
.NET Core 2.0 Libuv + 3.2.7	100,683
.NET Core 2.0 Libuv + 4.0.0	123,990
.NET Core 2.1 Sockets + 3.2.7	152,406
.NET Core 2.1 Sockets + 4.0.0	217,057

With these numbers we should land on Top 10.

Next steps for Npgsql

• @divega to ask for a review on the lock free code
• @sebastienros is looking at some inconsistencies between the results we get in our own runs and TE CI.

Note that for both Npgsql and MySqlConnector:

1. We are still seeing consistently 15-20% faster perf on Windows
2. We can't get .NET Core 2.1 bits into the official benchmarks until we are at least in RC

EF Core

@anpete has checked in a few improvements for EF Core 2.1 that reduce allocations. We don't have results yet.

Query demultiplexing (aka gating, etc.)

We discussed briefly the concept of demultiplexing identical queries issues from different requests into a single query. A couple of variations of this have been raised by @sebastienros and @anpete. Although we don't know if this would be in the spirit of the benchmarks, it sounds like it could be potentially very useful for real-world workloads, and is something we would like to pursue.

[anpete]

The TE folks have confirmed that database request de-duping would violate the spirit of the test ¯\_(ツ)_/¯

[roji]

We are still seeing consistently 15-20% faster perf on Windows

Hmm... Do you guys have a plan for investigating this? Since the numbers above are for Linux, that means that there's an additional 15-20% speedup that can be done... If we manage to get this fixed we could land in an even better place, so I'm guessing this should be pretty high priority?

There's also considerable potential here to optimize things for everyone using dotnet on Linux, not just data.

BTW I still intend to do cross-platform socket benchmarks (#30) but my work list is long and time is short...

We can't get .NET Core 2.1 bits into the official benchmarks until we are at least in RC

I'm assuming that's some sort of TE rule? What does it mean for Npgsql 4.0.0-preview1, should I be planning an RC (or RTM) for when you guys RC?

Lock free connection pool (~3% improvement expected)

That's odd, I saw significantly more when switching to lock-free (7-8% IIRC). In any case, I'll be looking into making my work available as an external component so that maybe it can be shared with the MySQL provider.

Prepared and auto-prepared statements (~15% improvement expected)

Here as well there's some potential for at least some code-copying across... Feel free to involve me when you start working on this @bgrainger.

[roji]

PS opened #39 to track and discuss the 15-20% Windows/Linux gap.

[niemyjski]

This is amazing! Keep up the good work!

[tmds]

@divega The Sockets numbers are higher than Libuv, is that due to comparing .NET Core 2.1 with .NET Core 2.0? Does .NET Core 2.1 Libuv perform even better?

[divega]

@tmds I heard from @DamianEdwards and @sebastienros that sockets is actually faster in this scenario, but switching to sockets doesn't explain the whole delta. There have been various other improvements in .NET Core and ASP.NET Core 2.1.

[tmds]

@tmds I heard from @DamianEdwards and @sebastienros that sockets is actually faster in this scenario

That's surprising and interesting! My expectation for this test is libuv/sockets should perform very similar and libuv should be on top. Can you share the numbers for libuv 2.1?

[roji]

Just to make sure everyone's on the same page, Npgsql itself is of course always on managed sockets, the libuv vs. sockets affects only the web side.

[tmds]

Exactly. That is why I expect similar results since most of the work is the same. My expectation libuv performs a bit better comes from it performing better on all other benchmarks.

[divega]

Can you share the numbers for libuv 2.1?

@sebastienros can you?

[sebastienros]

No, I obviously didn't do that on purpose ;) because I ran them when I built the tables and the differences where about noise level. And also, I chose Libuv for 2.0 and Socket for 2.1 because this is what customers are actually on (decided by default on WebHostBuilder).

[sebastienros]

If you want to see the differences between Libuv and Sockets on common scenarios I can refer you to these charts: https://aka.ms/aspnet/benchmarks

[tmds]

and the differences where about noise level

ok, that matches with my expectations.

We are still seeing consistently 15-20% faster perf on Windows

A large part of this probably is due to Npgsql using managed sockets. If it could leverage the Kestrel Transport, then for Libuv we'd see better performance.

[roji]

We are still seeing consistently 15-20% faster perf on Windows

A large part of this probably is due to Npgsql using managed sockets. If it could leverage the Kestrel Transport, then for Libuv we'd see better performance.

I'm not sure why that should be... Npgsql uses managed sockets on both Windows and Linux, they should be on par unless something's off in the .NET Linux socket implementation... Or it could be somewhere completely different in the runtime that's less optimized on Linux - we keep assuming it's socket-related but that remains to be shown. I'm also told that with the latest 2.1 managed socket performance is supposed to be very similar to libuv (at least on web-only perf).

At some point we did discuss testing Npgsql with libuv. But the problem is that Npgsql is by nature a client networking model - there typically isn't any event loop to hook into...

@sebastienros, do you see the same 15-20% Linux/Windows gap on plaintext, where no database is being used? This is in order to isolate whether the Linux/Windows gap is a result of something Npgsql does or just a general overall platform difference.