Proxy supports multiple algorithms for load balancing across destinations

[samsp-msft]

Complete the set of common load balancing schemes

• Round Robin Implement Round-Robin Load Balancing #72
• Random Implement Random Load-Balancing #73
• Least loaded of 2 random "The Power of Two Choices" Load-Balancing Algorithm #75

• First healthy endpoint (already implemented)

[Tratcher]

Current implementation:
https://github.com/microsoft/reverse-proxy/blob/be96d64cb0fb732e5a31aebb91e0b9350798d62a/src/ReverseProxy.Core/Service/Proxy/LoadBalancer.cs#L21-L36

Note that load balancing is configurable per backend set. Do we need that level of granularity, or could this be specified server wide?

[analogrelay]

Oh it should definitely be per backend. It's very common for different services to have different load profiles and benefit from different kinds of load balancing. It's also something Envoy configures per-cluster which serves as some prior art.

[Tratcher]

Another one: Least busy
We have the metadata we need for this, it just requires one pass through the list.

[Kahbazi]

Is it possible to add a feature for blue green deployment to load balancer? or is it out of the scope of load balancer?

[Tratcher]

Is it possible to add a feature for blue green deployment to load balancer? or is it out of the scope of load balancer?

I think that's a different feature from load balancing that would be handled earlier at the routing phase. Load balancing assumes all nodes are equivalent, plus or minus health/load.

[analogrelay]

I'm going to break this up into smaller issues for individual algorithms. @Tratcher and I were talking about some of them being fairly easy and high-value for preview 1 (while others are more complicated).

[analogrelay]

Created #72 #73 #74 #75

Closing this out now.