How to handle routes with logic to different Clusters?

[donaldgray]

What is the best way to conditionally choose a different Cluster to proxy request to?

For example, ideally I'd like to have an application with 2 Clusters (foo and bar) and a route with "clusterId": "foo" and I want to, depending on some logic applied to an incoming request, use cluster bar. Is this possible?

I'm currently using the Direct Forwarding but that means I lose out on load balancing to Clusters. I can apply any logic I need and change path/destination but without load balancing etc. Is it possible to get access to something that handles this, similar to how IHttpForwarder is used? With this method it looks like I'd need to send the request to another load-balancer for bar and let that do the load balancing.

I had a look at ITransformFactory but that looks like it is transforming requests/responses to/from a specified Cluster, rather than changing the target cluster.

Is this possible?

[Tratcher]

Can you give an example of the selection criteria you'd use to pick the cluster? Today routing must be used to select a cluster, and that's only so flexible.

IHttpForwarder is the most flexible for dynamic decisions. Our load balancing algorithms are pretty simple, you should be able to replicate them:
https://github.com/microsoft/reverse-proxy/blob/219a2fe8ad7e4e2d26ac1456aa3004c31a7f8419/src/ReverseProxy/LoadBalancing/PowerOfTwoChoicesLoadBalancingPolicy.cs#L29-L35

[donaldgray]

Can you give an example of the selection criteria you'd use to pick the cluster?

The example application serves images (on /image/{**req}). The consumer of the application can request various sizes, rotations, crops (according to IIIF Image spec). These requests are proxied to an image-server, which parses the request and generates the desired image.
We also have pregenerated thumbnails for popular sizes. Thumbnails can be requested directly on (on /thumb/{**req}) but I'd like to redirect /image/{**req} if we know we can satisfy the image with a known thumbnail as there's a lot less work going on to satisfy that request.

So, an example config could be:

"Routes": {
  "thumbnail": {
    "ClusterId": "thumb",
    "Match": {
      "Path": "/thumb/{**catch-all}",
    }
  },
  "Clusters": {
    "img": {
      "Destinations": {
        "img/one": { "Address": "http://localhost:5001", },
        "img/two": { "Address": "http://localhost:5002", }
      }
    },
    "thumb": {
      "Destinations": {
        "thumb/one": { "Address": "http://localhost:5011", },
        "thumb/two": { "Address": "http://localhost:5012", }
      }
    }
  }
}

Then something like the Direct Forwarding for implementing logic:

var clusters = endpoints.ServiceProvider.GetService<ICanGiveYouClusters>();

endpoints.Map("/image/{**req}", async httpContext =>
{
    var parsed = ImageRequest.parse(httpContext.Request.Path);
    if (IsRequestForKnownThumbSize(parsed)
    {
        var targetCluster = clusters.GetCluster("thumb");
        await targetCluster.Client.SendAsync(httpContext, requestOptions, transformer);
    }
    else
    {  
        var targetCluster = clusters.GetCluster("img");
        await targetCluster.Client.SendAsync(httpContext, requestOptions, transformer);
    }

Thanks for the link to LB algorithms - I'll have a look at those and replicate.

[Tratcher]

Interesting. One way to achieve that would be a url rewrite before routing.

app.Use((context, next) =>
{
  if (context.Request.Path.StartsWithSegments("/image", out remainder) && IsRequestForKnownThumbSize(remainder))
  {
    context.Request.Path = "/thumb" + remainder;
  }
  return _next();
});

[samsp-msft]

@Tratcher - Should we make ProxyConfigManager public? I could potentially then write code along the lines of:

        public Task ImageRedirect(HttpContext context, Func<Task> next)
        {
            var parsed = ImageRequest.parse(httpContext.Request.Path);
            if (IsRequestForKnownThumbSize(parsed))
            {
                var cfg = context.RequestServices?.GetService(typeof(ProxyConfigManager)) as ProxyConfigManager;
                var proxyFeature = context.Features.Get<IReverseProxyFeature>();
                proxyFeature.AvailableDestinations = cfg._clusters["img"].Destinations.Values.ToList();
            }
            return next();
        }

This step could then be put at the front of the pipeline and would change the list of destinations.

[Tratcher]

Should we make ProxyConfigManager public?

Making the whole config manager public just to get to the clusters seems like exposing too much surface area. We do have IClusterChangeListener to get clusters, but you have to track the resulting list yourself. There's room for improvement there.

You'd also need to update more than the available destinations list, the rest of the request state is still pointing at the config for the other route and cluster. If we want to make this kind of retargeting first class then I think we'd want an API you could call to swap everything together.

[samsp-msft]

I think switching out the clusters is not going to be uncommon (think A/B) scenarios so its something we need to have a better story for. Please excuse any typo’s this message was fat fingered on my phone.

…

________________________________ From: Chris Ross ***@***.***> Sent: Thursday, July 22, 2021 11:24:35 AM To: microsoft/reverse-proxy ***@***.***> Cc: Sam Spencer ***@***.***>; Comment ***@***.***> Subject: Re: [microsoft/reverse-proxy] How to handle routes with logic to different Clusters? (#1159) Should we make ProxyConfigManager public? Making the whole config manager public just to get to the clusters seems like exposing too much surface area. We do have IClusterChangeListener to get destinations, but you have to track the resulting list yourself. There's room for improvement there. You'd also need to update more than the available destinations list, the rest of the request state is still pointing at the config for the other route and cluster. If we want to make this kind of retargeting first class then I think we'd want an API you could call to swap everything together. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Freverse-proxy%2Fissues%2F1159%23issuecomment-885136648&data=04%7C01%7Csam.spencer%40microsoft.com%7C52c6bfe6019948b0eebc08d94d3df5b1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637625750781560700%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4lWUuPfgETjOkBbo4hjtrJKbAcNGNmrgxZMQj5kFV%2FM%3D&reserved=0>, or unsubscribe<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FANC7AWRQYYSKFW4B4NTYLBLTZBO6HANCNFSM5AYG4IZA&data=04%7C01%7Csam.spencer%40microsoft.com%7C52c6bfe6019948b0eebc08d94d3df5b1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637625750781570659%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=zrPH44iNQBls3ovri7x2Ls2v6EY9c5ODfrJfSXobZIs%3D&reserved=0>.

[Tratcher]

True, this dynamic cluster selection is pretty similar to A/B requirements (#126). We might resolve this as a duplicate.

[donaldgray]

One way to achieve that would be a url rewrite before routing.

I tried this out this morning but I have a routing requirement where I want to route to X rather than Y but the path remains the same. For that reason I think I'll use the "Direct Forwarding" approach and mimic any loadbalancing logic.

The dynamic cluster A/B requirements look promising, I'll subscribe to that issue and keep an eye on how it progresses. Also, the final comment on that is about getting customers to help validate the design - I'd be more than happy to help out if I can.

[karelz]

Triage: Interesting idea, we should find out how many more customers would use it. Moving to Backlog for now.