Proxy can act as a Kubernetes Ingress Controller

[galvesribeiro]

Hello folks!

First of all, great work! Congratz for the first preview release! It is indeed something .Net Community was lagging behind and had for too long to deal with clumsy things like nginx.

So, I'm coming to a point where I'd like to make a POC with YARP for a controlled small set of microservices in Kubernetes and that comes to a need to make it an ingestion controller.

With that in mind, I would like to ask if the Kube integration is something you guys plan to have on this repo, on some other under MSFT umbrella, or if it is something that the community will have to implemente for their own needs.

If it is something MSFT will (or already is) develop, ok, all we have todo is wait. If that is something that community will have to build, is that something that would be acceptable on this repo as a PR? If we should go to this PR, I'de like to use this issue to discuss the approach we should take to contribute it.

Thanks! Keep it going!

[cwoolum]

I was just thinking it would be pretty cool to have an operator for this. I don't think it would make sense in this repo though. Also, I don't see there even being a docker image based on this repo since this is only a library.

The operator would probably want to provide guidance on building an image.. maybe a Dockerfile?

[davidfowl]

I think it should be in this repository as another project.

[galvesribeiro]

Yeah, I think that too. The other project can contain the Dockerfile and whatever is required for the controller to run.

[samsp-msft]

Acting as a K8s ingestion controller is something we have been thinking about and is on the backlog, but we don't yet know where it would fit in the timeline.

The first step is what are your requirements?

[galvesribeiro]

@samsp-msft Well, the current proxy functionality that the framework has in the current preview, has everything we need for now.

What would be generally required for people using Kubernetes is an ingress controller that can be configured using kubernetes config objects, like YAML, and that can spin up/down pods that will handle the traffic and at same time, provide configuration to those pods in a transparent way so whenever you change the config of the ingress deployment, it would "replicate" the config to the running proxy pods.

I'll dig into the specifics until this weekend and will post back here some sort of proposal.

Will keep in touch.

[davidfowl]

We have a prototype of Kestrel as an ingress written by @jkotalik so we have a pretty good idea of what's required to integrate YARP.

[jkotalik]

Yeah, I was tempted to port that over soon as a starting piece that can be improved upon. Code is here: https://github.com/jkotalik/kestrel-ingress

[galvesribeiro]

Hey @jkotalik that is awesome :) That follow more or less the design that I had in mind.

Implement it using YARP would be easier. The only thing I'm concerned is the Kubernetes API client... I've being researching multiple clients to use back in the day when I was writing Orleans.Clustering.Kubernetes and all of them was really a mess by either not being maintained or lacking auth/credentials support, or any other problem that made me just create one with the APIs that I need to use.

Unless we plan on having a Microsoft.Extensions.KubernetesClient to be reused and maintained under .Net Foundation (just a joke - we do have a client for pure Docker with Docker.DotNet btw), I think we should do the same, use HttpClientFactory nicities and invoke only the necessary kube APIs for this.

So, regardless of the way we go with it, is this something you wanna tackle and bring the initial stuff in? Or is it something we should discuss more, build a plan and reimplement it leveraging what you have?

[davidfowl]

No we’re not writing a k8s client. The C# client is maintained and we’ll contribute to it as needed. I’m sure @brendandburns would appreciate it. Turns out it would also be continent to have informers implemented for this PR (kubernetes-client/csharp#394) would be helpful @macsux

[galvesribeiro]

Okey, no problem. I dropped that client back in the day I don't remember why but I hope it got better. Glad to know it is having informers support.