API proposal: IMultiplexedConnectionListenerFactory and company

[jkotalik]

Today, when accepting a connection from an IConnectionTransport, you get a ConnectionContext that has information about the connection. However, for scenarios like QUIC, after you get the connection, you need to accept again in order to get a QUIC stream on said connection. This is fairly awkward and isn't intuitive as the System.IO.PipeReader/PipeWriter will be unusable.

When implementing HTTP/3, right now we have the following:

public interface IConnectionListenerFactory
{
    ValueTask<IConnectionListener> BindAsync(EndPoint endpoint, CancellationToken cancellationToken = default);
}

public interface IConnectionListener : IAsyncDisposable
{
    EndPoint EndPoint { get; }
    ValueTask<ConnectionContext> AcceptAsync(CancellationToken cancellationToken = default);
    ValueTask UnbindAsync(CancellationToken cancellationToken = default);
}

public interface IQuicStreamListenerFeature
{
    ValueTask<ConnectionContext> AcceptAsync(CancellationToken cancellationToken = default);
}

Usage:

IConnectionListener connectionListener = connectionListenerFactory.BindAsync(new IPEndPoint());
ConnectionContext context = await connectionListener.AcceptAsync(); 
IQuicStreamListenerFeature feature = context.ConnectionFeatures.Get<IQuicStreamListenerFeature>();
ConnectionContext streamContext = await feature.AcceptAsync();

I'd like to propose a new API for accepting connections and streams from a multiplexed connection. Though this may only apply to QUIC for now, the notion of streams/connections will probably apply to other scenarios.

public interface IMultiplexedConnectionListenerFactory : IConnectionListenerFactory
{
}
public interface IMultiplexedConnectionListener : IConnectionListener // Need to downcast
{
    ValueTask<MultiplexedConnectionContext> AcceptMultiplexedAsync(CancellationToken cancellationToken = default);
}
public class MultiplexedConnectionContext : ConnectionContext
{
    ValueTask<StreamConnectionContext> AcceptStreamAsync();
    ValueTask<StreamConnectionContext> ConnectStreamAsync(bool unidirectional); // undirectional or bidirectional stream, like half open/half close
}
public class StreamConnectionContext : ConnectionContext
{
    int StreamId { get; }
    bool Unidirectional { get; }
}

Also, we could see how IMultiplexedConnectionListenerFactory without implementing IConnectionListenerFactory.

Usage:

IMultiplexedConnectionListener connectionListener = connectionListenerFactory.BindAsync(new IPEndPoint()) as IMultiplexedConnectionListener;
MultiplexedConnectionContext context = await connectionListener.AcceptAsync(); 
ConnectionContext streamContext = await context.AcceptAsync();

To me this feels a bit cleaner that what we are doing with features.

[jkotalik]

cc @davidfowl @halter73 @scalablecory thoughts?

[halter73]

Should the StreamConnectionContext indicate in what way the stream is unidirectional (i.e send-only or receive-only)?

[jkotalik]

Good point, we may need it to be an enum instead stating which direction.

[halter73]

Does IMultiplexedConnectionListenerFactory exist just so you can tell early what type of listener you'll get from it? It doesn't add any methods, so BindAsync will still return a plain old IConnectionListener which you'd need to cast to a IMultiplexedConnectionListener either way. You might as well save the as-casting until you get the listener.

Or better yet, keep IMultiplexedConnectionListenerFactory but don't have it derive from IConnectionListener at all. Kestrel could inject both kinds of listeners and use the right one where appropriate.

Also, do you think MultiplexedConnectionContext should derive from ConnectionContext? Before we had no choice since it was returned from IConnectionLister.AcceptAsync(). But now that it's returned from IMultiplexedConnectionListener.AcceptMultiplexedAsync(), we could reconsider that. One thing I like about this design is we could avoid having ConnectionContext that would can be neither read from nor written to. Even though the only property you don't want from ConnectionContext is probably the Transport IDuplexPipe, I still think it will be better if the type system prevents people from thinking the Transport IDuplexPipe will work.

Having a StreamConnectionContext that derives from ConnectionContext makes a lot of sense even if we stick with something closer to the old design with a IQuicStreamListenerFeature.

[davidfowl]

It has to be a connection context. Do you really want another middleware pipeline that only works for multiplexed connections?

[jkotalik]

Does IMultiplexedConnectionListenerFactory exist just so you can tell early what type of listener you'll get from it?

As you mentioned, I'd like for kestrel to inject both listener factories. You can resolve IMultiplexedConnectionListenerFactory and IConnectionListenerFactory individually.

Or better yet, keep IMultiplexedConnectionListenerFactory but don't have it derive from IConnectionListener at all. Kestrel could inject both kinds of listeners and use the right one where appropriate.

That could work. To make sure I'm on the same page, it would look like:

public interface IMultiplexedConnectionListenerFactory
{
}
public interface IMultiplexedConnectionListener
{
    ValueTask<MultiplexedConnectionContext> AcceptMultiplexedAsync(CancellationToken cancellationToken = default);
}
public class MultiplexedConnectionContext : ConnectionContext
{
    ValueTask<StreamConnectionContext> AcceptStreamAsync();
    ValueTask<StreamConnectionContext> ConnectStreamAsync(bool unidirectional); // undirectional or bidirectional stream, like half open/half close
}
public class StreamConnectionContext : ConnectionContext
{
    int StreamId { get; }
    Direction Direction{ get; }
}
// Wordsmith this pls.
public enum Direction
{
   Bidirectional,
   UnidirectionalInbound,
   UnidirectionalOutbound
}

And as @davidfowl mentioned, everything we return needs to return a ConnectionContext, so I'll keep both of these deriving from ConnectionContext.

[analogrelay]

Do you really want another middleware pipeline that only works for multiplexed connections?

Spicy question (though I'm not entirely sold either way yet): Is there middleware that can work on both a Multiplexed and a Non-Multiplexed connection?

I suppose I can think of some that just operate on features, such as a filtering middleware that rejects connections based on some conditions (remote IP, TLS cipher selections, etc.).

What about pulling up a new base class or interface that doesn't have a body pipe for those shared middleware to use? We're really early in bedrock, so it's a good time to think about this kind of refactoring (if we're ever going to do it).

public class MultiplexedConnectionContext : ConnectionContext
{
    ValueTask<StreamConnectionContext> AcceptStreamAsync();
    ValueTask<StreamConnectionContext> ConnectStreamAsync(bool unidirectional); // undirectional or bidirectional stream, like half open/half close
}
public class StreamConnectionContext : ConnectionContext
{
    int StreamId { get; }
    Direction Direction{ get; }
}

These APIs are too rigid IMO. QUICv1 is likely to only have the one "attribute" associated with a stream (its directionality), but what if QUICv2 adds a new attribute that should be specified during "Connect"? What about other multiplexed protocols? Will they all have a concept of directionality?

Also, we talked in corefx about using the term Open for streams since there isn't really a connection handshake involved.

[davidfowl]

I suppose I can think of some that just operate on features, such as a filtering middleware that rejects connections based on some conditions (remote IP, TLS cipher selections, etc.).

Maybe we can ask more questions here, What does this mean for SignalR? How would we go about changing it to work in a world where there are multiplexed streams? How far does it need to bleed?

What about pulling up a new base class or interface that doesn't have a body pipe for those shared middleware to use? We're really early in bedrock, so it's a good time to think about this kind of refactoring (if we're ever going to do it).

Make sense, but what would take the interface? Where would it be used?

These APIs are too rigid IMO. QUICv1 is likely to only have the one "attribute" associated with a stream (its directionality), but what if QUICv2 adds a new attribute that should be specified during "Connect"? What about other multiplexed protocols? Will they all have a concept of directionality?

That's what features are for. I think if we're going to do a multiplexed stream type, it needs to be lowest common denominator (like ConnectionContext currently is).

[analogrelay]

That's what features are for. I think if we're going to do a multiplexed stream type, it needs to be lowest common denominator (like ConnectionContext currently is).

Right but the ConnectStreamAsync takes a single argument. How would that scale to other attributes?

[davidfowl]

We have this same problem on the client side BTW, ConnectAsync should take a property bag.