net/http: add Transport.MaxConnsPerHost knob, including dial-in-progress conns

[trajber]

As far as I know, http2 clients multiplex multiple requests on the same http2 connection.
To test this scenario I created a simple http2 client and server (listed bellow) using go1.6-beta2.
The server handler prints the request.RemoteAddr to ensure that the client is using the same host:port. Until then everything seemed to work properly because all log messages prints the same host and port.
But when I capture the network packets (or just run netstat) I see different client ports in use. The client does not seem multiplexing on the same TCP socket.

I don't know if I'm missing something or if this is a real issue.

Client - http://play.golang.org/p/oKBRCXGbOL
Server - http://play.golang.org/p/AmG6J9qRBv

[bradfitz]

Verified.

With client.go:

package main

import (
    "crypto/tls"
    "io/ioutil"
    "log"
    "net/http"
    "sync"
)

func main() {
    tr := &http.Transport{
        TLSClientConfig:    &tls.Config{InsecureSkipVerify: true},
        DisableCompression: false,
    }
    client := &http.Client{Transport: tr}

    var wg sync.WaitGroup
    for i := 0; i < 10; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            resp, err := client.Get("https://localhost:54321/hello")
            if err != nil {
                log.Println(err)
            }
            defer resp.Body.Close()
            slurp, err := ioutil.ReadAll(resp.Body)
            if err != nil {
                log.Fatal(err)
            }
            log.Printf("Got: %q", slurp)
        }()
    }
    wg.Wait()
}

And server:

package main

import (
    "fmt"
    "log"
    "net/http"
    "time"
)

func HelloServer(w http.ResponseWriter, req *http.Request) {
    time.Sleep(1 * time.Second)
    fmt.Fprintf(w, "You are %v, %v", req.RemoteAddr, req.Proto)
}

func main() {
    http.HandleFunc("/hello", HelloServer)
    err := http.ListenAndServeTLS(":54321", "cert.pem", "key.pem", nil)
    if err != nil {
        log.Fatal("ListenAndServe: ", err)
    }
}

I see that http2 does in fact send all requests to the same connection, but it ends up creating those other connections anyway (even though it never uses them):

bradfitz@laptop ~$ GODEBUG=http2debug=2 go run client.go 2>&1 | grep -E 'creating|Framer' 
2016/01/14 11:49:39 http2: Transport creating client conn to [::1]:54321
2016/01/14 11:49:39 http2: Framer 0xc820182b00: wrote SETTINGS len=18, settings: ENABLE_PUSH=0, INITIAL_WINDOW_SIZE=4194304, MAX_HEADER_LIST_SIZE=10485760
2016/01/14 11:49:39 http2: Framer 0xc820182b00: wrote WINDOW_UPDATE len=4 (conn) incr=1073741824
2016/01/14 11:49:39 http2: Transport creating client conn to [::1]:54321
2016/01/14 11:49:39 http2: Framer 0xc820182dc0: wrote SETTINGS len=18, settings: ENABLE_PUSH=0, INITIAL_WINDOW_SIZE=4194304, MAX_HEADER_LIST_SIZE=10485760
2016/01/14 11:49:39 http2: Framer 0xc820182dc0: wrote WINDOW_UPDATE len=4 (conn) incr=1073741824
2016/01/14 11:49:39 http2: Transport creating client conn to [::1]:54321
2016/01/14 11:49:39 http2: Framer 0xc820183080: wrote SETTINGS len=18, settings: ENABLE_PUSH=0, INITIAL_WINDOW_SIZE=4194304, MAX_HEADER_LIST_SIZE=10485760
2016/01/14 11:49:39 http2: Framer 0xc820183080: wrote WINDOW_UPDATE len=4 (conn) incr=1073741824
2016/01/14 11:49:39 http2: Framer 0xc820182b00: read SETTINGS len=18, settings: MAX_FRAME_SIZE=1048576, MAX_CONCURRENT_STREAMS=250, MAX_HEADER_LIST_SIZE=1048896
2016/01/14 11:49:39 http2: Framer 0xc820182b00: wrote SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Framer 0xc820182b00: wrote HEADERS flags=END_STREAM|END_HEADERS stream=1 len=42
2016/01/14 11:49:39 http2: Framer 0xc820182b00: read SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Transport creating client conn to [::1]:54321
2016/01/14 11:49:39 http2: Framer 0xc820183340: wrote SETTINGS len=18, settings: ENABLE_PUSH=0, INITIAL_WINDOW_SIZE=4194304, MAX_HEADER_LIST_SIZE=10485760
2016/01/14 11:49:39 http2: Framer 0xc820183340: wrote WINDOW_UPDATE len=4 (conn) incr=1073741824
2016/01/14 11:49:39 http2: Transport creating client conn to [::1]:54321
2016/01/14 11:49:39 http2: Framer 0xc820183600: wrote SETTINGS len=18, settings: ENABLE_PUSH=0, INITIAL_WINDOW_SIZE=4194304, MAX_HEADER_LIST_SIZE=10485760
2016/01/14 11:49:39 http2: Framer 0xc820183600: wrote WINDOW_UPDATE len=4 (conn) incr=1073741824
2016/01/14 11:49:39 http2: Framer 0xc820183080: read SETTINGS len=18, settings: MAX_FRAME_SIZE=1048576, MAX_CONCURRENT_STREAMS=250, MAX_HEADER_LIST_SIZE=1048896
2016/01/14 11:49:39 http2: Framer 0xc820183080: wrote SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Framer 0xc820182b00: wrote HEADERS flags=END_STREAM|END_HEADERS stream=3 len=6
2016/01/14 11:49:39 http2: Framer 0xc820183080: read SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Framer 0xc820183340: read SETTINGS len=18, settings: MAX_FRAME_SIZE=1048576, MAX_CONCURRENT_STREAMS=250, MAX_HEADER_LIST_SIZE=1048896
2016/01/14 11:49:39 http2: Framer 0xc820183340: wrote SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Framer 0xc820182b00: wrote HEADERS flags=END_STREAM|END_HEADERS stream=5 len=6
2016/01/14 11:49:39 http2: Framer 0xc820183340: read SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Framer 0xc820182dc0: read SETTINGS len=18, settings: MAX_FRAME_SIZE=1048576, MAX_CONCURRENT_STREAMS=250, MAX_HEADER_LIST_SIZE=1048896
2016/01/14 11:49:39 http2: Framer 0xc820182dc0: wrote SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Framer 0xc820182b00: wrote HEADERS flags=END_STREAM|END_HEADERS stream=7 len=6
2016/01/14 11:49:39 http2: Framer 0xc820182dc0: read SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Transport creating client conn to [::1]:54321
2016/01/14 11:49:39 http2: Framer 0xc8201838c0: wrote SETTINGS len=18, settings: ENABLE_PUSH=0, INITIAL_WINDOW_SIZE=4194304, MAX_HEADER_LIST_SIZE=10485760
2016/01/14 11:49:39 http2: Framer 0xc8201838c0: wrote WINDOW_UPDATE len=4 (conn) incr=1073741824
2016/01/14 11:49:39 http2: Framer 0xc820183600: read SETTINGS len=18, settings: MAX_FRAME_SIZE=1048576, MAX_CONCURRENT_STREAMS=250, MAX_HEADER_LIST_SIZE=1048896
2016/01/14 11:49:39 http2: Framer 0xc820183600: wrote SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Framer 0xc820182b00: wrote HEADERS flags=END_STREAM|END_HEADERS stream=9 len=6
2016/01/14 11:49:39 http2: Framer 0xc820183600: read SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Transport creating client conn to [::1]:54321
2016/01/14 11:49:39 http2: Framer 0xc820183b80: wrote SETTINGS len=18, settings: ENABLE_PUSH=0, INITIAL_WINDOW_SIZE=4194304, MAX_HEADER_LIST_SIZE=10485760
2016/01/14 11:49:39 http2: Framer 0xc820183b80: wrote WINDOW_UPDATE len=4 (conn) incr=1073741824
2016/01/14 11:49:39 http2: Framer 0xc8201838c0: read SETTINGS len=18, settings: MAX_FRAME_SIZE=1048576, MAX_CONCURRENT_STREAMS=250, MAX_HEADER_LIST_SIZE=1048896
2016/01/14 11:49:39 http2: Framer 0xc8201838c0: wrote SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Framer 0xc820182b00: wrote HEADERS flags=END_STREAM|END_HEADERS stream=11 len=6
2016/01/14 11:49:39 http2: Framer 0xc8201838c0: read SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Framer 0xc820183b80: read SETTINGS len=18, settings: MAX_FRAME_SIZE=1048576, MAX_CONCURRENT_STREAMS=250, MAX_HEADER_LIST_SIZE=1048896
2016/01/14 11:49:39 http2: Framer 0xc820183b80: wrote SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Transport creating client conn to [::1]:54321
2016/01/14 11:49:39 http2: Framer 0xc8200da8f0: wrote SETTINGS len=18, settings: ENABLE_PUSH=0, INITIAL_WINDOW_SIZE=4194304, MAX_HEADER_LIST_SIZE=10485760
2016/01/14 11:49:39 http2: Framer 0xc8200da8f0: wrote WINDOW_UPDATE len=4 (conn) incr=1073741824
2016/01/14 11:49:39 http2: Transport creating client conn to [::1]:54321
2016/01/14 11:49:39 http2: Framer 0xc820016420: wrote SETTINGS len=18, settings: ENABLE_PUSH=0, INITIAL_WINDOW_SIZE=4194304, MAX_HEADER_LIST_SIZE=10485760
2016/01/14 11:49:39 http2: Framer 0xc820182b00: wrote HEADERS flags=END_STREAM|END_HEADERS stream=13 len=6
2016/01/14 11:49:39 http2: Framer 0xc820016420: wrote WINDOW_UPDATE len=4 (conn) incr=1073741824
2016/01/14 11:49:39 http2: Framer 0xc820183b80: read SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Framer 0xc820016420: read SETTINGS len=18, settings: MAX_FRAME_SIZE=1048576, MAX_CONCURRENT_STREAMS=250, MAX_HEADER_LIST_SIZE=1048896
2016/01/14 11:49:39 http2: Framer 0xc820016420: wrote SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Framer 0xc820182b00: wrote HEADERS flags=END_STREAM|END_HEADERS stream=15 len=6
2016/01/14 11:49:39 http2: Framer 0xc8200da8f0: read SETTINGS len=18, settings: MAX_FRAME_SIZE=1048576, MAX_CONCURRENT_STREAMS=250, MAX_HEADER_LIST_SIZE=1048896
2016/01/14 11:49:39 http2: Framer 0xc820016420: read SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Framer 0xc8200da8f0: wrote SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Transport creating client conn to [::1]:54321
2016/01/14 11:49:39 http2: Framer 0xc820201760: wrote SETTINGS len=18, settings: ENABLE_PUSH=0, INITIAL_WINDOW_SIZE=4194304, MAX_HEADER_LIST_SIZE=10485760
2016/01/14 11:49:39 http2: Framer 0xc820201760: wrote WINDOW_UPDATE len=4 (conn) incr=1073741824
2016/01/14 11:49:39 http2: Framer 0xc8200da8f0: read SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Framer 0xc820182b00: wrote HEADERS flags=END_STREAM|END_HEADERS stream=17 len=6
2016/01/14 11:49:39 http2: Framer 0xc820201760: read SETTINGS len=18, settings: MAX_FRAME_SIZE=1048576, MAX_CONCURRENT_STREAMS=250, MAX_HEADER_LIST_SIZE=1048896
2016/01/14 11:49:39 http2: Framer 0xc820201760: wrote SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Framer 0xc820201760: read SETTINGS flags=ACK len=0
2016/01/14 11:49:39 http2: Framer 0xc820182b00: wrote HEADERS flags=END_STREAM|END_HEADERS stream=19 len=6
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read HEADERS flags=END_HEADERS stream=1 len=49
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read HEADERS flags=END_HEADERS stream=7 len=4
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read HEADERS flags=END_HEADERS stream=5 len=4
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read HEADERS flags=END_HEADERS stream=13 len=4
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read HEADERS flags=END_HEADERS stream=11 len=4
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read HEADERS flags=END_HEADERS stream=15 len=4
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read HEADERS flags=END_HEADERS stream=19 len=4
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read HEADERS flags=END_HEADERS stream=9 len=4
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read HEADERS flags=END_HEADERS stream=3 len=4
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read HEADERS flags=END_HEADERS stream=17 len=4
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read DATA flags=END_STREAM stream=1 len=29 data="You are [::1]:55709, HTTP/2.0"
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read DATA flags=END_STREAM stream=11 len=29 data="You are [::1]:55709, HTTP/2.0"
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read DATA flags=END_STREAM stream=5 len=29 data="You are [::1]:55709, HTTP/2.0"
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read DATA flags=END_STREAM stream=17 len=29 data="You are [::1]:55709, HTTP/2.0"
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read DATA flags=END_STREAM stream=7 len=29 data="You are [::1]:55709, HTTP/2.0"
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read DATA flags=END_STREAM stream=13 len=29 data="You are [::1]:55709, HTTP/2.0"
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read DATA flags=END_STREAM stream=19 len=29 data="You are [::1]:55709, HTTP/2.0"
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read DATA flags=END_STREAM stream=3 len=29 data="You are [::1]:55709, HTTP/2.0"
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read DATA flags=END_STREAM stream=9 len=29 data="You are [::1]:55709, HTTP/2.0"
2016/01/14 11:49:40 http2: Framer 0xc820182b00: read DATA flags=END_STREAM stream=15 len=29 data="You are [::1]:55709, HTTP/2.0"

[bradfitz]

Oh, right, I remember the issue here.

The fundamental problem is that at the time the http.Transport gets a RoundTrip call, it doesn't know which protocol the peer supports.

If two goroutines ask a URL at the same time, there are two possible best answers, depending on which protocol the peer supports (which we don't know)

If the peer only supports HTTP/1, the best strategy is to start two TCP connects immediately, so both requests can be sent once both TCP setups are complete.

If the peer supports HTTP/2, the best strategy (at least to minimize resources), is to only have 1 TCP connect in flight, wait for it to finish, and then send both requests on the same connection.

Unfortunately, we don't know. And we don't have a place to keep state to learn over time, either (not that we'd necessarily want to).

Various solutions:

1. hard-code popular site hostnames known to support http2. Kinda gross, wastes space, maintenance burden, doesn't scale.
2. always bound max connects at 1. penalizes HTTP/1, but is best for http2.
3. always bound max connects at N. perhaps 2 or 3. Good compromise?
4. do 3 + also learn over time, at least storing the state only in memory? But once you're connected, you tend to stay connected. and you have to bound memory usage somehow. which hosts do you remember support http2? this is too big

We should at least shut down those idle http2 connections once they're connected, though. Right now we just sit on them and they stay idle, running and replying to pings, etc. That's a waste.

[bradfitz]

(re-titled the bug, as this has nothing to do with x/net/http2 which does properly merge connections. The problem is solely in the integration with the http1 Transport)

[trajber]

In some situations we do know if the peer supports http2.
What if http.Transport had an attribute to always bound max conns at 1 (your second solution)? It will be used when we know that the server supports http2.
In other situations the http.Transport tries to discover what the peer supports.

[bradfitz]

In some situations we do know if the peer supports http2.

If you know your peer supports http2, then use https://godoc.org/golang.org/x/net/http2#Transport directly.

What if http.Transport had an attribute to always bound max conns at 1 (your second solution)? It will be used when we know that the server supports http2.

We're not adding any new API surface (no new knobs) this late in the release cycle.

We're probably not even going to bound the number of TCP connects we do for Go 1.6. I have a change to shut down the unneeded connections at least.

[bradfitz]

Sent https://go-review.googlesource.com/#/c/18675/ for closing unneeded connections and https://go-review.googlesource.com/#/c/18676/ to bring it into std, with tests.

[gopherbot]

CL https://golang.org/cl/18676 mentions this issue.

[gopherbot]

CL https://golang.org/cl/18675 mentions this issue.

[bradfitz]

Okay, those two CLs are probably enough for Go 1.6. If it's easy, I might limit the max pending dials per host to 5 or 6, like browsers, but it's not a huge deal if it slips. It's been how it is for ages.