net/http: Post http://example.com: net/http: HTTP/1 transport connection broken: readLoopPeekFailLocked: EOF

[erikdubbelboer]

Please answer these questions before submitting your issue. Thanks!

1. What version of Go are you using (go version)?

go version devel +d78c84c Mon Apr 25 23:22:56 2016 +0000 linux/amd64

1. What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/erik"
GORACE=""
GOROOT="/home/erik/src/github.com/golang/go"
GOTOOLDIR="/home/erik/src/github.com/golang/go/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build769409056=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"

1. What did you do?

Send HTTP POST requests to a host that seems to close the connection some specific time after the response. We see the error Post http://example.com: net/http: HTTP/1 transport connection broken: readLoopPeekFailLocked: EOF multiple times per second when doing around 100 requests per second.

It's hard to debug on live traffic and I haven't been able to reproduce this in a clean example.

The error seems to come from roundTrip() which means the request is already in flight. But it also comes from readLoopPeekFailLocked() which is called from readLoop() in the case of no expected responses.

I'm guessing the connection is closed after a request. Because pc.readLoopPeekFailLocked(err) is called after err is wrapped in a beforeRespHeaderError{err} the peekErr == io.EOF in readLoopPeekFailLocked() is false and the error gets assigned to pc.closed while the connection is already in use. After that I'm not sure, the connection must have somehow already been chosen for the next request. Otherwise it wouldn't make sense that pc.numExpectedResponses == 0 while we are inside the loop to wait for a response.

Is there any debugging (logging things) I can do on the live traffic to help find this bug?

[bradfitz]

Thanks for the report. I think this should be fixable with the information provided.

[owenthereal]

@bradfitz Is this fixed on master?

[bradfitz]

Not yet. This bug is still open. Commits will be referenced from here.

[gopherbot]

CL https://golang.org/cl/23160 mentions this issue.

[gopherbot]

CL https://golang.org/cl/25153 mentions this issue.