net/http: document NoBody change in release notes

[jasdel]

What version of Go are you using (go version)?

1.8beta1 and devel +2912544

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build416877118=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

Make HTTP request with Go 1.8beta1 and Go tip with a zero length HTTP request body.

• Generic example: https://play.golang.org/p/1IllAJN-_j
• AWS SDK for Go specific example: https://play.golang.org/p/tTBz9ArwgE

What did you expect to see?

GET http://route53.amazonaws.com/2013-04-01/hostedzone HTTP/1.1
Host: route53.amazonaws.com
User-Agent: aws-sdk-go/1.6.1 (go1.7.4; darwin; amd64)
Accept-Encoding: gzip

What did you see instead?

GET http://route53.amazonaws.com/2013-04-01/hostedzone HTTP/1.1
Host: route53.amazonaws.com
User-Agent: aws-sdk-go/1.6.1 (go1.8beta1; darwin; amd64)
Transfer-Encoding: chunked
Accept-Encoding: gzip

0

In aws/aws-sdk-go#984 it was discovered that Go 1.8beta1 and tip will send HTTP requests with Transfer-Ecoding: chunked if the request's body is zero length. This seems to be confusing some web servers that are waiting on content from the client. Eventually the the socket is closed by the server.

I think this is related to a HTTP server waiting for the last-chunk value. I have a feeling that the request content encoded body should be sent as, but not familiar with transfer-encoding spec to have a definitive suggesting.

0  <- chunk-size
0  <- last-chunk

https://tools.ietf.org/html/rfc2616#page-25

With that said, it would be preferable for the previous functionality preserved where transfer-encoding is only used for non-zero length bodies.

[bradfitz]

Whoa, good find! I'm surprised that the nothing in the gauntlet of net/http tests caught that.

Thanks!

[bradfitz]

Oh, sorry, I'm blind. I missed the most important lines of your repro and I got fixated on worrying about something unrelated.

In your repro, you write:

	req, err := http.NewRequest("GET", server.URL, nil)

	body := bytes.NewReader([]byte{})
	req.Body = ioutil.NopCloser(body)

The first line, NewRequest ends up setting req.Body to nil, and req.ContentLength to 0. So far so good.

Then you set Body non-nil. The only part of the http package with magic that uses the type of the Body is NewRequest. At this point on, the http Transport (as used by the Client) just sees that your Request.ContentLength is 0 (which means nothing for client requests) and sees a non-nil Body, so it assumes it might have content.

In Go 1.7 and some earlier releases, but not all, the Transport first did a test-read of 1 byte to determine whether the ContentLength was actually 0, or was only 0 because it was never set by the client.

That auto-byte-sniffing behavior has caused so many problems, we finally removed it in Go 1.8.

It now does exactly what you tell it to.

A new https://beta.golang.org/pkg/net/http/#NoBody variable can be used to signal that your ContentLength of 0 is actually zero, and not just unknown. It's now used by NewRequest internally. So if you did instead:

	body := bytes.NewReader([]byte{})
	req, err := http.NewRequest("GET", server.URL, body)

... then it would work as you expect, since NewRequest would check the size of the body and set NoBody automatically.

I never added this to https://beta.golang.org/doc/go1.8#net_http because I thought it was too much of an internal detail & more of a bug fix. Apparently not. I'll update the docs.

[bradfitz]

I forgot to mention: there is also #13722 open, to support GET-with-body more officially in Go 1.9. Consider this a baby step in that directory. Do you depend on GET with body in general? If not, the safest option for you to support any Go version is to never set Body to non-nil for GET requests.

[bradfitz]

I've sent https://go-review.googlesource.com/34210 as a documentation update.

Is that sufficient?

Let me know if I'm missing something.

[jasdel]

Thanks for the update! This use case the code is setting the body arbitrarily for all requests as they are built. In all GET et co, request no body bytes are expected to be sent. This is why the examples had zero length bodies.

Arguably the code shouldn't be setting the request's Body at all if the length of the bytes is zero, which is a change I should make.

My biggest concern was if users update Go to 1.8 the AWS SDK for Go would start to fail to make requests because an empty body was set on Request.Body, which worked fine in Go 1.7.

[bradfitz]

Maybe I can make an exception for GET requests.

That might be the good opt-in answer to #13722 anyway: for GET requests, require that ContentLength be non-zero for a body to be sent.

Would that work?

[gopherbot]

CL https://golang.org/cl/34210 mentions this issue.