net/http: StripPrefix regression from fix for #30165

[twmb]

The fix for #30165 breaks code relying on the stripped prefix to elide a leading /. If code relies on equality testing of r.URL.Path after stripping a leading /path/, that code will break due to the new leading /.

I raise this only because I have a test that ensures a response cannot change; that test failed on tip.

What did you do?

https://play.golang.org/p/84ozdbcJ8dm

What did you expect to see?

=== RUN   TestPathStrip
--- PASS: TestPathStrip (0.00s)
PASS

What did you see instead?

--- FAIL: TestPathStrip (0.00s)
    junk_test.go:26: got "/foo" != exp "foo"
FAIL
exit status 1

Does this issue reproduce with the latest release (go1.12.4)?

No, tip only.

System details

go version devel +e40dffe55a Mon Apr 22 23:03:04 2019 +0000 linux/amd64
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/twmb/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/twmb/go"
GOPROXY=""
GORACE=""
GOROOT="/home/twmb/go/go"
GOTMPDIR=""
GOTOOLDIR="/home/twmb/go/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/twmb/testing/go.mod"
GOROOT/bin/go version: go version devel +e40dffe55a Mon Apr 22 23:03:04 2019 +0000 linux/amd64
GOROOT/bin/go tool compile -V: compile version devel +e40dffe55a Mon Apr 22 23:03:04 2019 +0000
uname -sr: Linux 4.15.0-47-generic
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.2 LTS
Release:	18.04
Codename:	bionic
gdb --version: GNU gdb (Ubuntu 8.1-0ubuntu3) 8.1.0.20180409-git

[dmitshur]

I suspect this is an unfortunate but intended consequence of fixing #30165. Code relying on that should be updated not to rely on it. That is, instead of:

http.Handle("/foo/", http.StripPrefix("/foo/", h))

It makes more sense to me to be writing:

http.Handle("/foo/", http.StripPrefix("/foo", h))

That leads to a more consistent behavior. Handlers almost always receive absolute request paths, and trying to handle or support paths that don't begin with a "/" is often going to be problematic or impossible.

But I'll let Brad see if anything else can/should be done. /cc @bradfitz

[twmb]

I agree it's a consequence, but it it is misleading to start with a cleaned path, strip a known prefix, and end up with a result where the full prefix was not stripped. Handlers that are coded to only be behind StripPrefix now have to also know that maybe a slash is not stripped, even if requested.

The original problem exists because the file server is relying on r.URL.Path to be non-empty; the fix should be to check that it is non-empty before indexing into the path. The original code should also likely just be using FileServer.

[dmitshur]

I think the original snippet be simplified to https://play.golang.org/p/qIYFprNaibv.

[twmb]

Is there a decision on this? #30165 changed the behavior in this release cycle, and either a revert should to happen before 1.13, or a behavior change note should be made.

[dmitshur]

A decision hasn't been made, and I agree it needs to be made by 1.13. Adding release-blocker label to make a decision on how to proceed here (to roll back the original fix, find a solution that keeps the original fix without introducing this regression, or to document the change in behavior).

[dmitshur]

The change in CL 161738 unconditionally cleans the path, but to fix the original issue #30165, it would be sufficient to take action only when the post-strip path is the empty string. Cleaning the entire path in seems like it might be too large of a behavior change to make for http.StripPrefix.

I think a safe thing to do here would be to roll back the CL and apply a more targeted fix (perhaps for 1.14).

[dmitshur]

In the past, poor interaction between http.StripPrefix and other parts of http were resolved by keeping http.StripPrefix behavior and changing other code. E.g., see commit 3745716, which changed http.FileServer when it worked poorly under specific http.StripPrefix usage.

By now, the bar for changing http.StripPrefix behavior should be even higher, so I think we need to resolve this by restoring previous http.StripPrefix behavior of not re-inserting leading / in the general case.

We'll roll back the original CL, reopen #30165, and try again.

[gopherbot]

Change https://golang.org/cl/180498 mentions this issue: net/http: roll back "clean the path of the stripped URL by StripPrefix"

[gopherbot]

Change https://golang.org/cl/180499 mentions this issue: net/http: don't panic serving dir in ServeFile with empty Request.URL.Path