net/http: ServeFile panics when StripPrefix over-strips and results in empty path

[ggicci]

What version of Go are you using (go version)?

$ go version
go version go1.11.4 darwin/amd64

Does this issue reproduce with the latest release?

YES

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ggicci/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/ggicci/workspace"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.11.4/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.11.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/ht/hdwl47w16m10r45w89drkkph0000gn/T/go-build233590303=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I did use http.StripPrefix accompanied with http.ServeFile to strip URL prefix and serve a folder. And the HTTP server will panic and the web page will receive nothing if I opened http://localhost:8080/download/.

Here's a code snippet to reproduce the problem:

package main

import "net/http"

var myHandler = http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
	http.ServeFile(rw, r, "./files")
})

func main() {
	http.Handle("/download/", http.StripPrefix("/download/", myHandler))
	http.ListenAndServe(":8080", nil)
}

What did you expect to see?

I should see contents from my files folder. File list or index page content.

What did you see instead?

No HTTP response, but panics information from stderr.

2019/02/11 17:43:17 http: panic serving [::1]:56154: runtime error: index out of range
goroutine 19 [running]:
net/http.(*conn).serve.func1(0xc00008a960)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc00011a000, 0xc000122000, 0x12fdce0, 0xc000010030, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc00011a000, 0xc000122000, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc00011a000, 0xc000122000)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc00011a000, 0xc000122000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc00011a000, 0xc00010c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc00011a000, 0xc00010c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc00011a000, 0xc00010c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc00011a000, 0xc00010c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc00008a960, 0x1300100, 0xc000096200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5
2019/02/11 17:43:17 http: panic serving [::1]:56174: runtime error: index out of range
goroutine 20 [running]:
net/http.(*conn).serve.func1(0xc00008aa00)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc00015e000, 0xc00016c000, 0x12fdce0, 0xc00015a010, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc00015e000, 0xc00016c000, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc00015e000, 0xc00016c000)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc00015e000, 0xc00016c000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc00015e000, 0xc00010c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc00015e000, 0xc00010c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc00015e000, 0xc00010c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc00015e000, 0xc00010c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc00008aa00, 0x1300100, 0xc000096300)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5
2019/02/11 17:43:17 http: panic serving [::1]:56175: runtime error: index out of range
goroutine 21 [running]:
net/http.(*conn).serve.func1(0xc00008aaa0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc000194000, 0xc00010c300, 0x12fdce0, 0xc000088d50, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc000194000, 0xc00010c300, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc000194000, 0xc00010c300)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc000194000, 0xc00010c300)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc000194000, 0xc00010c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc000194000, 0xc00010c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc000194000, 0xc00010c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc000194000, 0xc00010c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc00008aaa0, 0x1300100, 0xc000096480)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5
2019/02/11 17:43:17 http: panic serving [::1]:56176: runtime error: index out of range
goroutine 5 [running]:
net/http.(*conn).serve.func1(0xc0001ae000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc00011a0e0, 0xc000122300, 0x12fdce0, 0xc000010080, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc00011a0e0, 0xc000122300, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc00011a0e0, 0xc000122300)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc00011a0e0, 0xc000122300)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc00011a0e0, 0xc000122100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc00011a0e0, 0xc000122100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc00011a0e0, 0xc000122100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc00011a0e0, 0xc000122100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc0001ae000, 0x1300100, 0xc00005e200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5
2019/02/11 17:44:47 http: panic serving [::1]:56544: runtime error: index out of range
goroutine 35 [running]:
net/http.(*conn).serve.func1(0xc0001c2000)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1746 +0xd0
panic(0x1269ae0, 0x14af4d0)
	/usr/local/Cellar/go/1.11.4/libexec/src/runtime/panic.go:513 +0x1b9
net/http.serveFile(0x12fff00, 0xc00015e0e0, 0xc00016c200, 0x12fdce0, 0xc00015a070, 0x12b95f1, 0x5, 0x0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:586 +0xac9
net/http.ServeFile(0x12fff00, 0xc00015e0e0, 0xc00016c200, 0x12b95ef, 0x7)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/fs.go:681 +0x13f
main.glob..func1(0x12fff00, 0xc00015e0e0, 0xc00016c200)
	/Users/ggicci/workspace/src/ggicci.me/go/reproduce-gohttostripprefix/main.go:6 +0x54
net/http.HandlerFunc.ServeHTTP(0x12cc038, 0x12fff00, 0xc00015e0e0, 0xc00016c200)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.StripPrefix.func1(0x12fff00, 0xc00015e0e0, 0xc00016c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2003 +0x18b
net/http.HandlerFunc.ServeHTTP(0xc00008ecc0, 0x12fff00, 0xc00015e0e0, 0xc00016c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1964 +0x44
net/http.(*ServeMux).ServeHTTP(0x14ba820, 0x12fff00, 0xc00015e0e0, 0xc00016c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2361 +0x127
net/http.serverHandler.ServeHTTP(0xc000091040, 0x12fff00, 0xc00015e0e0, 0xc00016c100)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2741 +0xab
net/http.(*conn).serve(0xc0001c2000, 0x1300100, 0xc0001580c0)
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:1847 +0x646
created by net/http.(*Server).Serve
	/usr/local/Cellar/go/1.11.4/libexec/src/net/http/server.go:2851 +0x2f5

[gopherbot]

Change https://golang.org/cl/161738 mentions this issue: net/http: StipPrefix clean URL path

[gopherbot]

Change https://golang.org/cl/180498 mentions this issue: net/http: roll back "clean the path of the stripped URL by StripPrefix"

[dmitshur]

I investigated this and here's some more information about why this happens.

The URL path for a GET /download/ request currently ends up being the empty string after http.StripPrefix("/download/", myHandler) strips the prefix.

Calling http.ServeFile on a *http.Request with the empty URL path panics in http.serveFile here:

// redirect if the directory name doesn't end in a slash
if d.IsDir() {
	url := r.URL.Path
	if url[len(url)-1] != '/' { // panic here, since url is empty string
		localRedirect(w, r, path.Base(url)+"/")
		return
	}
}

/cc @bradfitz

[gopherbot]

Change https://golang.org/cl/180499 mentions this issue: net/http: don't panic serving dir in ServeFile with empty Request.URL.Path

[dmitshur]

Re-opening this because the fix was rolled back in CL 180498.

For (my) reference, the code where panic happens was added in CL 20128 to fix #13996. It didn't take into account that r.URL.Path can be empty string when http.StripPrefix is used in a way that over-strips the prefix.

When fixing this, need to make sure it won't introduce an infinite or otherwise unwanted redirects. The analysis in the comment #13996 (comment) is good and should be taken into account.