net/textproto: canonicalMIMEHeaderKey returns a "malformed MIME header line:" for header containing /

[kb4rodrigog]

Go version

go1.22.5

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/user/Library/Caches/go-build'
GOENV='/Users/user/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/user/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/user/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.22.5'
GCCGO='gccgo'
AR='ar'
CC='clang'
CXX='clang++'
CGO_ENABLED='1'
GOMOD='/Users/user/projects/project/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/g0/r8qb78z13c3_x62jqrr_6xqm0000gp/T/go-build1410450898=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

I tried to get the mime headers passing an email header

	tr := textproto.NewReader(bufio.NewReader(bytes.NewReader(b)))
	headers, err := tr.ReadMIMEHeader()

What did you see happen?

I got this error:
malformed MIME header line: X-Hdr-a/b

What did you expect to see?

I expect to see this mime header along with the others in the headers object.

The issue happens because the function validHeaderFieldByte is missing the / character in the list of valid ones

https://datatracker.ietf.org/doc/html/rfc2822#section-3.6.8

optional-field  =       field-name ":" unstructured CRLF

field-name      =       1*ftext

ftext           =       %d33-57 /               ; Any character except
                        %d59-126                ;  controls, SP, and
                                                ;  ":".

The validation was already missing the character in version 1.19, but the canonicalMIMEHeaderKey function was not returning the boolean value to indicate as a !ok to the readMIMEHeader function.

// validHeaderFieldByte reports whether b is a valid byte in a header
// field name. RFC 7230 says:
//
//	header-field   = field-name ":" OWS field-value OWS
//	field-name     = token
//	tchar = "!" / "#" / "$" / "%" / "&" / "'" / "*" / "+" / "-" / "." /
//	        "^" / "_" / "`" / "|" / "~" / DIGIT / ALPHA
//	token = 1*tchar
func validHeaderFieldByte(b byte) bool {
	return int(b) < len(isTokenTable) && isTokenTable[b]
}

// canonicalMIMEHeaderKey is like CanonicalMIMEHeaderKey but is
// allowed to mutate the provided byte slice before returning the
// string.
//
// For invalid inputs (if a contains spaces or non-token bytes), a
// is unchanged and a string copy is returned.
func canonicalMIMEHeaderKey(a []byte) string {
	// See if a looks like a header key. If not, return it unchanged.
	for _, c := range a {
		if validHeaderFieldByte(c) {
			continue
		}
		// Don't canonicalize.
		return string(a)
	}
  ...
}

[gabyhelp]

Related Issues and Documentation

• net/mail: characters allowed in RFC 5322 are invalid while parsing email header #58862 (closed)
• net/textproto: can write binary header it can't read #60674
• net/textproto: fails when headers contain invalid headers #53855 (closed)
• net/textproto: ability to skip conversion for canonical format #29965 (closed)
• net/textproto: missing/incorrect arg in error string. #43408
• net/textproto: mime header Get expects canonical header form #6252 (closed)
• net/textproto: CanonicalMIMEHeaderKey panic on strings with spaces #6712 (closed)
• net/textproto: NewConn fails to initialize commonHeader prior to use #46363 (closed)
• net/textproto: missing argument to printf formatting in error string #37467 (closed)
• mime: WordDecoder#DecodeHeader accept space as a part of 'encoded-word' #19417 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[dcormier]

Seeing as the textproto package docs state it implements support for the likes of SMTP, it seems like headers from those relevant RFCs should be supported.

Package textproto implements generic support for text-based request/response protocols in the style of HTTP, NNTP, and SMTP.

[ianlancetaylor]

For #53188 we restricted the set of permitted characters in the textproto package to conform to RFC 7230. We aren't going to go back on that. For the specific case of mail messages we added CL 504416 to permit more characters for mail messages, as part of #58862 and #60332.

I don't think there is anything to do here. Sorry.

[dcormier]

The docs on textproto state it's intended for use with SMTP. If that's not the case, should those docs be updated to reflect that?

[gopherbot]

Change https://go.dev/cl/601555 mentions this issue: net/textproto: document enforcement of RFC 7230 for headers

[gopherbot]

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)

[dmitshur]

Closed by merging CL 601555 (commit 3b9d10c).