x/mod/modfile: parser includes newlines in comment text

[smowton]

NB. this is probably the wrong issue tracker; github.com/golang/mod describes itself as a [mirror] but doesn't say what it's mirroring or where to contact the real maintainers, and googling doesn't reveal any obvious primary repo, so if that's the case please let me know where to refile and/or add a note to the mirror repo's description

What version of Go are you using (go version)?

1.14

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Don't currently have access to that box, but a generic Win32 Github Actions VM

What did you do?

Parse a go.mod file including comments using golang.org/x/mod/modfile

What did you expect to see?

Comments should not include newline characters

What did you see instead?

On Win32, the \r of the CRLF line-ending is included in the comment's Token field. \r characters are mostly skipped (https://github.com/golang/mod/blob/master/modfile/read.go#L504) but the code to specifically find the end of a comment (https://github.com/golang/mod/blob/master/modfile/read.go#L522) does not account for a CRLF line-ending. That could be a deliberate choice (the documentation makes no claim either way), but it doesn't seem likely.

[ianlancetaylor]

CC @jayconrod

[jayconrod]

It looks like this is a regression in v0.3.0. This playground reproduces the problem. It doesn't reproduce with v0.2.0. The documentation indicates the newline should not be included (neither CR nor LF).

This doesn't seem like a Windows-specific issue, though it might have been in v0.2.0. The lexer reads everything up to the LF and includes that in the comment token.

[gopherbot]

Change https://golang.org/cl/240557 mentions this issue: modfile: remove trailing newline from comment tokens

[smowton]

Thanks @jayconrod!