cmd/go: internal error: duplicate loads of xxxxxxx

[pierrre]

I use this command to update all my GOPATH:
go get -v -d -u .../ (or replace ".../" by "all")
It worked with Go 1.5 (and I don't remember to see any error with Go 1.6 beta/rc)

With Go 1.6, it prints this at the end:

internal error: duplicate loads of runtime/internal/sys
internal error: duplicate loads of unsafe
internal error: duplicate loads of runtime/internal/atomic
internal error: duplicate loads of runtime
internal error: duplicate loads of errors
internal error: duplicate loads of internal/race
internal error: duplicate loads of sync/atomic
internal error: duplicate loads of sync
internal error: duplicate loads of io
internal error: duplicate loads of unicode
internal error: duplicate loads of unicode/utf8
internal error: duplicate loads of bytes
internal error: duplicate loads of bufio
internal error: duplicate loads of math
internal error: duplicate loads of strconv
internal error: duplicate loads of reflect
internal error: duplicate loads of encoding/binary
internal error: duplicate loads of syscall
internal error: duplicate loads of time
internal error: duplicate loads of os
internal error: duplicate loads of fmt
internal error: duplicate loads of sort
internal error: duplicate loads of strings
internal error: duplicate loads of path/filepath
internal error: duplicate loads of compress/flate
internal error: duplicate loads of hash
internal error: duplicate loads of path
internal error: duplicate loads of io/ioutil
internal error: duplicate loads of flag
internal error: duplicate loads of log
internal error: duplicate loads of cmd/internal/obj
internal error: duplicate loads of cmd/internal/obj/arm
internal error: duplicate loads of cmd/internal/obj/arm64
internal error: duplicate loads of cmd/internal/obj/mips
internal error: duplicate loads of cmd/internal/obj/ppc64
internal error: duplicate loads of cmd/internal/obj/x86
internal error: duplicate loads of cmd/asm/internal/arch
internal error: duplicate loads of cmd/asm/internal/flags
internal error: duplicate loads of text/scanner
internal error: duplicate loads of cmd/asm/internal/lex
internal error: duplicate loads of cmd/asm/internal/asm
internal error: duplicate loads of math/rand
internal error: duplicate loads of cmd/compile/internal/big
internal error: duplicate loads of cmd/internal/gcprog
internal error: duplicate loads of crypto
internal error: duplicate loads of crypto/md5
internal error: duplicate loads of text/tabwriter
internal error: duplicate loads of runtime/pprof
internal error: duplicate loads of cmd/compile/internal/gc
internal error: duplicate loads of cmd/compile/internal/amd64
internal error: duplicate loads of cmd/compile/internal/arm
internal error: duplicate loads of cmd/compile/internal/arm64
internal error: duplicate loads of cmd/compile/internal/mips64
internal error: duplicate loads of cmd/compile/internal/ppc64
internal error: duplicate loads of cmd/compile/internal/x86
internal error: duplicate loads of hash/crc32
internal error: duplicate loads of cmd/internal/goobj
internal error: duplicate loads of cmd/internal/unvendor/golang.org/x/arch/arm/armasm
internal error: duplicate loads of cmd/internal/unvendor/golang.org/x/arch/x86/x86asm
internal error: duplicate loads of hash/adler32
internal error: duplicate loads of compress/zlib
internal error: duplicate loads of debug/dwarf
internal error: duplicate loads of debug/elf
internal error: duplicate loads of debug/gosym
internal error: duplicate loads of debug/macho
internal error: duplicate loads of debug/pe
internal error: duplicate loads of debug/plan9obj
internal error: duplicate loads of regexp/syntax
internal error: duplicate loads of regexp
internal error: duplicate loads of cmd/internal/objfile
internal error: duplicate loads of crypto/sha1
internal error: duplicate loads of os/exec
internal error: duplicate loads of cmd/link/internal/ld
internal error: duplicate loads of cmd/link/internal/amd64
internal error: duplicate loads of cmd/link/internal/arm
internal error: duplicate loads of cmd/link/internal/arm64
internal error: duplicate loads of cmd/link/internal/mips64
internal error: duplicate loads of cmd/link/internal/ppc64
internal error: duplicate loads of cmd/link/internal/x86
internal error: duplicate loads of compress/gzip
internal error: duplicate loads of cmd/pprof/internal/profile
internal error: duplicate loads of cmd/pprof/internal/plugin
internal error: duplicate loads of encoding
internal error: duplicate loads of encoding/base64
internal error: duplicate loads of unicode/utf16
internal error: duplicate loads of encoding/json
internal error: duplicate loads of html
internal error: duplicate loads of net/url
internal error: duplicate loads of text/template/parse
internal error: duplicate loads of text/template
internal error: duplicate loads of html/template
internal error: duplicate loads of cmd/pprof/internal/report
internal error: duplicate loads of cmd/pprof/internal/svg
internal error: duplicate loads of cmd/pprof/internal/tempfile
internal error: duplicate loads of cmd/pprof/internal/commands
internal error: duplicate loads of cmd/pprof/internal/driver
internal error: duplicate loads of container/list
internal error: duplicate loads of crypto/subtle
internal error: duplicate loads of crypto/cipher
internal error: duplicate loads of crypto/aes
internal error: duplicate loads of crypto/des
internal error: duplicate loads of math/big
internal error: duplicate loads of crypto/elliptic
internal error: duplicate loads of crypto/sha512
internal error: duplicate loads of encoding/asn1
internal error: duplicate loads of crypto/ecdsa
internal error: duplicate loads of crypto/hmac
internal error: duplicate loads of internal/syscall/unix
internal error: duplicate loads of crypto/rand
internal error: duplicate loads of crypto/rc4
internal error: duplicate loads of crypto/rsa
internal error: duplicate loads of crypto/sha256
internal error: duplicate loads of crypto/dsa
internal error: duplicate loads of crypto/x509/pkix
internal error: duplicate loads of encoding/hex
internal error: duplicate loads of encoding/pem
internal error: duplicate loads of internal/singleflight
internal error: duplicate loads of runtime/cgo
internal error: duplicate loads of net
internal error: duplicate loads of crypto/x509
internal error: duplicate loads of crypto/tls
internal error: duplicate loads of internal/golang.org/x/net/http2/hpack
internal error: duplicate loads of mime
internal error: duplicate loads of mime/quotedprintable
internal error: duplicate loads of net/textproto
internal error: duplicate loads of mime/multipart
internal error: duplicate loads of net/http/internal
internal error: duplicate loads of net/http
internal error: duplicate loads of cmd/pprof/internal/fetch
internal error: duplicate loads of cmd/pprof/internal/symbolizer
internal error: duplicate loads of cmd/pprof/internal/symbolz
internal error: duplicate loads of cmd/vet/internal/whitelist
internal error: duplicate loads of container/heap
internal error: duplicate loads of encoding/xml
internal error: duplicate loads of go/token
internal error: duplicate loads of go/scanner
internal error: duplicate loads of go/ast
internal error: duplicate loads of go/doc
internal error: duplicate loads of go/parser
internal error: duplicate loads of go/build
internal error: duplicate loads of go/constant
internal error: duplicate loads of go/printer
internal error: duplicate loads of go/format
internal error: duplicate loads of go/types
internal error: duplicate loads of go/internal/gccgoimporter
internal error: duplicate loads of go/internal/gcimporter
internal error: duplicate loads of go/importer
internal error: duplicate loads of internal/trace
internal error: duplicate loads of os/signal

[ianlancetaylor]

Appears to have been introduced by https://golang.org/cl/18978.

[tmwh]

https://golang.org/cl/18978 introduced changes that fixed the errors appearing earlier then these, so it did not break anything.
The issue here is that go get -u all is trying to update commands too. The commands cache was not clearing and their packages were not rebuilt using new packages cache for imports. That led to duplicated standard packages in the packageList and that's why we were getting these errors. I've added command cache clearing and test for that into https://golang.org/cl/19899

[gopherbot]

CL https://golang.org/cl/19899 mentions this issue.

[gopherbot]

CL https://golang.org/cl/22040 mentions this issue.