"failed to set up "", error Unable to retrieve local repo information" when doing `dep ensure -update`

[tomwilkie]

Mac OS X Seirra.

$ dep ensure -update
solve error: No valid source could be created:
	failed to set up "", error Unable to retrieve local repo information	failed to set up "", error Unable to update repository	failed to set up "", error Unable to update repository	failed to set up "", error Unable to update repository
ensure Solve(): No valid source could be created:
	failed to set up "", error Unable to retrieve local repo information	failed to set up "", error Unable to update repository	failed to set up "", error Unable to update repository	failed to set up "", error Unable to update repository

I tried deleting some of the repos in the cache:

$ for dir in $(find $GOPATH/pkg/dep/sources -maxdepth 1); do echo $dir; git -C $dir remote -v; done 
/Users/twilkie/Documents/pkg/dep/sources/git---git.colasdn.top-kr-logfmt
origin  git://github.com/kr/logfmt (fetch)
origin  git://github.com/kr/logfmt (push)
/Users/twilkie/Documents/pkg/dep/sources/git---git.colasdn.top-sercand-kuberesolver
origin  git://github.com/sercand/kuberesolver (fetch)
origin  git://github.com/sercand/kuberesolver (push)
/Users/twilkie/Documents/pkg/dep/sources/git-gopkg.in-inf.v0
origin  git://github.com/go-inf/inf (fetch)
origin  git://github.com/go-inf/inf (push)
...

But then I got:

$ dep ensure --update
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0x133ed4c]

goroutine 275 [running]:
github.com/golang/dep/vendor/github.com/sdboyer/gps.(*SourceMgr).SyncSourceFor(0xc420018c00, 0xc4201b4240, 0x10, 0x0, 0x0, 0x0, 0x0)
	/Users/twilkie/Documents/src/github.com/golang/dep/vendor/github.com/sdboyer/gps/source_manager.go:461 +0x10c
github.com/golang/dep/vendor/github.com/sdboyer/gps.(*bridge).breakLock.func1(0xc4203a4f00, 0xc4201b4240, 0x10, 0x0, 0x0, 0x1614200, 0xc4203d0640)
	/Users/twilkie/Documents/src/github.com/golang/dep/vendor/github.com/sdboyer/gps/bridge.go:341 +0x68
created by github.com/golang/dep/vendor/github.com/sdboyer/gps.(*bridge).breakLock
	/Users/twilkie/Documents/src/github.com/golang/dep/vendor/github.com/sdboyer/gps/bridge.go:345 +0x240

[tomwilkie]

I'm getting the panic no matter what I do now - I've delete everything & re-inited, and I still get:

$ dep ensure -v -update
Root project is "github.com/weaveworks/cortex"
 27 transitively valid internal packages
 70 external packages imported from 29 projects
(0)   ✓ select (root)
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0x133ed4c]

goroutine 301 [running]:
github.com/golang/dep/vendor/github.com/sdboyer/gps.(*SourceMgr).SyncSourceFor(0xc420018c00, 0xc4201b4260, 0x10, 0x0, 0x0, 0x0, 0x0)
	/Users/twilkie/Documents/src/github.com/golang/dep/vendor/github.com/sdboyer/gps/source_manager.go:461 +0x10c
github.com/golang/dep/vendor/github.com/sdboyer/gps.(*bridge).breakLock.func1(0xc420392f00, 0xc4201b4260, 0x10, 0x0, 0x0, 0x1614200, 0xc42019f040)
	/Users/twilkie/Documents/src/github.com/golang/dep/vendor/github.com/sdboyer/gps/bridge.go:341 +0x68
created by github.com/golang/dep/vendor/github.com/sdboyer/gps.(*bridge).breakLock
	/Users/twilkie/Documents/src/github.com/golang/dep/vendor/github.com/sdboyer/gps/bridge.go:345 +0x240

[tomwilkie]

I had a bit of a play and got it to run without panicing: https://github.com/golang/dep/compare/master...tomwilkie:hack-it-up?expand=1

Not found the root problem yet.

[sdboyer]

Ahhh well we totally are leaking file handles there. That's a big derp. Will fix that right now.

I suspect some of this is fixed by sdboyer/gps#187, which we've yet to merge into dep just yet. (same bug as #250).

However, this is concerning:

$ dep ensure -update
solve error: No valid source could be created:
	failed to set up "", error Unable to retrieve local repo information	failed to set up "", error Unable to update repository	failed to set up "", error Unable to update repository	failed to set up "", error Unable to update repository

I need to work on the error formatting there, for sure, but the big question would be what state the local repos were in that they couldn't be read from. The "Unable to update repository failed to set up" error is emanating from the underlying vcs library that gps uses.

The empty string is obnoxious, and totally just a mistake on my part. sdboyer/gps#192 is to fix up these errors so they're less useless.

[tomwilkie]

I've just opened a PR to improve the error reporting: sdboyer/gps#193 (and one for the FD leak)

[tomwilkie]

I think the root cause was poorly configured git ssh keys on my part. What causes dep to use ssh instead of http?

[tomwilkie]

Okay I see its trying https and ssh... its being really unreliable for me.

[tomwilkie]

Okay the unreliability is too many fds (I added some more debugging to Mastermind/vcs):

solve error: No valid source could be created:
	failed to set up "https://github.com/ugorji/go", error Unable to retrieve local repo information: pipe: too many open files, 	failed to set up "ssh://git@github.com/ugorji/go", error Unable to update repository: pipe: too many open files, 	failed to set up "git://github.com/ugorji/go", error Unable to retrieve local repo information: pipe: too many open files, 	failed to set up "http://github.com/ugorji/go", error Unable to update repository: pipe: too many open files, 
ensure Solve(): No valid source could be created:

[tomwilkie]

Opened Masterminds/vcs#67 for more helpful errors from Mastermind.

[sdboyer]

That change isn't strictly necessary - the data's all made accessible via methods. We could add the Original() bit to gps' unwrapVcsErr(); I think I omitted it initially because it didn't seem to add much, though I don't recall my reasoning now.

[amenzhinsky]

I'm still getting the same error

$ dep init 
Unable to cache golang.org/x/net
Cached gopkg.in/yaml.v2
Cached github.com/go-sql-driver/mysql
Cached github.com/lib/pq
Cached github.com/mattn/go-sqlite3
solve error: Unable to retrieve local repo information: 
Unable to retrieve local repo information: