dep ensure fails with grouped write of manifest, lock and vendor: cannot delete #1407

hectorj2f · 2017-11-28T11:02:10Z

What version of `dep` are you using (`dep version`)?

v0.3.2

What `dep` command did you run?

dep ensure -v

I always get this error one out of 2 runs.

(20/22) Wrote github.com/pelletier/[email protected]
(21/22) Wrote gopkg.in/mattn/[email protected]
(22/22) Wrote gopkg.in/cheggaaa/[email protected]
grouped write of manifest, lock and vendor: cannot delete /pkg/src/github.com/project/kubernetes/cli/vendor: remove /pkg/src/github.com/project/kubernetes/cli/vendor/golang.org/x/sys/unix: directory not empty

The text was updated successfully, but these errors were encountered:

sdboyer · 2017-11-28T13:13:10Z

hi! any chance you're running on WSL? #1327

hectorj2f · 2017-11-28T13:14:04Z

Nope, I am running on Ubuntu

sdboyer · 2017-11-28T13:21:19Z

hmm, that's a first.

can you provide more detailed reproduction steps?

hectorj2f · 2017-11-28T13:29:28Z

I have a clean project without any vendor directory, so I run dep ensure. Everything goes smoothly. Then I run it again and I get the mentioned error and I can see a new directory vendor.orig. If I run it a third time, everything works as when I ran it at the beginning.

sdboyer · 2017-11-28T13:33:09Z

"i have a clean project" is not a step i can reproduce. i need to know what imports exist, what's in the Gopkg.toml, etc. reporting the output of dep hash-inputs would probably be sufficient, though if you're working on a public project you could point me to, that would be preferable.

hectorj2f · 2017-11-28T14:54:52Z

"i have a clean project" is not a step i can reproduce

Unfortunately I cannot point you to the our private project.

Output of Gopkg.toml

[[constraint]]
  name = "github.com/mesosphere/dcos-commons"
  version = "0.30.3"

[[constraint]]
  branch = "v3-unstable"
  name = "gopkg.in/alecthomas/kingpin.v3-unstable"

[[constraint]]
  name = "github.com/yudai/gojsondiff"
  version = "v1.0.0"

[[constraint]]
  name = "gopkg.in/cheggaaa/pb.v1"
  version = "v1.0.18"

Output of Gopkg.lock


[[projects]]
  branch = "master"
  name = "github.com/alecthomas/units"
  packages = ["."]
  revision = "2efee857e7cfd4f3d0138cc3cbb1b4966962b93a"

[[projects]]
  name = "github.com/davecgh/go-spew"
  packages = ["spew"]
  revision = "346938d642f2ec3594ed81d874461961cd0faa76"
  version = "v1.1.0"

[[projects]]
  name = "github.com/mattn/go-runewidth"
  packages = ["."]
  revision = "9e777a8366cce605130a531d2cd6363d07ad7317"
  version = "v0.0.2"

[[projects]]
  name = "github.com/mesosphere/dcos-commons"
  packages = ["cli","cli/client","cli/commands","cli/config"]
  revision = "7ae5d99f5e48817e1bdd9a68c3c6179e377ad74a"
  version = "0.30.3"

[[projects]]
  name = "github.com/nicksnyder/go-i18n"
  packages = ["i18n","i18n/bundle","i18n/language","i18n/translation"]
  revision = "0dc1626d56435e9d605a29875701721c54bc9bbd"
  version = "v1.10.0"

[[projects]]
  name = "github.com/pelletier/go-toml"
  packages = ["."]
  revision = "16398bac157da96aa88f98a2df640c7f32af1da2"
  version = "v1.0.1"

[[projects]]
  name = "github.com/pmezard/go-difflib"
  packages = ["difflib"]
  revision = "792786c7400a136282c1664665ae0a8db921c6c2"
  version = "v1.0.0"

[[projects]]
  branch = "master"
  name = "github.com/sergi/go-diff"
  packages = ["diffmatchpatch"]
  revision = "1744e2970ca51c86172c8190fadad617561ed6e7"

[[projects]]
  name = "github.com/stretchr/testify"
  packages = ["assert","require","suite"]
  revision = "69483b4bd14f5845b5a1e55bca19e954e827f1d0"
  version = "v1.1.4"

[[projects]]
  name = "github.com/yudai/gojsondiff"
  packages = ["."]
  revision = "7b1b7adf999dab73a6eb02669c3d82dbb27a3dd6"
  version = "1.0.0"

[[projects]]
  branch = "master"
  name = "github.com/yudai/golcs"
  packages = ["."]
  revision = "ecda9a501e8220fae3b4b600c3db4b0ba22cfc68"

[[projects]]
  branch = "v3-unstable"
  name = "gopkg.in/alecthomas/kingpin.v3-unstable"
  packages = ["."]
  revision = "c9d19ded11d4843b42b60700409da7259460cb65"

[[projects]]
  name = "gopkg.in/cheggaaa/pb.v1"
  packages = ["."]
  revision = "657164d0228d6bebe316fdf725c69f131a50fb10"
  version = "v1.0.18"

[[projects]]
  branch = "v2"
  name = "gopkg.in/yaml.v2"
  packages = ["."]
  revision = "287cf08546ab5e7e37d55a84f7ed3fd1db036de5"

[solve-meta]
  analyzer-name = "dep"
  analyzer-version = 1
  inputs-digest = "0273ecdf202ad698ab023fb64f91dd9e472ed2bbd9e61fafdc17cea113b3d869"
  solver-name = "gps-cdcl"
  solver-version = 1

Output of dep hash-inputs:

dep  hash-inputs
-CONSTRAINTS-
github.com/mesosphere/dcos-commons
svc-^0.30.3
github.com/yudai/gojsondiff
svc-^1.0.0
gopkg.in/alecthomas/kingpin.v3-unstable
b-v3-unstable
gopkg.in/cheggaaa/pb.v1
svc-^1.0.18
-IMPORTS/REQS-
github.com/mesosphere/dcos-commons/cli
github.com/mesosphere/dcos-commons/cli/client
github.com/mesosphere/dcos-commons/cli/commands
github.com/mesosphere/dcos-commons/cli/config
github.com/stretchr/testify/assert
github.com/stretchr/testify/suite
github.com/yudai/gojsondiff
gopkg.in/alecthomas/kingpin.v3-unstable
gopkg.in/cheggaaa/pb.v1
-IGNORES-
-OVERRIDES-
-ANALYZER-
dep
1

hectorj2f · 2017-11-29T10:58:22Z

@sdboyer can you reproduce it ?

xjtian · 2017-11-30T06:28:14Z

I'm running into this issue as well. It seems somewhat non-deterministic, but only happens if I run dep ensure with an existing vendor/ directory. Happens once every 2-3 runs I'd say, with the same error:

cannot delete .../src/PROJECT/vendor: remove .../src/PROJECT/vendor/golang.org/x/sys/unix: directory not empty

System Info:

Ubuntu 16.04 VM, managed by Virtualbox/Vagrant (bento/ubuntu-16.04 box)
MacOS 10.12.6 host
VM environment has DEPNOLOCK=1 set (dep init freeze for hours #947)

Gopkg.toml:

required = ["github.com/golang/protobuf/protoc-gen-go", "github.com/go-swagger/go-swagger/cmd/swagger"]

[[constraint]]
  name = "github.com/go-swagger/go-swagger"
  version = "0.12.0"

[[constraint]]
  branch = "master"
  name = "github.com/go-openapi/strfmt"

[[constraint]]
  branch = "master"
  name = "github.com/golang/glog"

[[constraint]]
  branch = "master"
  name = "github.com/golang/protobuf"

[[constraint]]
  name = "github.com/labstack/echo"
  version = "3.2.5"

[[constraint]]
  branch = "master"
  name = "github.com/lib/pq"

[[constraint]]
  name = "github.com/mattn/go-sqlite3"
  version = "1.4.0"

[[constraint]]
  name = "github.com/prometheus/client_golang"
  version = "0.8.0"

[[constraint]]
  branch = "master"
  name = "github.com/prometheus/client_model"

[[constraint]]
  name = "github.com/stretchr/testify"
  version = "1.1.4"

[[constraint]]
  branch = "master"
  name = "golang.org/x/net"

[[constraint]]
  name = "google.golang.org/grpc"
  version = "1.8.0"

dep hash-inputs:

-CONSTRAINTS-
github.com/go-openapi/strfmt
b-master
github.com/go-swagger/go-swagger
svc-^0.12.0
github.com/golang/glog
b-master
github.com/golang/protobuf
b-master
github.com/labstack/echo
svc-^3.2.5
github.com/lib/pq
b-master
github.com/mattn/go-sqlite3
svc-^1.4.0
github.com/prometheus/client_golang
svc-^0.8.0
github.com/prometheus/client_model
b-master
github.com/stretchr/testify
svc-^1.1.4
golang.org/x/net
b-master
google.golang.org/grpc
svc-^1.8.0
-IMPORTS/REQS-
github.com/go-openapi/errors
github.com/go-openapi/strfmt
github.com/go-openapi/swag
github.com/go-openapi/validate
github.com/go-swagger/go-swagger/cmd/swagger
github.com/golang/glog
github.com/golang/protobuf/jsonpb
github.com/golang/protobuf/proto
github.com/golang/protobuf/protoc-gen-go
github.com/golang/protobuf/ptypes
github.com/golang/protobuf/ptypes/any
github.com/golang/protobuf/ptypes/duration
github.com/golang/protobuf/ptypes/timestamp
github.com/golang/protobuf/ptypes/wrappers
github.com/labstack/echo
github.com/lib/pq
github.com/mattn/go-sqlite3
github.com/prometheus/client_golang/prometheus
github.com/prometheus/client_model/go
github.com/stretchr/testify/assert
github.com/stretchr/testify/mock
golang.org/x/net/context
google.golang.org/grpc
google.golang.org/grpc/codes
google.golang.org/grpc/metadata
google.golang.org/grpc/status
-IGNORES-
-OVERRIDES-
-ANALYZER-
dep
1

Let me know if you need any more info. Thanks for looking into this!

cove · 2018-02-01T17:14:30Z

I'm able to consistently reproduce this issue.

Env:

Windows 10 Pro (10.0.16299 Build 16299)
Docker 17.12.0-ce-win47 (15139) using Linux containers
Go builder container (golang:1.9.2)

Steps:

Download https://github.com/AccelByte/dep-issue-1407
Type "make build" twice

The 2nd build results in the following error:

docker run --rm -e PROJECT_ROOT=depbug -v C:/ab/depbug:/go/src/depbug -w /go/src/depbug -v /var/run/docker.sock:/var/run/docker.sock  depbug dep ensure -v
Gopkg.lock was already in sync with imports and Gopkg.toml
(1/3) Wrote github.com/go-ini/[email protected]
(2/3) Wrote github.com/jmespath/go-jmespath@0b12d6b5
(3/3) Wrote github.com/aws/[email protected]
grouped write of manifest, lock and vendor: cannot delete /go/src/depbug/vendor: remove /go/src/depbug/vendor/github.com/aws/aws-sdk-go/models/apis: directory not empty
make: *** [build] Error 1

djumpen · 2018-05-30T22:18:35Z

@sdboyer Any chance to get answer on this issue?
I faced this issue under vagrant box. On host machine dep works OK, but from under the vagrant box ($GOPATH mounted to Ubuntu box) dep causes errors and creates vendor.orig. The only workaround is to remove vendor folder before every dep ensure

djumpen · 2018-05-31T17:08:10Z

The problem was caused by interaction of Go and virtualbox fs.
NFS is a solution
config.vm.synced_folder "~", "~", :nfs => { :mount_options => ["dmode=777","fmode=777"] }

JesseRhoads · 2018-07-31T13:54:02Z

I have the same environment as xjtian (macos host, virtualbox vm, DEPNOLOCK=1) and this breaks my build every time.
What is bizarre to me is that I can literally delete the files before running ensure and it still hits it, so it's almost like there's an intermediate step in the ensure process that causes this.
And sorry, but removing the vendor directory each time or using NFS mounts are not great solutions.

sinramyeon · 2018-08-27T06:41:49Z

macOs Sierra 10.12.6 dep installed by brew and got same problem

export PATH=$PATH:/usr/local/opt/go/libexec/bin
kimseolhwaui-Mac-mini:engine ksh$ dep ensure
grouped write of manifest, lock and vendor: could not stat file that VerifyVendor claimed existed: stat /Users/ksh/dev/workspace/go/src/engine/vendor/gopkg.in/mgo.v2: no such file or directory

joonas-fi · 2018-08-30T12:26:16Z

Yeah same error here with Win10 + Vagrant-Virtualbox shared folder mounted inside a Docker container in Ubuntu guest.

Virtualbox shared folders have been notoriously buggy (probably file locks) many years for me no matter using Windows 7 or 10 as a host.

* Use s3 head instead of get to retrieve item metadata * remove stow dependency that dep added * Fix dep problems * Remove vendor directory before running dep ensure because of golang/dep#1407

0x7061 · 2018-10-07T02:25:00Z

Same here on Ubuntu, however I feel like it only happens inside Docker!?!?

rebrendov · 2018-11-13T13:16:49Z

Hey, I stumbled on a very similar problem and for me the culprit was the chown root vendor made by one of the tools I used.
Besides that I would check if the user has a write permission for vendor directory.

markzeras · 2018-11-28T13:35:33Z

What solved for me was removing /vendor

kamal-github · 2019-02-21T15:25:13Z

But is this the permanent solution @mrkmat ?

woyoutlz · 2019-02-25T03:28:05Z

solved by removing /vendor

papag00se · 2019-03-19T23:19:34Z

Hey, I stumbled on a very similar problem and for me the culprit was the chown root vendor made by one of the tools I used.
Besides that I would check if the user has a write permission for vendor directory.

This is what worked for me. In my case I had to open Git Bash on Window as Administrator

sdboyer added the bug label Nov 28, 2017

kevinburke closed this as completed Jun 9, 2019

ericbottard mentioned this issue Aug 8, 2019

Add scaler implementation for liiklus kedacore/keda#289

Merged

dep ensure fails with grouped write of manifest, lock and vendor: cannot delete #1407

dep ensure fails with grouped write of manifest, lock and vendor: cannot delete #1407

Comments

hectorj2f commented Nov 28, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What version of dep are you using (dep version)?

What dep command did you run?

sdboyer commented Nov 28, 2017

Uh oh!

hectorj2f commented Nov 28, 2017

Uh oh!

sdboyer commented Nov 28, 2017

Uh oh!

hectorj2f commented Nov 28, 2017

Uh oh!

sdboyer commented Nov 28, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hectorj2f commented Nov 28, 2017

Uh oh!

hectorj2f commented Nov 29, 2017

Uh oh!

xjtian commented Nov 30, 2017

Uh oh!

cove commented Feb 1, 2018

Uh oh!

djumpen commented May 30, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

djumpen commented May 31, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JesseRhoads commented Jul 31, 2018

Uh oh!

sinramyeon commented Aug 27, 2018

Uh oh!

joonas-fi commented Aug 30, 2018

Uh oh!

0x7061 commented Oct 7, 2018

Uh oh!

rebrendov commented Nov 13, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

markzeras commented Nov 28, 2018

Uh oh!

kamal-github commented Feb 21, 2019

Uh oh!

woyoutlz commented Feb 25, 2019

Uh oh!

papag00se commented Mar 19, 2019

Uh oh!

hectorj2f commented Nov 28, 2017 •

edited

Loading

What version of `dep` are you using (`dep version`)?

What `dep` command did you run?

sdboyer commented Nov 28, 2017 •

edited

Loading

djumpen commented May 30, 2018 •

edited

Loading

djumpen commented May 31, 2018 •

edited

Loading

rebrendov commented Nov 13, 2018 •

edited

Loading