yarn install overwrites linked modules with cached version

[lukebennett]

Do you want to request a feature or report a bug?

Bug

What is the current behavior?

Running yarn install for an app that has linked modules results in yarn copying its cached version of each module to the symlinked directory, overwriting the local copy in the process.

If the current behavior is a bug, please provide the steps to reproduce.

I struggled to reproduce this from scratch, it's possible that it's down to a specific combination of factors with my setup. This involves local copies of private/scoped npm modules yarn linked from an application, and referenced in package.json via file:../xxx. The version of the modules yarn has in its cache appears to be from when I first installed and ran it a few days ago (there have been numerous changes to the modules since), so it's possible the problem lies with its caching strategy as much as its installation strategy.

One observation I've made is that the problem occurs even when using yarn install --force to supposedly bypass the cache.

Happy to answer any questions and/or try anything to help pinpoint the problem.

What is the expected behavior?

yarn install should skip linked modules like npm intstall does. Performing the same steps above using npm install does not result in this problem.

Please mention your node.js, yarn and operating system version.

node v6.2.2
yarn v0.16.0
Mac OS X v10.10.5

[bmayen]

This isn't specific to cached packages. I have a published package foo@1.0.0. Locally I link an unpublished package foo@2.0.0. My project has a package.json dependency for foo@2.0.0 and I link to that. When I run yarn after linking, it tries to install foo@2.0.0 from the repo instead of using the linked version and it fails because foo@2.0.0 hasn't been published yet.

[jimdebeer]

really need this

[mstdokumaci]

this is breaking

[haggholm]

This applies even when you install different packages. E.g.

cd @my/local-dep
yarn link
cd ../../@my/main-project
yarn link @my/local-dep
yarn add lodash

Even though I didn’t even explicitly do anything with @my/local-dep, yarn still goes through the modules and installs anything not locally installed, even if it’s linked.

This happens even if I specify --no-lockfile.

It’s a particularly nasty problem because it overwrites my local working copy of @my/local-dep, so it will overwrite anything you haven’t committed yet, effectively erasing local work.

$ yarn --version
0.16.1

[joefiorini]

I've noticed this issue as well. In my testing it seems to be limited to scoped modules, but that may not be true for everyone. My use case:

Our team works in a monorepo. To get shared packages locally, the only way to have an efficient development process is to symlink them. Since we can't save symlinked installs to package.json, we use file: specifiers with relative paths, followed by a library called linklocal that goes finds and symlinks local dependencies.

It's kind of a hack, but it works very well. However, npm shrinkwrap is so painful for us that switching to yarn is extremely advantageous, even if we have to work around a few warts due to early adoption.

This issue is very bad for that above workflow though. Anytime yarn tries to install we run the risk of losing work.

What exactly is the advantage of using the cache for local dependencies? Since they would never hit the network anyway, couldn't yarn just treat anything that's a file: reference as a cache miss and either skip copying it entirely (as npm does) or copy it in from the local dir?

[haggholm]

I’d be much happier if Yarn just left linked dependencies alone—if it’s a symlink, either do nothing on the assumption that I want to work with it as a symlink, or if for some reason that can’t be done, issue an error and refuse to do anything. Even deleting the symlinks and replacing them with normal dependencies would be better—still a nuisance, but at least not as bad as the current behaviour.

1. If I run yarn link, I should never have to worry about Yarn changing my files.
2. There’s no reason why Yarn should be modifying files outside of node_modules (and its own caches).

IMHO, the preferred behaviour would be to leave symlinked directories alone, unless there’s a version mismatch, in which case error out with a descriptive message.

[bmayen]

unless there’s a version mismatch, in which case error out with a descriptive message.

What would be the advantage of this? Why not allow linked packages to be any version? I believe that's how NPM currently works.

[damonmaria]

Having the same issue. It seems like it's replacing the linked module with the published version. As tho it's installing the package into the directory without realising it's a symlinked package. I don't think yarn's cache is the reason.

[joefiorini]

I was able to get around this issue for time being by using yarn link and then not adding the dependencies to my package.json at all. The script that starts our dev server & installs deps for CI/production builds also handles running yarn link for our shared packages, which ensures it happens everywhere we need to build our code.

[wclr]

This is very confusing behaviour of yarn, a big source of hassle, have you managed fix the logic behind it? I believe yarn should not check for cache when dealing with file: modules (it also refers to other non-versioned sources like remote git repo).

[wclr]

I've described inconsistent behaviour or yarn that leads to the issue: #1794

[wclr]

Btw in new 0.17.6 it seems that yarn doesn't replace content of simlinked folder (in node_modules) with version from cache, but installs new dependencies (that changed in linked package.json - which is nice)

Can others confirm that this not an issue anymore?
@lukebennett @joefiorini @haggholm