Skip to content

Commit e90b4ae

Browse files
Greg Magolanalexeagle
Greg Magolan
authored and
alexeagle
committed
feat: support for multi-linked first party dependencies
Adds the ability to link first party deps to subdirectories and adds a new `links` to `yarn_install` and `npm_install` which simplifies linking first party deps into `yarn_install` and `npm_install` managed `node_modules` trees. The first release of the multi-linker, which landed in 3.2.0, supported linking only 3rd party deps to subdirectories. ------------------------------ This new feature is enabled by new `npm_link` which can add a `LinkablePackageInfo` to any target. If a target already provides a `LinkablePackageInfo`, it provides a new `LinkablePackageInfo` with a new provided values for `package_path` and `package_name` which informs the linker into which `${package_path}/node_modules/${package_name}` folder to link the package into. For example, given ``` lib_a/BUILD.bazel: js_library( name = "lib_a", srcs = [ "index.js", "package.json", ], package_name = "@somescope/lib-a" ) ``` which makes `//lib_a:lib_a` link to the root node_modules at `node_modules/@somescope/lib-a` `npm_link` can be used to create a new `lib_a` target that links elsewhere: ``` sub/BUILD.bazel: npm_link( name = "lib_a", target = "//lib_a", package_path = "sub", package_name = "@somescope/lib-a", ) ``` which makes `//sub:lib_a` linked to `sub/node_modules/@somescope/lib-a`. Note: a target may depend on both `//lib_a:lib_a` and `//sub:lib_a` in its `deps` and the linker will link the library to both `node_modules/@somescope/lib-a` and `sub/node_modules/@somescope/lib-a` respectively. Alternately, `npm_link` can add a `LinkablePackageInfo` provider to a target that doesn't yet have a `package_name` associated with it, For example, ``` lib_b/BUILD.bazel: js_library( name = "lib_b", srcs = [ "index.js", "package.json", ], ) ``` ``` sub/BUILD.bazel: npm_link( name = "lib_b", target = "//lib_b", package_path = "sub", package_name = "@somescope/lib-b", ) ``` which makes `//sub:lib_b` linked to `sub/node_modules/@somescope/lib-b` while `//lib_b:lib_b` is not linked at all. The linked target can be of any type, including a simple filegroup. For example, ``` lib_c/BUILD.bazel: filegroup( name = "lib_c", srcs = [ "index.js", "package.json", ], ) ``` ``` sub/BUILD.bazel: npm_link( name = "lib_c", target = "//lib_c", package_path = "sub", package_name = "@somescope/lib-c", ) ``` which makes `//sub:lib_c` linked to `sub/node_modules/@somescope/lib-c` while `//lib_c:lib_c` is not linked at all. ------------------------------ With the above abilities, we've also added syntactical sugar to `yarn_install` and `npm_install` which uses `npm_link` under the hood. For example, given a `sub/package.json` we can define its `yarn_install` as, ``` WORKSPACE: yarn_install( name = “@sub_npm_deps”, package_json = "//sub:package.json", package_path = "sub", links = { "@somescope/lib-a": "//lib_a:lib_a", "@somescope/lib-b": "//lib_b:lib_b", } ) ``` which generates two `npm_link` under the hood: ``` @sub_npm_deps//@somescope/lib-a:BUILD.bazel: npm_link( name = "lib-a", target = "//lib_a:lib_a", package_path = "sub", package_name = "@somescope/lib-a", ) ``` ``` @sub_npm_deps//@somescope/lib-b:BUILD.bazel: npm_link( name = "lib-b", target = "//lib_b:lib_b", package_path = "sub", package_name = "@somescope/lib-b", ) ``` which are both linked to `sub/node_modules` (`sub/node_modules/@somescope/lib-a` and `sub/node_modules/@somescope/lib-b` respectively). This allows the downstream syntactical sugar of depending on first party deps by their package names from the external workspace they are "linked" to. ``` sub/BUILD.bazel: nodejs_binary( name = "bin", entry_point = "bin.js", data = [ "bin.js" "@sub_npm_deps//@somescope/lib-a", "@sub_npm_deps//@somescope/lib-b", ] ) ``` and those deps will be available to the application in `sub/node_modules` where you would expect them to be if they had been defined in the `sub/package.json` itself using yarn workspaces outside of bazel. `sub/bin.js` can then require those libs with, ``` const liba = require('@somescope/lib-a') const libb = require('@somescope/lib-b') ``` and those will be resolved to `sub/node_modules/@somescope/lib-a/index.js` and `sub/node_modules/@somescope/lib-b/index.js` with standard node_modules resolution.
1 parent be184c2 commit e90b4ae

File tree

119 files changed

+12786
-446
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

119 files changed

+12786
-446
lines changed

‎WORKSPACE‎

Lines changed: 188 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,8 @@ workspace(
1818
"@angular_deps": ["packages/angular/node_modules"],
1919
# cypress_deps must be a managed directory to ensure it is downloaded before cypress_repository is run.
2020
"@cypress_deps": ["packages/cypress/test/node_modules"],
21+
"@internal_test_multi_linker_sub_deps": ["internal/linker/test/multi_linker/sub/node_modules"],
2122
"@npm": ["node_modules"],
22-
"@npm_internal_linker_test_multi_linker": ["internal/linker/test/multi_linker/node_modules"],
2323
"@npm_node_patches": ["packages/node-patches/node_modules"],
2424
},
2525
)
@@ -54,23 +54,145 @@ yarn_install(
5454
environment = {
5555
"SOME_USER_ENV": "yarn is great!",
5656
},
57+
links = {
58+
"@test_multi_linker/lib-a": "//internal/linker/test/multi_linker/lib_a",
59+
"@test_multi_linker/lib-a2": "//internal/linker/test/multi_linker/lib_a",
60+
"@test_multi_linker/lib-b": "@//internal/linker/test/multi_linker/lib_b",
61+
"@test_multi_linker/lib-b2": "@//internal/linker/test/multi_linker/lib_b",
62+
"@test_multi_linker/lib-c": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_c",
63+
"@test_multi_linker/lib-c2": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_c",
64+
"@test_multi_linker/lib-d": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_d",
65+
"@test_multi_linker/lib-d2": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_d",
66+
},
5767
package_json = "//:package.json",
5868
yarn_lock = "//:yarn.lock",
5969
)
6070

6171
yarn_install(
62-
name = "npm_internal_linker_test_multi_linker",
72+
name = "internal_test_multi_linker_deps",
73+
links = {
74+
"@test_multi_linker/lib-a": "@//internal/linker/test/multi_linker/lib_a",
75+
"@test_multi_linker/lib-a2": "@//internal/linker/test/multi_linker/lib_a",
76+
"@test_multi_linker/lib-b": "@//internal/linker/test/multi_linker/lib_b",
77+
"@test_multi_linker/lib-b2": "@//internal/linker/test/multi_linker/lib_b",
78+
"@test_multi_linker/lib-c": "@//internal/linker/test/multi_linker/lib_c",
79+
"@test_multi_linker/lib-c2": "@//internal/linker/test/multi_linker/lib_c",
80+
"@test_multi_linker/lib-d": "@//internal/linker/test/multi_linker/lib_d",
81+
"@test_multi_linker/lib-d2": "@//internal/linker/test/multi_linker/lib_d",
82+
},
6383
package_json = "//internal/linker/test/multi_linker:package.json",
6484
package_path = "internal/linker/test/multi_linker",
85+
symlink_node_modules = False,
6586
yarn_lock = "//internal/linker/test/multi_linker:yarn.lock",
6687
)
6788

6889
yarn_install(
69-
name = "onepa_npm_deps",
70-
package_json = "//internal/linker/test/multi_linker/onepa:package.json",
71-
package_path = "internal/linker/test/multi_linker/onepa",
90+
name = "internal_test_multi_linker_test_a_deps",
91+
links = {
92+
"@test_multi_linker/lib-a": "@//internal/linker/test/multi_linker/lib_a",
93+
"@test_multi_linker/lib-a2": "@//internal/linker/test/multi_linker/lib_a",
94+
"@test_multi_linker/lib-b": "@//internal/linker/test/multi_linker/lib_b",
95+
"@test_multi_linker/lib-b2": "@//internal/linker/test/multi_linker/lib_b",
96+
"@test_multi_linker/lib-c": "@//internal/linker/test/multi_linker/lib_c",
97+
"@test_multi_linker/lib-c2": "@//internal/linker/test/multi_linker/lib_c",
98+
"@test_multi_linker/lib-d": "@//internal/linker/test/multi_linker/lib_d",
99+
"@test_multi_linker/lib-d2": "@//internal/linker/test/multi_linker/lib_d",
100+
},
101+
package_json = "//internal/linker/test/multi_linker/test_a:package.json",
102+
package_path = "internal/linker/test/multi_linker/test_a",
103+
symlink_node_modules = False,
104+
yarn_lock = "//internal/linker/test/multi_linker/test_a:yarn.lock",
105+
)
106+
107+
yarn_install(
108+
name = "internal_test_multi_linker_test_b_deps",
109+
package_json = "//internal/linker/test/multi_linker/test_b:package.json",
110+
package_path = "internal/linker/test/multi_linker/test_b",
111+
symlink_node_modules = False,
112+
yarn_lock = "//internal/linker/test/multi_linker/test_b:yarn.lock",
113+
)
114+
115+
yarn_install(
116+
name = "internal_test_multi_linker_test_c_deps",
117+
package_json = "//internal/linker/test/multi_linker/test_c:package.json",
118+
package_path = "internal/linker/test/multi_linker/test_c",
119+
symlink_node_modules = False,
120+
yarn_lock = "//internal/linker/test/multi_linker/test_c:yarn.lock",
121+
)
122+
123+
yarn_install(
124+
name = "internal_test_multi_linker_test_d_deps",
125+
package_json = "//internal/linker/test/multi_linker/test_d:package.json",
126+
package_path = "internal/linker/test/multi_linker/test_d",
127+
symlink_node_modules = False,
128+
yarn_lock = "//internal/linker/test/multi_linker/test_d:yarn.lock",
129+
)
130+
131+
yarn_install(
132+
name = "internal_test_multi_linker_lib_b_deps",
133+
# transitive deps for this first party lib should not include dev dependencies
134+
args = ["--production"],
135+
package_json = "//internal/linker/test/multi_linker/lib_b:package.json",
136+
package_path = "internal/linker/test/multi_linker/lib_b",
72137
symlink_node_modules = False,
73-
yarn_lock = "//internal/linker/test/multi_linker/onepa:yarn.lock",
138+
yarn_lock = "//internal/linker/test/multi_linker/lib_b:yarn.lock",
139+
)
140+
141+
yarn_install(
142+
name = "internal_test_multi_linker_lib_c_deps",
143+
# transitive deps for this first party lib should not include dev dependencies
144+
args = ["--production"],
145+
package_json = "//internal/linker/test/multi_linker/lib_c:lib/package.json",
146+
package_path = "internal/linker/test/multi_linker/lib_c/lib",
147+
symlink_node_modules = False,
148+
yarn_lock = "//internal/linker/test/multi_linker/lib_c:lib/yarn.lock",
149+
)
150+
151+
yarn_install(
152+
name = "internal_test_multi_linker_sub_dev_deps",
153+
links = {
154+
"@test_multi_linker/lib-a": "//internal/linker/test/multi_linker/lib_a",
155+
"@test_multi_linker/lib-a2": "//internal/linker/test/multi_linker/lib_a",
156+
"@test_multi_linker/lib-b": "//internal/linker/test/multi_linker/lib_b",
157+
"@test_multi_linker/lib-b2": "//internal/linker/test/multi_linker/lib_b",
158+
"@test_multi_linker/lib-c": "//internal/linker/test/multi_linker/lib_c",
159+
"@test_multi_linker/lib-c2": "//internal/linker/test/multi_linker/lib_c",
160+
"@test_multi_linker/lib-d": "//internal/linker/test/multi_linker/lib_d",
161+
"@test_multi_linker/lib-d2": "//internal/linker/test/multi_linker/lib_d",
162+
},
163+
package_json = "//internal/linker/test/multi_linker/sub:package.json",
164+
package_path = "internal/linker/test/multi_linker/sub/dev",
165+
symlink_node_modules = False,
166+
yarn_lock = "//internal/linker/test/multi_linker/sub:yarn.lock",
167+
)
168+
169+
yarn_install(
170+
name = "internal_test_multi_linker_sub_deps",
171+
# transitive deps for this first party lib should not include dev dependencies
172+
args = ["--production"],
173+
links = {
174+
"@test_multi_linker/lib-a": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_a",
175+
"@test_multi_linker/lib-a2": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_a",
176+
"@test_multi_linker/lib-b": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_b",
177+
"@test_multi_linker/lib-b2": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_b",
178+
"@test_multi_linker/lib-c": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_c",
179+
"@test_multi_linker/lib-c2": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_c",
180+
"@test_multi_linker/lib-d": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_d",
181+
"@test_multi_linker/lib-d2": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_d",
182+
},
183+
package_json = "//internal/linker/test/multi_linker/sub:package.json",
184+
package_path = "internal/linker/test/multi_linker/sub",
185+
yarn_lock = "//internal/linker/test/multi_linker/sub:yarn.lock",
186+
)
187+
188+
yarn_install(
189+
name = "internal_test_multi_linker_onep_a_deps",
190+
# transitive deps for this first party lib should not include dev dependencies
191+
args = ["--production"],
192+
package_json = "//internal/linker/test/multi_linker/onep_a:package.json",
193+
package_path = "internal/linker/test/multi_linker/onep_a",
194+
symlink_node_modules = False,
195+
yarn_lock = "//internal/linker/test/multi_linker/onep_a:yarn.lock",
74196
)
75197

76198
npm_install(
@@ -273,6 +395,61 @@ yarn_install(
273395
".json",
274396
".proto",
275397
],
398+
links = {
399+
"@some-scope/some-target-b": "@//some/target/b",
400+
"@some-scope/some-target-b2": "@//some/target/b",
401+
"some-target-a": "//some/target/a",
402+
"some-target-a2": "//some/target/a",
403+
},
404+
manual_build_file_contents = """
405+
filegroup(
406+
name = "golden_files",
407+
srcs = [
408+
"//:BUILD.bazel",
409+
"//:manual_build_file_contents",
410+
"//:WORKSPACE",
411+
"//@angular/core:BUILD.bazel",
412+
"//@gregmagolan:BUILD.bazel",
413+
"//@gregmagolan/test-a/bin:BUILD.bazel",
414+
"//@gregmagolan/test-a:BUILD.bazel",
415+
"//@gregmagolan/test-a:index.bzl",
416+
"//@gregmagolan/test-b:BUILD.bazel",
417+
"//ajv:BUILD.bazel",
418+
"//jasmine/bin:BUILD.bazel",
419+
"//jasmine:BUILD.bazel",
420+
"//jasmine:index.bzl",
421+
"//rxjs:BUILD.bazel",
422+
"//unidiff:BUILD.bazel",
423+
"//zone.js:BUILD.bazel",
424+
"//some-target-a:BUILD.bazel",
425+
"//some-target-a2:BUILD.bazel",
426+
"//@some-scope/some-target-b:BUILD.bazel",
427+
"//@some-scope/some-target-b2:BUILD.bazel",
428+
],
429+
)""",
430+
package_json = "//:tools/fine_grained_goldens/package.json",
431+
symlink_node_modules = False,
432+
yarn_lock = "//:tools/fine_grained_goldens/yarn.lock",
433+
)
434+
435+
yarn_install(
436+
name = "fine_grained_goldens_multi_linked",
437+
included_files = [
438+
"",
439+
".js",
440+
".jst",
441+
".ts",
442+
".map",
443+
".d.ts",
444+
".json",
445+
".proto",
446+
],
447+
links = {
448+
"@some-scope/some-target-b": "@//some/target/b",
449+
"@some-scope/some-target-b2": "@//some/target/b",
450+
"some-target-a": "@build_bazel_rules_nodejs//some/target/a",
451+
"some-target-a2": "@build_bazel_rules_nodejs//some/target/a",
452+
},
276453
manual_build_file_contents = """
277454
filegroup(
278455
name = "golden_files",
@@ -293,9 +470,14 @@ filegroup(
293470
"//rxjs:BUILD.bazel",
294471
"//unidiff:BUILD.bazel",
295472
"//zone.js:BUILD.bazel",
473+
"//some-target-a:BUILD.bazel",
474+
"//some-target-a2:BUILD.bazel",
475+
"//@some-scope/some-target-b:BUILD.bazel",
476+
"//@some-scope/some-target-b2:BUILD.bazel",
296477
],
297478
)""",
298479
package_json = "//:tools/fine_grained_goldens/package.json",
480+
package_path = "tools/fine_grained_goldens",
299481
symlink_node_modules = False,
300482
yarn_lock = "//:tools/fine_grained_goldens/yarn.lock",
301483
)

‎examples/user_managed_deps/BUILD.bazel‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,15 @@
11
load("@build_bazel_rules_nodejs//:index.bzl", "js_library", "nodejs_binary", "nodejs_test")
22

3-
# This is "user-managed" dependencies - Bazel knows nothing about our package manager.
4-
# We assume that developers will install the `node_modules` directory themselves and
5-
# keep it up-to-date.
6-
# This rule simply exposes files in the node_modules directory to Bazel using js_library
7-
# with external_npm_package set to True so it can read them as inputs to processes it spawns.
3+
# This is "user-managed" dependencies - Bazel knows nothing about our package
4+
# manager. We assume that developers will install the `node_modules` directory
5+
# themselves and keep it up-to-date. This rule simply exposes files in the
6+
# node_modules directory to Bazel using js_library with package_name set to
7+
# "node_modules" so it can read them as inputs to processes it spawns.
88
js_library(
99
name = "node_modules",
10+
# Special value to provide ExternalNpmPackageInfo which is used by downstream
11+
# rules that use these npm dependencies
12+
package_name = "$node_modules$",
1013
srcs = glob(
1114
include = [
1215
"node_modules/**/*.js",
@@ -25,9 +28,6 @@ js_library(
2528
"node_modules/**/* *",
2629
],
2730
),
28-
# Provide ExternalNpmPackageInfo which is used by downstream rules
29-
# that use these npm dependencies
30-
external_npm_package = True,
3131
)
3232

3333
nodejs_binary(

0 commit comments

Comments
 (0)