rules_rust 0.57.1 Private Crates repo not pulling

[ericmcbride]

• Seems like there was a regression with private repositories and rules_rust 0.57.1
• I can revert back to 0.48.0 and it will pull from artifactory no issue
• I try to bump to 0.56.0 and theres an issue with vendor and manifest keys.
• This is blocking me from fixing this issue

[UebelAndre]

How are you pointing to artifactory?

[ericmcbride]

I am using the .cargo/config.toml and

[registries.my_artifactory]
index = "sparse+https://my_artifactory/artifactory/api/cargo/my_crates/index/"

[registry]
default = "my_artifactory"
global-credential-providers = ["cargo:token"]

And using the credentials.toml / netrc for pulling. It works with cargo fetch but no go with bazel

[UebelAndre]

what does your crate_universe integration look like?

[ericmcbride]

MODULE.bazel

crate = use_extension(
    "@rules_rust//crate_universe:extension.bzl",
    "crate",
)
crate.from_cargo(
    name = "crates_index",
    cargo_config = "//:.cargo/config.toml",
    cargo_lockfile = "//:Cargo.lock",
    manifests = [
        "//:Cargo.toml",
       /// alot of manifests
    ],
    
)
use_repo(crate, "crates_index")

[UebelAndre]

As far as I can tell this should be working. It's a hard thing for me to test though. Can you bisect and find the exact release or commit where it stopped working?

[ericmcbride]

I can do a bisect later but 0.54 works with artifactory, 0.55 and 0.56 i cant tell due to the vendor / manifest issue, and 0.57 / 0.57.1 artifactory doesnt work

[UebelAndre]

What is the vendor / manifest issue you see?

[ericmcbride]

Error in fail: /private/var/tmp/_bazel_eric.mcbride/79b277eea1629c6c0e66b29d7b082943/modextwd/rules_rust~~crate/cargo-bazel splice --output-dir /private/var/tmp/_bazel_eric.mcbride/79b277eea1629c6c0e66b29d7b082943/modextwd/rules_rust~~crate/crates_index/splicing-output --config /private/var/tmp/_bazel_eric.mcbride/79b277eea1629c6c0e66b29d7b082943/modextwd/rules_rust~~crate/crates_index/config.json --splicing-manifest /private/var/tmp/_bazel_eric.mcbride/79b277eea1629c6c0e66b29d7b082943/modextwd/rules_rust~~crate/crates_index/splicing_manifest.json --nonhermetic-root-bazel-workspace-dir /Users/eric.mcbride/Programming/<redacted>/<redacted> --cargo-lockfile /Users/eric.mcbride/Programming/<redacted>/<redacted>/Cargo.lock --cargo /private/var/tmp/_bazel_eric.mcbride/79b277eea1629c6c0e66b29d7b082943/external/rules_rust~~rust_host_tools~rust_host_tools/bin/cargo --rustc /private/var/tmp/_bazel_eric.mcbride/79b277eea1629c6c0e66b29d7b082943/external/rules_rust~~rust_host_tools~rust_host_tools/bin/rustc returned with exit code 1:

Error: Failed to splice workspace Caused by:
    Some manifests are not being tracked.
    Please add the following labels to the `manifests` key:
     //crate_universe:Cargo.toml
     //crate_universe:test_data/test_data_passing_crate/Cargo.toml
     //crate_universe:test_data/workspace_examples/non-ws/Cargo.toml
     //crate_universe:test_data/workspace_examples/symlinked/Cargo.toml
     //crate_universe:test_data/workspace_examples/ws1/Cargo.toml
     //crate_universe:test_data/workspace_examples/ws1/ws1c1/Cargo.toml
     //crate_universe:test_data/workspace_examples/ws1/ws1c1/ws1c1c1/Cargo.toml
     //crate_universe:test_data/workspace_examples/ws1/ws1c2/Cargo.toml
     //crate_universe:test_data/workspace_examples/ws2/Cargo.toml
     //crate_universe:test_data/workspace_examples/ws2/ws2c1/Cargo.toml
     //crate_universe:test_data/workspace_examples/ws2/ws2excluded/Cargo.toml
     //crate_universe:test_data/workspace_examples/ws2/ws2excluded/ws2excluded2/Cargo.toml
     //crate_universe:test_data/workspace_examples/ws2/ws2excluded/ws2included/Cargo.toml
     //crate_universe/tools/cross_installer:Cargo.toml
     //crate_universe/tools/urls_generator:Cargo.toml
     //:Cargo.toml
     //third-party-in-workspace:Cargo.toml
     //third-party-without-workspace:Cargo.toml
     //:Cargo.toml

This appears whenever i run bazel build //... on 0.55 and 0.56

[aignas]

Created #3261.

[ericmcbride]

@UebelAndre it seems like the default argument of isolated = true that was added in a recent MR to the extensions.bzl file is the culprit.

I was able to run CARGO_BAZEL_ISOLATED=false bazel build //... and it was able to pull from artifactory.

[UebelAndre]

@UebelAndre it seems like the default argument of isolated = true that was added in a recent MR to the extensions.bzl file is the culprit.

I was able to run CARGO_BAZEL_ISOLATED=false bazel build //... and it was able to pull from artifactory.

Ah, good find! That was definitely not an intended change in that PR but it is something that I think is gonna be important to have in a bzlmod world. Previously I thought it crate_repository was too clunky to use transitively in WORKSPACE files but with bzlmod, it'll be very easy to use that and have your project consumable. I think in cases where a bazel_dep is using crate universe, it should be own the root module to determine if splicing should be isolated or not. Curious if folks out there have encountered similar patterns.

[ericmcbride]

@UebelAndre it seems like the default argument of isolated = true that was added in a recent MR to the extensions.bzl file is the culprit.
I was able to run CARGO_BAZEL_ISOLATED=false bazel build //... and it was able to pull from artifactory.

Ah, good find! That was definitely not an intended change in that PR but it is something that I think is gonna be important to have in a bzlmod world. Previously I thought it crate_repository was too clunky to use transitively in WORKSPACE files but with bzlmod, it'll be very easy to use that and have your project consumable. I think in cases where a bazel_dep is using crate universe, it should be own the root module to determine if splicing should be isolated or not. Curious if folks out there have encountered similar patterns.

My question is what happens if Isolated = True and theres no way for the isolated environment to access the credentials.toml file? Should we provide a way with crate_repository to pull in the credentials file like we do with the cargo file or would that be a security concern? Is there anyway we can resolve this with netrc?

[aignas]

netrc is usually accessible by the bazel downloader irrespective of what arguments one passes. I would personally say that if you want to keep the isolated behaviour then there should be a way to somehow configure credentials for cargo.

As a user I am not sure how to do this. Maybe I just down know how to do it and this is a documentation issue?