Mark downloaded repository as reproducible

[trevorgray]

#515 only makes the config repository reproducible. The downloaded toolchains are store in a separate repository. Since this separate repository is not marked as reproducible, the extracted files are currently not cached.

This change fixes that by marking the repositority that contains the downloaded toolchain also reproducible.

Similar to the git repo rule here: https://github.com/bazelbuild/bazel/blob/648f991bd8653408ba6b4c24c6ff8fa4fce51eba/tools/build_defs/repo/git.bzl#L219

[trevorgray]

Fixed the missing attr_dict import which I missed when copying the changes to this branch.

[fmeum]

A bunch of CI jobs are failing due to increased disk usage. Did you see any of that in your local testing? I wouldn't have expected disk usage to change since the repo cache moves extracted dirs.

[trevorgray]

The extracted data gets put into the repo cache but then the data is copied to the external directory as normally would happen. They aren't moved out of the repo cache or hardlinked. This change just prevents re-downloading and extracting them each time.

You can set --experimental_repository_cache_hardlinks to hardlink the files which is suppose to fix the copying issue but I've tried setting this and the files were still copied.

[fmeum]

That's true for the repository cache, but shouldn't be the case for the new repo contents cache. It does move the directory and then symlinks to it: https://cs.opensource.google/bazel/bazel/+/f03468522dac6a2e5989ee13cdd540de9349c195:src/main/java/com/google/devtools/build/lib/bazel/repository/cache/RepoContentsCache.java;l=174

[trevorgray]

Yes. You are right. I was looking inside the llvm tools repo directory so I did not see the symlink since it was symlinking the whole directory and not the individual files.

[trevorgray]

toolchains_llvm/tests/scripts/run_tests.sh

Line 70 in 6a2ba86

# WebAssembly tests use a separate (newer) version of LLVM to exercise support

WebAssembly tests use a separate (newer) version of LLVM to exercise support for experimental features such as wasm64, which can cause the CI environment to run out of disk space. Mitigate this by expunging the workspace before trying to build Wasm targets.

So the problem is that when you run bazel clean --expunge, the command doesn't clean out the repo cache. Before the llvm toolchains were being cleaned since they were in the work tree. Now they are in the repo cache and not the work tree. The fix seems to be to just delete the repo cache before running those tests right after the bazel clean

[fmeum]

Nice, thanks for digging!

[trevorgray]

I've updated the code to fix missing $USER variable on some variants of linux. I've also updated the path on mac os since apparently it's different (https://bazel.build/remote/output-directories)