Tracking issue for config-include

[ehuss]

Implementation: #7649
Original proposal: #6699
Documentation: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#config-include
Issues: Z-config-include Nightly: `include` config key

Summary

Adds the include config key to include other config files.

Unresolved issues

• Should there be any restriction on the filename? Currently there isn't. I was initially considering restricting to .cargo/config-* in case we want to safely use other filenames in .cargo/ for other purposes. However, it seems like an unnatural restriction. Another restriction is that it should maybe end in .toml?
  • The team has decided on putting a restriction on allowing .toml extension only. See Tracking issue for config-include #7723 (comment)
• Inconsistent relative path behavior in configurations: Inconsistent relative path behavior between config include and other fields #15769
• Optional include supports and syntax: Tracking issue for config-include #7723 (comment).

FCP： #16284 (comment)

[ensc]

Some additional wishes (from #9306):

• allow to include multiple files

  include = "foo.toml"
  include = "bar.toml"
  include = "conf.d/*.toml"

• allow to ignore missing files; e.g. either by a special syntax like leading "-":

  include = "-can-be-missing.toml"
  include = "required.toml"

  or by using special keywords (e.g. required vs include)

  This feature is required for local configuration (e.g. the setup of build.target-dir) within a more compliex environment

• substitute placeholders within the filename; e.g.

  • %h ... local hostname
  • %u ... local username
  • %H ... hash of absolute Cargo.toml path
  • ...

[jonhoo]

Some quick thoughts on this:

Restrictions on the path. What's the motivation for restricting the paths? Cargo potentially having special files in .cargo/ in the future is a good one, though I also see an upside in being able to stash other config files under .cargo/ rather than adding another top-level directory to a project like .cargo-includes/. One option might be to say that we allow any path, except a path under .cargo/ that isn't prefixed with config-. So myconfig.toml, myconfig, .cargo/config-my.toml, .cargo/config-my are all okay, but .cargo/my.toml is not. Restrictions beyond that seem unwarranted to me (e.g. requiring .toml).

Multiple includes. I think the ability to have multiple includes is table-stakes for a feature like this. We could get that solely through globs (see below), but just allowing enumeration feels better. I think we should lean into TOML here though and say that the value can be either a string or a list of strings, rather than allowing the key to appear multiple times in a given context (it can, of course, appear multiple times in different tables).

Include globs. I'm torn on allowing globs. They tend to cause headaches as much as it fixes them. One option might be to instead say that Cargo implicitly has a top-level include for each file in .cargo/config.d/ instead? Which I think is a feature that could land independently of this. So my vote is to not include glob support.

Missing files. I think we should initially have just include, and say that it warns, but does not error, if the target file does not exist. We could then in the future add something like require (I don't love that name for this) if the need is great enough.

Placeholders. This feels like something worth punting on until we see use in practice, and not something that should block config-include. It may turn out to be useful, but I think realistically it could be added later. We may want to include checks in the code for config-include that checks the path for % though and warn about possible future-incompatibility (or just straight up disallow it).

[ensc]

• my main reason for this feature is local customization. E.g. I want .cargo/config as part of the project in git, but local setup (e.g. for build.target-dir) should be possible and only in my environment. This setup must be optional (e.g. no warnings when included files are not existing).

  Atm, my git work tree is always dirty in rust projects because there is some local setup (paths (see above) or other default features on Cargo.toml). Moving these settings in optional files would be very helpful

• when using distinct keywords (include for optional, require for mandatory includes), the syntax must allow multiple same keys in the same context. E.g.

   include: optional-00.conf        ## sets A = 0
   require: mandatory-00.conf       ## sets A = 1
   include: optional-01.conf        ## sets A = 2

  has a different load order than

   include:
   - optional-00.conf        ## sets A = 0
   - optional-01.conf        ## sets A = 2
   require:
   - mandatory-00.conf       ## sets A = 1

  "A" will be 2 in the first example but "1" in the latter one.

  That's why, some indication whether mandatory or not should be part of the file name. Perhaps

  include:
  - path: optional-00.conf
    optional: true
  - mandatory-00.conf
  - path: optional-01.conf
    optional: true

  Or just the - prefix (which is used e.g. by GNU Make in -include, or systemd in EnvironmentFile to mark optional files)

[jonhoo]

• This setup must be optional (e.g. no warnings when included files are not existing).

Why must it produce no warnings?

• paths (see above) or other default features on Cargo.toml

I don't think this feature will help you with includes for Cargo.toml. It's only for Cargo's configuration files as I understand it.

• the syntax must allow multiple same keys in the same context

I think that's simply just not allowed by the TOML spec, though I could be wrong. At the very least it would require some custom de/serialization to handle duplicate keys. Your alternative proposal of introducing table entries in the include array seems like a nice one to me, and crucially also seems like something that could be an extension of the feature, rather than something that's there from the get-go.

• Or just the - prefix (which is used e.g. by GNU Make in -include, or systemd in EnvironmentFile to mark optional files)

I personally strongly prefer explicit syntax (required: true) over "magic" strings. The latter are, among other things, much harder to search for, and not at all immediately obvious to an unfamiliar reader.

[ensc]

This setup must be optional (e.g. no warnings when included files are not existing).
Why must it produce no warnings?

Because the setup is not in git (it is local). When other people clone the project they will either be spammed by warnings about non-existing files, or they have to create them manually.