Project References Wildcards

[eabrouwer3]

🔍 Search Terms

"project references" "glob" "wildcard" "repetitive" "monorepo"

✅ Viability Checklist

• This wouldn't be a breaking change in existing TypeScript/JavaScript code
• This wouldn't change the runtime behavior of existing JavaScript code
• This could be implemented without emitting different JS based on the types of the expressions
• This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
• This feature would agree with the rest of our Design Goals: https://github.com/Microsoft/TypeScript/wiki/TypeScript-Design-Goals

⭐ Suggestion

Project References currently have to be written out explicitly. It would be super nice if we could use a wildcard/glob like this, where I could simply have all of my tsconfig.json files look like this:

{
  "extends": ["../../../tsconfig.base.json"],
  "references": [{"path": "../../libs/*"}]
}

Or, even something like this:

{
  "extends": ["../../../tsconfig.base.json"],
  "references": [{"path": "../../libs/*/tsconfig.build.json"}]
}

We can discuss below whether supporting globstars (**) is prudent or not.

📃 Motivating Example

I've got a project with 8 libs and 3 apps using project references. It follows this basic structure:

tsconfig.base.json
workspaces/
  apps/
    app-1/
      tsconfig.json
    app-2/
      tsconfig.json
    .../
  libs/
    lib-1/
      tsconfig.json
    lib-2/
      tsconfig.json
    .../

There's a little more complexity, but that's the basic idea. However, I have a ton of repetition between all my tsconfig.json files that basically all look like this:

{
  "extends": ["../../../tsconfig.base.json"],
  "references": [
    {
      "path": "../../libs/lib-1"
    },
    {
      "path": "../../libs/lib-2"
    },
    // ...
  ]
}

I can't rely on the extends functionality from tsconfig.base.json because of things like @RyanCavanaugh's explanation here: #27098 (comment).

💻 Use Cases

1. What do you want to use this for?

• A monorepo with a number of shared libraries and apps

2. What shortcomings exist with current approaches?

• Lots of repetition of lengthy amounts of JSON

3. What workarounds are you using in the meantime?

• Specifying each lib in the "references" portion for every single app/lib that needs to be able to import shared libraries.

[jakebailey]

Projects must be acyclic; how would that work if one globs over every project in the dependencies of every project?

Even if cycles were legal, it seems like a bad idea to allow configuration that makes it easy for every project to depend on every other project; at that point it's back to a monolith that all has to be recompiled. Though, I suppose with the potential for alternative config options, at least.

[jonaskello]

I would agree with @jakebailey. If you want to build only app-2 and it's related libs then it would not be possible since everything is referencing everything else. In the small example this may not be a problem but if you have 10 apps and >100 libs it will.

If you ware using pnpm/yarn/npm workspaces you still need to specify the exact dependencies in package.json and then having the references in tsconfig.json becomes a non-issue becuase you can use a tool like workspaces-to-typescript-project-references to automatically fill the references from package.json into tsconfig.json.

I would of course be nice if this functionality was built into tsc with something like tsc --sync-refs :-).

[jakebailey]

I'll note that if you're using declaration maps and a package manager with monorepo support (with explicit deps), it may be entirely possible to not use references at all, and just build each project individually. There are downsides, of course, but at that point with say pnpm's dependency aware script running, you're basically just using packages as they'd be viewed when published (but locally).

[jonaskello]

Yes, in this small example building each package separately may be feasible. However having to restart tsc many times is really slow if you have many apps/libs. You may also take a look at nx batch mode to see the difference. From what I understand that also builds the tsconfig refs dynamically.

[eabrouwer3]

You guys all make great points. What if we had one of the following options to reduce the repetitiveness?

{
  "extends": ["../../../tsconfig.base.json"],
  "references": "inherit"

^^ This leads to the same problems you're all talking about, but it might be nice to have the option to inherit. Especially in cases where (in my project) I have a tsconfig.json and a tsconfig.build.json where I have to specify all of the references in both places.

The other option would be basically a shorthand for the "references" key, so you don't have to specify an object with "path" every time if you don't need to specify any other options.

{
  "extends": "../../../tsconfig.base.json",
  "references": ["../../libs/lib-1", "../../libs/lib-2", ...]
}

But we could make it a dual typed field. It's an array of either string, or an options object with "path" and the other options. That at least cuts down how big the repetition is.

[jakebailey]

I have a tsconfig.json and a tsconfig.build.json where I have to specify all of the references in both places.

What in what situation do you need both of these? I can only think of a situation where you have a tsconfig to load all workspace code to avoid project reference overhead, but that only helps when you have a significantly large number of packages.

[eabrouwer3]

tsconfig.json compiles everything, including my test files. tsconfig.build.json excludes all test files and anything else like a db seed file or whatever that I don't want included in the production build. I feel like this pattern is fairly common, as in I've seen it around in a number of places, but maybe I've been misguided.

[jakebailey]

I have personally more often seen tests in separate projects (what we do in TS, dt-tools), or one project that always builds tests with exclusions elsewhere (e.g., npmignore, what I do in my packages). But, I of course have my own narrow lens.

[RyanCavanaugh]

My thinking was that this is useful (only) for the top-level "solution" tsconfig

tsconfig.json <- specifies references: packages/*
packages/
  foo/
    tsconfig.json
  bar/
    tsconfig.json

so that you don't have to list out all the package/ subdirs individually

[jonaskello]

We co-locate the tests with the other code which I think is quite common in the javascript community. If you change the application code you may break the compile of the test code and vice versa so IMO it does not make sense to compile them separately.

It does make sense to not distribute the test files in production builds, but I see compile and distribution are separate things. However as mentioned all these points are personal preferences.

[Qrokqt]

Instead of having to specify the references param, could it instead infer it from the package.json dependencies which use the workspace syntax? Right now I'm basically copying both lists back and forth to keep them in sync.

[jonaskello]

@Qrokqt If you are manually keeping tsconfig references and package.json in sync you may be interested in automating that with meta-updater or workspaces-to-typescript-project-references