Discussion: Setting up ts_project with custom path mapping in a monorepo

[lencioni]

I've been working to set up Bazel with ts_project in a large, custom monorepo (i.e. not using something like Lerna or yarn workspaces). We recently resolved a challenge and I'd like to publicly document it here in case it helps someone else (or if someone sees a better solution for us). Feel free to close this issue.

Our monorepo is set up so that we have a "frontend" directory with a lot of projects as subdirectories under "frontend". The structure is roughly like this:

frontend/
frontend/project-a/
frontend/project-b/
frontend/project-c/
...

Imports inside the same project use relative paths:

import foo from './foo';

And imports from other projects use custom colon-import paths:

import someOtherProject from ':some-other-project';
import anotherThing from ':a-different-project/anotherThing';

We've been running TypeScript without Bazel for a while, and had these custom imports wired up via the following path mapping in our base tsconfig:

{
  "compilerOptions": {
    "paths": {
      ":*": [
        "./frontend/*",
      ],
    }
  }
}

When setting up Bazel, we initially expanded this path mapping to look more like this:

{
  "compilerOptions": {
    "paths": {
      ":*": [
        "./frontend/*",

        // Bazel generates the .d.ts files in the bazel-out directory, so we
        // need to tell TypeScript that it might be able to find them there.
        //
        // This hack was inspired by formatjs:
        // https://github.com/formatjs/formatjs/blob/268d6aef6fc08180a0be691b631eb1cdfb68ca31/tsconfig.json#L19-L41
        // And the rootDirs note in the docs here:
        // https://bazelbuild.github.io/rules_nodejs/TypeScript.html#ts_project
        "./bazel-out/darwin-fastbuild/bin/frontend/*",
        "./bazel-out/k8-fastbuild/bin/frontend/*",
        "./bazel-out/x64_windows-fastbuild/bin/frontend/*",
        "./bazel-out/darwin-dbg/bin/frontend/*",
        "./bazel-out/k8-dbg/bin/frontend/*",
        "./bazel-out/x64_windows-dbg/bin/frontend/*",
      ],
    }
  }
}

This seemed to work okay for some of our projects, however, once we started getting into more complex projects that depended on types that were built from a different project, we started seeing errors in our Bazel ts_project output like this:

frontend/complex-project/foo/bar.ts(27,16): error TS2571: Object is of type 'unknown'.

It seemed that these types ended up as unknown instead of the actual types we expected.

We looked at the .d.ts files that ts_project built for these types, and found code that started like this:

import Foo from ':some-dependency/foo/bar';
// ...

Was built into this:

// In a file in frontend/complex-project
// ...
export declare const FooType: import("../../bazel-out/darwin-fastbuild/bin/frontend/some-dependency/foo/bar").Foo<Bar>;
// ...

We expected the import path to look something like "../../some-dependency/foo/bar" but instead got "../../bazel-out/darwin-fastbuild/bin/frontend/some-dependency/foo/bar", which doesn't really exist. As a result, TypeScript infers it as any which probably gets converted to an unknown somewhere along the line via the generics used here.

After a bit of messing around, we believe that we were able to arrive at an okay solution: add rootDirs and an extra paths mapping like this:

{
  "compilerOptions": {
    "rootDirs": [
      "../..",
      // Ensure that the relative paths in types produced by bazel get turned into
      // "frontend/" paths.
      "../../bazel-out/darwin-fastbuild/bin",
      "../../bazel-out/k8-fastbuild/bin",
      "../../bazel-out/x64_windows-fastbuild/bin",
      "../../bazel-out/darwin-dbg/bin",
      "../../bazel-out/k8-dbg/bin",
      "../../bazel-out/x64_windows-dbg/bin",
    ],

    "paths": {
      ":*": [
        "./frontend/*",

        // Bazel generates the .d.ts files in the bazel-out directory, so we
        // need to tell TypeScript that it might be able to find them there.
        //
        // This hack was inspired by formatjs:
        // https://github.com/formatjs/formatjs/blob/268d6aef6fc08180a0be691b631eb1cdfb68ca31/tsconfig.json#L19-L41
        // And the rootDirs note in the docs here:
        // https://bazelbuild.github.io/rules_nodejs/TypeScript.html#ts_project
        "./bazel-out/darwin-fastbuild/bin/frontend/*",
        "./bazel-out/k8-fastbuild/bin/frontend/*",
        "./bazel-out/x64_windows-fastbuild/bin/frontend/*",
        "./bazel-out/darwin-dbg/bin/frontend/*",
        "./bazel-out/k8-dbg/bin/frontend/*",
        "./bazel-out/x64_windows-dbg/bin/frontend/*",
      ],

      // WARNING: Don't use frontend/ imports, use colon instead. This is for bazel build
      // type resolution: The rootDirs above produce "frontend/" paths in the .d.ts files
      // produced by bazel, and then we remap those paths back into bazel-out here.
      // Note this does mean you can never install a package named "frontend", but that's
      // probably fine since this package is unlikely to be something we'd install:
      // https://www.npmjs.com/package/frontend
      "frontend/*": [
        "./bazel-out/darwin-fastbuild/bin/frontend/*",
        "./bazel-out/k8-fastbuild/bin/frontend/*",
        "./bazel-out/x64_windows-fastbuild/bin/frontend/*",
        "./bazel-out/darwin-dbg/bin/frontend/*",
        "./bazel-out/k8-dbg/bin/frontend/*",
        "./bazel-out/x64_windows-dbg/bin/frontend/*",
      ],
    }
  }
}

This configuration causes our special colon imports to be converted from ":some-project" to "frontend/some-project", which then works because of the second "frontend/*" path mapping.

The main caveat that we've thought of with this approach is that it means we can't just install and use a package named "frontend", but that seems unlikely to cause us any problems.

[mgenov]

@lencioni could you provide more details where tsconfig.json files are defined?

[lencioni]

Sure thing @mgenov!

For the most part, we currently have a tsconfig.json file in each of our frontend projects. So with the structure I mentioned above, that would be:

frontend/project-a/tsconfig.json
frontend/project-b/tsconfig.json
frontend/project-c/tsconfig.json

We are using project references, and each of these tsconfig.json files extend from a more centralized config in one of our projects that has our global settings in it like the rootDirs and paths stuff I mentioned above. Here's an example of one of these project tsconfig.json files:

{
  "extends": "../typescript/tsconfig.dts.json",
  "include": [".", "./**/*.json"],

  "compilerOptions": {
    "types": ["jest", "node"]
  },

  "references": [
    {
      "path": "../project-b"
    }
  ]
}

We have created some internal tooling that scans the code in our repo for dependencies (currently using jest-haste-map) and keeps parts of these files like the references and compilerOptions.types up to date automatically.

In a few cases today we have some tsconfig.json files more deeply nested in projects. This is mostly to support things in our repo like cypress, which have some different global types that would conflict with other things.

frontend/project-a/tsconfig.json
frontend/project-b/tsconfig.json
frontend/project-b/cypress/tsconfig.json
frontend/project-c/tsconfig.json

We plan to add a lot more tsconfig.json files in our repo, such as in all of our test directories, so that we can improve TypeScript performance and avoid dragging in test-specific global types into production code, but first we have some things we need to solve, such as upgrading our tooling to manage the exclude field for these deeply nested tsconfig.json files. As we roll this out, I believe we will also have some cycles between our test directories that we will need to resolve as well.

[mgenov]

Thanks for your feedback @lencioni

I've made a showcase project to try to illustrate this usage but I encountered the following compilation error:

frontend/project-a/index.ts(1,24): error TS2307: Cannot find module ':project-b' or its corresponding type declarations.

when executing

bazel build //frontend/project-a:tsconfig

Could you take a look what could cause the error?

Code showcase is located at: https://github.com/mgenov/bazel-typescript-showcase

[lencioni]

I think the baseUrl needs to be a bit different in your repo. I opened a PR that I think fixes it: mgenov/bazel-typescript-showcase#1

[mgenov]

Yes, with this change everything is working as expected. Thanks!