module: add --experimental-transform-types flag

PR-URL: #54283
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Zeyu "Alex" Yang <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Paolo Insogna <[email protected]>

Author: marco-ippolito

doc/api/cli.md

@@ -923,6 +923,17 @@ files with no extension will be treated as WebAssembly if they begin with the
 WebAssembly magic number (`\0asm`); otherwise they will be treated as ES module
 JavaScript.
 
+### `--experimental-transform-types`
+
+<!-- YAML
+added: REPLACEME
+-->
+
+> Stability: 1.0 - Early development
+
+Enables the transformation of TypeScript-only syntax into JavaScript code.
+Implies `--experimental-strip-types` and `--enable-source-maps`.
+
 ### `--experimental-eventsource`
 
 <!-- YAML
@@ -2961,6 +2972,7 @@ one is included in the list below.
 * `--experimental-sqlite`
 * `--experimental-strip-types`
 * `--experimental-top-level-await`
+* `--experimental-transform-types`
 * `--experimental-vm-modules`
 * `--experimental-wasi-unstable-preview1`
 * `--experimental-wasm-modules`

doc/api/typescript.md

@@ -1,5 +1,14 @@
 # Modules: TypeScript
 
+<!-- YAML
+changes:
+  - version: REPLACEME
+    pr-url: https://github.com/nodejs/node/pull/54283
+    description: Added `--experimental-transform-types` flag.
+-->
+
+> Stability: 1.0 - Early development
+
 ## Enabling
 
 There are two ways to enable runtime TypeScript support in Node.js:
@@ -44,13 +53,15 @@ added: v22.6.0
 > Stability: 1.0 - Early development
 
 The flag [`--experimental-strip-types`][] enables Node.js to run TypeScript
-files that contain only type annotations. Such files contain no TypeScript
-features that require transformation, such as enums or namespaces. Node.js will
-replace inline type annotations with whitespace, and no type checking is
-performed. TypeScript features that depend on settings within `tsconfig.json`,
+files. By default Node.js will execute only files that contain no
+TypeScript features that require transformation, such as enums or namespaces.
+Node.js will replace inline type annotations with whitespace,
+and no type checking is performed.
+To enable the transformation of such features
+use the flag [`--experimental-transform-types`][].
+TypeScript features that depend on settings within `tsconfig.json`,
 such as paths or converting newer JavaScript syntax to older standards, are
-intentionally unsupported. To get fuller TypeScript support, including support
-for enums and namespaces and paths, see [Full TypeScript support][].
+intentionally unsupported. To get full TypeScript support, see [Full TypeScript support][].
 
 The type stripping feature is designed to be lightweight.
 By intentionally not supporting syntaxes that require JavaScript code
@@ -82,20 +93,24 @@ The `tsconfig.json` option `allowImportingTsExtensions` will allow the
 TypeScript compiler `tsc` to type-check files with `import` specifiers that
 include the `.ts` extension.
 
-### Unsupported TypeScript features
+### TypeScript features
 
 Since Node.js is only removing inline types, any TypeScript features that
-involve _replacing_ TypeScript syntax with new JavaScript syntax will error.
-This is by design. To run TypeScript with such features, see
-[Full TypeScript support][].
+involve _replacing_ TypeScript syntax with new JavaScript syntax will error,
+unless the flag [`--experimental-transform-types`][] is passed.
 
-The most prominent unsupported features that require transformation are:
+The most prominent features that require transformation are:
 
 * `Enum`
-* `experimentalDecorators`
 * `namespaces`
+* `legacy module`
 * parameter properties
 
+Since Decorators are currently a [TC39 Stage 3 proposal](https://github.com/tc39/proposal-decorators)
+and will soon be supported by the JavaScript engine,
+they are not transformed and will result in a parser error.
+This is a temporary limitation and will be resolved in the future.
+
 In addition, Node.js does not read `tsconfig.json` files and does not support
 features that depend on settings within `tsconfig.json`, such as paths or
 converting newer JavaScript syntax into older standards.
@@ -132,8 +147,9 @@ TypeScript syntax is unsupported in the REPL, STDIN input, `--print`, `--check`,
 ### Source maps
 
 Since inline types are replaced by whitespace, source maps are unnecessary for
-correct line numbers in stack traces; and Node.js does not generate them. For
-source maps support, see [Full TypeScript support][].
+correct line numbers in stack traces; and Node.js does not generate them.
+When [`--experimental-transform-types`][] is enabled, source-maps
+are enabled by default.
 
 ### Type stripping in dependencies
 
@@ -145,6 +161,7 @@ a `node_modules` path.
 [ES Modules]: esm.md
 [Full TypeScript support]: #full-typescript-support
 [`--experimental-strip-types`]: cli.md#--experimental-strip-types
+[`--experimental-transform-types`]: cli.md#--experimental-transform-types
 [`tsx`]: https://tsx.is/
 [`verbatimModuleSyntax`]: https://www.typescriptlang.org/tsconfig/#verbatimModuleSyntax
 [file extensions are mandatory]: esm.md#mandatory-file-extensions

doc/node.1

@@ -194,6 +194,9 @@ Enable snapshot testing in the test runner.
 .It Fl -experimental-strip-types
 Enable experimental type-stripping for TypeScript files.
 .
+.It Fl -experimental-transform-types
+Enable transformation of TypeScript-only syntax into JavaScript code.
+.
 .It Fl -experimental-eventsource
 Enable experimental support for the EventSource Web API.
 .

lib/internal/main/eval_string.js

@@ -14,7 +14,7 @@ const {
   markBootstrapComplete,
 } = require('internal/process/pre_execution');
 const { evalModuleEntryPoint, evalScript } = require('internal/process/execution');
-const { addBuiltinLibsToObject, tsParse } = require('internal/modules/helpers');
+const { addBuiltinLibsToObject, stripTypeScriptTypes } = require('internal/modules/helpers');
 
 const { getOptionValue } = require('internal/options');
 
@@ -24,7 +24,7 @@ markBootstrapComplete();
 
 const code = getOptionValue('--eval');
 const source = getOptionValue('--experimental-strip-types') ?
-  tsParse(code) :
+  stripTypeScriptTypes(code) :
   code;
 
 const print = getOptionValue('--print');

lib/internal/modules/cjs/loader.js

@@ -1362,8 +1362,8 @@ function loadESMFromCJS(mod, filename) {
     if (isUnderNodeModules(filename)) {
       throw new ERR_UNSUPPORTED_NODE_MODULES_TYPE_STRIPPING(filename);
     }
-    const { tsParse } = require('internal/modules/helpers');
-    source = tsParse(source);
+    const { stripTypeScriptTypes } = require('internal/modules/helpers');
+    source = stripTypeScriptTypes(source, filename);
   }
   const cascadedLoader = require('internal/modules/esm/loader').getOrInitializeCascadedLoader();
   const isMain = mod[kIsMainSymbol];
@@ -1576,9 +1576,9 @@ function loadCTS(module, filename) {
     throw new ERR_UNSUPPORTED_NODE_MODULES_TYPE_STRIPPING(filename);
   }
   const source = getMaybeCachedSource(module, filename);
-  const { tsParse } = require('internal/modules/helpers');
-  const content = tsParse(source);
-  module._compile(content, filename, 'commonjs');
+  const { stripTypeScriptTypes } = require('internal/modules/helpers');
+  const code = stripTypeScriptTypes(source, filename);
+  module._compile(code, filename, 'commonjs');
 }
 
 /**
@@ -1592,8 +1592,8 @@ function loadTS(module, filename) {
   }
   // If already analyzed the source, then it will be cached.
   const source = getMaybeCachedSource(module, filename);
-  const { tsParse } = require('internal/modules/helpers');
-  const content = tsParse(source);
+  const { stripTypeScriptTypes } = require('internal/modules/helpers');
+  const content = stripTypeScriptTypes(source, filename);
   let format;
   const pkg = packageJsonReader.getNearestParentPackageJSON(filename);
   // Function require shouldn't be used in ES modules.
@@ -1613,7 +1613,7 @@ function loadTS(module, filename) {
     if (Module._cache[parentPath]) {
       let parentSource;
       try {
-        parentSource = tsParse(fs.readFileSync(parentPath, 'utf8'));
+        parentSource = stripTypeScriptTypes(fs.readFileSync(parentPath, 'utf8'), parentPath);
       } catch {
         // Continue regardless of error.
       }

lib/internal/modules/esm/get_format.js

@@ -162,9 +162,8 @@ function getFileProtocolModuleFormat(url, context = { __proto__: null }, ignoreE
         // but this gets called again from `defaultLoad`/`defaultLoadSync`.
         let parsedSource;
         if (source) {
-          // We do the type stripping only if `source` is not falsy.
-          const { tsParse } = require('internal/modules/helpers');
-          parsedSource = tsParse(source);
+          const { stripTypeScriptTypes } = require('internal/modules/helpers');
+          parsedSource = stripTypeScriptTypes(source, url);
         }
         const detectedFormat = detectModuleFormat(parsedSource, url);
         // When source is undefined, default to module-typescript.

lib/internal/modules/esm/translators.js

@@ -38,7 +38,7 @@ const { readFileSync } = require('fs');
 const { dirname, extname, isAbsolute } = require('path');
 const {
   loadBuiltinModule,
-  tsParse,
+  stripTypeScriptTypes,
   stripBOM,
   urlToFilename,
 } = require('internal/modules/helpers');
@@ -309,7 +309,7 @@ translators.set('require-commonjs', (url, source, isMain) => {
 translators.set('require-commonjs-typescript', (url, source, isMain) => {
   emitExperimentalWarning('Type Stripping');
   assert(cjsParse);
-  const code = tsParse(stringify(source));
+  const code = stripTypeScriptTypes(stringify(source), url);
   return createCJSModuleWrap(url, code);
 });
 
@@ -526,7 +526,7 @@ translators.set('wasm', async function(url, source) {
 translators.set('commonjs-typescript', function(url, source) {
   emitExperimentalWarning('Type Stripping');
   assertBufferSource(source, false, 'load');
-  const code = tsParse(stringify(source));
+  const code = stripTypeScriptTypes(stringify(source), url);
   debug(`Translating TypeScript ${url}`);
   return FunctionPrototypeCall(translators.get('commonjs'), this, url, code, false);
 });
@@ -535,7 +535,7 @@ translators.set('commonjs-typescript', function(url, source) {
 translators.set('module-typescript', function(url, source) {
   emitExperimentalWarning('Type Stripping');
   assertBufferSource(source, false, 'load');
-  const code = tsParse(stringify(source));
+  const code = stripTypeScriptTypes(stringify(source), url);
   debug(`Translating TypeScript ${url}`);
   return FunctionPrototypeCall(translators.get('module'), this, url, code, false);
 });

lib/internal/modules/helpers.js

@@ -25,6 +25,7 @@ const path = require('path');
 const { pathToFileURL, fileURLToPath } = require('internal/url');
 const assert = require('internal/assert');
 
+const { Buffer } = require('buffer');
 const { getOptionValue } = require('internal/options');
 const { setOwnProperty } = require('internal/util');
 const { inspect } = require('internal/util/inspect');
@@ -300,7 +301,21 @@ function getBuiltinModule(id) {
   return normalizedId ? require(normalizedId) : undefined;
 }
 
+/**
+ * TypeScript parsing function, by default Amaro.transformSync.
+ *  @type {Function}
+ */
 let typeScriptParser;
+/**
+ * The TypeScript parsing mode, either 'strip-only' or 'transform'.
+ * @type {string}
+ */
+let typeScriptParsingMode;
+/**
+ * Whether source maps are enabled for TypeScript parsing.
+ * @type {boolean}
+ */
+let sourceMapEnabled;
 
 /**
  * Load the TypeScript parser.
@@ -318,22 +333,44 @@ function loadTypeScriptParser(parser) {
   } else {
     const amaro = require('internal/deps/amaro/dist/index');
     // Default option for Amaro is to perform Type Stripping only.
-    const defaultOptions = { __proto__: null, mode: 'strip-only' };
+    typeScriptParsingMode = getOptionValue('--experimental-transform-types') ? 'transform' : 'strip-only';
+    sourceMapEnabled = getOptionValue('--enable-source-maps');
     // Curry the transformSync function with the default options.
-    typeScriptParser = (source) => amaro.transformSync(source, defaultOptions);
+    typeScriptParser = amaro.transformSync;
   }
   return typeScriptParser;
 }
 
 /**
+ * @typedef {object} TransformOutput
+ * @property {string} code The compiled code.
+ * @property {string} [map] The source maps (optional).
+ *
  * Performs type-stripping to TypeScript source code.
  * @param {string} source TypeScript code to parse.
- * @returns {string} JavaScript code.
+ * @param {string} filename The filename of the source code.
+ * @returns {TransformOutput} The stripped TypeScript code.
  */
-function tsParse(source) {
+function stripTypeScriptTypes(source, filename) {
   assert(typeof source === 'string');
   const parse = loadTypeScriptParser();
-  const { code } = parse(source);
+  const options = {
+    __proto__: null,
+    mode: typeScriptParsingMode,
+    sourceMap: sourceMapEnabled,
+    filename,
+    // Transform option is only applied in transform mode.
+    transform: {
+      verbatimModuleSyntax: true,
+    },
+  };
+  const { code, map } = parse(source, options);
+  if (map) {
+    // TODO(@marco-ippolito) When Buffer.transcode supports utf8 to
+    // base64 transformation, we should change this line.
+    const base64SourceMap = Buffer.from(map).toString('base64');
+    return `${code}\n\n//# sourceMappingURL=data:application/json;base64,${base64SourceMap}`;
+  }
   return code;
 }
 
@@ -353,7 +390,7 @@ module.exports = {
   loadBuiltinModule,
   makeRequireFunction,
   normalizeReferrerURL,
-  tsParse,
+  stripTypeScriptTypes,
   stripBOM,
   toRealPath,
   hasStartedUserCJSExecution() {

src/node_options.cc

@@ -808,6 +808,14 @@ EnvironmentOptionsParser::EnvironmentOptionsParser() {
             &EnvironmentOptions::experimental_strip_types,
             kAllowedInEnvvar);
   Implies("--experimental-strip-types", "--experimental-detect-module");
+
+  AddOption("--experimental-transform-types",
+            "enable transformation of TypeScript-only"
+            "syntax into JavaScript code",
+            &EnvironmentOptions::experimental_transform_types,
+            kAllowedInEnvvar);
+  Implies("--experimental-transform-types", "--experimental-strip-types");
+  Implies("--experimental-transform-types", "--enable-source-maps");
   AddOption("--interactive",
             "always enter the REPL even if stdin does not appear "
             "to be a terminal",

src/node_options.h

@@ -241,6 +241,7 @@ class EnvironmentOptions : public Options {
   std::vector<std::string> preload_esm_modules;
 
   bool experimental_strip_types = false;
+  bool experimental_transform_types = false;
 
   std::vector<std::string> user_argv;