Feature Request: Documented Utility Type

[harrysolovay]

Search Terms

Docs, documentation, re-key, utility, type, extract, literal, update, tsdoc, comment, source, text

Suggestion

I'd love to see documentation as a first-class citizen of the type system. My suggestion is a new utility type Documented, which allows users to document and unwrap documentation at the type-level.

The type could look as follows: Documented<V, D extends string> = V.

This:

type X = {
  a: Documented<string, "Some documentation for `a`">;
};

... would be equivalent to this:

type X = {
  /**
   * Some documentation for `a`
   */
  a: string;
};

While it would be preferable for consistency to specify the documentation attached to the key (as this is how mapped types transfer documentation), index signature and other contextual restrictions apply. Aka., this is no good:

type X = {
  [Documented<"a", "Some documentation for `a`">]: string;
}

Additionally, accessing the symbol of the literal of a specific key can be arduous (Extract<keyof ..., ...>). Whereas accessing the symbol of the field is simple (X["a"]).

ALSO: apologies if I'm butchering the terminology.

Use Cases

This utility type would enable users to extract documentation as string literal types, which they could then further manipulate and use elsewhere. They could build up higher-order generic types to assist with and type-check the documentation process.

Examples

type X = {
  /**
   * Some documentation for `a`
   */
  a: string;
};

type XaDocs = X["a"] extends Documented<any, infer D> ? D : undefined;
declare const xaDocs: XaDocs; // "Some documentation for `a`"

// update (prefix) the string literal type of the documentation
type XaDocsUpdated = XaDocs extends string ? `USE WITH CAUTION: ${XaDocs}` : undefined;

// utilize the `Documented` utility to attach the new documentation
type XUpdated1 = {
  a: Documented<string, XaDocsUpdated>;
};

// perhaps within a mapped type
type XUpdated2 = {
  [K in keyof X]: K extends "a" ? Documented<X[K], XaDocsUpdated> : X[K];
};

In either of the cases above, a will be documented as "USE WITH CAUTION: Some documentation for a".

The real power of this utility is not in direct usage as demoed above, but rather in higher-order utilities. For instance, you could imagine a utility library that lets users specify flags that correspond to expected documentation.

import {EnsureDocumentationExists} from "some-doc-utility-lib";
import {HasDescriptionAndExampleDocs, HasDescriptionDocs} from "./my-own-potentially-poorly-documented-types";

export type A = EnsureDocumentationExists<HasDescriptionAndExampleDocs, {description: true; example: true}>;
export type B = EnsureDocumentationExists<HasDescriptionDocs, {description: true; example: true}>; // type-error
export type C = EnsureDocumentationExists<HasDescriptionDocs, {description: true}>;

Checklist

My suggestion meets these guidelines:

• This wouldn't be a breaking change in existing TypeScript/JavaScript code
• This wouldn't change the runtime behavior of existing JavaScript code
• This could be implemented without emitting different JS based on the types of the expressions
• This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)
• This feature would agree with the rest of TypeScript's Design Goals.

[sam-goodwin]

Documentation can be bound to both types and properties:

/**
 * Type-bound docs.
 */
interface A {
  /**
   * Property-bound docs.
   */
  prop: string;
}

How would they be differentiated?

[harrysolovay]

We could create a differently-documented A interface like so:

interface AWithNewDocs extends Documented<A, "New type-bound docs."> {
  prop: Documented<A["prop"], "New property-bound docs.">;
}

Or create the type from scratch.

type AWithDocs = Documented<{
  prop: Documented<string, "Property-bound docs.">
}, "Type-bound docs.">

[harrysolovay]

It just occurred to me, this would be extremely useful for developers of type-level DSL parsers (which I'm guessing is the next wave of TS tooling thanks to ^4.1's string literal parsing).

I've been experimenting with parsing GraphQL source string types into GraphQL AST types (and soon hopefully into TS types). For now, it's tough not to provoke the recursion limiter. A source string with more than a few statements gives TS error 2589. Soon, however, I'd hope it will be possible to parse and extract type information from large samples of many embedded languages.

So how could we make use of GraphQL documentation ("block strings")? Ideally we could extract that documentation from the GraphQL source string type, and reuse it for js/tsdocs.

Let's look at this hypothetical GraphQL-in-TS experience:

import {Parse, CorrespondingTsType} from "type-level-graphql";

type SchemaAst = Parse<`
  """
  Some `User` type description.
  """
  type User {
    """
    Some `User.name` field description.
    """
    name: String!
    """
    Some `User.favoriteLanguage` description.
    """
    favoriteLanguage: String!
  }

  type Query {
    users: [User!]!
  }

  schema {
    query: Query
  }
`>;

type Schema = CorrespondingTsType<SchemaAst>;

declare const schema: Schema;

schema.query.users.?[0]; // documentation is "Some `User` type description."
schema.query.users.?[0].name; // documentation is "Some `User.name` field description."
schema.query.users.?[0].favoriteLanguage; // documentation is "Some `User.favoriteLanguage` description."

Here, we would want for the documentation in the GraphQL source string to flow through into Schema.

A Documented utility type would enable the creation of such an experience.

[micnic]

I might be mistaken, but comments are not actual values in JS/TS or any other programming language and therefore they cannot have a type in anyway, so the syntax you propose is impossible to implement.

[harrysolovay]

@micnic I'm not sure why having or not having a JS value complement would impose whether a syntax is possible. For instance, type params which act as type variables for reusability in others.

type WithVars<
  UserProvided,
  SynthesizedA = SomeComputationA<UserProvided>, // <--
  SynthesizedB = SomeComputationB<UserProvided>, // <--
  SynthesizedC = SomeComputationC<SynthesizedA, SynthesizedB>
> = SynthesizedC;

I'm curious to hear your thoughts on the following arguments for a Documented type:

Js/tsdoc comments are already treated (somewhat) as members of the language. They abide by a specific control flow so that users can re-alias and map between documented types with their documentation preserved. Another similarity with the language: documentation is not critical to the code's runtime execution and can be compiled away (just like static types).

Documentation is a pillar of good software. Yet it's often difficult to give this process the time of day. This is––in part––because documentation cannot be easily modularized, composed, and constrained (not to mention, it involves configuring often-custom workflow tools, which spawn new processes). These challenges can be addressed well within the language. Existing documentation validation tools' flexibility pales in comparison to that of the type system. A Documented utility type bridges the divide.

[tjjfvi]

A few days ago I submitted #41023, which has a similar intent. If this were to be implemented I would suggest a Documentation<T> (or similar) type, that could be used like Documented<T, Documentation<U>>, to accomplish the T & Docs<U> I had suggested there.

As a slight aside, I don't really care for the Documented<T, "blah">, and would advocate for either:

• T & Docs<"blah"> and T & DocsOf<U> or
• WithDocs<T, "blah"> and WithDocs<T, DocsOf<U>>

[harrysolovay]

@tjjfvi it does seem that we're requesting the same thing! Apologies for not finding your issue.

Your proposed solution is certainly more readable!:

Yours

type AWithDocs = {
  prop: string & Docs<"Property-bound docs.">;
} & Docs<"Type-bound docs.">;

Vs.

type AWithDocs = Documented<{
  prop: Documented<string, "Property-bound docs.">
}, "Type-bound docs.">

If we intersect with Docs<"blah">, the result contains a new field (the documentation string keyed by a unique symbol). This means that the Docs utility type would look something like this: type Docs<S extends string> = {[docsKey]: S}.

If we were to document some value...

type A = string & Docs<"Documentation for A.">;

And then utilize that type...

const a: A = "hello"; // TS error 2322: Type 'string' is not assignable to type 'A'.

We'd get a type error.

Would the error––in the case of the Docs utility––be suppressed? (doesn't seem like the right path)

There's also the matter of js/tsdoc -> types.

/**
 * Some description.
 */
type A = number;

Does the signature of A become number & Docs<"Some description.">? (also doesn't seem like the right path)

What I proposed is also flawed, as Documented is essentially an identity, and the documentation string type isn't accessible on the type it's documenting (only through unwrapping with infer). Nothing else in the type system behaves this way. (aka. this also doesn't seem like the right path... but maybe documentation is the exception)

At this point, I have no strong preference for the experience. Just a belief that manipulating documentation within the type system could open some cool & useful doors.

Hopefully we get some more feedback and ideas.

[tjjfvi]

In my mind, the Docs<T> would be "magic" type like ThisType<T>; in lib.d.ts, it would be written as type Docs<T> = unknown, but would receive special casing in the compiler to add the jsdoc and handle the intersection properly.

/**
 * Some description.
 */
type A = number;

Does the signature of A become number & Docs<"Some description.">? (also doesn't seem like the right path)

No; in my mind, both A and B (below) would display as number, but with the Some description. displayed below like it is currently for A.

// In lib.d.ts, with magic special handling
type Docs<T> = unknown;

/**
 * Some description.
 */
type A = number; // Hover `A`

type B = number & Docs<"Some description.">; // Hover `B`; with my proposal it should look identical to `A`.

Playground Link

Originally, I thought that the the DocsOf<T> utility I proposed would be handled with an intrinsic utility type (#40580). However, using conditional types and infer to unwrap it is pleasing, and would be one less "magic" type. Thus, I would likely advocate for the DocsOf<T> to be defined in lib.d.ts as T extends Docs<infer D> ? D : never.

What I proposed is also flawed, as Documented is essentially an identity, and the documentation string type isn't accessible on the type it's documenting (only through unwrapping with infer). Nothing else in the type system behaves this way. (aka. this also doesn't seem like the right path... but maybe documentation is the exception)

IMO, the special handling makes sense for documentation; it isn't really type information, as T & Docs<"A"> === T & Docs<"B">. I could however see an argument that the DocsOf<T> violates this, as it takes "not really type information", and converts it into a literal string type; if this was a concern, I would propose the DocsOf<T> be a similar magic unknown with special handling, and it would be used like A & DocsOf<B>.

While I think people may be resistant to bringing documentation information into the type system, I think it is essential with the complex types typescript is increasingly allowing us to create, as many of these complex type modification lose documentation, like I demonstrated in #41023.

[harrysolovay]

Yes!: it would be an intrinsic utility type, so I suppose there's no need to get too fixated on the "magic" as bad. One more question about your proposal:

/**
 * Some documentation for A.
 */
type A = {aField: string};

type AAndB = {bField: string} & A;

declare const aAndB: AAndB;

In this case, aAndB doesn't contain the documentation of the right-hand of its intersection.

While introducing an intrinsic utility makes sense... I'm not so sure that its usage should enforce a convention for applying the documentation data of Docs<"...">.

If this is the desired experience, I'd hope that––in the above example––aAndB would contain the docs of A. However, what then do we do in this situation?:

/**
 * Some documentation for A.
 */
type A = {aField: string};

/**
 * Some documentation for B.
 */
type B = {bField: string};

type AAndB = A & B;

declare const aAndB: AAndB;

This leads me to believe that you're proposing some new documentation control flow behavior in addition to the intrinsic utility type. Is this the case?

[tjjfvi]

No; rather I was suggesting a special casing wrt intersection types and the Docs<T>/DocsOf<T>, so that A & Docs<"blah"> would transfer the documentation, while B & AWithDocs would not.

I agree that this inconsistency seems non-ideal, but I think it is reasonable:

• For the clarity of the syntax; I think A & Docs<"test"> reads cleaner than Documented<A, "test">
• Otherwise, it would be changing existing behavior wrt intersection and documentation
• As you pointed out, merging two docs in an intersection would be problematic.

This is the "magic" I was talking about in my earlier comments; similar to how ThisType is handled specially.

[tjjfvi]

See also rbuckton's recently submitted #41220

[tjjfvi]

So, after digging around for a while in typescript's source, I have started to figure out how the documentation stuff gets handled.

AFAICT, when it is retrieving the documentation, it:

• Starts at getQuickInfoAtPosition in src/services/services.ts
• Goes through a very roundabout process
• Reaches one of four places: (Note that these functions could have different implementations, as they are checked based on e.g. Symbol and not SymbolObject, but I didn't see any instance of this in the source)
  • return []
  • SignatureObject.getDocumentationComment
    • Essentially becomes getDocumentationComment([this.declaration], this.checker)
  • SymbolObject.getDocumentationComment
    • This has some special handling in the case of a tuple label, which I am going to ignore for right now, as it is a bit of an edge case
    • Returns getDocumentationComment(this.declarations, checker)
  • SymbolObject.getContextualDocumentationComment
    • Essentially boils down to the above, but with some slight special handling in the case of get/set stuff (I think).
• And then all paths seem to reach one glorious function: getDocumentationComment

// ...
    function getDocumentationComment(declarations: readonly Declaration[] | undefined, checker: TypeChecker | undefined): SymbolDisplayPart[] {
        if (!declarations) return emptyArray;

        let doc = JsDoc.getJsDocCommentsFromDeclarations(declarations);
        if (doc.length === 0 || declarations.some(hasJSDocInheritDocTag)) {
            forEachUnique(declarations, declaration => {
                const inheritedDocs = findInheritedJSDocComments(declaration, declaration.symbol.name, checker!); // TODO: GH#18217
                // TODO: GH#16312 Return a ReadonlyArray, avoid copying inheritedDocs
                if (inheritedDocs) doc = doc.length === 0 ? inheritedDocs.slice() : inheritedDocs.concat(lineBreakPart(), doc);
            });
        }
        return doc;
    }
// ...

I attempted to start inspecting what the getDocumentationComment function was being passed, but have not yet been able to get it working.

[kaya3]

This could perhaps be implemented by creating a unique symbol type (e.g. DocComment), and declaring:

type Docs<S extends string> = {[K in DocComment]?: S}
type WithDocs<T, S extends string> = T & Docs<S>
type DocsOf<T> = T[DocComment & keyof T] & string

The changes to the compiler required would then be:

• Declare an intrinsic unique symbol type with the name DocComment,
• Transform literal doc comments from the source code into DocComment properties on the type, and
• When type information is shown to the user, hide DocComment properties and instead show them as documentation.

In the meantime, you can get something close to the desired behaviour in your own projects without compiler support, by declaring the DocComment symbol yourself, and tolerating the less-ergonomic IntelliSense experience.

Playground Link

[tjjfvi]

That wouldn't work, because it would mean that

/** Foo */
type Foo = { a: number }
/** Bar */
type Bar = { a: number }

declare const foo: Foo
const bar: Bar = foo // error, property `DocComment` isn't assignable

Further, adding a property to every type with documentation would be a breaking change, as it would also affect things like keyof.

[harrysolovay]

Another use case: mapping a TS type into a constraint for a JSON schema. For instance, imagine a JsonSchemaOf utility type:

const jsonSchema: JsonSchemaOf<{
  primitive: number;
}> = {
  primitive: {
    type: "number",
  },
};

The JsonSchemaOf utility type could leverage a Documented utility type for extracting metadata, such as default values.

const jsonSchema: JsonSchemaOf<{
  /**
   * @default 101
   */
  primitive: number;
}> = {
  primitive: {
    type: "number",
    default: 102,
//           ^
//           This should produce a type error, as we've documented `primitive` with a `@default` of `101` 
  },
};

Without the Documented utility type, we must encode the constraint in less desirable ways...

const DefaultKey: unique symbol = Symbol();
type DefaultKey = typeof DefaultKey;

type WithDefault<T, Default> = T & {[DefaultKey]: Default};

const jsonSchema: JsonSchemaOf<{
  primitive: WithDefault<number, 101>;
}> = {
  primitive: {
    type: "number",
    default: 101,
  },
};

... this of course requires that we then remove the attached metadata before using elsewhere...

interface MyType {
  primitive: WithDefault<number, 101>;
}

type MyTypeSanitized = {
  [Key in keyof MyType]: Omit<MyType, DefaultKey>
};

Anyways, just thought I'd share this additional use case. Still very much hoping the Documented utility type is given consideration. Thanks!

[entropitor]

@harrysolovay Isn't the above comment the reverse of this ticket? Being able to access documentation of a type (latest comment) instead of being able to attach documentation to a type (the original issue)?

[tjjfvi]

@entropitor The issue is about both, to a degree; see this example from the original post:

type XaDocs = X["a"] extends Documented<any, infer D> ? D : undefined;

[AlCalzone]

I have another use case for this. In a library of mine, I am defining an extensive object through a high level abstraction to avoid a lot of boilerplate code. For example, this input:

const AssociationCCValues = Object.freeze({
	...V.defineStaticCCValues(CommandClasses.Association, {
		/** Whether the node has a lifeline association */
		...V.staticProperty("hasLifeline", { internal: true }),
		/** How many association groups the node has */
		...V.staticProperty("groupCount", { internal: true }),
	}),

	...V.defineDynamicCCValues(CommandClasses.Association, {
		/** The maximum number of nodes in an association group */
		...V.dynamicPropertyAndKeyWithName(
			"maxNodes",
			"maxNodes",
			(groupId: number) => groupId,
			{ internal: true },
		),
		/** The node IDs currently belonging to an association group */
		...V.dynamicPropertyAndKeyWithName(
			"nodeIds",
			"nodeIds",
			(groupId: number) => groupId,
			{ internal: true },
		),
	}),
});

which is being defined in a similar way in over 50 other files with more or less definitions results in an object with the following type, all created through some generic type magic:

declare const AssociationCCValues: Readonly<{
    nodeIds: (groupId: number) => {
        readonly id: {
            commandClass: CommandClasses.Association;
            property: "nodeIds";
            propertyKey: number;
        };
        readonly endpoint: (endpoint?: number | undefined) => {
            readonly commandClass: CommandClasses.Association;
            readonly endpoint: number;
            readonly property: "nodeIds";
            readonly propertyKey: number;
        };
        readonly meta: {
            readonly readable: true;
            readonly writable: true;
            readonly minVersion: 1;
            readonly stateful: true;
            readonly secret: false;
            readonly internal: true;
        };
    };
    maxNodes: (groupId: number) => {
        readonly id: {
            commandClass: CommandClasses.Association;
            property: "maxNodes";
            propertyKey: number;
        };
        readonly endpoint: (endpoint?: number | undefined) => {
            readonly commandClass: CommandClasses.Association;
            readonly endpoint: number;
            readonly property: "maxNodes";
            readonly propertyKey: number;
        };
        readonly meta: {
            readonly readable: true;
            readonly writable: true;
            readonly minVersion: 1;
            readonly stateful: true;
            readonly secret: false;
            readonly internal: true;
        };
    };
    groupCount: {
        readonly id: {
            commandClass: CommandClasses.Association;
            property: "groupCount";
        };
        readonly endpoint: (endpoint?: number | undefined) => {
            readonly commandClass: CommandClasses.Association;
            readonly endpoint: number;
            readonly property: "groupCount";
        };
        readonly meta: {
            readonly readable: true;
            readonly writable: true;
            readonly minVersion: 1;
            readonly stateful: true;
            readonly secret: false;
            readonly internal: true;
        };
    };
    hasLifeline: {
        readonly id: {
            commandClass: CommandClasses.Association;
            property: "hasLifeline";
        };
        readonly endpoint: (endpoint?: number | undefined) => {
            readonly commandClass: CommandClasses.Association;
            readonly endpoint: number;
            readonly property: "hasLifeline";
        };
        readonly meta: {
            readonly readable: true;
            readonly writable: true;
            readonly minVersion: 1;
            readonly stateful: true;
            readonly secret: false;
            readonly internal: true;
        };
    };
}>

While this is very helpful at the usage site, being able to preserve the comments as JSDoc on the resulting object properties nodeIds etc. would be a big improvement.

Just the comments would be enough in my case, I don't see a need for JSDoc tags at the moment.

[magwas]

@MartinJohns I am confused about whether and how this proposal would allow documentation generated by external tools to be inserted such that it appears at the IDE at the end of the day, as of the motivation of #53205

For example if I have

export const plankLength: number = 5

I could of course generate a barrel like

import {plankLength as plankLength_orig} from "original"
const plankLength: WithDocs<number, 'The length of a wooden plank, in metres'> = plankLenght_orig;

but I would rather not get involved in generating the barrel, as my responsibility is to check contracts and document them, not determining what will be exported or not.

something like the below would be okay, but I do not see how MagicallyInsertDocumentation would work, and what other steps should be done to actually have the comment in the IDE.

import {plankLength} from "original"
MagicallyInsertDocumentation<plankLength, 'The length of a wooden plank, in metres'>