[graphql-js] require opt-in to allow defer/stream

[robrichard]

Context

asked by @glasser graphql/graphql-js#2848 (comment)

What this would mean in practice (assuming defer and stream are part of schemas in graphql@16 by default) is that anybody trying to combine graphql@16 with an Apollo Server transport that doesn't support incremental delivery would get a startup error and they'd have to tell graphql@16 to disable defer and stream. This might be a bit of a harsh experience for upgrades but at least it would communicate clearly to folks that a new cool feature is available but their chosen transport doesn't support it.

I might suggest that it would be easiest for the broader ecosystem if graphql@16 graphql@17 required opting your schema in to defer/stream andgraphql@17 graphql@18 required opting out.

Decision

• graphql-js v17 will release without defer & stream enabled by default
• To enable defer & stream, the directives must be explicitly passed to GraphQLSchema

Consequences

• defer/stream will not be listed as a "specified directives" in graphql-js, because the spec does not require it to be present
• It will be included in the SDL created by printSchema
• it will show up in schema introspection results to communicate to clients it is supported

[robrichard]

@glasser The defer-stream branch has been updated with a schema flag enableDeferStream, false by default.

[n1ru4l]

@IvanGoncharov Keeping it in sync for the SDL first flow is as easy as doing the following. Unfortunately, there is no way to get the Directive AST with graphql.js 😢

import { GraphQLDeferDirective, GraphQLStreamDirective } from "graphql"
import { makeExecuteableSchema } from "@graphql-tools/schema";
import { astFromDirective } from "@graphql-tools/utils";

const deferDirectiveAST = astFromDirective(GraphQLDeferDirective)
const streamDirectiveAST = astFromDirective(GraphQLStreamDirective)

const schema = makeExecuteableSchema({
 typeDefs: [...userTypeDefs, deferDirectiveAST, streamDirectiveAST],
 resolvers,
});

The current feature flag implementation also does not allow enabling only defer or only stream.

[n1ru4l]

It might also be important to get the opinion of the other major schema builder libraries.

@hayes for GiraphQL
@jasonkuhrt for Nexus
@sikanhe for gqtx

[maraisr]

That's so verbose @n1ru4l 👀

If we think about it, there is 2 things at play. The schema and what that means to the executor.

1. You opt into the directive in anyway that your schema dialect wants, SDL, directives, nexus codegen, whatever.
   directives: [GraphQLDeferDirective]

and call it a day, everything else is up to the executor, on what that means.

2. You tell the executor what to do, coz the rest is all in revolvers right. in graphql-js that means revolvers can return an AsyncIterable or not. Once this lands in stable, without the executor flag, it'll just await that all as opposed to yielding

execute({schema, document, enableIncrementalDelivery: true})

this api also allows for TypeScript to type arrow, and not offer the AsyncIterable as a union member.

means, the schema can be defined within the current eco system, and then the executor just opts in.

And everyone is happy, and the world makes sense again. Absolutely no reason to over complicate things.

[n1ru4l]

I still think that the (experimental_)enableIncrementalDelivery flag is redundant and can be computed by doing a schema.getDirectives().some(directive => directive === GraphQLDeferDirective || directive === GraphQLStreamDirective) check within the execute logic.

@maraisr What you are proposing is to have both the developer provide the directives to the schema, and then also have the developer provide the enableIncrementalDelivery property for execute?

The example I showed is explicitly for usage with @graphql-tools/schema as @IvanGoncharov raised the concern that people have to maintain their own defer/stream directive definitions. We could adjust the API to do the verboseness under the hood (astFromDirective calls). This, however, should probably not be discussed here and is a different concern.

[maraisr]

The spec clearly states that its a SHOULD not a MUST, so there is cases where you want the directive but not the server counter part. To blindly infer the presence of the behavior purely based on the presence of a directive is a smell, and one we should avoid.

What you are proposing is to have both the developer provide the directives to the schema, and then also have the developer provide the enableIncrementalDelivery property for execute?

Correct.

We can never enforce this behavior on users, as it 100% requires a client that also supports the transport medium. SO until there is an eco-system buy-in, and is majorly used, both need to be provided.

The example I showed is explicitly for usage with @graphql-tools/schema

With all due respect, I don't think we should optimize the api of graphql-js to suffice your library.

You can 100% if youd like during your, what, makeExecuteableSchema infer all the properties you'd like :)

[jasonkuhrt]

Also @tgriesser @Sytten for Nexus.

[n1ru4l]

Since the original discussion got a bit heated, emotional, and too vague (picking certain aspects and ignoring others), I tried to gather all mentioned ideas and concerns, outline the pros and cons, while also adding my personal conclusion.

The problem

The core question that started the debate is the following:

How can we disable/disallow defer & stream usage as a GraphQL Server library maintainer that wraps graphql-js?

Use-case: apollo-server wants to support the latest graphql-js versions as a (peer-)dependency as soon as possible, without the need to support all the new features supported by it, like incremental delivery.

Proposed Solutions

1. Feature Flag within graphql-js GraphQLSchema

With those requirements in mind, the idea came up to disable defer and stream via an enableDeferStream feature flag passed to the GraphQLSchema constructor.

The default value of enableDeferStream is false.

const schema = new GraphQLSchema({
  enableDeferStream: true,
  ...remainingParams,
});

This flag modifies the directive value that can also be passed separate to GraphQLSchema.

const schema = new GraphQLSchema({
  enableDeferStream: true,
  // incrementalDelivery: true will result in adding the stream directive anyways
  // -> It is impossible to only allow defer or stream this way.
  directive: specifiedDirectives.filter((directive) => directive.name !== “stream”),
});

Pros:

• Easy to use API for people that want to enable incremental delivery

Cons:

• Introduces a special feature flag that conflicts with or mutates other config options

• Impossible to allow only defer or only stream

• The schema contains “configuration/flags” for the execution algorithm

• increased complexity within graphql-js

• This does not solve the core issue for library maintainers, as people can pass a GraphQLSchema where enableDeferStream is set to true to the ApolloServerconstructur.
  Thus if library maintainers want to force disable defer and stream they need additional assertion logic (as described in approach 2.) or map the input GraphQLSchema.

  type ServerArgs = {
      schema: GraphQLSchema
      port?: number
      hostname?: string
  }
  
  function createServer({ schema: inputSchema, ...rest }: ServerArgs) {
    const schema = new GraphQLSchema({
      ...inputSchema.toConfig(),
      enableDeferStream: false
    });
    /// ... other logic
  
    createHTTPServer()
  }

2. Assert the GraphQL schema with code

type ServerArgs = {
    schema: GraphQLSchema
    port?: number
    hostname?: string
}

function createServer({ schema, ...rest }: ServerArgs) {
  if (schema.getDirectives().some(directive => directive.name === "defer" || directive.name === "stream")) {
    throw new Error("This framework does not support defer and stream.")
  }

  /// ... other logic

  createHTTPServer()
}

Pros:

• backward compatible with older GraphQL versions
• no instanceof checks are required (we can check whether defer and stream are in the schema via the directive name)
• no graphql-js adjustments required

Cons:

• When servers add support for the next GraphQL.js major version they need to add this check somewhere in their code-base (same as with 1.)
• The schema contains “configuration” for the execution algorithm (same as 1.)

3. Filter the GraphQL schema with code

type ServerArgs = {
    schema: GraphQLSchema
    port?: number
    hostname?: string
}

function createServer({ schema: inputSchema, ...rest }: ServerArgs) {
  const schemaConfig = inputSchema.toConfig()
  schemaConfig.directives = directives.filter(directive => directive.name !== "defer" && directive.name !== "stream"))
  const schema = new GraphQLSchema({ ...schemaConfig })

  /// ... other logic

  createHTTPServer()
}

Pros:

• backward compatible with older GraphQL versions
• no instanceof checks are required (we can check whether defer and stream are in the schema via the directive name)
• no graphql-js adjustments required

Cons:

• When servers add support for the next GraphQL.js major version they need to add this code somewhere in their code-base (same as with 1.)
• The schema contains “configuration” for the execution algorithm (same as 1.)

4. Feature Flag for execute

Maybe the schema should not really be the source of truth on whether defer and stream directives on the operations should yield a stream of results during execution.
Instead of providing the feature flag via the schema, it could be provided to the execute function instead.

execute({
  schema,
  document,
  experimental_enableDeferStream: true,
});

With this approach, the GraphQL schema can contain the defer and stream directives while defer/ stream can still be enabled/disabled via a separate flag for the execute function.

This could have additional benefits (e.g. disable defer/stream during execution on a request basis).

Pros:

• Separation of concerns
  The schema, which is often provided by the user (as there are many ways of building schemas SDL first, code first and each of those would have to introduce a way of proxying the feature flag), is not responsible for holding the configuration on whether defer and stream should be respected during execution

Cons:

• increased complexity within graphql-js
• Additional complexity for people that want to adopt defer/stream
  Users need to provide a flag when calling execute and additionally add the defer and stream directives to their schema.
  This conflicts with the concern raised by Ivan that it is too complicated to get started with defer and stream (having to provide directives to the schema + having to provide a feature flag to the execute function)
  A middle way could be to automatically add defer and stream to the specifiedDirectives array, but require people to provide the feature flag for execute to actually enable the magic.
  In the end, graphql-js is a library that many people do not use directly and only via a wrapper library, thus having a too complex API surface might not be a concern for graphql-js, but rather for the wrapper libraries.

Conclusion

It seems that the discussion drifted a bit apart into different use cases and scenarios. It seems like the current solution (1.) that has already been implemented within the defer-stream graphql-js branch does not really solve the issue it was intended for.

The use-case that should have been solved could have been achieved without having any additional changes within graphql-js and just adding a few assertion lines within the graphql-server libraries (such as apollo-server) that wrap graphql-js.

From the discussion, we can see that there is definitely interest within the community to have that flag for disabling defer and stream during execution (even though the directives are within the schema). Thus, it might be beneficial to have an enableDeferStream (enableIncrementalDelivery) flag for the execute function.

It also seems that the intention for the configuration option was only in temporary nature. However, not every server might wanna support both defer and stream in the future. It might make sense to also give developers the power to disable/enable the functionality even after defer/stream becomes a stable feature.

My proposed solution

By combining all the discussions the following approach might be a viable solution.

1. Defer and Stream Directives are NOT within the specifiedDirectives array starting with GraphQL.js 17 that introduces opt-in stream/defer support
   For adding those a function with for composing them to the base specifiedDirectives should be added.

   import {
     specifiedDirectives,
     experimental_withIncrementalDeliveryDirectives,
     GraphQLSchema,
   } from "graphql";
   
   const schema = new GraphQLSchema({
     ...opts
     directives: experimental_withIncrementalDeliveryDirectives(
      specifiedDirectives
     ),
   });

   The implementation for experimental_withIncrementalDeliveryDirectives could look like the following:

   function experimental_withIncrementalDeliveryDirectives(
     baseDirectives: Array<GraphQLDirective>
   ): Array<GraphQLDirective> {
     return [...baseDirectives, GraphQLDeferDirective, GraphQLStreamDirective]
   }

2. Whether incremental delivery should be done during the execution phase should be configured via a feature flag that is passed to the execute function starting with GraphQL.js 17

   import {
     execute
   } from "graphql";
   import { schema } from "./schema"
   
   execute({
     ...opts,
     schema,
     experimental_incrementalDelivery: true,
   })

HOWEVER, if we introduce these changes behind feature flags anyways, we don't even need to wait for GraphQL.js 17 to ship these changes, we can already do that starting with a GraphQL.js 16 version and then changing the defaults with GraphQL.js 17.

The changes for when this feature becomes stable would be the following:

• GraphQLStreamDirective and GraphQLDeferDirective are by default part of the specifiedDirectives array
• The experimental_withIncrementalDeliveryDirectives function is removed
• The execute flag experimental_enableIncrementalDelivery is renamed to enableIncrementalDelivery and the default value is updated from false to true

If there are too many concerns regarding this, the GraphQL.js 17 release could introduce the following changes instead:

• experimental_withIncrementalDeliveryDirectives is renamed to withIncrementalDeliveryDirectives
• The execute flag experimental_enableIncrementalDelivery is renamed to enableIncrementalDelivery

Then the GraphQL.js 18 release would introduce the following changes:

• GraphQLStreamDirective and GraphQLDeferDirective are by default part of the specifiedDirectives array
• withIncrementalDeliveryDirectives is deprecated
• The default value of the execute flag enableIncrementalDelivery is set to true

[robrichard]

@n1ru4l thank you for writing this up, I think it's very helpful.

I think one additional thing that needs to be considered is that introspection is the primary way for a client to know if their server supports defer/stream. The defer/stream directives should not be in the schema if the server does not support them.

The spec draft currently states

GraphQL implementations are not required to implement the @defer and @stream directives. If either or both of these directives are implemented, they must be implemented according to this specification. GraphQL implementations that do not support these directives must not make them available via introspection.

Separating the concerns of adding the directives and enabling their execution could make it very easy to create a server that's not compliant with the spec.

[yaacovCR]

The defer/stream directives currently only SHOULD be respected, might a server want to disable for an entire request based on high load or some other reason?

A flag on execute could help with that theoretical use case, although certainly could be checked as each directive is evaluated…

[yaacovCR]

Also think it’s important that we avoid flag on schema that skips default defer/stream and disables execution, as I will probably want to implement a custom stream directive that has a custom argument to include info on when individual streams end, for example, respected by graphql-executor, not (yet) by graphql-JS. When the argument is disabled, graphql executor and graphql JS should give identical response, but with current setup, graphql js would not honor stream at all.

i don’t think this is just about my single use case, library, I think it’s about the equation between whether the defer/stream directives are introspectable = whether the DEFAULT defer/stream directives are introspectable

[n1ru4l]

Regarding disabling defer and stream on a request basis, there are also these questions I raised some time ago: graphql/graphql-over-http#167

It might make sense that the server does not use incremental delivery in case the user does not accept multipart responses.

[benjie]

This was a really great summary @n1ru4l; thanks for putting the time into this!

---

TL;DR

In my opinion:

1. adding @defer to schema should be opt-in
2. adding @stream to schema should be opt-in
3. use an executeIncremental function to add incremental support to execution (similar to subscribe) - if you just use execute then incremental features should not be enabled

Reasoning follows...

---

From my perspective, I'm not sure that we necessarily ever want to enable @defer and @stream by default - they significantly increase the complexity level of GraphQL. I personally would have them always as "opt in".

Separately, subscriptions currently require a different method (subscribe) to use; maybe this is an appropriate method for defer/stream too - executeIncremental.

With regards to schema design (and specifically what is exposed to users to indicate the capabilities of the server), I think it makes sense to allow @stream and @defer to be supported separately from each other. However, once execution comes around, I think incremental should be sufficient - I don't think you'd expose both directives but then only support one of them during execution. This lines up with @n1ru4l's experimental_enableDeferStream flag to execute (though I'd rather using a separate function - the simplicity of graphql / execute always returning an object or promise to an object is desirable to me, particularly from a types perspective).

[robrichard]

@IvanGoncharov @n1ru4l @saihaj @maraisr @glasser

I added this to the agenda of the next GraphQL-JS WG meeting on Jan 26.

https://github.com/graphql/graphql-js-wg/blob/main/agendas/2022-01-26.md#agenda

[n1ru4l]

The result from the Working Group discussion:

• We remove the feature flag
• Stream/Defer is enabled by adding the corresponding directives to the GraphQLSchema directives
• GraphQLStreamDirective and GraphQLDeferDirective are not part of specifiedDirectives
• Until we have more real-world use-cases we don't introduce feature flags for execute/subscribe

[glasser]

As one of the original people asking about this, that works for me. Two clarifying questions:

• Does that mean that a schema could have GraphQLDeferDirective without GraphQLStreamDirective, in which case only one would work? (I think that would be reasonable and perhaps even helpful; I know that we've heard a lot more excitement around the simpler @defer than the more complex @stream, though certainly at a transport level it should be straightforward enough to allow both.)
• The plan is still for execute to return PromiseOrValue<ExecutionResult | AsyncGenerator<AsyncExecutionResult, void, void>>; however, it would be valid for a caller that has verified that the schema does not contain GraphQLDeferDirective or GraphQLStreamDirective to assert that the (post-Promise) value is an ExecutionResult?

[n1ru4l]

Yes, a schema can only contain defer or only stream. I also think that defer will reach broader adoption than stream.

Ideally a server would verify a schema does not contain the defer and the stream directive if it does not support incremental deivery or raises an error if execute does not return a Promise.

Additionally it would be an additional layer of security if the AsyncGenerator from execute is lazy aka only starts setting up the underlying event listeners upon the first .next() call on the iterable. That could avoid potential memory leaks.

We would consider adding an executeAsync function that rejects in case the execution results in an AsyncGenerator instead of a promise if really necessary.

[robrichard]

I updated the defer-stream branch with the agreed changes. I also added a tutorial to the new docs site with an example of how to pass the directives to the schema.

[maraisr]

@robrichard how does one get push rights to graphql-js repo 🤔

[robrichard]

What would be the correct way for users who use SDL schemas to enable @defer & @stream before it is added by default? They will need to add the directive to their schema like so

directive @defer(if: Boolean, label: String) on FRAGMENT_SPREAD | INLINE_FRAGMENT
directive @stream(if: Boolean, label: String, initialCount: Int = 0) on FIELD

# rest of schema
type Query {
}

As mentioned previously, this leaves room for error, as the directive definition should match exactly what is in the spec and what graphql-js is expecting.

There is a private function printDirective in pintSchema.ts, should this be exposed publicly? Then we can document that the correct way to enable defer/stream is:

import {GraphQLDeferDirective, GraphQLStreamDirective, printDirective} from 'graphql';

const schema = `
${printDirective(GraphQLDeferDirective)}
${printDirective(GraphQLStreamDirective)}

# rest of schema
type Query {
}
`;

[maraisr]

The directives probably doesn't need to be provided in the template literal, as if you're using graphql-js, they can be provided directly to the buildSchema methods 🤔 so both as-code and as-sdl folk, can both supply the directives in the same way. 🤝

[robrichard]

@maraisr can you show me an example?

[maraisr]

https://github.com/graphql/graphql-js/blob/da5723860e87c97831c02a1137e9431d96c14239/src/type/schema.ts#L388

So;

new GraphQLSchema({
   directives: [GraphQLDeferDirective],
   query: graphql`type Query { ... }`
});

---

Granted my as-code looks a little more like

import { buildGraphQLSchema } from "gqtx";
import { GraphQLDeferDirective, specifiedDirectives } from "graphql";

import { mutation, query } from "./types";

const directives = [...specifiedDirectives, GraphQLDeferDirective];

export const makeSchema = () => {
	const schema = buildGraphQLSchema({
		directives,
		query,
		mutation,
	});

	
	schema._enableDeferStream = true;

	return schema;
};