Testing Subscriptions using Websocket Client not working

[sgrannan]

What is the proper way to test subscriptions using the WebSocketGraphqlTester client? Do you have documentation/samples that actually show a full test with the WebSocketGraphqlTester implementation?

I could not seem to find any examples that fit my needs in the repos. I believe I need to test with a test client because I have custom Filter and custom Interceptors that have logic for handling authentication and logging of requests. I would like to ensure they are executed within each test.

I have followed the documentation (here) that seems to be based on a ReactorNettyWebSocketClient to try to get this tested so far, but I am unable to get it to work because I receive errors. Here is the client setup in my test:

String authToken = "Bearer <token>";
  String url = "http://localhost:" + testPort + "/subscriptions";
  ReactorNettyWebSocketClient client = new ReactorNettyWebSocketClient();
  WebSocketGraphQlTester tester = WebSocketGraphQlTester.create(URI.create(url), client);

  var response = tester.mutate().headers(headers -> headers.add(HttpHeaders.AUTHORIZATION, authToken)).build()
          .document(readFile(deviceSubscription)) // Process the graphqls file into string (ex: subscription { device (id: $deviceId) { id label } }
          .variable("deviceId", "12345") // Add ID as variable
          .executeSubscription().toFlux("device", Device.class);

StepVerifier.create(response).verifyComplete();  // ERROR thrown here

Here are my Spring Boot dependencies:

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-graphql</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-websocket</artifactId>
</dependency>

And Test Dependencies:

<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-test</artifactId>
	<scope>test</scope>
</dependency>
<dependency>
	<groupId>io.projectreactor</groupId>
	<artifactId>reactor-test</artifactId>
	<scope>test</scope>
</dependency>
<dependency>
	<groupId>org.springframework.graphql</groupId>
	<artifactId>spring-graphql-test</artifactId>
	<scope>test</scope>
</dependency>
<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-webflux</artifactId>
	<scope>test</scope>
</dependency>

However, I get a 403 error: org.springframework.graphql.client.GraphQlTransportException: GraphQlTransport error: Invalid handshake response getStatus: 403

With this stack trace:

Error has been observed at the following site(s):
*__________Mono.error ⇢ at org.springframework.graphql.client.DefaultGraphQlClient$DefaultRequestSpec.lambda$execute$1(DefaultGraphQlClient.java:161)
*__Mono.onErrorResume ⇢ at org.springframework.graphql.client.DefaultGraphQlClient$DefaultRequestSpec.lambda$execute$2(DefaultGraphQlClient.java:159)
*______Mono.flatMap ⇢ at org.springframework.graphql.client.DefaultGraphQlClient$DefaultRequestSpec.execute(DefaultGraphQlClient.java:158)
| Mono.cast ⇢ at org.springframework.graphql.test.tester.AbstractGraphQlTesterBuilder$1.execute(AbstractGraphQlTesterBuilder.java:167)
| Mono.map ⇢ at org.springframework.graphql.test.tester.DefaultGraphQlTester$DefaultRequest.execute(DefaultGraphQlTester.java:156)

I should point out that I was required to use the Webflux dependency in test to get the WebsocketGraphqlTester properly configured. I believe this may be a source of the issue as some security configuration has been autoconfigured now? My project is more Web MVC like, even with Subscriptions implementation. Note: This is a conversion from spring-boot-graphql-kickstarter.

Can you point me in the right direction that fits my needs or does Spring GraphQL not support this kind of testing yet?

[sgrannan]

I have figured out that whatever annotations are processed with @SpringBootTest, the GraphQlWebMvcAutoConfiguration.WebSocketConfiguration is not autowired under these circumstances, and such no WebSocketGraphqlHandler is wired up to map to the /subscriptions endpoint that is set.

This is all that my test class uses:

@SpringBootTest(
        classes = {MySocketsApplication.class},
        webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT,
        properties = {"spring.profiles.active=unit-test"}
)

For a client to work, it would need the handler registration at the least. Why is there no autoconfiguration happening for the WebSocketConfiguration? Is there something manual I can do to force configuration?

[sgrannan]

I found my problem after spending a whole day on it. Turns out its directly related to #308 because the main GraphqlAutoconfiguration was never happening since test configuration could not find my schema on in the main/resources/graphql folder.

I think it would be much appreciated it if you could fix that by having it check classpath*: and release it in any upcoming 1.2.x release.

[bclozel]

I'm sorry you spent so much time finding the cause here.

I don't think we'll change the behavior for #308 as this was solved with #338 - we've separated the default locations for the schema file and the query files. If we were to mix both, you could run into situations where the schema is different in your tests than in production (by having an additional schema file in your test resources). Worse, your application could pick up a schema file that's mistakenly contributed by one of your dependencies.

If you're running into similar problem in the future, I would suggest first looking at the startup logs as you should see GraphQL related statements:

[  restartedMain] [                                                 ] efaultSchemaResourceGraphQlSourceBuilder : Loaded 1 resource(s) in the GraphQL schema.
[  restartedMain] [                                                 ] s.b.a.g.s.GraphQlWebMvcAutoConfiguration : GraphQL endpoint HTTP POST /graphql

If this is not the case, you can add debug=true to your application.properties (or run your app with the DEBUG env variable). You should then get the auto-configuration report that will explain why auto-configurations matched or didn't match:

============================
CONDITIONS EVALUATION REPORT
============================


Positive matches:
-----------------

...

   GraphQlAutoConfiguration matched:
      - @ConditionalOnClass found required classes 'graphql.GraphQL', 'org.springframework.graphql.execution.GraphQlSource' (OnClassCondition)
      - @ConditionalOnGraphQlSchema found schema 'file [/Users/bclozel/workspace/observing-graphql-in-action/build/resources/main/graphql/schema.graphqls]'; @ConditionalOnGraphQlSchema did not find GraphQlSourceBuilderCustomizer (DefaultGraphQlSchemaCondition)

In your case, you should have seen that no schema was found in the detected location.

[sgrannan]

@bclozel I appreciate your response. However you should understand that the error that I was originally getting was a red herring which led me down the wrong path thinking it was something else to do with Spring Security configuration. I did have graphql set to debug, which is how I eventually found the problem with the report items you noted.

Because I was not having any issue running the application regularly, there was no reason to believe that the schema was a problem. In other graphql implementations, if no schema is found, they would throw an error no matter if for tests or for production. The user shouldn't have to dig into debug logs to realize that a critical requirement is missing. I would encourage you to rethink your strategy in surfacing such issues.

Lastly, why is there no documentation on #338 that describes the default configuration for GraphqlTesters on any of the Spring GraphQL Testing ([here])(https://docs.spring.io/spring-graphql/reference/testing.html) pages?

[bclozel]

@sgrannan thanks for the feedback. It's still not clear to me what was the intent behind adding a graphql folder in your test resources. Can you elaborate?

[bclozel]

Lastly, why is there no documentation on #338 that describes the default configuration for GraphqlTesters on any of the Spring GraphQL Testing ([here])(https://docs.spring.io/spring-graphql/reference/testing.html) pages?

You mean something like this?

[bclozel]

In other graphql implementations, if no schema is found, they would throw an error no matter if for tests or for production.

We could do this, but this would prevent anyone from using the GraphQL client in an application that only consumes an external API. If the application itself doesn't provide a schema, we would fail in all cases. Thoughts?

The user shouldn't have to dig into debug logs to realize that a critical requirement is missing. I would encourage you to rethink your strategy in surfacing such issues.

I agree, configuring debug logs on graphql-java is not an effective strategy for understanding setup issues. What about the auto-configuration report as described above? Wouldn't this help in this case?

[sgrannan]

@sgrannan thanks for the feedback. It's still not clear to me what was the intent behind adding a graphql folder in your test resources. Can you elaborate?

The graphql folder under my test resources houses all specific queries that are performed during unit tests, thereby keeping inline query/mutation/subscription strings out of my unit test classes. It also allows me to re-use queries in other tests just by changing the variables.

[sgrannan]

Lastly, why is there no documentation on #338 that describes the default configuration for GraphqlTesters on any of the Spring GraphQL Testing ([here])(https://docs.spring.io/spring-graphql/reference/testing.html) pages?

You mean something like this?

For example, given a file called projectReleases.graphql in src/main/resources/graphql-test, with content:

I try not to pollute the main portion of my applications with test data. There are a separation of duties, and test data stays in the test packages. Do you not feel that your configuration suggestion for using graphql-test is rather opinionated when it should not be?

To give you some background, I am coming as a conversion from spring-boot-graphql-kickstart library, and I'm sure many other people will be as well since it has been archived and Spring GraphQL is now the blessed approach.

[bclozel]

I try not to pollute the main portion of my applications with test data. There are a separation of duties, and test data stays in the test packages.

I think it's a typo, it should be src/test/resources/graphql-test.

Do you not feel that your configuration suggestion for using graphql-test is rather opinionated when it should not be?

That's just a shortcut - you can load the document however you want and provide it with the document(String) method. We could make that more flexible, but at the price of adding an unnecessary complex setup for the tester.