Multi-part upload of a byte[] with WebTestClient is Base64 encoded (?) and assigned a JSON content type [SPR-16350]

[spring-projects-issues]

Andy Wilkinson opened SPR-16350 and commented

Consider the following code:

MultiValueMap<String, Object> multipartData = new LinkedMultiValueMap<>();
multipartData.add("file", new byte[] { 'a', 'b', 'c', 'd' });
WebTestClient.bindToRouterFunction(RouterFunctions.route(POST("/foo"), (req) -> {
	MultiValueMap<String, Part> parts = req.body(BodyExtractors.toMultipartData())
			.block();
	Part filePart = parts.getFirst("file");
	ByteArrayOutputStream contentStream = new ByteArrayOutputStream();
	DataBufferUtils.write(filePart.content(), contentStream).blockFirst();
	System.out.println(new String(contentStream.toByteArray()));
	System.out.println(filePart.headers());
	return null;
})).configureClient().baseUrl("http://localhost").build().post().uri("/foo")
		.body(BodyInserters.fromMultipartData(multipartData)).exchange()
		.expectBody().returnResult();

With recent 5.0.3 snapshots it produces the following output:

["YWJjZA=="]
{content-disposition=[form-data; name="file"], content-type=[application/json;charset=UTF-8]}

With 5.0.2.RELEASE the following output is produced:

abcd
{content-disposition=[form-data; name="file"]}

There also appears to be a related regression when using a ByteArrayResource that results in an exception, presumably because it's being treated as JSON:

reactor.core.Exceptions$ErrorCallbackNotImplemented: org.springframework.core.codec.CodecException: Type definition error: [simple type, class java.io.ByteArrayInputStream]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class java.io.ByteArrayInputStream and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: java.util.LinkedList[0]->org.springframework.restdocs.webtestclient.WebTestClientRequestConverterTests$1["inputStream"])
Caused by: org.springframework.core.codec.CodecException: Type definition error: [simple type, class java.io.ByteArrayInputStream]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class java.io.ByteArrayInputStream and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: java.util.LinkedList[0]->org.springframework.restdocs.webtestclient.WebTestClientRequestConverterTests$1["inputStream"])
	at org.springframework.http.codec.json.AbstractJackson2Encoder.encodeValue(AbstractJackson2Encoder.java:136)
	at org.springframework.http.codec.json.AbstractJackson2Encoder.lambda$encode$0(AbstractJackson2Encoder.java:100)
	at org.springframework.http.codec.json.AbstractJackson2Encoder$$Lambda$129/1362546706.apply(Unknown Source)
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:107)
	at reactor.core.publisher.FluxJust$WeakScalarSubscription.request(FluxJust.java:91)
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:156)
	at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:1463)
	at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:1337)
	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
	at reactor.core.publisher.FluxJust.subscribe(FluxJust.java:68)
	at reactor.core.publisher.FluxMapFuseable.subscribe(FluxMapFuseable.java:63)
	at reactor.core.publisher.Flux.subscribe(Flux.java:6621)
	at reactor.core.publisher.FluxConcatArray$ConcatArraySubscriber.onComplete(FluxConcatArray.java:200)
	at reactor.core.publisher.FluxConcatArray.subscribe(FluxConcatArray.java:80)
	at reactor.core.publisher.MonoIgnoreElements.subscribe(MonoIgnoreElements.java:37)
	at reactor.core.publisher.Mono.subscribe(Mono.java:3008)
	at reactor.core.publisher.Mono.subscribeWith(Mono.java:3116)
	at reactor.core.publisher.Mono.subscribe(Mono.java:2893)
	at org.springframework.http.codec.multipart.MultipartHttpMessageWriter.encodePart(MultipartHttpMessageWriter.java:262)
	at org.springframework.http.codec.multipart.MultipartHttpMessageWriter.lambda$encodePartValues$3(MultipartHttpMessageWriter.java:225)
	at org.springframework.http.codec.multipart.MultipartHttpMessageWriter$$Lambda$127/57497692.apply(Unknown Source)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.LinkedList$LLSpliterator.forEachRemaining(LinkedList.java:1235)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:512)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:502)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
	at org.springframework.http.codec.multipart.MultipartHttpMessageWriter.encodePartValues(MultipartHttpMessageWriter.java:225)
	at org.springframework.http.codec.multipart.MultipartHttpMessageWriter.lambda$writeMultipart$2(MultipartHttpMessageWriter.java:209)
	at org.springframework.http.codec.multipart.MultipartHttpMessageWriter$$Lambda$80/576020159.apply(Unknown Source)
	at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.drain(FluxConcatMap.java:357)
	at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onSubscribe(FluxConcatMap.java:210)
	at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:128)
	at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:61)
	at reactor.core.publisher.FluxConcatMap.subscribe(FluxConcatMap.java:121)
	at reactor.core.publisher.Flux.subscribe(Flux.java:6621)
	at reactor.core.publisher.FluxConcatArray$ConcatArraySubscriber.onComplete(FluxConcatArray.java:200)
	at reactor.core.publisher.FluxConcatArray.subscribe(FluxConcatArray.java:80)
	at reactor.core.publisher.FluxPeek.subscribe(FluxPeek.java:83)
	at reactor.core.publisher.FluxPeek.subscribe(FluxPeek.java:83)
	at reactor.core.publisher.FluxPeek.subscribe(FluxPeek.java:83)
	at reactor.core.publisher.FluxPeek.subscribe(FluxPeek.java:83)
	at reactor.core.publisher.Flux.subscribe(Flux.java:6621)
	at reactor.core.publisher.Flux.subscribeWith(Flux.java:6788)
	at reactor.core.publisher.Flux.subscribe(Flux.java:6614)
	at reactor.core.publisher.Flux.subscribe(Flux.java:6578)
	at org.springframework.http.codec.multipart.SynchronossPartHttpMessageReader$SynchronossPartGenerator.accept(SynchronossPartHttpMessageReader.java:133)
	at org.springframework.http.codec.multipart.SynchronossPartHttpMessageReader$SynchronossPartGenerator.accept(SynchronossPartHttpMessageReader.java:109)
	at reactor.core.publisher.FluxCreate.subscribe(FluxCreate.java:92)
	at reactor.core.publisher.MonoCollect.subscribe(MonoCollect.java:66)
	at reactor.core.publisher.MonoMapFuseable.subscribe(MonoMapFuseable.java:59)
	at reactor.core.publisher.Mono.block(Mono.java:1161)
	at org.springframework.restdocs.webtestclient.WebTestClientRequestConverterTests.lambda$8(WebTestClientRequestConverterTests.java:187)
	at org.springframework.restdocs.webtestclient.WebTestClientRequestConverterTests$$Lambda$10/1638215613.handle(Unknown Source)
	at org.springframework.web.reactive.function.server.RouterFunctions.lambda$null$3(RouterFunctions.java:232)
	at org.springframework.web.reactive.function.server.RouterFunctions$$Lambda$116/945722724.get(Unknown Source)
	at org.springframework.web.reactive.function.server.RouterFunctions.wrapException(RouterFunctions.java:240)
	at org.springframework.web.reactive.function.server.RouterFunctions.lambda$null$4(RouterFunctions.java:232)
	at org.springframework.web.reactive.function.server.RouterFunctions$$Lambda$110/769798433.apply(Unknown Source)
	at reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:141)
	at reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:53)
	at reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:60)
	at reactor.core.publisher.MonoOnErrorResume.subscribe(MonoOnErrorResume.java:44)
	at reactor.core.publisher.Mono.subscribe(Mono.java:3008)
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.drain(MonoIgnoreThen.java:167)
	at reactor.core.publisher.MonoIgnoreThen.subscribe(MonoIgnoreThen.java:56)
	at reactor.core.publisher.Mono.subscribe(Mono.java:3008)
	at reactor.core.publisher.Mono.subscribeWith(Mono.java:3116)
	at reactor.core.publisher.Mono.subscribe(Mono.java:3002)
	at reactor.core.publisher.Mono.subscribe(Mono.java:2969)
	at reactor.core.publisher.Mono.subscribe(Mono.java:2941)
	at org.springframework.test.web.reactive.server.HttpHandlerConnector.lambda$connect$1(HttpHandlerConnector.java:90)
	at org.springframework.test.web.reactive.server.HttpHandlerConnector$$Lambda$65/1108924067.apply(Unknown Source)
	at org.springframework.mock.http.client.reactive.MockClientHttpRequest.lambda$null$2(MockClientHttpRequest.java:125)
	at org.springframework.mock.http.client.reactive.MockClientHttpRequest$$Lambda$95/945591847.get(Unknown Source)
	at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:44)
	at reactor.core.publisher.Mono.subscribe(Mono.java:3008)
	at reactor.core.publisher.FluxConcatIterable$ConcatIterableSubscriber.onComplete(FluxConcatIterable.java:141)
	at reactor.core.publisher.FluxConcatIterable.subscribe(FluxConcatIterable.java:60)
	at reactor.core.publisher.MonoSourceFlux.subscribe(MonoSourceFlux.java:47)
	at reactor.core.publisher.Mono.subscribe(Mono.java:3008)
	at reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:172)
	at reactor.core.publisher.MonoFlatMap.subscribe(MonoFlatMap.java:53)
	at reactor.core.publisher.Mono.subscribe(Mono.java:3008)
	at reactor.core.publisher.Mono.subscribeWith(Mono.java:3116)
	at reactor.core.publisher.Mono.subscribe(Mono.java:3002)
	at reactor.core.publisher.Mono.subscribe(Mono.java:2969)
	at reactor.core.publisher.Mono.subscribe(Mono.java:2941)
	at org.springframework.test.web.reactive.server.HttpHandlerConnector.connect(HttpHandlerConnector.java:101)
	at org.springframework.test.web.reactive.server.WiretapConnector.connect(WiretapConnector.java:73)
	at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.exchange(ExchangeFunctions.java:74)
	at org.springframework.web.reactive.function.client.DefaultWebClient$DefaultRequestBodyUriSpec.exchange(DefaultWebClient.java:326)
	at org.springframework.test.web.reactive.server.DefaultWebTestClient$DefaultRequestBodyUriSpec.exchange(DefaultWebTestClient.java:282)
	at org.springframework.restdocs.webtestclient.WebTestClientRequestConverterTests.multipartUploadFromResource(WebTestClientRequestConverterTests.java:191)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class java.io.ByteArrayInputStream and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: java.util.LinkedList[0]->org.springframework.restdocs.webtestclient.WebTestClientRequestConverterTests$1["inputStream"])
	at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
	at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
	at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:312)
	at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.failForEmpty(UnknownSerializer.java:71)
	at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.serialize(UnknownSerializer.java:33)
	at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
	at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
	at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145)
	at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107)
	at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25)
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:400)
	at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1392)
	at com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(ObjectWriter.java:1120)
	at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:950)
	at org.springframework.http.codec.json.AbstractJackson2Encoder.encodeValue(AbstractJackson2Encoder.java:133)
	... 117 more

---

Affects: 5.0.3

Issue Links:

• Support Publishers for multipart data in BodyInserters [SPR-16307] #20854 Support Publishers for multipart data in BodyInserters

Referenced from: commits 93a522f

[spring-projects-issues]

Andy Wilkinson commented

The first problem can be avoided by explicitly setting the part's content type:

MultipartBodyBuilder bodyBuilder = new MultipartBodyBuilder();
bodyBuilder.part("file", new byte[] { 1, 2, 3, 4 },
		MediaType.APPLICATION_OCTET_STREAM);
WebTestClient.bindToRouterFunction(RouterFunctions.route(POST("/foo"), (req) -> {
	req.body(BodyExtractors.toMultipartData()).block();
	return null;
})).configureClient().baseUrl("http://localhost").build().post().uri("/foo")
		.body(BodyInserters.fromObject(bodyBuilder.build())).exchange()
		.expectBody().returnResult();

[spring-projects-issues]

Andy Wilkinson commented

As can the second:

MultipartBodyBuilder bodyBuilder = new MultipartBodyBuilder();
bodyBuilder.part("file", new ByteArrayResource(new byte[] { 1, 2, 3, 4 }) {

	@Override
	public String getFilename() {
		return "image.png";
	}

}, MediaType.IMAGE_PNG);
WebTestClient.bindToRouterFunction(RouterFunctions.route(POST("/foo"), (req) -> {
	req.body(BodyExtractors.toMultipartData()).block();
	return null;
})).configureClient().baseUrl("http://localhost").build().post().uri("/foo")
		.body(BodyInserters.fromObject(bodyBuilder.build())).exchange()
		.expectBody().returnResult();

However, this rather seems to defeat the purpose of using a Resource where, in 5.0.2, the content type of the part was inferred from its file name.

[spring-projects-issues]

Rossen Stoyanchev commented

Repro project https://github.com/spring-projects/spring-framework-issues/tree/master/SPR-16350.

[spring-projects-issues]

Rossen Stoyanchev commented

Indeed this was introduced very recently with #7035ee when BodyInserters was updated to use MultipartBodyBuilder internally. The part method is only prepared to take a single value but we were passing the value from a MultiValueMap instead (i.e. a List) which caused Jackson to get involved. Thanks for catching it before it was even released!