Netty4ClientHttpRequestFactory buffers (aggregates) all requests/responses [SPR-12631]

[spring-projects-issues]

Francisco Lozano opened SPR-12631 and commented

The usage of HttpObjectAggregator is convenient for a quick implementation, but it makes it unfeasible to handle big uploads/downloads.

A streaming-friendly solution would be very useful for handling big requests.

---

Affects: 4.1.4

Issue Links:

• Netty4ClientHttpRequestFactory always allocates "maxRequestSize" bytes for each request [SPR-12623] #17224 Netty4ClientHttpRequestFactory always allocates "maxRequestSize" bytes for each request
• RestTemplate POST and PUT don't work with Netty4ClientHttpRequestFactory [SPR-14860] #19426 RestTemplate POST and PUT don't work with Netty4ClientHttpRequestFactory

0 votes, 6 watchers

[spring-projects-issues]

Brian Clozel commented

After discussing this issue with the team, we've decided to resolve this issue as "won't fix".

First, the HttpObjectAggregator is not just tailored for quick implementations, but is fairly used in many netty-driven HTTP clients out there. By using this, we're indeed buffering the whole request/response in memory.

You're right though, I've seen netty-driven HTTP clients also adopt streaming use cases; they're using dedicated types and methods on their public facing API and a different set of Netty handlers.

We could in theory support the streaming use case on the request side at least, with a streaming variant of our request implementation for Netty. But the overall design of our RestTemplate support (and RestTemplate itself) is a bit at odds with the way things are designed in Netty.

For example, we don't currently support persistent connections with the Netty client (we're closing the connection for each request/response exchange; i.e. not reusing them).

For now, we think that the changes required to improve the support are far from trivial and we'd rather rely on the existing streaming support already available through other implementations.

Note that, as mentioned in #19448, RestTemplate is not meant to provide users with the streaming response. So the streaming response use case isn't supported in any other implementation. Of course, you can still write your own HttpMessageConverter or ResponseExtractor implementation to work with the live response within a RestTemplate call.

If you'd like to look into an HTTP client that supports streaming both ways, you can give a try to the new WebClient (see reference doc here) - to be released with Spring 5.

Thanks!

[spring-projects-issues]

Francisco Lozano commented

If you are covering this functionality in a newer API, I agree it's perfectly reasonable not to fix this in RestTemplate.

When I said 'quick' I didn't mean it as a bad thing, I think it's good to use HttpObjectAggregator, as long as you know the payload sizes and are comfortable with having them fully allocated in memory.

Thanks for giving it thoughtful consideration!