HTTPClient POST larger files doesn't work anymore.

[comino]

Basic Infos
ESP8266HTTPClient: TCP socket hangs up at sending larger files with.
With stable version is was working just fine even with very large files so it was introduced lately.

Hardware
Hardware: ESP-WROOM2
Core Version: latest github

Description
Im using HTTPClient.sendRequest("POST", file).
With smaller files <1kB it works just fine. With larger files ~50kB it failes with HTTPC_ERROR_SEND_PAYLOAD_FAILED after a long timeout.

The error occurs in this part of the code:

L492-499 ESP8266HTTPClient.cpp

bytesWrite = _tcp->write((const uint8_t *) (buff + bytesWrite), leftBytes);
bytesWritten += bytesWrite;

if(bytesWrite != leftBytes) {
DEBUG_HTTPCLIENT("[HTTP-Client][sendRequest] short write, asked for %d but got %d failed.\n", leftBytes, bytesWrite);
free(buff);
return returnError(HTTPC_ERROR_SEND_PAYLOAD_FAILED);
}

Output
"short write, asked for 1460 but got 0 failed"

So there is no more data send via "_tcp->write()" after ~4kB

Any major change in the WiFiClient which may cause this?

[igrr]

Yes, lots of major changes in WiFiClient.
Can you please mention how you are sending the file? From file system or SD or some other stream?

[comino]

sure. Im just creating a minimal example. Will come back as I know more!

[comino]

I cant reproduce it currently and assume the problem is outside the arduino code.

[comino]

OK its getting more confusing ! I tried the same code in 4 different networks - in 3 of 4 it works just fine.
In 1 of the 4 networks the tcp connection "hangs up" like described above with increasing file size.

Switched the router and still the same.. hard to track this one down, but with stable version it works in 4 of 4.

Any hints how to debug welcome.. Im a bit lost here.

[comino]

The issue appears in this part of ClientContext.h

    size_t _write_from_source(DataSource* ds)
    {
        assert(_datasource == nullptr);
        _datasource = ds;
        _written = 0;
        _write_some();
        while (_datasource && !_noblock) {
            _send_waiting = true;
            esp_yield();
        }
        _send_waiting = false;
        return _written;
    }

And to be more exact - in the "write_some" function -

    void _write_some()
    {
        if (!_datasource || !_pcb) {
            return;
        }

        size_t left = _datasource->available();
        size_t can_send = tcp_sndbuf(_pcb); // <<<<<<<< THIS RETURNS 0 AFTER SOME PACKETS

        if (_pcb->snd_queuelen >= TCP_SND_QUEUELEN) {
            can_send = 0;
        }
        size_t will_send = (can_send < left) ? can_send : left;
        if (will_send) {
            const uint8_t* buf = _datasource->get_buffer(will_send);
            err_t err = tcp_write(_pcb, buf, will_send, TCP_WRITE_FLAG_COPY);
            _datasource->release_buffer(buf, will_send);
            if (err == ERR_OK) {
                _written += will_send;
                tcp_output(_pcb);
            }
        }

The tcp_sndbuf returns 0 ( size_t can_send = tcp_sndbuf(_pcb);) after some packets and following the connections times out.

This one is going deeper into the rabbit hole... So the tcp_write is appearently filling the buffer until its full... Any ideas ?

[me-no-dev]

I'm having similar reports on the Async side with some networks and over 3G. So far the problem is that the other end does not respond with ACK and lwip is holding the packets.

[comino]

ok glad Im not alone ;)

Link to the according AsyncTCP issue: me-no-dev/ESPAsyncTCP#20

[me-no-dev]

what are our options though? If I figure out how to clear the lwip buffer and let other data be sent, what is the assurance that that will work and what will happen in cases where the other end is actually gone and there is nobody to take the data.

[me-no-dev]

@comino can you take tcpdump with the packets so we can look at such transfer?

[me-no-dev]

@comino is the router that have problems a pppoe router? Can you check it's MTU?

[comino]

Im just trying to convince wireshark to show me any packets. Somehow I cant capture the packets from. I will oost them here as I get it to work.

To be honest I have very little information about the router - its: CH7466CE and Google doenst tell me what conenction method its using.

Edit:
MTU:1500 !

[comino]

It's been some time, but afaik the issue is still alive.
I tried to hunt the issue down in lwip, but it went far down the rabbit hole and I gave up.

The issue has been discussed here:
me-no-dev/ESPAsyncTCP#20

Currently I have this issue in only one network I have access to. Its using a Cisco Router with an MTU of 1450.

Citation of @me-no-dev :
"packet fragmentation is not the issue as neither side can really know what the MSS is through the connection in between. The limiting router can be somewhere externally for both and both to have full 1460 available locally. Issue is that lwip does not even have the code for handling "packet needed fragmentation" ICMP messages. The only code for ICMP is for ping and I am still not that familiar with all the goes inside. I do know that newer versions of LwIP have that taken care of and I know that we will update ESP8266... "

Might be solved with:
#3206