Default header receive buffer size CONFIG_HTTPD_MAX_REQ_HDR_LEN=512 too small

[craigdetter]

The default in build for the sdkconfig configuration parameter is:
CONFIG_HTTPD_MAX_REQ_HDR_LEN=512 (bytes).

On Google Chrome latest version running on MacOS on the latest Catalina OS:
using Wireshark trace - shows the "post" header is 554 bytes long (not incl. CR/LF)

It needs to be set at least 768 bytes, but 1024 would be better.

See details here:
https://github.com/espressif/arduino-esp32/issues/5969

[craigdetter]

The 554 byte header size referenced in the original post uses a plain old HTML5 form with a method="post".

Out of curiosity, I also captured a Wireshark trace that employs a Javascript XMLHttpRequest to send data to the ESP32. Writing a quick test page a Javascript XMLHttpRequest currently uses a 495-byte header (just under 512) - again using the Chrome browser. If the Chrome team makes a small change, sayonara.

The Javascript test code used:

var gHttpReq;

function postHttpData() {
	
	if (window.XMLHttpRequest) { 
	  gHttpReq = new XMLHttpRequest();
	}
	else {
	  gHttpReq = new ActiveXObject("Microsoft.XMLHTTP");
	}
	
	gHttpReq.open("POST", "/post/setup_form_content?"); 
	gHttpReq.onload = function(event){ 
	    alert("Success, server responded with: " + event.target.response); 
	}; 
	
	var formData = new FormData(document.getElementById("setup-form")); 
	gHttpReq.send(formData);
}

The Wireshark trace captured this header:

Host: 192.168.4.1\r\n
Connection: keep-alive\r\n
Content-Length: 592\r\n
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36\r\n
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryPKtEBbYhTg3YzO0z\r\n
Accept: /\r\n
Origin: http://192.168.4.1\r\n
Referer: http://192.168.4.1/\r\n
Accept-Encoding: gzip, deflate\r\n
Accept-Language: en-US,en;q=0.9,pl;q=0.8\r\n
\r\n

See packet 1 in the Wireshark trace screen snapshot below....

[LenShustek]

This problem has been talked about for years in various forums. Will it ever get fixed?

Until this is fixed, a server using arduino-esp32 that implements any web page buttons will fail when the client browser is Chrome or Edge. It works with Firefox and the (obsolete) Internet Explorer, but a that's not a requirement you can impose on users. Please help, or we need to abandon the Arduino IDE!

[me-no-dev]

Will it ever get fixed?

Done. Sorry for the delay :)

[LenShustek]

Wonderful, thanks! When will the change appear in espressif/arduino-esp32 so we can use it?

[me-no-dev]

Next time that there is something new in ESP-IDF v4.4, the lib-builder will rebuild and create PR to arduino-esp32. When we merge that PR, it will be available through git.

[Patsjemoe]

Hi,
I was using Firefox, so could get around it.
But now I am using Webview in Android Studio, and have again the problem of headers too long.
Changing the HTTPD_MAX value in sdkconfig does not solve the issue.
Is there a "real" solution in the mean time ?
thanks in advance
best regards
Ludo

[DonMorr]

Hi, I was using Firefox, so could get around it. But now I am using Webview in Android Studio, and have again the problem of headers too long. Changing the HTTPD_MAX value in sdkconfig does not solve the issue. Is there a "real" solution in the mean time ? thanks in advance best regards Ludo

Hi @Patsjemoe , did you have any luck sorting this? I'm hitting the exact same issue. Cheers, Donal

[LenShustek]

I believe it was fixed in version 2.0.4 of the Espressif Arduino core at https://github.com/espressif/arduino-esp32. I see they're up to 2.0.5 now.