Excessively large number of StringBuilder objects allocated in FormReader for big input data #2722
Comments
|
From @yuwaMSFT on Monday, March 14, 2016 12:16:41 PM @muratg It's not. It's a stress app |
|
From @sajayantony on Monday, March 14, 2016 12:19:06 PM @muratg @Eilon Do you want to mark this as label this as stress. FormReader is currently failing stress mix. So I've asked @yuwaMSFT to remove that scenario for now and see how far along it can progress. |
|
From @Tratcher on Monday, March 14, 2016 12:23:36 PM We can pool StringBuilders. aspnet/HttpAbstractions#578 will help a little by removing the backing buffer. A workaround for large forms is using the WebUtilties libraries to read them in a streamed fashion without building up a large collection. |
|
From @yuwaMSFT on Monday, March 14, 2016 12:45:08 PM Also for large forms, they could stack up in Gen 2 quickly since the request takes time to finish. I ran the test for <5mins and disconnected all clients. Checked the StringBuilder objs in heap: |
|
From @Tratcher on Monday, March 14, 2016 12:46:53 PM The string builders are only used in the initial parsing, they aren't held for the duration of the request are they? |
|
From @yuwaMSFT on Monday, March 14, 2016 12:56:40 PM From the code it's supposed to be cleared after we read the complete key/value pair...But if the form data is big it may take a while to finish reading I guess. Also since there are more allocations, we may do GC more frequently and push the objs to the next generate faster? |
|
From @Tratcher on Monday, March 14, 2016 1:31:12 PM What does your data look like? form-url-encoded? Lots of small keys/values, or large keys/values? |
|
From @yuwaMSFT on Monday, March 14, 2016 2:00:47 PM Yes, the default content type. It's a single pair with big value. |
|
From @halter73 on Monday, March 14, 2016 3:59:41 PM @dougbu Based on you dotpeek screenshots it looks like the strings from For some reason, @yuwaMSFT is seeing tens of thousands of StringBuilders (2000 per 16MB post and that's StringBuilders not strings) remaining in the heap leading to the process running out of memory and being killed by the Linux kernel. |
|
From @Tratcher on Monday, March 14, 2016 4:01:00 PM Is it because he's only sending a single big value and it's taking a while to receive and build? |
|
From @halter73 on Monday, March 14, 2016 4:21:08 PM I'm assuming @yuwaMSFT is hitting TextContentController.AddContent with this type of request. So probably. @yuwaMSFT Is this correct? I assume EnableRewind is also exacerbating this issue. |
|
From @Tratcher on Monday, March 14, 2016 4:25:09 PM Note EnableRewind is being removed: aspnet/HttpAbstractions#578 |
|
This should have been fixed by @jkotalik 's refactoring to use pipelines |
ghost
locked as resolved and limited conversation to collaborators
amcasey
added
area-networking
From @yuwaMSFT on Friday, March 11, 2016 10:54:07 AM
In one stress run on Linux we hit OOM. During investigation, we saw excessive StringBuilder objects allocated in FormReader. The implementation will keep reading the stream until reaching a separator. In the test, we were posting faked large data ( > 16MB), which could cause the allocation of 2000 StringBuilder objects (8K chunk for each). If keep them all in memory we could only support very small number of clients. We probably can either reject big form or save data to temp cache file.
Copied from original issue: aspnet/HttpAbstractions#583
The text was updated successfully, but these errors were encountered: