Skip to content

Memory issues on large uploads #2

Closed
@etherwvlf

Description

@etherwvlf

I am experiencing strange issues where memory load becomes monstrous when uploading large files.
It starts normal and slowly but steadily process memory grows to explicit amounts.

This for example is the result when uploading 10GB file, around 80-90% of upload completion:
DeepinScreenshot_select-area_20220524233104

Sometimes memory load hovers around 8-10GB sometimes it goes all the way up to:
DeepinScreenshot_select-area_20220526213225

I am using this config for testing, but I don't think it is related to an error in the config, as everything works fine and dandy for small files.

{
	http_port 8888
	admin off
	order upload before file_server
}

:8888 {
	root .

	encode zstd gzip

	file_server /upload/* {
		browse upload.htm
	}

	file_server /* browse

	@mypost method POST
	upload @mypost {
		dest_dir upload
		max_filesize 15G
		response_template upload-resp.txt
	}

	log {
		output file access.log
	}
}

As long as you use something big and upload it continuously for enough time this can be reproduced, as I have tried with different size and types of files. Doesn't matter if I use my JS upload form or simply POST-ing with curl the result is the same.
After completing upload, the process retains this memory loaded state for some time and returns to around 400-900MB memory usage after a while which also seems rather much to me for an idle process.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions