Added guidelines for scaling firecracker VMs count #396
Conversation
ustiugov
force-pushed
the
scaling_readme
branch
from
dee1d2b to
e697652
Compare
|
@IRCody Thank you for the corrections, please check. |
docs/scaling.md
Outdated
| sudo sh -c "echo \"* soft nofile 1000000\" >> /etc/security/limits.conf" | ||
| sudo sh -c "echo \"* hard nofile 1000000\" >> /etc/security/limits.conf" | ||
| sudo sh -c "echo \"root soft nofile 1000000\" >> /etc/security/limits.conf" | ||
| sudo sh -c "echo \"root hard nofile 1000000\" >> /etc/security/limits.conf" | ||
| sudo sh -c "echo \"* soft nproc 4000000\" >> /etc/security/limits.conf" | ||
| sudo sh -c "echo \"* hard nproc 4000000\" >> /etc/security/limits.conf" | ||
| sudo sh -c "echo \"root soft nproc 4000000\" >> /etc/security/limits.conf" | ||
| sudo sh -c "echo \"root hard nproc 4000000\" >> /etc/security/limits.conf" | ||
| sudo sh -c "echo \"* soft stack 65536\" >> /etc/security/limits.conf" | ||
| sudo sh -c "echo \"* hard stack 65536\" >> /etc/security/limits.conf" | ||
| sudo sh -c "echo \"root soft stack 65536\" >> /etc/security/limits.conf" | ||
| sudo sh -c "echo \"root hard stack 65536\" >> /etc/security/limits.conf" |
There was a problem hiding this comment.
For this section, it would make sense to show that as a file, rather then the sequence of echo commands like https://github.com/firecracker-microvm/firecracker-containerd/blob/master/docs/getting-started.md#configure-firecracker-containerd-binary.
Regarding the configuration itself, I have a few questions.
- Do we need to have
rootand*separately? Doesn't*coverroot? - Assuming that Firecracker is running as a separate Linux user, not root, can we specifically target the user instead of essentially removing the limits for all of them?
There was a problem hiding this comment.
@kzys Thank you for the feedback, I have applied it.
- root and the rest of the users are managed separately in Ubuntu, please see here for details.
- The orchestrator (like the one in
examples/taskworkflow.go) runs with sudo (at least, in the quickstart/getting-started example commands) and it creates a lot of threads and processes. As a result, it can hit one of the system limits as well as VMs themselves.
There was a problem hiding this comment.
Thanks! I didn't know that * doesn't include root.
I can understand why we need more nofile and nproc, but not so sure about stack. Does something consume more stack by having more VMs?
ustiugov
force-pushed
the
scaling_readme
branch
2 times, most recently
from
fbb1252 to
635b243
Compare
|
Thanks for the update @ustiugov. The CI is failing on checking DCO/subject length. I think if you amend your commit and add a line between your commit subject and the Signed-off-by line it will pass. |
Signed-off-by: Dmitrii Ustiugov <[email protected]>
I have put together a set of guidelines to configure the host OS to scale the number of microVMs past one thousand (tested with 1500 microVMs on a host with 256GB memory). Feedback is welcome.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.