Skip to content

Commit 313eb77

Browse files
author
Sam Kleinman
committed
DOCS-471: ulimit page draft
1 parent b3c65f1 commit 313eb77

File tree

1 file changed

+193
-0
lines changed

1 file changed

+193
-0
lines changed

‎draft/administration/ulimit.txt

Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
=========================
2+
Linux ``ulimit`` Settings
3+
=========================
4+
5+
The Linux kernel provides a system to limit and control the number of
6+
threads, connections, open files, on a per-process and per-user
7+
basis. These limits prevent renegade processes from using too many
8+
system resources. Sometimes, these limits, as configured by the
9+
distribution developers, are too low for MongoDB and can cause a
10+
number of issues in the course of normal MongoDB operation.
11+
12+
Resource Utilization
13+
--------------------
14+
15+
:program:`mongod` and :program:`mongos` each use threads and file
16+
descriptors to track connections, and manage internal operations. This
17+
section outlines the general resource utilization patterns for
18+
MongoDB. Use these figures in combination with the actual
19+
20+
``mongod``
21+
~~~~~~~~~~
22+
23+
- 1 thread for each incoming connection from clients and, for
24+
:term:`replica sets <replica set>`, other :program:`mongod` instances.
25+
26+
- 1 thread for each outgoing connection to another :program:`mongod`
27+
when running in a replica set.
28+
29+
- 1 file descriptor for each incoming connection.
30+
31+
- 1 file descriptor for each data file in used by the
32+
:program:`mongod` instance.
33+
34+
- 1 file descriptor for each journal file used by the
35+
:program:`mongod` instance (when :setting:`journal` is ``true``.)
36+
37+
:program:`mongod` uses threads for a number of internal processes
38+
including :ref:`TTL collections <ttl-collections>` and replication
39+
which may require a small number of additional resources.
40+
41+
``mongos``
42+
~~~~~~~~~~
43+
44+
- 1 thread for each incoming connection from clients.
45+
46+
- 1 file descriptor for each incoming connection.
47+
48+
- 1 file descriptor for each connection to each member of each
49+
shard.
50+
51+
For :program:`mongos`, consider the following behaviors:
52+
53+
- :program:`mongos` instances maintain a connection pool to each shard
54+
so that the :program:`mongos` can reuse connections and quickly
55+
fulfill requests without needing to create new connections.
56+
57+
- The size of the connection pool is configurable with the
58+
:setting:`maxConns` runtime options (i.e. :option:`--maxConns
59+
<mongos --maxConns>`.)
60+
61+
Review and Set Resource Limits
62+
------------------------------
63+
64+
65+
``ulimit``
66+
~~~~~~~~~~
67+
68+
You can use the ``ulimit`` command at the system prompt to check
69+
system limits, as in the following example
70+
71+
.. code-block:: sh
72+
73+
$ ulimit -a
74+
-t: cpu time (seconds) unlimited
75+
-f: file size (blocks) unlimited
76+
-d: data seg size (kbytes) unlimited
77+
-s: stack size (kbytes) 8192
78+
-c: core file size (blocks) 0
79+
-m: resident set size (kbytes) unlimited
80+
-u: processes 192276
81+
-n: file descriptors 21000
82+
-l: locked-in-memory size (kb) 40000
83+
-v: address space (kb) unlimited
84+
-x: file locks unlimited
85+
-i: pending signals 192276
86+
-q: bytes in POSIX msg queues 819200
87+
-e: max nice 30
88+
-r: max rt priority 65
89+
-N 15: unlimited
90+
91+
``ulimit`` refers to the per-*user* limitations for various
92+
resources. Therefore, if your :program:`mongod` instance executes as a
93+
user that is also running other processes, you can see some contention
94+
for these resources. Also, be aware that the ``processes`` value
95+
(i.e. ``-u``) refers to the combined number of distinct processes and
96+
sub-process threads.
97+
98+
You can change ``ulimit`` settings by issuing a command in the
99+
following form: ::
100+
101+
.. code-block:: sh
102+
103+
ulimit -n <value>
104+
105+
Substitute the ``-n`` option for any possible value in the output of
106+
``ulimit -a``.
107+
108+
.. note::
109+
110+
After changing the ``ulimit`` settings, you *must* restart the
111+
process to take advantage of the modified settings. You can use the
112+
``/proc`` file system to see the current limitations on a running
113+
process.
114+
115+
``/proc`` File System
116+
~~~~~~~~~~~~~~~~~~~~~
117+
118+
The ``/proc`` file-system stores the per-process limits in the
119+
file system object located at ``/proc/<pid>/limits``, where ``<pid>``
120+
is the processes :term:`PID` or process identifier. You can use the
121+
following ``bash`` function to return the content of the ``limits``
122+
object for a process or processes with a given name:
123+
124+
.. code-block:: sh
125+
126+
return-limits(){
127+
128+
for process in $@; do
129+
process_pids=`ps -C $process -o pid --no-headers | cut -d " " -f 2`
130+
131+
if [ -z $@ ]; then
132+
echo "[no $process running]"
133+
else
134+
for pid in $process_pids; do
135+
echo "[$process #$pid -- limits]"
136+
cat /proc/$pid/limits
137+
done
138+
fi
139+
140+
done
141+
142+
}
143+
144+
You can copy and paste this function into a current shell session, or
145+
load it as part of a script. Call the function with one the following
146+
invocations:
147+
148+
.. code-block:: sh
149+
150+
return-limits mongod
151+
return-limits mongos
152+
return-limits mongod mongos
153+
154+
The output of the first command may resemble the following:
155+
156+
.. code-block:: sh
157+
158+
[mongod #6809 -- limits]
159+
Limit Soft Limit Hard Limit Units
160+
Max cpu time unlimited unlimited seconds
161+
Max file size unlimited unlimited bytes
162+
Max data size unlimited unlimited bytes
163+
Max stack size 8720000 unlimited bytes
164+
Max core file size 0 unlimited bytes
165+
Max resident set unlimited unlimited bytes
166+
Max processes 192276 192276 processes
167+
Max open files 1024 4096 files
168+
Max locked memory 40960000 40960000 bytes
169+
Max address space unlimited unlimited bytes
170+
Max file locks unlimited unlimited locks
171+
Max pending signals 192276 192276 signals
172+
Max msgqueue size 819200 819200 bytes
173+
Max nice priority 30 30
174+
Max realtime priority 65 65
175+
Max realtime timeout unlimited unlimited us
176+
177+
Recommended Settings
178+
--------------------
179+
180+
Every deployment may have unique requirements and settings; however,
181+
the following thresholds and settings are particularly important for
182+
:program:`mongod` and :program:`mongos` deployments:
183+
184+
- ``-f`` (file size): ``unlimited``
185+
- ``-t`` (cpu time): ``unlimited``
186+
- ``-v`` (virtual memory): ``unlimited``
187+
- ``-n`` (open files): ``21000``
188+
- ``-m`` (memory size): ``unlimited``
189+
- ``-u`` (processes/threads): ``32000``
190+
191+
Always remember to restart your :program:`mongod` and
192+
:program:`mongos` instances after changing the ``ulimit`` settings to
193+
make sure that the settings change takes effect.

0 commit comments

Comments
 (0)