Skip to content

Pad 1 #523

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jan 11, 2013
Merged

Pad 1 #523

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions source/core/write-operations.txt
Original file line number Diff line number Diff line change
Expand Up @@ -341,6 +341,92 @@ write operation that affects multiple documents using the
To isolate a sequence of write operations from other read and write
operations, see :doc:`/tutorial/perform-two-phase-commits`.

.. _write-operations-padding-factor:

Padding Factor
--------------

When updating a document, if an update operation does not cause the
document to increase in size, the update occurs in-place. If, however,
the update causes the document to increase in size, for instance if you
use the :operator:`$push` operator, **and** exceeds its allocated space,
then the document is relocated on disk to a new location with enough
contiguous space to fit the larger document. Since the move will
require an update of all the indexes for the document, if the
collection has many indexes, the move will impact write performance.

To minimize document movements, MongoDB employs padding. MongoDB
adaptively learns if documents in a collection tend to grow, and if
they do, adds a :stats:`paddingFactor` so that the documents have room
to grow on subsequent writes. The :stats:`paddingFactor` indicates the
padding for new inserts and moves.

.. versionadded:: 2.2
You can use the :dbcommand:`collMod` command with the
:collflag:`usePowerOf2Sizes` flag so that MongoDB allocates document
space in sizes that are powers of 2. This helps ensure that the
space freed due to either deletions or document relocations is
reused efficiently. As with all padding, using powers of 2
minimizes, but does not eliminate, document movements.

To check the current :stats:`paddingFactor` on a collection, you can
run the :dbcommand:`db.collection.stats()` command in the
:program:`mongo` shell, as in the following example:

.. code-block:: javascript

db.myCollection.stats()

Since each document is written at a different point in time, the
padding for each document will not be the same. You can calculate the
padding size by subtracting 1 from the paddingFactor:

.. code-block:: none

padding size = (paddingFactor - 1) * <document size>.

For example, a ``paddingFactor`` of ``1.0`` specifies no padding
whereas a paddingFactor of ``1.5`` specifies a padding size of ``0.5`` or 50
percent (50%) of the document size.

Because the :stats:`paddingFactor` is relative to the size of each
document, you cannot calculate the exact amount of padding for a
collection based on the average document size and padding factor.

If an update operation causes the document to *decrease* in size, for
instance if you perform an :operator:`$unset` or a :operator:`$pop`
update, the document remains in place and effectively has more
padding. If the document remains this size, the space is not reclaimed
until you perform a :dbcommand:`compact` or a
:dbcommand:`repairDatabase` operation.

.. note::

Padding is removed when you perform :dbcommand:`compact`,
:dbcommand:`repairDatabase`, or initial replica sync operations.
However, with the :dbcommand:`compact` command, you can run the
command with a ``paddingFactor`` or a ``paddingBytes`` parameter.

Padding is also removed if you :program:`export <mongoexport>` from
a collection. If you :program:`import <mongoimport>` into a new
collection, the new collection is created without padding. If you
:program:`import <mongoimport>` into an existing collection with
padding, the existing padding is maintained.

If padding is removed during these operations, updates may be slower
although the size requirements for storage are lower.

.. COMMENT -- not sure if we really need this manual padding info or if
it belongs here, but ...

.. seealso::

- :ref:`faq-developers-manual-padding`

- `Fast Updates with MongoDB (update-in-place )
<http://blog.mongodb.org/post/248614779/fast-updates-with-mongodb-u
pd ate-in-place>`_

Architecture
------------

Expand Down
46 changes: 46 additions & 0 deletions source/faq/developers.txt
Original file line number Diff line number Diff line change
Expand Up @@ -660,3 +660,49 @@ information on indexes.
.. Commenting out.. If your small documents are approximately the page
cache unit size, there is no benefit for ram cache efficiency, although
embedding will provide some benefit regarding random disk i/o.

.. _faq-developers-manual-padding:

Can I manually pad documents to prevent moves during updates?
-------------------------------------------------------------

An update can cause a document to move on disk if the document grows in
size. To *minimize* document movements, MongoDB uses
:ref:`padding <write-operations-padding-factor>`.

You should not have to pad manually because MongoDB handles
:ref:`padding automatically <write-operations-padding-factor>`. MongoDB
adaptively learns if documents in a collection tend to grow, and if
they do, adds a :stats:`paddingFactor` so that documents have room to
grow on subsequent writes. You can change the default
:stats:`paddingFactor` calculation by using the the
:dbcommand:`collMod` command with the :collflag:`usePowerOf2Sizes`
flag. The :collflag:`usePowerOf2Sizes` flag ensures that MongoDB
allocates document space in sizes that are powers of 2, which helps
ensure that the space freed due to either deletions or document
relocations is reused efficiently.

However, in those exceptions where you must pad manually, you can use
the strategy of first adding a temporary field to a document and then
:operator:`$unsetting <$unset>` the field, as in the following example:

.. code-block:: javascript

var myTempPadding = [ "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"];

db.myCollection.insert( { _id: 5, paddingField: myTempPadding } );

db.myCollection.update( { _id: 5 },
{ $unset: { paddingField: "" } }
)

db.myCollection.update( { _id: 5 },
{ $set: { realField: "Some text that I might have needed padding for" } }
)

.. seealso::

:ref:`write-operations-padding-factor`
1 change: 0 additions & 1 deletion themes/mongodb/static/mongodb-docs.css_t
Original file line number Diff line number Diff line change
Expand Up @@ -727,7 +727,6 @@ div.section > h2, div.section > h3,div.section > h4 {

div.admonition p {
line-height:1.5em;
padding: 1em 0;
}

dd > div.admonition { margin-left: 0; }
Expand Down