From 05e4c21c58581433890776649bbcf0fcdc0e0115 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Sat, 2 Jul 2022 00:46:24 +0200 Subject: [PATCH 1/9] =?UTF-8?q?gh-94635:=20Reorder=20sqlite3=20docs=20a=20?= =?UTF-8?q?la=20Di=C3=A1taxis?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- Doc/library/sqlite3.rst | 282 ++++++++++++++++++++++------------------ 1 file changed, 155 insertions(+), 127 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index d3b7d0723991bd..fe37291eff936a 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -8,7 +8,11 @@ **Source code:** :source:`Lib/sqlite3/` --------------- + +.. _sqlite3-intro: + +Introduction +------------ SQLite is a C library that provides a lightweight disk-based database that doesn't require a separate server process and allows accessing the database @@ -21,6 +25,12 @@ The sqlite3 module was written by Gerhard Häring. It provides an SQL interface compliant with the DB-API 2.0 specification described by :pep:`249`, and requires SQLite 3.7.15 or newer. + +.. _sqlite3-tutorial: + +Tutorial +-------- + To use the module, start by creating a :class:`Connection` object that represents the database. Here the data will be stored in the :file:`example.db` file:: @@ -113,10 +123,15 @@ both styles: PEP written by Marc-André Lemburg. +.. _sqlite3-reference: + +Reference +--------- + .. _sqlite3-module-contents: Module functions and constants ------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. data:: apilevel @@ -398,7 +413,7 @@ Module functions and constants .. _sqlite3-connection-objects: Connection Objects ------------------- +~~~~~~~~~~~~~~~~~~ .. class:: Connection @@ -867,7 +882,7 @@ Connection Objects .. _sqlite3-cursor-objects: Cursor Objects --------------- +~~~~~~~~~~~~~~ .. class:: Cursor @@ -1023,7 +1038,7 @@ Cursor Objects .. _sqlite3-row-objects: Row Objects ------------ +~~~~~~~~~~~ .. class:: Row @@ -1089,7 +1104,7 @@ Now we plug :class:`Row` in:: .. _sqlite3-blob-objects: Blob Objects ------------- +~~~~~~~~~~~~ .. versionadded:: 3.11 @@ -1143,7 +1158,7 @@ Blob Objects .. _sqlite3-exceptions: Exceptions ----------- +~~~~~~~~~~ The exception hierarchy is defined by the DB-API 2.0 (:pep:`249`). @@ -1230,60 +1245,15 @@ The exception hierarchy is defined by the DB-API 2.0 (:pep:`249`). ``NotSupportedError`` is a subclass of :exc:`DatabaseError`. -.. _sqlite3-types: +.. _sqlite3-guides: -SQLite and Python types ------------------------ - - -Introduction -^^^^^^^^^^^^ - -SQLite natively supports the following types: ``NULL``, ``INTEGER``, -``REAL``, ``TEXT``, ``BLOB``. - -The following Python types can thus be sent to SQLite without any problem: - -+-------------------------------+-------------+ -| Python type | SQLite type | -+===============================+=============+ -| :const:`None` | ``NULL`` | -+-------------------------------+-------------+ -| :class:`int` | ``INTEGER`` | -+-------------------------------+-------------+ -| :class:`float` | ``REAL`` | -+-------------------------------+-------------+ -| :class:`str` | ``TEXT`` | -+-------------------------------+-------------+ -| :class:`bytes` | ``BLOB`` | -+-------------------------------+-------------+ - - -This is how SQLite types are converted to Python types by default: - -+-------------+----------------------------------------------+ -| SQLite type | Python type | -+=============+==============================================+ -| ``NULL`` | :const:`None` | -+-------------+----------------------------------------------+ -| ``INTEGER`` | :class:`int` | -+-------------+----------------------------------------------+ -| ``REAL`` | :class:`float` | -+-------------+----------------------------------------------+ -| ``TEXT`` | depends on :attr:`~Connection.text_factory`, | -| | :class:`str` by default | -+-------------+----------------------------------------------+ -| ``BLOB`` | :class:`bytes` | -+-------------+----------------------------------------------+ - -The type system of the :mod:`sqlite3` module is extensible in two ways: you can -store additional Python types in an SQLite database via object adaptation, and -you can let the :mod:`sqlite3` module convert SQLite types to different Python -types via converters. +Guides +------ +.. _sqlite3-adapters: Using adapters to store custom Python types in SQLite databases -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ SQLite supports only a limited set of data types natively. To store custom Python types in SQLite databases, *adapt* them to one of the @@ -1322,8 +1292,10 @@ This function can then be registered using :func:`register_adapter`. .. literalinclude:: ../includes/sqlite3/adapter_point_2.py +.. _sqlite3-converters: + Converting SQLite values to custom Python types -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Writing an adapter lets you convert *from* custom Python types *to* SQLite values. @@ -1362,40 +1334,10 @@ The following example illustrates the implicit and explicit approaches: .. literalinclude:: ../includes/sqlite3/converter_point.py -Default adapters and converters -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -There are default adapters for the date and datetime types in the datetime -module. They will be sent as ISO dates/ISO timestamps to SQLite. - -The default converters are registered under the name "date" for -:class:`datetime.date` and under the name "timestamp" for -:class:`datetime.datetime`. - -This way, you can use date/timestamps from Python without any additional -fiddling in most cases. The format of the adapters is also compatible with the -experimental SQLite date/time functions. - -The following example demonstrates this. - -.. literalinclude:: ../includes/sqlite3/pysqlite_datetime.py - -If a timestamp stored in SQLite has a fractional part longer than 6 -numbers, its value will be truncated to microsecond precision by the -timestamp converter. - -.. note:: - - The default "timestamp" converter ignores UTC offsets in the database and - always returns a naive :class:`datetime.datetime` object. To preserve UTC - offsets in timestamps, either leave converters disabled, or register an - offset-aware converter with :func:`register_converter`. - - .. _sqlite3-adapter-converter-recipes: Adapter and Converter Recipes -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This section shows recipes for common adapters and converters. @@ -1437,55 +1379,42 @@ This section shows recipes for common adapters and converters. sqlite3.register_converter("timestamp", convert_timestamp) -.. _sqlite3-controlling-transactions: +.. _sqlite3-default-adapters: -Controlling Transactions ------------------------- +Default adapters and converters +""""""""""""""""""""""""""""""" -The ``sqlite3`` module does not adhere to the transaction handling recommended -by :pep:`249`. +There are default adapters for the date and datetime types in the datetime +module. They will be sent as ISO dates/ISO timestamps to SQLite. -If the connection attribute :attr:`~Connection.isolation_level` -is not :const:`None`, -new transactions are implicitly opened before -:meth:`~Cursor.execute` and :meth:`~Cursor.executemany` executes -``INSERT``, ``UPDATE``, ``DELETE``, or ``REPLACE`` statements. -Use the :meth:`~Connection.commit` and :meth:`~Connection.rollback` methods -to respectively commit and roll back pending transactions. -You can choose the underlying `SQLite transaction behaviour`_ — -that is, whether and what type of ``BEGIN`` statements ``sqlite3`` -implicitly executes – -via the :attr:`~Connection.isolation_level` attribute. +The default converters are registered under the name "date" for +:class:`datetime.date` and under the name "timestamp" for +:class:`datetime.datetime`. -If :attr:`~Connection.isolation_level` is set to :const:`None`, -no transactions are implicitly opened at all. -This leaves the underlying SQLite library in `autocommit mode`_, -but also allows the user to perform their own transaction handling -using explicit SQL statements. -The underlying SQLite library autocommit mode can be queried using the -:attr:`~Connection.in_transaction` attribute. +This way, you can use date/timestamps from Python without any additional +fiddling in most cases. The format of the adapters is also compatible with the +experimental SQLite date/time functions. -The :meth:`~Cursor.executescript` method implicitly commits -any pending transaction before execution of the given SQL script, -regardless of the value of :attr:`~Connection.isolation_level`. +The following example demonstrates this. -.. versionchanged:: 3.6 - :mod:`sqlite3` used to implicitly commit an open transaction before DDL - statements. This is no longer the case. +.. literalinclude:: ../includes/sqlite3/pysqlite_datetime.py -.. _autocommit mode: - https://www.sqlite.org/lang_transaction.html#implicit_versus_explicit_transactions +If a timestamp stored in SQLite has a fractional part longer than 6 +numbers, its value will be truncated to microsecond precision by the +timestamp converter. -.. _SQLite transaction behaviour: - https://www.sqlite.org/lang_transaction.html#deferred_immediate_and_exclusive_transactions +.. note:: + The default "timestamp" converter ignores UTC offsets in the database and + always returns a naive :class:`datetime.datetime` object. To preserve UTC + offsets in timestamps, either leave converters disabled, or register an + offset-aware converter with :func:`register_converter`. -Using :mod:`sqlite3` efficiently --------------------------------- +.. _sqlite3-shortcut-methods: Using shortcut methods -^^^^^^^^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~~~~~~~~ Using the nonstandard :meth:`execute`, :meth:`executemany` and :meth:`executescript` methods of the :class:`Connection` object, your code can @@ -1498,8 +1427,10 @@ directly using only a single call on the :class:`Connection` object. .. literalinclude:: ../includes/sqlite3/shortcut_methods.py +.. _sqlite3-columns-by-name: + Accessing columns by name instead of by index -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ One useful feature of the :mod:`sqlite3` module is the built-in :class:`sqlite3.Row` class designed to be used as a row factory. @@ -1513,7 +1444,7 @@ case-insensitively by name: .. _sqlite3-connection-context-manager: Using the connection as a context manager -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A :class:`Connection` object can be used as a context manager that automatically commits or rolls back open transactions when leaving the body of @@ -1535,6 +1466,103 @@ the context manager is a no-op. .. literalinclude:: ../includes/sqlite3/ctx_manager.py +.. _sqlite3-concepts: + +Concepts +-------- + +.. _sqlite3-types: + +SQLite and Python types +~~~~~~~~~~~~~~~~~~~~~~~ + +SQLite natively supports the following types: ``NULL``, ``INTEGER``, +``REAL``, ``TEXT``, ``BLOB``. + +The following Python types can thus be sent to SQLite without any problem: + ++-------------------------------+-------------+ +| Python type | SQLite type | ++===============================+=============+ +| :const:`None` | ``NULL`` | ++-------------------------------+-------------+ +| :class:`int` | ``INTEGER`` | ++-------------------------------+-------------+ +| :class:`float` | ``REAL`` | ++-------------------------------+-------------+ +| :class:`str` | ``TEXT`` | ++-------------------------------+-------------+ +| :class:`bytes` | ``BLOB`` | ++-------------------------------+-------------+ + + +This is how SQLite types are converted to Python types by default: + ++-------------+----------------------------------------------+ +| SQLite type | Python type | ++=============+==============================================+ +| ``NULL`` | :const:`None` | ++-------------+----------------------------------------------+ +| ``INTEGER`` | :class:`int` | ++-------------+----------------------------------------------+ +| ``REAL`` | :class:`float` | ++-------------+----------------------------------------------+ +| ``TEXT`` | depends on :attr:`~Connection.text_factory`, | +| | :class:`str` by default | ++-------------+----------------------------------------------+ +| ``BLOB`` | :class:`bytes` | ++-------------+----------------------------------------------+ + +The type system of the :mod:`sqlite3` module is extensible in two ways: you can +store additional Python types in an SQLite database via +:ref:`object adapters `, +and you can let the :mod:`sqlite3` module convert SQLite types to +Python types via :ref:`converters `. + + +.. _sqlite3-controlling-transactions: + +Controlling Transactions +~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``sqlite3`` module does not adhere to the transaction handling recommended +by :pep:`249`. + +If the connection attribute :attr:`~Connection.isolation_level` +is not :const:`None`, +new transactions are implicitly opened before +:meth:`~Cursor.execute` and :meth:`~Cursor.executemany` executes +``INSERT``, ``UPDATE``, ``DELETE``, or ``REPLACE`` statements. +Use the :meth:`~Connection.commit` and :meth:`~Connection.rollback` methods +to respectively commit and roll back pending transactions. +You can choose the underlying `SQLite transaction behaviour`_ — +that is, whether and what type of ``BEGIN`` statements ``sqlite3`` +implicitly executes – +via the :attr:`~Connection.isolation_level` attribute. + +If :attr:`~Connection.isolation_level` is set to :const:`None`, +no transactions are implicitly opened at all. +This leaves the underlying SQLite library in `autocommit mode`_, +but also allows the user to perform their own transaction handling +using explicit SQL statements. +The underlying SQLite library autocommit mode can be queried using the +:attr:`~Connection.in_transaction` attribute. + +The :meth:`~Cursor.executescript` method implicitly commits +any pending transaction before execution of the given SQL script, +regardless of the value of :attr:`~Connection.isolation_level`. + +.. versionchanged:: 3.6 + :mod:`sqlite3` used to implicitly commit an open transaction before DDL + statements. This is no longer the case. + +.. _autocommit mode: + https://www.sqlite.org/lang_transaction.html#implicit_versus_explicit_transactions + +.. _SQLite transaction behaviour: + https://www.sqlite.org/lang_transaction.html#deferred_immediate_and_exclusive_transactions + + .. rubric:: Footnotes .. [#f1] The sqlite3 module is not built with loadable extension support by From 175f82360a11428986bab1316886143e3fa75bfe Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Wed, 27 Jul 2022 14:56:36 +0200 Subject: [PATCH 2/9] Syntax fix --- Doc/library/sqlite3.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index 8a74b40cd3068e..f4ba86504e0c99 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -26,11 +26,12 @@ compliant with the DB-API 2.0 specification described by :pep:`249`, and requires SQLite 3.7.15 or newer. This document includes four main sections: + * :ref:`sqlite3-tutorial`: teaches how to use the sqlite3 module * :ref:`sqlite3-reference`: describes the classes and methods defined in the module * :ref:`sqlite3-howtos`: explains how to solve specific problems -* :ref:`sqlites-concepts`: provides more background and insights about the +* :ref:`sqlite3-concepts`: provides more background and insights about the design of the module From dbc4079e99508159b631bc29e29020ee12a342fa Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Wed, 27 Jul 2022 16:04:46 +0200 Subject: [PATCH 3/9] How-to guides a la diataxis --- Doc/library/sqlite3.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index f4ba86504e0c99..f3cca41d8c9766 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -1387,8 +1387,8 @@ Python types via :ref:`converters `. .. _sqlite3-howtos: -Guides ------- +How-to guides +------------- .. _sqlite3-adapters: From c529b7fef363ab1fbe58a7b368c6112ac4a07f34 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Thu, 28 Jul 2022 08:26:04 +0200 Subject: [PATCH 4/9] Partially adress CAM's review --- Doc/library/sqlite3.rst | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index f3cca41d8c9766..33eb7d56904503 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -27,12 +27,12 @@ requires SQLite 3.7.15 or newer. This document includes four main sections: -* :ref:`sqlite3-tutorial`: teaches how to use the sqlite3 module -* :ref:`sqlite3-reference`: describes the classes and methods defined in the - module -* :ref:`sqlite3-howtos`: explains how to solve specific problems -* :ref:`sqlite3-concepts`: provides more background and insights about the - design of the module +* :ref:`sqlite3-tutorial` teaches how to use the sqlite3 module. +* :ref:`sqlite3-reference` describes the classes and functions this module + defines. +* :ref:`sqlite3-howtos` explains how to handle specific tasks. +* :ref:`sqlite3-explanation` provides in-depth understanding of transaction + control. .. _sqlite3-tutorial: @@ -1613,8 +1613,8 @@ the context manager is a no-op. .. _sqlite3-uri-tricks: -SQLite URI tricks -^^^^^^^^^^^^^^^^^ +Working with SQLite URIs +^^^^^^^^^^^^^^^^^^^^^^^^ Some useful URI tricks include: @@ -1642,7 +1642,7 @@ can be found in the `SQLite URI documentation`_. .. _SQLite URI documentation: https://www.sqlite.org/uri.html -.. _sqlite3-concepts: +.. _sqlite3-explanation: Concepts -------- From 27802916fbe79383ea95ee3eee294d14c56de8f7 Mon Sep 17 00:00:00 2001 From: Erlend Egeberg Aasland Date: Fri, 29 Jul 2022 08:20:21 +0200 Subject: [PATCH 5/9] Update Doc/library/sqlite3.rst Co-authored-by: CAM Gerlach --- Doc/library/sqlite3.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index 33eb7d56904503..7dc96950b5f0fa 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -1644,8 +1644,8 @@ can be found in the `SQLite URI documentation`_. .. _sqlite3-explanation: -Concepts --------- +Explanation +----------- .. _sqlite3-controlling-transactions: From aee26ed114e10977fab2ada9fc037732b3b3cd13 Mon Sep 17 00:00:00 2001 From: Erlend Egeberg Aasland Date: Fri, 29 Jul 2022 08:20:30 +0200 Subject: [PATCH 6/9] Update Doc/library/sqlite3.rst Co-authored-by: CAM Gerlach --- Doc/library/sqlite3.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index 7dc96950b5f0fa..9e4a3f299d8b5d 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -30,7 +30,7 @@ This document includes four main sections: * :ref:`sqlite3-tutorial` teaches how to use the sqlite3 module. * :ref:`sqlite3-reference` describes the classes and functions this module defines. -* :ref:`sqlite3-howtos` explains how to handle specific tasks. +* :ref:`sqlite3-howtos` details how to handle specific tasks. * :ref:`sqlite3-explanation` provides in-depth understanding of transaction control. From d4cdbee5f548002549517a7f71b0a2a472f0382b Mon Sep 17 00:00:00 2001 From: Erlend Egeberg Aasland Date: Fri, 29 Jul 2022 08:20:36 +0200 Subject: [PATCH 7/9] Update Doc/library/sqlite3.rst Co-authored-by: CAM Gerlach --- Doc/library/sqlite3.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index 9e4a3f299d8b5d..49c87ef495e71b 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -31,8 +31,8 @@ This document includes four main sections: * :ref:`sqlite3-reference` describes the classes and functions this module defines. * :ref:`sqlite3-howtos` details how to handle specific tasks. -* :ref:`sqlite3-explanation` provides in-depth understanding of transaction - control. +* :ref:`sqlite3-explanation` provides in-depth background on + transaction control. .. _sqlite3-tutorial: From d53d11d4b1d362252c1bae2423e6c5ecaa0a7fd4 Mon Sep 17 00:00:00 2001 From: Erlend Egeberg Aasland Date: Sat, 30 Jul 2022 23:32:53 +0200 Subject: [PATCH 8/9] Don't link to the module inside the module docs --- Doc/library/sqlite3.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index 49c87ef495e71b..39f55779884c0f 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -1381,7 +1381,7 @@ This is how SQLite types are converted to Python types by default: The type system of the :mod:`sqlite3` module is extensible in two ways: you can store additional Python types in an SQLite database via :ref:`object adapters `, -and you can let the :mod:`sqlite3` module convert SQLite types to +and you can let the ``sqlite3`` module convert SQLite types to Python types via :ref:`converters `. From 56e01399cfdaf97ede3f5e167e5323b17d1222e6 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Sun, 31 Jul 2022 00:38:51 +0200 Subject: [PATCH 9/9] Fix merge --- Doc/library/sqlite3.rst | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index 455d07eb55643f..a0b5c73d7a4ad4 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -1610,13 +1610,14 @@ This section shows recipes for common adapters and converters. sqlite3.register_converter("timestamp", convert_timestamp) -.. _sqlite3-shortcut-methods: +.. _sqlite3-connection-shortcuts: -Using shortcut methods -^^^^^^^^^^^^^^^^^^^^^^ +Using connection shortcut methods +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Using the nonstandard :meth:`execute`, :meth:`executemany` and -:meth:`executescript` methods of the :class:`Connection` object, your code can +Using the :meth:`~Connection.execute`, +:meth:`~Connection.executemany`, and :meth:`~Connection.executescript` +methods of the :class:`Connection` class, your code can be written more concisely because you don't have to create the (often superfluous) :class:`Cursor` objects explicitly. Instead, the :class:`Cursor` objects are created implicitly and these shortcut methods return the cursor