Skip to content

Commit a24bb47

Browse files
DOCSP-10462 doc for sql support (#27)
* DOCSP-10462 doc for sql support
1 parent 535d150 commit a24bb47

24 files changed

+1623
-9
lines changed

snooty.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ title = "MongoDB Atlas Data Lake"
33

44
intersphinx = ["https://docs.mongodb.com/manual/objects.inv"]
55

6-
toc_landing_pages = ["/config/config-data-lake", "/tutorial/getting-started"]
6+
toc_landing_pages = ["/config/config-data-lake", "/tutorial/getting-started", "/admin/query-with-sql"]
77

88
[constants]
99
adl = "Atlas Data Lake"

source/admin/query-with-sql.txt

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
.. _query-with-sql:
2+
3+
=================
4+
Querying with SQL
5+
=================
6+
7+
.. default-domain:: mongodb
8+
9+
.. include:: /includes/fact-sql-beta-message.rst
10+
11+
{+adl+} supports SQL format queries through the :ref:`JDBC driver
12+
<jdbc-driver>` for {+adl+} and using the :ref:`adl-sql-stage`
13+
:ref:`aggregation pipeline <adl-aggregation-pipeline>` stage. To support
14+
SQL format queries, {+adl+} automatically creates a |json| schema that maps
15+
to a relational schema of columns, tables, and databases for all new
16+
collections, except wildcard (``*``) collections, and views in the {+dl+}
17+
storage configuration. To learn more about the schema, see
18+
:ref:`sql-schema-format`.
19+
20+
{+dl+} automatically generates a schema for a new non-wildcard collection or
21+
view in the storage configuration when you:
22+
23+
- :ref:`Create <dl-create-collection-views-cmd>` the collection or view
24+
in the storage configuration.
25+
26+
- :ref:`Rename <dl-rename-collection-cmd>` a collection or view that does not
27+
already have a schema. If you rename a collection or view that already has
28+
a schema, the schema is also renamed. {+dl+} does not generate a new schema
29+
for a renamed collection or view if it already exists.
30+
31+
- :ref:`Set <datalake-setstorageconfig>` the storage configuration.
32+
33+
.. include:: /includes/fact-schema-for-existing-collections.rst
34+
35+
By default, {+dl+} samples data from only one randomly selected document in
36+
your non-wildcard collection or view to generate a |json| schema. If your
37+
collection or view contains polymorphic data, you can provide a larger
38+
sampling size to {+dl+} to generate a new schema or you can manually
39+
construct and set the schema.
40+
41+
You can manually generate schemas for all collections and views using the
42+
:ref:`sqlgenerateschema-cmd` command, set or update the schema for your
43+
collections or views using the :ref:`sqlsetschema-cmd` command, and view
44+
the stored schema using the :ref:`sqlgetschema-cmd` command.
45+
46+
Once the SQL schema is set up, you can query your {+adl+} collections or views
47+
through the :ref:`JDBC driver <jdbc-driver>` for {+adl+} and using the
48+
:ref:`adl-sql-stage` :ref:`aggregation pipeline <adl-aggregation-pipeline>`
49+
stage.
50+
51+
You can manually delete a schema for a collection or view by running the
52+
:ref:`sqlsetschema-cmd` command with an empty schema document. {+dl+}
53+
automatically removes the schema for a collection or view when you:
54+
55+
- :ref:`Drop the collection or view <dl-drop-collection-views-cmd>` from the
56+
storage configuration.
57+
- :ref:`Modify the storage configuration <datalake-setstorageconfig>` to
58+
remove the collection or view from the storage configuration.
59+
- :ref:`Drop the database <dl-drop-database-cmd>` that contains the collection
60+
or view from the storage configuration.
61+
62+
.. class:: hidden
63+
64+
.. toctree::
65+
:titlesonly:
66+
67+
SQL Schema Format </reference/config-files/sql-schema-format>
68+
Generate SQL Schema </reference/cli/sql/sqlgenerateschema>
69+
Set SQL Schema </reference/cli/sql/sqlsetschema>
70+
Retrieve SQL Schema </reference/cli/sql/sqlgetschema>
71+
Connect Using JDBC Driver </tutorial/jdbc-driver>
Loading
Loading

source/images/scalar-lattice.png

62.7 KB
Loading
147 KB
Loading
295 KB
Loading
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
Consider a collection named ``egData`` in a database named
2+
``sampleDB`` with the following documents:
3+
4+
.. code-block:: json
5+
6+
{"a": {"b": {"c": [1, 2, 3]}}, "s": 1}
7+
{"a": {"b": {"c": [4, 5, 6]}}, "s": 2}
8+
{"a": {"b": [7, 8, 9]}, "s": 3}
9+
{"a": {"b": {"c": []}}, "s": 4}
10+
{"a": {"b": {"c": "hello"}}, "s": 5}
11+
{"a": {"b": {"c": {"d": 1}}}, "s": 6}
12+
{"a": {"b": {"c": null}}}
13+
{"s": 7}
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
.. note::
2+
3+
{+dl+} automatically generates schemas for only new collections and
4+
views in the :ref:`storage configuration <datalake-configuration-file>`.
5+
Existing :manual:`namespaces </reference/limits/#faq-dev-namespace>`
6+
will not have auto-generated schemas. If you want {+dl+} to automatically
7+
generate schemas for your existing non-wildcard collections and views in
8+
the storage configuration, :ref:`remove <datalake-setstorageconfig>` the :datalakeconf:`databases` in your {+dl+} storage configuration and then
9+
:ref:`update <datalake-setstorageconfig>` your {+dl+} storage
10+
configuration with the old configuration.
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
.. admonition:: Beta
2+
:class: note
3+
4+
The support for SQL format queries is available as a Beta feature.
5+
The feature and the corresponding documentation may change at any
6+
time during the Beta stage.

0 commit comments

Comments
 (0)