Skip to content

Commit 064e86c

Browse files
DOCSP-12356 doc for maxWildcardCollections (#77)
* DOCSP-12356 doc for maxWildcardCollections * Update source/includes/extracts-common-conf-params.yaml Co-authored-by: Devin Hilly <[email protected]> * Update source/includes/extracts-common-conf-params.yaml Co-authored-by: Devin Hilly <[email protected]> * Update source/reference/cli/collections/create-collections-views.txt Co-authored-by: Devin Hilly <[email protected]> * Update source/reference/format/data-lake-configuration.txt Co-authored-by: Devin Hilly <[email protected]> * DOCSP-12356 fix for display error Co-authored-by: Devin Hilly <[email protected]>
1 parent 3052444 commit 064e86c

File tree

4 files changed

+51
-4
lines changed

4 files changed

+51
-4
lines changed

source/includes/extracts-common-conf-params.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,12 @@ content: |
2525
2626
:ref:`data-lake-data-formats`
2727
---
28+
ref: param-max-wildcard-collections
29+
content: |
30+
*Optional.* Maximum number of wildcard ``*`` collections in the database.
31+
Each wildcard collection can have only one data source. Value can be between
32+
``1`` and ``1000``, inclusive. If omitted, defaults to ``100``.
33+
---
2834
ref: cli-param-collection-name
2935
content: |
3036
Either the name of the collection to which {+data-lake-short+} maps the data

source/query/query-data-lake.txt

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,8 @@ To query data on |s3|, your {+dl+} storage :ref:`configuration
6666
}
6767
]
6868
}
69-
]
69+
],
70+
"maxWildcardCollections" : <number-of-wildcard-collections>
7071
}
7172
]
7273
}

source/reference/cli/collections/create-collections-views.txt

Lines changed: 25 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,24 @@ Parameters
126126
- Description
127127
- Required?
128128

129-
* - ``<collection-name>|*``
129+
* - .. tabs::
130+
:hidden:
131+
132+
.. tab:: S3 Configuration
133+
:tabid: s3
134+
135+
``<collection-name>|*``
136+
137+
.. tab:: Atlas Configuration
138+
:tabid: atlas
139+
140+
``<collection-name>|*``
141+
142+
.. tab:: HTTP Configuration
143+
:tabid: http
144+
145+
``<collection-name>``
146+
130147
- string
131148
- .. tabs::
132149
:hidden:
@@ -136,11 +153,16 @@ Parameters
136153

137154
.. include:: /includes/extracts/cli-param-collection-name.rst
138155

139-
140156
You can generate collection names dynamically from file
141157
paths by specifying ``*`` for the collection name and the
142158
``collectionName()`` function in the
143-
``dataSources.collection`` field.
159+
``dataSources.collection`` field. By default, {+adl+}
160+
creates up to ``100`` wildcard collections. You can
161+
customize the maximum number of wildcard collections that
162+
{+adl+} automatically generates using the
163+
:datalakeconf:`databases.[n].maxWildcardCollections`
164+
parameter. Note that each wildcard collection can contain
165+
only one ``dataSource``.
144166

145167
.. tab:: Atlas Configuration
146168
:tabid: atlas

source/reference/format/data-lake-configuration.txt

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -360,6 +360,7 @@ The {+data-lake-short+} configuration has the following format:
360360
]
361361
}
362362
],
363+
"maxWildcardCollections" : <integer>,
363364
"views" : [
364365
{
365366
"name" : "<string>",
@@ -688,6 +689,7 @@ The {+data-lake-short+} configuration has the following format:
688689
]
689690
}
690691
],
692+
"maxWildcardCollections" : <integer>,
691693
"views" : [
692694
{
693695
"name" : "<string>",
@@ -886,6 +888,22 @@ The {+data-lake-short+} configuration has the following format:
886888
If omitted, {+dl+} uses the :datalakeconf:`~stores.[n].urls` in the
887889
specified :datalakeconf:`~databases.[n].collections.[n].dataSources.[n].storeName`.
888890

891+
.. tabs::
892+
:hidden:
893+
894+
.. tab:: S3
895+
:tabid: s3
896+
897+
.. datalakeconf:: databases.[n].maxWildcardCollections
898+
899+
.. include:: /includes/extracts/param-max-wildcard-collections.rst
900+
901+
.. tab:: Atlas
902+
:tabid: atlas
903+
904+
.. tab:: HTTP
905+
:tabid: http
906+
889907
.. datalakeconf:: databases.[n].views
890908

891909
Array of objects where each object represents an :manual:`aggregation

0 commit comments

Comments
 (0)