Merge #1759

1759: v0.28: Pagination settings r=maryamsulemani97 a=guimachiavelli

Closes #1722 

Note for reviewers: you may ignore the failed CI checks, as they are currently failing due to a link to a document created in another PR.

Co-authored-by: gui machiavelli <[email protected]>
Co-authored-by: gui machiavelli <[email protected]>
Co-authored-by: Maryam Sulemani <[email protected]>
Co-authored-by: Maryam <[email protected]>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>

Author: 3 people

.code-samples.meilisearch.yaml

@@ -183,6 +183,9 @@ update_settings_1: |-
           "twoTypos": 10
         },
         "disableOnAttributes": ["title"]
+      },
+      "pagination": {
+        "maxTotalHits": 5000
       }
     }'
 reset_settings_1: |-
@@ -958,3 +961,23 @@ getting_started_typo_tolerance: |-
     --data-binary '{
       "minWordSizeForTypos": { "oneTypo": 4 }
   }'
+settings_guide_pagination_1: |-
+  curl \
+    -X PATCH 'http://localhost:7700/indexes/movies/settings/pagination' \
+    -H 'Content-Type: application/json' \
+    --data-binary '{
+      "maxTotalHits": 50
+    }'
+get_pagination_settings_1: |-
+  curl \
+    -X GET 'http://localhost:7700/indexes/books/settings/pagination'
+update_pagination_settings_1: |-
+  curl \
+    -X PATCH 'http://localhost:7700/indexes/books/settings/pagination' \
+    -H 'Content-Type: application/json' \
+    --data-binary '{
+      "maxTotalHits": 100
+    }'
+reset_pagination_settings_1: |-
+  curl \
+    -X DELETE 'http://localhost:7700/indexes/books/settings/pagination'

.vuepress/config.js

@@ -324,6 +324,7 @@ module.exports = {
                 '/reference/api/displayed_attributes',
                 '/reference/api/distinct_attribute',
                 '/reference/api/filterable_attributes',
+                '/reference/api/pagination',
                 '/reference/api/ranking_rules',
                 '/reference/api/searchable_attributes',
                 '/reference/api/sortable_attributes',

.vuepress/public/sample-template.yaml

@@ -146,3 +146,7 @@ updating_guide_retrieve_documents_old: |-
 updating_guide_update_settings_old: |-
 updating_guide_add_documents_old: |-
 getting_started_typo_tolerance: |-
+settings_guide_pagination_1: |-
+get_pagination_settings_1: |-
+update_pagination_settings_1: |-
+reset_pagination_settings_1: |-

learn/advanced/known_limitations.md

@@ -97,6 +97,6 @@ user = 1 OR user = 2 […] OR user = 1500 OR user = 1501 […] OR user = 2000 OR
 
 ## Maximum number of results per search
 
-**Limitation:** Meilisearch returns up to 1000 documents per search.
+**Limitation:** By default, Meilisearch returns up to 1000 documents per search.
 
-**Explanation:** This non-customizable limit ensures the database is protected from malicious scraping. This limit only applies to the [search route](/reference/api/search.md). If you want to get all documents in your database, you can use the [get documents endpoint](/reference/api/documents.md#get-documents) instead.
+**Explanation:** Meilisearch limits the maximum amount of returned search results to protect your database from malicious scraping. You may change this by using the `maxTotalHits` property of the [pagination index settings](/reference/api/pagination.md#maxtotalhits). `maxTotalHits` only applies to the [search route](/reference/api/search.md) and has no effect on the [get documents endpoint](/reference/api/documents.md#get-documents).

learn/configuration/settings.md

@@ -7,6 +7,7 @@ This page describes the **index-level settings** available in Meilisearch and ho
 | **[displayedAttributes](/learn/configuration/settings.md#displayed-attributes)**   | Fields displayed in the returned documents                                       | All attributes found in the documents                                                       |
 | **[distinctAttribute](/learn/configuration/settings.md#distinct-attribute)**       | Search returns documents with distinct (different) values of the given field     | `null`                                                                                      |
 | **[filterableAttributes](/learn/configuration/settings.md#filterable-attributes)** | List of attributes that can be used for filtering                                | `null`                                                                                      |
+| **[pagination](/learn/advanced/pagination.md)**                                           | Pagination settings                                                       | `{}`
 | **[rankingRules](/learn/configuration/settings.md#ranking-rules)**                 | List of ranking rules sorted by order of importance                              | [A list of ordered built-in ranking rules](/learn/core_concepts/relevancy.md#built-in-rules) |
 | **[searchableAttributes](/learn/configuration/settings.md#searchable-attributes)** | Fields in which to search for matching query words sorted by order of importance | All attributes found in the documents                                                       |                                                     |
 | **[sortableAttributes](/learn/configuration/settings.md#sortable-attributes)**     |  List of attributes to use when sorting search results                           | `[]`                                                                         |
@@ -95,6 +96,22 @@ To be able to filter search results on `director` and `genres` in a movie databa
 
 <CodeSamples id="faceted_search_update_settings_1" />
 
+## Pagination
+
+The maximum number of results Meilisearch can return. By default, this value is `1000` which means you cannot access results beyond `1000`.
+
+[Learn more about pagination in our dedicated guide.](/learn/advanced/pagination.md)
+
+### Example
+
+The code sample below updates `maxTotalHits` to `50`:
+
+<CodeSamples id="settings_guide_pagination_1" />
+
+::: note
+`maxTotalHits` takes priority over search parameters such as [`limit`](/reference/api/search.md#limit) and [`offset`](/reference/api/search.md#offset).
+:::
+
 ## Ranking rules
 
 Built-in ranking rules that **ensure relevancy in search results**. Ranking rules are applied in a default order which can be changed in the settings. You can add or remove rules and change their order of importance.

learn/core_concepts/indexes.md

@@ -59,6 +59,7 @@ You can customize the following index settings:
 - [Stop words](#stop-words)
 - [Displayed and searchable attributes](#displayed-and-searchable-attributes)
 - [Typo tolerance](#typo-tolerance)
+- [Pagination](#pagination)
 
 To change index settings, use the [update settings endpoint](/reference/api/settings.md#update-settings) or any of the [child routes](/reference/api/settings.md#all-settings).
 
@@ -130,3 +131,9 @@ Typo tolerance is a built-in feature that helps you find relevant results even w
 You can update the typo tolerance settings using the [update settings endpoint](/reference/api/settings.md#update-settings) or the [update typo tolerance endpoint](/reference/api/typo_tolerance.md#update-typo-tolerance).
 
 [Learn more about typo tolerance](/learn/configuration/typo_tolerance.md)
+
+### Pagination
+
+To protect your database from malicious scraping, Meilisearch only returns up to `1000` results for a search query. You can change this limit using the [update settings endpoint](/reference/api/settings.md#update-settings) or the [update pagination settings endpoint](/reference/api/pagination.md#update-pagination-settings).
+
+[Learn more about pagination](/learn/advanced/pagination.md)

reference/api/pagination.md

@@ -0,0 +1,98 @@
+# Pagination
+
+_Child route of the [settings route](/reference/api/settings.md)._
+
+This route allows you to configure the pagination settings for an index.
+
+Pagination settings can also be updated directly through the [global settings route](/reference/api/settings.md#update-settings) along with the other settings.
+
+To learn more about paginating search results with Meilisearch, refer to our [dedicated guide](/learn/advanced/pagination.md).
+
+::: warning
+Updating the settings means overwriting the default settings of Meilisearch. You can reset to default values using the `DELETE` routes.
+:::
+
+## Get pagination settings
+
+<RouteHighlighter method="GET" route="/indexes/{index_uid}/settings/pagination"/>
+
+Get the pagination settings of an index. The index [`uid`](/learn/core_concepts/indexes.md#index-uid) is required.
+
+### Example
+
+<CodeSamples id="get_pagination_settings_1" />
+
+#### Response: `200 OK`
+
+```json
+{
+  "maxTotalHits": 1000
+}
+```
+
+### Returned fields
+
+#### `maxTotalHits`
+
+The maximum number of results Meilisearch can return.
+
+## Update pagination settings
+
+<RouteHighlighter method="PATCH" route="/indexes/{index_uid}/settings/pagination"/>
+
+Partially update the pagination settings for an index. The index [`uid`](/learn/core_concepts/indexes.md#index-uid) is required.
+
+### Body
+
+#### `maxTotalHits`
+
+**Type:** integer
+**Default value:** `1000`
+
+An integer indicating the maximum number of search results Meilisearch can return. `maxTotalHits` takes priority over search parameters such as `limit` and `offset`.
+
+For example, if you set `maxTotalHits` to 100, you will not be able to access search results beyond 100 no matter the value configured for `offset`.
+
+::: note
+Setting `maxTotalHits` to a high value might negatively impact performance and expose instance data to malicious scraping.
+:::
+
+#### Example
+
+<CodeSamples id="update_pagination_settings_1" />
+
+#### Response: `200 OK`
+
+```json
+{
+    "taskUid": 1,
+    "indexUid": "books",
+    "status": "enqueued",
+    "type": "settingsUpdate",
+    "enqueuedAt": "2022-04-14T20:56:44.991039Z"
+}
+```
+
+You can use the returned `taskUid` to get more details on [the status of the task](/reference/api/tasks.md#get-task).
+
+## Reset pagination settings
+
+Reset an index's pagination settings to their default value. The index [`uid`](/learn/core_concepts/indexes.md#index-uid) is required.
+
+#### Example
+
+<CodeSamples id="reset_pagination_settings_1" />
+
+#### Response: `200 OK`
+
+```json
+{
+    "taskUid": 1,
+    "indexUid": "books",
+    "status": "enqueued",
+    "type": "settingsUpdate",
+    "enqueuedAt": "2022-04-14T20:53:32.863107Z"
+}
+```
+
+You can use the returned `taskUid` to get more details on [the status of the task](/reference/api/tasks.md#get-task).

reference/api/search.md

@@ -20,7 +20,7 @@ Other than the differences mentioned above, the two routes are strictly equivale
 Search for documents matching a specific query in the given index. The index [`uid`](/learn/core_concepts/indexes.md#index-uid) is required.
 
 ::: note
-This endpoint has a [non-customizable limit of 1000 results](/learn/advanced/known_limitations.md#maximum-number-of-results-per-search). If you want to scrape your database, use the [get documents endpoint](/reference/api/documents.md#get-documents) instead.
+By default, [this endpoint returns a maximum of 1000 results](/learn/advanced/known_limitations.md#maximum-number-of-results-per-search). If you want to scrape your database, use the [get documents endpoint](/reference/api/documents.md#get-documents) instead.
 :::
 
 This is the preferred route to perform search when an API key is required, as it allows for [preflight requests](https://developer.mozilla.org/en-US/docs/Glossary/Preflight_request) to be cached. Caching preflight requests **considerably improves search speed**.
@@ -111,7 +111,7 @@ Query terms enclosed in double quotes are treated as [phrase searches](#query-q)
 Search for documents matching a specific query in the given index. The index [`uid`](/learn/core_concepts/indexes.md#index-uid) is required.
 
 :::note
-This endpoint has a [non-customizable limit of 1000 results](/learn/advanced/known_limitations.md#maximum-number-of-results-per-search). If you want to scrape your database, you can use the [get documents endpoint](/reference/api/documents.md#get-documents) instead.
+By default, [this endpoint returns a maximum of 1000 results](/learn/advanced/known_limitations.md#maximum-number-of-results-per-search). If you want to scrape your database, use the [get documents endpoint](/reference/api/documents.md#get-documents) instead.
 :::
 
 This route should only be used when no API key is required. If an API key is required, use the POST route instead.