Skip to content

Conversation

timgrein
Copy link
Contributor

@timgrein timgrein commented Sep 23, 2025

Summary

This PR adds a simplified way of doing a full-text-search over all text fields in an index using ES|QL.

The new syntax looks like the following:

FROM idx | WHERE "query string"

instead of:

FROM idx | WHERE MULTI_MATCH("query string", field_one, ..., field_n)

This functionality inherits the underlying restrictions of multi_match (not working with semantic_text out of the box). I've intentionally decided against handling edge cases like raising errors, when multiple text fields, which include one or more semantic_text fields, are queried as this should probably be the responsibility of the multi_match function itself. It would also be not too straightforward as the information, if a text field is of type semantic_text is not known in the analysis/verification phase, but only during physical optimization AFAIU. It felt wrong to hack this into earlier phases of query processing, but happy to discuss more - could've missed something of course :)

Querying a single semantic_text field works as the query will be translated to use MATCH instead of MULTI_MATCH, if only one text (including semantic_text) field is present in the queried index.

Requests for testing:

Create index with multiple text fields:

curl --location --request PUT 'http://localhost:9200/example_full_text_index_multiple_fields' \
--header 'Content-Type: application/json' \

--data '{
    "mappings": {
        "properties": {
            "field_one": {
                "type": "text"
            },
            "field_two": {
                "type": "text"
            },
            "field_three": {
                "type": "long"
            }
        }
    }
}'

Indexing sample data:

curl --location 'http://localhost:9200/_bulk' \
--header 'Content-Type: application/json' \
--data '{ "index": { "_index": "example_full_text_index_multiple_fields", "_id": "1" } }
{ "field_one": "Star Wars" }
{ "index": { "_index": "example_full_text_index_multiple_fields", "_id": "2" } }
{ "field_one": "Lord of the Rings" }
{ "index": { "_index": "example_full_text_index_multiple_fields", "_id": "3" } }
{ "field_one": "Harry Potter" }
{ "index": { "_index": "example_full_text_index_multiple_fields", "_id": "4" } }
{ "field_one": "Pulp Fiction", "field_two": "Star Trek" }
'

Normal multi-match ES|QL search:

curl --location 'http://localhost:9200/_query?format=txt' \
--header 'Content-Type: application/json' \
--data '{
    "query": "FROM example_full_text_index_multiple_fields | WHERE MULTI_MATCH(\"Star\", field_one, field_two)"
}'

New simplified ES|QL full-text-search:

curl --location 'http://localhost:9200/_query?format=txt' \
--header 'Content-Type: application/json' \
--data '{
    "query": "FROM example_full_text_index_multiple_fields | WHERE \"Star\""
}'

@timgrein timgrein changed the title Allow simplified full text search using string in where condition ES|QL: Allow simplified full text search using string in where condition Sep 23, 2025
@timgrein timgrein changed the title ES|QL: Allow simplified full text search using string in where condition ES|QL: Allow simplified full text search using text in where clause Sep 23, 2025
@timgrein timgrein added the test-release Trigger CI checks against release build label Sep 25, 2025
Expression condition = filter.condition();

// Replace every instance of FullTextSearch
if (condition instanceof FullTextSearch fts) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively we could check, if MultiMatch contains one field, which is an instance of UnresolvedStar instead of introducing a separate "marker class" FullTextSearch.


if (condition instanceof MultiMatch multiMatch && multiMatch.fields().size() == 1) {
// MULTI_MATCH and MATCH have a subset of common options
Expression filteredOptions = filterOptionsForMatch(multiMatch.options());
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will add a test making sure options are passed correctly from MultiMatch to Match

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test-release Trigger CI checks against release build v9.3.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants