Skip to content

Conversation

kderusso
Copy link
Member

@kderusso kderusso commented May 2, 2025

Adds MATCH_PHRASE support to ES|QL.

Some examples of how to use MATCH_PHRASE:

POST _query?format=txt
{
  "query": """
  FROM books METADATA _score
  | WHERE MATCH_PHRASE(title, "return of the king")
  | KEEP title, _score
  | SORT _score
  | LIMIT 5
  """
}

POST _query?format=txt
{
  "query": """
  FROM books METADATA _score
  | WHERE MATCH_PHRASE(title, "return of king", {"slop": 5})
  | KEEP title, _score
  | SORT _score
  | LIMIT 5
  """
}

POST _query?format=txt
{
  "query": """
  FROM books METADATA _score
  | WHERE MATCH_PHRASE(title, "return of king", {"analyzer": "whitespace"})
  | KEEP title, _score
  | SORT _score
  | LIMIT 5
  """
}

POST _query?format=txt
{
  "query": """
  FROM books METADATA _score
  | WHERE MATCH_PHRASE(title, "", {"zero_terms_query": "all"})
  | KEEP title, _score
  | SORT _score
  | LIMIT 5
  """
}

@kderusso kderusso force-pushed the kderusso/esql-match-phrase branch from dabef06 to 13ccda7 Compare May 6, 2025 19:32
@kderusso kderusso added :SearchOrg/Relevance Label for the Search (solution/org) Relevance team >enhancement labels May 7, 2025
@elasticsearchmachine
Copy link
Collaborator

Hi @kderusso, I've created a changelog YAML for you.

Copy link
Member

@carlosdelest carlosdelest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's important that we register the match_phrase as snapshot and get some tests ready for release builds.

Besides that, some small nits.

- match: {esql.functions.coalesce: $functions_coalesce}
- gt: {esql.functions.categorize: $functions_categorize}
- length: {esql.functions: 134} # check the "sister" test above for a likely update to the same esql.functions length check
- length: {esql.functions: 135} # check the "sister" test above for a likely update to the same esql.functions length check
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hopefully those errors get corrected when registering the function under snapshot functions 🤞

- match: {esql.functions.coalesce: $functions_coalesce}
- gt: {esql.functions.categorize: $functions_categorize}
- length: {esql.functions: 134} # check the "sister" test above for a likely update to the same esql.functions length check
- length: {esql.functions: 135} # check the "sister" test above for a likely update to the same esql.functions length check
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you'll need to add match_phrase capability to these tests, otherwise they will fail for release as the function will not be registered

@ioanatia ioanatia removed the test-release Trigger CI checks against release build label Jun 4, 2025
@ioanatia
Copy link
Contributor

ioanatia commented Jun 4, 2025

@elasticmachine test this please

@ioanatia
Copy link
Contributor

ioanatia commented Jun 4, 2025

I checked the release tests failures - there is nothing pertaining to match_phrase - I removed the test-release label since all the match_phrase failures for release tests were fixed.

@kderusso kderusso requested a review from carlosdelest June 4, 2025 13:57
Copy link
Member

@carlosdelest carlosdelest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, great work! 💯

There are other tests that can be added to VerifierTests - look for EsqlCapabilities.Cap.MULTI_MATCH_FUNCTION.isEnabled() and you'll be able to add an equivalent one for match_phrase

@kderusso kderusso merged commit eee423a into elastic:main Jun 4, 2025
18 checks passed
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
8.19 Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 127661

kderusso added a commit to kderusso/elasticsearch that referenced this pull request Jun 10, 2025
* Initial commit of match_phrase

* Add MatchPhraseQueryTests

* First pass at CSV specs

* Update docs/changelog/127661.yaml

* Refactor so MatchPhrase doesn't use all fulltext test cases, just text only

* Fix tests

* Add some CSV test cases

* Fix test

* Update changelog

* Update tests

* Comment out MATCH_PHRASE in search-functions Markdown

* Minor PR feedback

* PR feedback - refactor/consolidate code

* Add some more tests

* Fix some tests

* [CI] Auto commit changes from spotless

* Fix tests

* PR feedback - add tests, support boost and numeric data

* Revert "PR feedback - add tests, support boost and numeric data"

This reverts commit 4e7a699.

* Apply testing/PR feedback outside numeric support only

* Regenerate docs

* Add negative test

* Update x-pack/plugin/esql/qa/testFixtures/src/main/resources/match-phrase-function.csv-spec

Co-authored-by: Carlos Delgado <[email protected]>

* Update x-pack/plugin/esql/qa/testFixtures/src/main/resources/match-phrase-function.csv-spec

Co-authored-by: Carlos Delgado <[email protected]>

* Update x-pack/plugin/esql/qa/testFixtures/src/main/resources/match-phrase-function.csv-spec

Co-authored-by: Carlos Delgado <[email protected]>

* PR feedback

* Fix auto-commit error

* Regenerate docs

* Update x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/MatchPhrase.java

Co-authored-by: Liam Thompson <[email protected]>

* Remove non text field types

* Fake test data

* Remove tests that no longer should pass without ip/date/version support

* Put real data in score tests now that I was able to engineer a failure

* Realized the scoring test might be flakey because how it was written, updated

* PR feedback

* PR feedback

* [CI] Auto commit changes from spotless

* Add check to MatchPhrase tests

* Fix merge errors

* [CI] Auto commit changes from spotless

* Test generated docs

* Add additional verifier tests

---------

Co-authored-by: elasticsearchmachine <[email protected]>
Co-authored-by: Carlos Delgado <[email protected]>
Co-authored-by: Liam Thompson <[email protected]>
(cherry picked from commit eee423a)

# Conflicts:
#	docs/reference/esql/functions/description/match_phrase.md
#	docs/reference/esql/functions/examples/match_phrase.md
#	docs/reference/esql/functions/kibana/definition/match_phrase.json
#	docs/reference/esql/functions/kibana/docs/match_phrase.md
#	docs/reference/query-languages/esql/_snippets/lists/search-functions.md
#	docs/reference/query-languages/esql/functions-operators/search-functions.md
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/EsqlFunctionRegistry.java
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/FullTextFunction.java
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/Match.java
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/QueryString.java
#	x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/analysis/VerifierTests.java
@kderusso
Copy link
Member Author

💚 All backports created successfully

Status Branch Result
8.19

Questions ?

Please refer to the Backport tool documentation

@kderusso
Copy link
Member Author

Backport PR: #129215

kderusso added a commit that referenced this pull request Jun 12, 2025
* [ES|QL] Add MATCH_PHRASE (#127661)

* Initial commit of match_phrase

* Add MatchPhraseQueryTests

* First pass at CSV specs

* Update docs/changelog/127661.yaml

* Refactor so MatchPhrase doesn't use all fulltext test cases, just text only

* Fix tests

* Add some CSV test cases

* Fix test

* Update changelog

* Update tests

* Comment out MATCH_PHRASE in search-functions Markdown

* Minor PR feedback

* PR feedback - refactor/consolidate code

* Add some more tests

* Fix some tests

* [CI] Auto commit changes from spotless

* Fix tests

* PR feedback - add tests, support boost and numeric data

* Revert "PR feedback - add tests, support boost and numeric data"

This reverts commit 4e7a699.

* Apply testing/PR feedback outside numeric support only

* Regenerate docs

* Add negative test

* Update x-pack/plugin/esql/qa/testFixtures/src/main/resources/match-phrase-function.csv-spec

Co-authored-by: Carlos Delgado <[email protected]>

* Update x-pack/plugin/esql/qa/testFixtures/src/main/resources/match-phrase-function.csv-spec

Co-authored-by: Carlos Delgado <[email protected]>

* Update x-pack/plugin/esql/qa/testFixtures/src/main/resources/match-phrase-function.csv-spec

Co-authored-by: Carlos Delgado <[email protected]>

* PR feedback

* Fix auto-commit error

* Regenerate docs

* Update x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/MatchPhrase.java

Co-authored-by: Liam Thompson <[email protected]>

* Remove non text field types

* Fake test data

* Remove tests that no longer should pass without ip/date/version support

* Put real data in score tests now that I was able to engineer a failure

* Realized the scoring test might be flakey because how it was written, updated

* PR feedback

* PR feedback

* [CI] Auto commit changes from spotless

* Add check to MatchPhrase tests

* Fix merge errors

* [CI] Auto commit changes from spotless

* Test generated docs

* Add additional verifier tests

---------

Co-authored-by: elasticsearchmachine <[email protected]>
Co-authored-by: Carlos Delgado <[email protected]>
Co-authored-by: Liam Thompson <[email protected]>
(cherry picked from commit eee423a)

# Conflicts:
#	docs/reference/esql/functions/description/match_phrase.md
#	docs/reference/esql/functions/examples/match_phrase.md
#	docs/reference/esql/functions/kibana/definition/match_phrase.json
#	docs/reference/esql/functions/kibana/docs/match_phrase.md
#	docs/reference/query-languages/esql/_snippets/lists/search-functions.md
#	docs/reference/query-languages/esql/functions-operators/search-functions.md
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/EsqlFunctionRegistry.java
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/FullTextFunction.java
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/Match.java
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/QueryString.java
#	x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/analysis/VerifierTests.java

* Take match_phrase out of snapshot and make tech preview (#128925)

* Take match_phrase out of snapshot and make tech preview

* Update docs/changelog/128925.yaml

* PR feedback

* Adding regenerated test data

* Update docs/changelog/128925.yaml

Co-authored-by: Carlos Delgado <[email protected]>

* [CI] Auto commit changes from spotless

* Checkstyle

* Correct docs

* Hopefully fix docs build

* Found one more bad docs link - here's hoping this now fixes the doc build

* OMG bitten by - vs _

---------

Co-authored-by: Carlos Delgado <[email protected]>
Co-authored-by: elasticsearchmachine <[email protected]>
Co-authored-by: Aurélien FOUCRET <[email protected]>

* Remove null example for match_phrase (#129173)

* Fix errors from merge

* Fix more errors from merge

* [CI] Auto commit changes from spotless

* Fix compile errors introduced through cherry-pick

* Fix test compilation

* Generate docs

* Register match_phrase as a function not a snapshot function (#129255)

* Register match_phrase as a function not a snapshot function

* Update usage

* Fix usage

---------

Co-authored-by: Carlos Delgado <[email protected]>
Co-authored-by: elasticsearchmachine <[email protected]>
Co-authored-by: Aurélien FOUCRET <[email protected]>
Co-authored-by: Ioana Tagirta <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL auto-backport Automatically create backport pull requests when merged >enhancement :SearchOrg/Relevance Label for the Search (solution/org) Relevance team Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) Team:Search - Relevance The Search organization Search Relevance team Team:SearchOrg Meta label for the Search Org (Enterprise Search) v8.19.0 v9.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants