ESQL: Improve Lookup Join performance with CachedDirectoryReader #137539
Conversation
elasticsearchmachine
added
v9.3.0
needs:triage
elasticsearchmachine
added
Team:Analytics
|
Pinging @elastic/es-analytical-engine (Team:Analytics) |
|
Hi @julian-elastic, I've created a changelog YAML for you. |
|
Buildkite benchmark this with esql-joins please |
|
|
||
| @Override | ||
| public void seekExact(BytesRef term, TermState state) throws IOException { | ||
| // TODO: when this can be true? |
There was a problem hiding this comment.
@dnhatn Do you need me to do any research on this one?
|
Buildkite benchmark this with esql-joins please |
| if (terms == null) { | ||
| return null; | ||
| } | ||
| // Return a FilterTerms that always creates a fresh TermsEnum iterator |
There was a problem hiding this comment.
@julian-elastic Sorry for the delay. I'm not sure this helps, since super.terms(k) likely gets it from a map as well. Is it possible to enable this cache only for a single term (depending on QueryList type), or to enable caching for DocValues only?
There was a problem hiding this comment.
You are right, I removed the termCache completely, and I still got the same improvement as without it.
|
Buildkite benchmark this with esql-joins please |
julian-elastic
force-pushed
the
CachedDirectoryReader
branch
from
684cc88 to
3e5e9ff
Compare
|
Buildkite benchmark this with esql-joins please |
💚 Build Succeeded
This build ran two esql-joins benchmarks to evaluate performance impact of this PR. History |
4 participants
Improve Lookup Join performance by caching objects needed for Lucene Queries.
Credit goes to @dnhatn