Skip to content

Add Cluster Feature for L2 Norm #129181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 37 commits into from
Jun 10, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
12fb2fa
propgating retrievers to inner retrievers
mridula-s109 Jun 2, 2025
81e99b6
test feature taken care of
mridula-s109 Jun 6, 2025
05fb0ab
Merge branch 'elastic:main' into main
mridula-s109 Jun 6, 2025
605c035
Small changes in concurrent multipart upload interfaces (#128977)
tlrx Jun 6, 2025
2dca633
Unmute FollowingEngineTests#testProcessOnceOnPrimary() test (#129054)
martijnvg Jun 6, 2025
4c0e3c9
[Build] Add support for publishing to maven central (#128659)
breskeby Jun 6, 2025
e2189e6
ESQL: Check for errors while loading blocks (#129016)
nik9000 Jun 6, 2025
aec1688
Make `PhaseCacheManagementTests` project-aware (#129047)
nielsbauman Jun 6, 2025
8c423ce
Vector test tools (#128934)
benwtrent Jun 6, 2025
df3ef0d
ES|QL: refactor generative tests (#129028)
luigidellaquila Jun 6, 2025
0eebc8c
Add a test of LOOKUP JOIN against a time series index (#129007)
bpintea Jun 6, 2025
b1e15f0
Make ILM `ClusterStateWaitStep` project-aware (#129042)
nielsbauman Jun 6, 2025
846b09a
Mute org.elasticsearch.xpack.esql.qa.mixed.MixedClusterEsqlSpecIT tes…
elasticsearchmachine Jun 6, 2025
a97d582
Remove `ClusterState` param from ILM `AsyncBranchingStep` (#129076)
nielsbauman Jun 6, 2025
763b502
Mute org.elasticsearch.xpack.esql.qa.mixed.MixedClusterEsqlSpecIT tes…
elasticsearchmachine Jun 6, 2025
8a660c8
Mute org.elasticsearch.upgrades.UpgradeClusterClientYamlTestSuiteIT t…
elasticsearchmachine Jun 6, 2025
aa16175
Mute org.elasticsearch.upgrades.UpgradeClusterClientYamlTestSuiteIT t…
elasticsearchmachine Jun 6, 2025
6e58b1e
Mute org.elasticsearch.packaging.test.DockerTests test081SymlinksAreF…
elasticsearchmachine Jun 7, 2025
05f70f0
Threadpool merge executor is aware of available disk space (#127613)
albertzaharovits Jun 8, 2025
713ab42
Add option to include or exclude vectors from _source retrieval (#128…
jimczi Jun 9, 2025
0776562
Remove direct minScore propagation to inner retrievers
mridula-s109 Jun 9, 2025
f145d26
cleaned up skip
mridula-s109 Jun 9, 2025
d8b6897
Mute org.elasticsearch.index.engine.ThreadPoolMergeExecutorServiceDis…
elasticsearchmachine Jun 9, 2025
82c7ab1
Add transport version for ML inference Mistral chat completion (#129033)
Jan-Kazlouski-elastic Jun 9, 2025
eca383d
Correct index path validation (#129144)
benwtrent Jun 9, 2025
fb6ec9a
Mute org.elasticsearch.index.engine.ThreadPoolMergeExecutorServiceDis…
elasticsearchmachine Jun 9, 2025
6806b24
Implemented completion task for Google VertexAI (#128694)
leo-hoet Jun 9, 2025
0ef36a1
Merge remote-tracking branch 'upstream/main'
mridula-s109 Jun 9, 2025
ece13d9
Merge remote-tracking branch 'upstream/main'
mridula-s109 Jun 9, 2025
36cd91e
Merge remote-tracking branch 'upstream/main'
mridula-s109 Jun 10, 2025
5716501
Added cluster feature to yaml
mridula-s109 Jun 10, 2025
00169b2
Node feature added
mridula-s109 Jun 10, 2025
34c3c27
Duplicate line - result of merge removed
mridula-s109 Jun 10, 2025
9a2a238
Merge branch 'main' into introduce-test_feature
mridula-s109 Jun 10, 2025
8efe176
Update docs/changelog/129181.yaml
mridula-s109 Jun 10, 2025
9bf88d4
Merge branch 'main' into introduce-test_feature
mridula-s109 Jun 10, 2025
fc979f7
Update 129181.yaml
mridula-s109 Jun 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/changelog/129181.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pr: 129181
summary: Add Cluster Feature for L2 Norm
area: "Search"
type: bug
issues: []
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
import java.util.Set;

import static org.elasticsearch.search.retriever.CompoundRetrieverBuilder.INNER_RETRIEVERS_FILTER_SUPPORT;
import static org.elasticsearch.xpack.rank.linear.L2ScoreNormalizer.LINEAR_RETRIEVER_L2_NORM;
import static org.elasticsearch.xpack.rank.linear.MinMaxScoreNormalizer.LINEAR_RETRIEVER_MINMAX_SINGLE_DOC_FIX;

public class RankRRFFeatures implements FeatureSpecification {
Expand All @@ -26,6 +27,6 @@ public Set<NodeFeature> getFeatures() {

@Override
public Set<NodeFeature> getTestFeatures() {
return Set.of(INNER_RETRIEVERS_FILTER_SUPPORT, LINEAR_RETRIEVER_MINMAX_SINGLE_DOC_FIX);
return Set.of(INNER_RETRIEVERS_FILTER_SUPPORT, LINEAR_RETRIEVER_MINMAX_SINGLE_DOC_FIX, LINEAR_RETRIEVER_L2_NORM);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
package org.elasticsearch.xpack.rank.linear;

import org.apache.lucene.search.ScoreDoc;
import org.elasticsearch.features.NodeFeature;

/**
* A score normalizer that applies L2 normalization to a set of scores.
Expand All @@ -25,6 +26,8 @@ public class L2ScoreNormalizer extends ScoreNormalizer {

private static final float EPSILON = 1e-6f;

public static final NodeFeature LINEAR_RETRIEVER_L2_NORM = new NodeFeature("linear_retriever.l2_norm");

public L2ScoreNormalizer() {}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -266,6 +266,9 @@ setup:

---
"should normalize initial scores with l2_norm":
- requires:
cluster_features: [ "linear_retriever.l2_norm" ]
reason: "Requires l2_norm normalization support in linear retriever"
- do:
search:
index: test
Expand Down Expand Up @@ -319,6 +322,9 @@ setup:

---
"should handle all zero scores in normalization":
- requires:
cluster_features: [ "linear_retriever.l2_norm" ]
reason: "Requires l2_norm normalization support in linear retriever"
- do:
search:
index: test
Expand Down
Loading