As a user, I want to search nested attributes using predicate-based path filtering

[jordanpadams]

Checked for duplicates

No - I haven't checked

🧑‍🔬 User Persona(s)

Data User, Instrument Scientist, Rover Operations Team, Navigation Specialist, Thermal Performance Analyst

💪 Motivation

Building on the simple hierarchical path search capability (#611), users need the ability to filter nested attributes based on predicates that match specific values within nested structures. This is particularly critical for:

1. Indexed array structures - Where multiple instances of the same class exist with different index identifiers (e.g., temperature readings from different named sensors)
2. Conditional nested filtering - Where users need to match specific instances within arrays based on identifying attributes
3. Multi-level predicate queries - Where multiple nested predicates must be combined to isolate the correct data

This requirement implements predicate-based nested query capabilities using XPath-like syntax with bracket notation for filtering nested structures.

📖 Additional Details

Problem Statement:
Simple hierarchical paths (#611) enable navigation like *.ns:Class.ns:attribute, but cannot filter to specific instances within repeated nested structures. Users need to query: "Find products where the temperature index with name 'NAVCAM LEFT' has value > 30" rather than "Find products where any temperature index has value > 30".

Primary Use Cases:

US1: Device Temperature by Sensor Name
As a thermal-performance analyst, I want to find Navcam images where the temperature index with index_id='NAVCAM LEFT CAMERA PLATE' has index_value_temperature > 30, so I can study that specific sensor's performance at high temperatures.

US2: Motion Counter Indexing
As a rover team geologist, I want to retrieve all images where the Motion_Counter_Index with index_id='SITE' has index_value_number == 33 AND the Motion_Counter_Index with index_id='DRIVE' has index_value_number == 366, so I can view images from that specific location/drive combination.

US3: Device Angle by Index ID
As a navigation-system specialist, I want to find records where the Device_Angle_Index with index_id='ELEVATION_INITIAL-ENCODER' has a value > 0.72, so I can isolate images when elevation exceeds that threshold.

US4: Multi-Attribute Nested Filtering
As an instrument calibration engineer, I want to query nested structures matching multiple attribute conditions within the same nested object (e.g., exposure_type='DARK' AND exposure_duration < 0.001), so I can extract specific calibration frames.

Proposed Solution:

• Support XPath-like predicate syntax using bracket notation: ns:Class[ns:attribute='value']
• Support predicate comparisons: =, !=, <, >, <=, >=
• Support multiple predicates on the same path
• Support nested predicates at different hierarchy levels
• Leverage OpenSearch nested queries with proper bool logic for predicate filtering
• Build upon the hierarchical path support from As a user, I want to search by a full/unique hierarchical path for a specific attribute #611

Example Query Syntax:

// Single predicate filter
geom:Device_Temperature_Index[geom:index_id='NAVCAM LEFT CAMERA PLATE']/geom:index_value_temperature > 30

// Multiple predicates (AND within same nested object)
geom:Motion_Counter/geom:Motion_Counter_Index[geom:index_id='SITE' AND geom:index_value_number == 33]

// Combining multiple nested predicates across different instances
geom:Motion_Counter/geom:Motion_Counter_Index[geom:index_id='SITE']/geom:index_value_number == 33
AND geom:Motion_Counter/geom:Motion_Counter_Index[geom:index_id='DRIVE']/geom:index_value_number == 366

// Nested predicates at multiple levels
img:Imaging[img:imaging_mode='FULL_FRAME']/img:Exposure[img:exposure_type='DARK']/img:exposure_duration < 0.001

OpenSearch Implementation:
These queries translate to OpenSearch nested queries with proper path scoping and bool logic. Each predicate becomes a nested query with must clauses for both the predicate match and the final attribute filter.

Dependencies:

• Requires hierarchical path support from As a user, I want to search by a full/unique hierarchical path for a specific attribute #611
• May require enhanced OpenSearch index mapping with nested field types

Acceptance Criteria

Given a PDS4 product with nested array structures containing index identifiers
When I perform a query using predicate-based path filtering (e.g., ns:Class[ns:id='VALUE']/ns:attribute > 10)
Then I expect the API to return only products where that specific nested instance (matching the predicate) satisfies the attribute condition

Given a query with multiple predicates on different nested instances
When I execute the query with AND logic across predicates
Then I expect the API to correctly combine the nested queries and return products matching all predicate conditions

Given a predicate with comparison operators (=, !=, <, >, <=, >=)
When I include the predicate in a nested path query
Then I expect the API to correctly evaluate the predicate condition within the nested context

⚙️ Engineering Details

Implementation considerations:

• Extend lexer/parser to support bracket notation and predicate expressions
• Implement predicate-to-OpenSearch-nested-query translator with proper bool wrapping
• Handle multiple predicates on same path (must be within same nested context)
• Handle multiple predicates across different paths (separate nested contexts)
• Ensure proper scoping of predicate conditions within nested queries
• Support full set of comparison operators
• Validate predicate syntax and nested path structure
• Performance optimization for complex multi-predicate queries

🎉 I&T

Test coverage should include:

• Single predicate queries on nested structures
• Multiple predicates within same nested object (AND logic)
• Multiple predicates across different nested instances
• All comparison operators (=, !=, <, >, <=, >=)
• Combined hierarchical paths with predicates
• Edge cases: non-existent predicate attributes, type mismatches, deeply nested predicates
• Performance testing with complex multi-predicate queries