DynamoDb query results in [Query condition missed key schema element (Service: DynamoDb, Status Code: 400]

[sharmanalin59]

Describe the issue

Set up data as below-:
`
aws dynamodb create-table
--table-name Music
--attribute-definitions
AttributeName=Artist,AttributeType=S
AttributeName=SongTitle,AttributeType=S
--key-schema AttributeName=Artist,KeyType=HASH AttributeName=SongTitle,KeyType=RANGE
--provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1 \

aws dynamodb put-item
--table-name Music
--item
'{"Artist": {"S": "Beatles"}, "SongTitle": {"S": "Hey Jude"}, "AlbumTitle": {"S": "White album"}}'
--return-consumed-capacity TOTAL --endpoint-url http://localhost:8000 --region ap-south-1

aws dynamodb put-item \
--table-name Music \
--item \
    '{"Artist": {"S": "Beatles"}, "SongTitle": {"S": "Something"}, "AlbumTitle": {"S": "White album"}}' \
--return-consumed-capacity TOTAL --endpoint-url http://localhost:8000 --region ap-south-1

aws dynamodb put-item \
--table-name Music \
--item \
    '{"Artist": {"S": "Beatles"}, "SongTitle": {"S": "Taxman"}, "AlbumTitle": {"S": "Revolver"}}' \
--return-consumed-capacity TOTAL --endpoint-url http://localhost:8000 --region ap-south-1

Querying from cmd gets the result-:
aws dynamodb query
--table-name Music
--projection-expression "Artist, AlbumTitle, SongTitle"
--key-condition-expression "Artist = :artist and SongTitle = :songTitle"
--expression-attribute-values '{":artist":{"S":"Beatles"},":songTitle":{"S":"Something"}}'
--endpoint-url http://localhost:8000 --region ap-south-1

`
{
"Items": [
{
"Artist": {
"S": "Beatles"
},
"SongTitle": {
"S": "Something"
},
"AlbumTitle": {
"S": "White album"
}
}
],
"Count": 1,
"ScannedCount": 1,
"ConsumedCapacity": null
}

Below is the aws 2 sdk code which results in exception-:

DynamoConfig config = DynamoConfig.builder()
.region(Region.AP_SOUTH_1.toString())
.dynamoDbServiceType(DynamoDbServiceType.LOCAL)
.dynamoDbServiceEndpoint("http://localhost:8000").build();

Music bean-:

@DynamoDbBean
public class Music {

@DynamoDbPartitionKey
public String getArtist() {
    return this.artist;
}

@DynamoDbSortKey
public String getSongTitle() {
    return this.songTitle;
}

//@DynamoDbSecondaryPartitionKey()
public String getAlbumDesc() {
    return this.albumDesc;
}

public void setArtist(String artist) {
    this.artist = artist;
}

public void setSongTitle(String songTitle) {
    this.songTitle = songTitle;
}

public void setAlbumDesc(String albumDesc) {
    this.albumDesc = albumDesc;
}

}

----------- code that causes problem--------------------:

`
QueryEnhancedRequest.Builder queryEnhancedRequestBuilder =
QueryEnhancedRequest.builder().consistentRead(consistentRead)
.scanIndexForward(scanIndexForward)
.limit(limit);

        Key.Builder keyBuilder = getKeyBuilder(partitionValue, sortValue);
        queryEnhancedRequestBuilder.queryConditional(QueryConditional.keyEqualTo(keyBuilder.build()));

        if(Objects.nonNull(attributeExpressions) && !attributeExpressions.isEmpty()) {
            queryEnhancedRequestBuilder
                    .filterExpression(getExpression(attributeExpressions).build());
        }

        //itemTable.query(r -> r.queryConditional(getQueryConditional(dynamoDbSortOperator, keyBuilder, partitionValueEnd, sortValueEnd))).items().iterator();
        // Get items in the Customer table and write out the ID value.
        //Iterator<Item> results = itemTable.query(r -> r.queryConditional(queryConditional).filterExpression(expression)).items().iterator();
        return itemTable.query(queryEnhancedRequestBuilder.build()).items().iterator();

`

Below is the exception-:
software.amazon.awssdk.services.dynamodb.model.DynamoDbException: Query condition missed key schema element (Service: DynamoDb, Status Code: 400, Request ID: c7aa90b7-cdaf-4a87-8f62-10f75d6780ba, Extended Request ID: null)
at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleErrorResponse(CombinedResponseHandler.java:123)
at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:79)
at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:59)
at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:40)
at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:40)
at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:30)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)
at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:50)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:36)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:48)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:31)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:193)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:167)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:82)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:175)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:76)
at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:56)
at software.amazon.awssdk.services.dynamodb.DefaultDynamoDbClient.query(DefaultDynamoDbClient.java:4084)
at software.amazon.awssdk.services.dynamodb.paginators.QueryIterable$QueryResponseFetcher.nextPage(QueryIterable.java:134)
at software.amazon.awssdk.services.dynamodb.paginators.QueryIterable$QueryResponseFetcher.nextPage(QueryIterable.java:125)
at software.amazon.awssdk.core.pagination.sync.PaginatedResponsesIterator.next(PaginatedResponsesIterator.java:58)
at software.amazon.awssdk.enhanced.dynamodb.internal.TransformIterator.next(TransformIterator.java:44)
at software.amazon.awssdk.core.pagination.sync.PaginatedItemsIterable$ItemsIterator.(PaginatedItemsIterable.java:58)
at software.amazon.awssdk.core.pagination.sync.PaginatedItemsIterable.iterator(PaginatedItemsIterable.java:48)

Links

(https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/examples-dynamodb-enhanced.html)

[debora-ito]

@sharmanalin59 how is the method getKeyBuilder(partitionValue, sortValue) implemented? Is it adding the sort key as expected?

[sharmanalin59]

`
private Key.Builder getKeyBuilder(String partitionValue, String sortValue) {
Key.Builder keyBuilder = Key.builder();
keyBuilder.partitionValue(DynamodbDbUtil.getAttributeValue(partitionValue).build());
keyBuilder.sortValue(DynamodbDbUtil.getAttributeValue(sortValue).build());

/*if(Objects.nonNull(partitionValue)) {
if(isDouble(partitionValue)) {
keyBuilder.partitionValue(Double.parseDouble(partitionValue));
}
else {
keyBuilder.partitionValue(partitionValue);
}
if(Objects.nonNull(sortValue)) {
if (isDouble(sortValue)) {
keyBuilder.sortValue(Double.parseDouble(sortValue));
} else {
keyBuilder.sortValue(sortValue);
}
}
}
*/
return keyBuilder;
}

public static AttributeValue.Builder getAttributeValue(Object attrValue) {
AttributeValue.Builder attributeValueBuilder = AttributeValue.builder();

    if(Objects.isNull(attrValue)) {
        return null;
    }
    if(attrValue.getClass().equals(String.class)) {
        if(isDouble((String) attrValue)) {
            attributeValueBuilder.n((String) attrValue);
        }
        else {
            attributeValueBuilder.s((String) attrValue);
        }
    }
    else if(attrValue.getClass().equals(Boolean.class)) {
        attributeValueBuilder.nul((Boolean) attrValue);
    }
    else if(attrValue.getClass().equals(String[].class)) {
        if(isDouble(((String[])attrValue)[0])) {
            attributeValueBuilder.ns((String[]) attrValue);
        }
        else {
            attributeValueBuilder.ss((String[]) attrValue);
        }
    }
    else if(attrValue instanceof Collection) {
        String[] array = Arrays.stream(((Collection)attrValue).toArray()).toArray(String[]::new);
        if(array.length > 0) {
            if (isDouble(array[0])) {
                attributeValueBuilder.ns(array);
            } else {
                attributeValueBuilder.ss(array);
            }
        }
    }
    return attributeValueBuilder;
}

`

[debora-ito]

The question still remains, is it adding the sort key as expected?

Another way to check is to enable the verbose wirelogs - see here for details: https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/logging-slf4j.html#sdk-java-logging-verbose

The wirelogs will show the content of what's being sent in the request. We don't recommend enabling in production environments though. Please make sure to redact any sensitive information the logs might contain.

[sharmanalin59]

Please have a look at the table schema as well as aws cli command, that is precisely I am trying to do, Find Beatles(key) Something(sort) document

[debora-ito]

I understand! That's why I asked for the wirelogs, the logs will show the content of the query request that is being sent to the service.

[debora-ito]

Just a heads up, I'm moving this to Github Discussion, it's the proper channel for guidance questions from now on.

[sharmanalin59]

Am I supposed to wait for an answer here or elsewhere?