DDB Enhanced: Support arbitrary map fields #1902
Comments
PatrickBuTaxdoo
added
feature-request
|
Hi @PatrickBuTaxdoo, DynamoDB Enhanced client won't work with You can see some examples in the DynamoDB README page and in the DynamoDB Enhanced test folder. |
|
I'll close this, feel free to reach out if you have any other question. |
|
I implemented a custom AttributeConverter that solved my issue without reflection. Its behaviour is similar to deserializing JSON using Gson to a Map (and back). Let me know if that could be a useful addition to this library. |
|
@PatrickBuTaxdoo , there is a request for something like that in #2162. |
|
@PatrickBuTaxdoo Can you please provide an example for the custom AttributeConverter that you wrote |
|
This attribute converter does not support all data types, but it was sufficient for my use case. Serialization from POJOs is not supported, only primitives, import software.amazon.awssdk.core.BytesWrapper;
import software.amazon.awssdk.enhanced.dynamodb.AttributeConverter;
import software.amazon.awssdk.enhanced.dynamodb.AttributeValueType;
import software.amazon.awssdk.enhanced.dynamodb.EnhancedType;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;
import java.math.BigDecimal;
import java.util.*;
import java.util.stream.Collectors;
public class JsonAttributeConverter implements AttributeConverter<Map<String, Object>> {
@Override
public AttributeValue transformFrom(Map<String, Object> input) {
return transformToAttributeValue(input);
}
@Override
public Map<String, Object> transformTo(AttributeValue input) {
return input.m().entrySet().stream()
.collect(Collectors.toMap(Map.Entry::getKey, e -> transformToObject(e.getValue())));
}
@Override
public EnhancedType<Map<String, Object>> type() {
return EnhancedType.mapOf(String.class, Object.class);
}
@Override
public AttributeValueType attributeValueType() {
return AttributeValueType.M;
}
/** Transforms the given object to an {@link AttributeValue} for DynamoDB. */
private static AttributeValue transformToAttributeValue(Object input) {
if (input instanceof Map) {
Map<String, Object> map = (Map<String, Object>) input;
return AttributeValue.builder()
.m(
map.entrySet().stream()
.collect(
Collectors.toMap(
Map.Entry::getKey,
e -> transformToAttributeValue(e.getValue()))))
.build();
}
if (input instanceof List) {
List<Object> input1 = (List<Object>) input;
List<AttributeValue> converted =
input1.stream()
.map(JsonAttributeConverter::transformToAttributeValue)
.collect(Collectors.toList());
return AttributeValue.builder().l(converted).build();
}
if (input instanceof Boolean) {
return AttributeValue.builder().bool((Boolean) input).build();
}
if (input instanceof String) {
return AttributeValue.builder().s((String) input).build();
}
if (input instanceof Number) {
return AttributeValue.builder().n(String.valueOf(input)).build();
}
throw new IllegalArgumentException("cannot serialize the given AttributeValue");
}
private static Object transformToObject(AttributeValue input) {
// map
if (input.hasM()) {
return input.m().entrySet().stream()
.collect(
Collectors.toMap(
Map.Entry::getKey, e -> transformToObject(e.getValue())));
}
// list
if (input.hasL()) {
return input.l().stream()
.map(JsonAttributeConverter::transformToObject)
.collect(Collectors.toList());
}
// string set
if (input.hasSs()) {
return input.ss();
}
// convert number sets to list of BigDecimals
if (input.hasNs()) {
return input.ns().stream().map(BigDecimal::new).collect(Collectors.toList());
}
// convert binary sets to list of byte arrays
if (input.hasBs()) {
return input.bs().stream().map(BytesWrapper::asByteArray).collect(Collectors.toList());
}
// string
if (input.s() != null) {
return input.s();
}
// convert number to BigDecimal
if (input.n() != null) {
return new BigDecimal(input.n());
}
// boolean
if (input.bool() != null) {
return input.bool();
}
// binary
if (input.b() != null) {
return input.b().asByteArray();
}
throw new IllegalArgumentException("cannot deserialize the given AttributeValue");
}
} |
Describe the Feature
If I have a class that I want to have mapped via the DynamoDB Enhanced client, I have to create a class like this:
This works fine, as long as the data structure is known.
I have a case where I need to support arbitrary Maps (or JSON objects). The key will always be a String, but the value type is unknown, so I use Object. I would like to have that mapped to a Map field in the DynamoDB item. My bean class now looks like this:
Trying to run
TableSchema.fromBean(Entry.class)now yields this error:The DDB Enhanced client seems to try to use a Map for the object type, and fails to get the class parameters.
Using just
Objectas seen hereyields the same error.
If there is another possibility to achieve that, please let me know.
Is your Feature Request related to a problem?
I want to store arbitrarily structured data in a map field, which is not possible by now.
Proposed Solution
Create a mapper for Objects, and use
instanceofto determine the type converter.Describe alternatives you've considered
JSON-encoding the data and store as string, but using this approach I would sacrifice searchability of the nested attributes.
Additional Context
Storing data with unknown structure, and still keep it searchable (via scan) would make a great addition!
Your Environment
The text was updated successfully, but these errors were encountered: