[Question] en/decryptData with CryptoMaterialsManager is deprecated, how to migrate to non-deprecated?

[StiviiK]

EDIT: Original problem solved, single question still open - please see #2033 (comment).

Problem:

I am trying to use the aws-encryption-sdk and followed the examples (which work fine), but the methods the examples are using are deprecated:

How can I migrate to the non-deprecated methods?

Source-Context:

[...]
 // Create a KMS master key provider
var masterKeyProvider = KmsMasterKeyProvider.builder()
        .buildStrict(kmsKeyId);

// Create a caching cryptographic materials manager
this.cachingMaterialsManager = CachingCryptoMaterialsManager.newBuilder()
        .withMasterKeyProvider(masterKeyProvider)
        .withCache(new LocalCryptoMaterialsCache(MAX_CACHE_ENTRIES))
        .withMaxAge(MAX_ENTRY_AGE_MILLISECONDS, TimeUnit.MILLISECONDS)
        .withMessageUseLimit(MAX_ENTRY_USES)
        .build();

[...]

var result = this.crypto.encryptData(
        this.cachingMaterialsManager,
        value.getBytes(StandardCharsets.UTF_8),
        new HashMap<>() // encryption context
);

[...]

// ToDO: Verify the encryption context
var result = this.crypto.decryptData(
        this.cachingMaterialsManager,
        encrypted
);

[marcindabrowski]

I think you have to use Keyrings as in the examples https://github.com/aws/aws-encryption-sdk-java/tree/master/src/examples/java/com/amazonaws/crypto/examples/keyrings.

[StiviiK]

I think you have to use Keyrings as in the examples https://github.com/aws/aws-encryption-sdk-java/tree/master/src/examples/java/com/amazonaws/crypto/examples/keyrings.

Interesting Idea, but it seems like this may be even older as it used the Java AWS SDK v1:
import com.amazonaws.encryptionsdk.kms.....;
vs.
import com.amazonaws.encryptionsdk.kmssdkv2....;

For context, I basically followed: https://github.com/aws/aws-encryption-sdk-java/blob/master/src/examples/java/com/amazonaws/crypto/examples/v2/SimpleDataKeyCachingExample.java

[marcindabrowski]

Well, I think it is the new API. Those tests where added recently, and it is working for me.
If you look into the API, only methods that uses keyring are not deprecated.

I think that version 3.0.0 is new, and documentation on Amazon site was not yet updated to point it.

[StiviiK]

You're right. But it can't figure out how to use caching using the Keyrings.
Additionally the non-deprecated API accepts also ICryptographicMaterialsManager, but I can't figure out how to use that interface.

[marcindabrowski]

I also don't know how to use the cache. But I think that somehow it is build in as default - i'm not sure, the code is somehow generated with Dafny. It have to be confirmed by someone from Amazon.

[StiviiK]

Thanks for your help, I have reimplemented it using Keyrings.

@repo-maintainers / amazon

I also don't know how to use the cache. But I think that somehow it is build in as default - i'm not sure, the code is somehow generated with Dafny. It have to be confirmed by someone from Amazon.

Could please somebody of you confirm / provide advise please?

[lucasmcdonald3]

Hi @StiviiK --

All new applications should use keyrings.
The preferred caching solution for new applications using keyrings is to use the AWS KMS Hierarchical Keyring.
This keyring caches intermediate branch key materials rather than data keys, improving caching performance.

Any applications that have already integrated with the caching cryptographic materials manager (CMM) are blocked from adopting keyrings until we (AWS Crypto Tools) release the caching CMM in the AWS Cryptographic Material Providers Library (MPL).

It looks like you are working on a new application, so I would suggest using the hierarchical keyring.

Let us know if you have further questions.
Thanks,
Lucas

[StiviiK]

@lucasmcdonald3 Awesome, thank you very much for your advice. I started implementing it using AWS KMS Hierarchical keyrings, but then another question came up for me.

Regarding Step 3, when do you recommend to create a new active branch key? We are basically building a Microservice Architecture with multiple REST APIs which encrypt/decrypt sensitive data on the fly.

[getaceres]

Hello. I'm trying to migrate an application from V2 to V3 and I have this problem too. I'm using a CachingCryptoMaterialsManager to minimize calls to KMS, since reusing the same key it's not a risk for us. I've read the documentation of the Hierarchical Keyring but it needs a DynamoDB table. However, our organization forbade us from using DynamoBD. ¿Is there a local implementation to replace the old CachingCryptoMaterialsManager just mantaining them in memory without any other dependency?

[seebees]

Regarding Step 3, when do you recommend to create a new active branch key?

@StiviiK This is a question for your threat model.
That said, Versioning (really, rotating) Branch Keys yearly is a good default.

@getaceres
We are planning on releasing the Caching CMM in the MPL.
However until we do the native Caching CMM is supported.

I have cut aws/aws-cryptographic-material-providers-library#354
to track support for other backing stores.

[StiviiK]

Regarding Step 3, when do you recommend to create a new active branch key?

@StiviiK This is a question for your threat model. That said, Versioning (really, rotating) Branch Keys yearly is a good default.

Thanks for the explanation regarding rotation.

Hi @ lucasmcdonald3:
I now have a follow-up question about branch key rotation. It seems that after I rotate the branch key, decryption of previously encrypted data no longer works.

Steps to Reproduce:

Step 1: I create a branch key and use it as input for branchKeyIdentifier:

public String createBranchKey() {
    // Create the branch key
    logger.info("Creating branch key");
    var keyId = keyStore.CreateKey(CreateKeyInput.builder().build()).branchKeyIdentifier();
    logger.info("Created key id: {}", keyId);
    return keyId;
}

public AwsKmsEncryptionComponent(KmsClient kmsClient, DynamoDbClient dynamoDbClient,
        AwsCryptoKeyStoreConfig keyStoreConfig, Kryo kryo) {
   crypto = AwsCrypto.builder().withCommitmentPolicy(CommitmentPolicy.RequireEncryptRequireDecrypt).build();
   serializer = kryo;

   keyStore = KeyStore.builder().KeyStoreConfig(KeyStoreConfig.builder().ddbClient(dynamoDbClient)
        .ddbTableName(keyStoreConfig.ddbname()).logicalKeyStoreName(keyStoreConfig.ddbname())
        .kmsClient(kmsClient)
        .kmsConfiguration(KMSConfiguration.builder().kmsKeyArn(keyStoreConfig.keyArn()).build()).build())
        .build();
   var materialsProvider = MaterialProviders.builder()
        .MaterialProvidersConfig(MaterialProvidersConfig.builder().build()).build();
   var kmsKeyringInput = CreateAwsKmsHierarchicalKeyringInput.builder().keyStore(keyStore)
        .branchKeyId(keyStoreConfig.branchKeyIdentifier()).ttlSeconds(24 * 60 * 60) // 24 hours
        .cache(CacheType.builder().Default(DefaultCache.builder().entryCapacity(1000).build()).build()).build();
   keyring = materialsProvider.CreateAwsKmsHierarchicalKeyring(kmsKeyringInput);
}

Step 2: I encrypt and decrypt data. Everything works as expected.

Step 3: I rotate the branch key by creating a new one using the method above and update the branchKeyIdentifier. (And reboot the application.)

Step 4: I attempt to decrypt old data and encounter the following error:

Caused by: software.amazon.cryptography.materialproviders.model.AwsCryptographicMaterialProvidersException: Unable to decrypt data key: No Encrypted Data Keys found to match. 
 Expected: 
KeyProviderId: aws-kms-hierarchy, KeyProviderInfo: 54d09f8d-b09a-44de-a2ac-e05e7857e042, BranchKeyVersion: 3826e188-e7df-4946-9770-653be0ffead4

However, inspecting DynamoDB shows that the key still exists and appears correct, so it should be able to resolve the old branch key.

Note: When I try to use the old identifier for the branchKeyIdentifier the previous data can be decrypted normally.

Question:

Am I missing something conceptually about branch key rotation? Shouldn’t the system be able to look up the correct branch key version in DynamoDB when decrypting older data?

Thanks in advance!

EDIT: I have realised my mistake. Generating a new branch key is the wrong approach, instead a new version should be generated based on the branch key: keyStore.VersionKey(VersionKeyInput.builder().branchKeyIdentifier("ID").build());.