DynamoDB tests spin forever and fail when running against DynamoDB Local

[jeremyong]

In TableOperationTest.cpp, I modified line 51 to look like

static const char* ENDPOINT_OVERRIDE = "localhost:8000";

In addition, I also ran DynamoDB Local following instructions in the following page: http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Tools.DynamoDBLocal.html#Tools.DynamoDBLocal.DownloadingAndRunning

After compiling and running runDynamoDBIntegrationTests, I get a large number of failed gtest assertions and consequently test failures. They appear to be taking an extremely long time to run as well, which I don't expect (> 15 minutes and still going?).

Please advise if I am doing something wrong as I am new to this particular database service and SDK in general. Thanks.

[prestomation]

Hi jeremyong,

I do not think we have ever tried the SDK with dynamodb-local, but it should work.

Some of the tests may wait for certain things to become consistent. if they never do, they will time out. This may be why the tests are taking long time to run, as the expected outcome never occurs.

Can you attach the log file generated by the dynamodb test executable? Please scrub any personal information or credentials in the logfile before attaching, if there is any.

[jeremyong]

/Users/jeremy/developer/aws-sdk-cpp/aws-cpp-sdk-dynamodb-integration-tests/TableOperationTest.cpp:237: Failure
Value of: deleteTableOutcome.GetError().GetErrorType()
Actual: 4-byte object <63-00 00-00>
Expected: DynamoDBErrors::RESOURCE_NOT_FOUND
Which is: 4-byte object <10-00 00-00>
/Users/jeremy/developer/aws-sdk-cpp/aws-cpp-sdk-dynamodb-integration-tests/TableOperationTest.cpp:237: Failure
Value of: deleteTableOutcome.GetError().GetErrorType()
Actual: 4-byte object <63-00 00-00>
Expected: DynamoDBErrors::RESOURCE_NOT_FOUND
Which is: 4-byte object <10-00 00-00>
/Users/jeremy/developer/aws-sdk-cpp/aws-cpp-sdk-dynamodb-integration-tests/TableOperationTest.cpp:237: Failure
Value of: deleteTableOutcome.GetError().GetErrorType()
Actual: 4-byte object <63-00 00-00>
Expected: DynamoDBErrors::RESOURCE_NOT_FOUND
Which is: 4-byte object <10-00 00-00>
/Users/jeremy/developer/aws-sdk-cpp/aws-cpp-sdk-dynamodb-integration-tests/TableOperationTest.cpp:237: Failure
Value of: deleteTableOutcome.GetError().GetErrorType()
Actual: 4-byte object <63-00 00-00>
Expected: DynamoDBErrors::RESOURCE_NOT_FOUND
Which is: 4-byte object <10-00 00-00>
/Users/jeremy/developer/aws-sdk-cpp/aws-cpp-sdk-dynamodb-integration-tests/TableOperationTest.cpp:237: Failure
Value of: deleteTableOutcome.GetError().GetErrorType()
Actual: 4-byte object <63-00 00-00>
Expected: DynamoDBErrors::RESOURCE_NOT_FOUND
Which is: 4-byte object <10-00 00-00>
/Users/jeremy/developer/aws-sdk-cpp/aws-cpp-sdk-dynamodb-integration-tests/TableOperationTest.cpp:237: Failure
Value of: deleteTableOutcome.GetError().GetErrorType()
Actual: 4-byte object <63-00 00-00>
Expected: DynamoDBErrors::RESOURCE_NOT_FOUND
Which is: 4-byte object <10-00 00-00>
[ RUN ] TableOperationTest.TestListTable
/Users/jeremy/developer/aws-sdk-cpp/aws-cpp-sdk-dynamodb-integration-tests/TableOperationTest.cpp:237: Failure
Value of: deleteTableOutcome.GetError().GetErrorType()
Actual: 4-byte object <63-00 00-00>
Expected: DynamoDBErrors::RESOURCE_NOT_FOUND
Which is: 4-byte object <10-00 00-00>
/Users/jeremy/developer/aws-sdk-cpp/aws-cpp-sdk-dynamodb-integration-tests/TableOperationTest.cpp:237: Failure
Value of: deleteTableOutcome.GetError().GetErrorType()
Actual: 4-byte object <63-00 00-00>
Expected: DynamoDBErrors::RESOURCE_NOT_FOUND
Which is: 4-byte object <10-00 00-00>
/Users/jeremy/developer/aws-sdk-cpp/aws-cpp-sdk-dynamodb-integration-tests/TableOperationTest.cpp:237: Failure
Value of: deleteTableOutcome.GetError().GetErrorType()
Actual: 4-byte object <63-00 00-00>
Expected: DynamoDBErrors::RESOURCE_NOT_FOUND
Which is: 4-byte object <10-00 00-00>
/Users/jeremy/developer/aws-sdk-cpp/aws-cpp-sdk-dynamodb-integration-tests/TableOperationTest.cpp:237: Failure
Value of: deleteTableOutcome.GetError().GetErrorType()
Actual: 4-byte object <63-00 00-00>
Expected: DynamoDBErrors::RESOURCE_NOT_FOUND
Which is: 4-byte object <10-00 00-00>
/Users/jeremy/developer/aws-sdk-cpp/aws-cpp-sdk-dynamodb-integration-tests/TableOperationTest.cpp:237: Failure
Value of: deleteTableOutcome.GetError().GetErrorType()
Actual: 4-byte object <63-00 00-00>
Expected: DynamoDBErrors::RESOURCE_NOT_FOUND
Which is: 4-byte object <10-00 00-00>
/Users/jeremy/developer/aws-sdk-cpp/aws-cpp-sdk-dynamodb-integration-tests/TableOperationTest.cpp:237: Failure
Value of: deleteTableOutcome.GetError().GetErrorType()
Actual: 4-byte object <63-00 00-00>
Expected: DynamoDBErrors::RESOURCE_NOT_FOUND
Which is: 4-byte object <10-00 00-00>
/Users/jeremy/developer/aws-sdk-cpp/aws-cpp-sdk-dynamodb-integration-tests/TableOperationTest.cpp:220: Failure
Value of: DynamoDBErrors::RESOURCE_IN_USE
Actual: 4-byte object <85-00 00-00>
Expected: createTableOutcome.GetError().GetErrorType()
Which is: 4-byte object <63-00 00-00>
/Users/jeremy/developer/aws-sdk-cpp/aws-cpp-sdk-dynamodb-integration-tests/TableOperationTest.cpp:303: Failure
Value of: listTablesOutcome.IsSuccess()
Actual: false
Expected: true
/Users/jeremy/developer/aws-sdk-cpp/aws-cpp-sdk-dynamodb-integration-tests/TableOperationTest.cpp:315: Failure
Value of: filteredTableNames.size()
Actual: 0
Expected: 1uL
Which is: 1
[ FAILED ] TableOperationTest.TestListTable (204761 ms)
[ RUN ] TableOperationTest.TestUpdateThroughput
/Users/jeremy/developer/aws-sdk-cpp/aws-cpp-sdk-dynamodb-integration-tests/TableOperationTest.cpp:220: Failure
Value of: DynamoDBErrors::RESOURCE_IN_USE
Actual: 4-byte object <85-00 00-00>
Expected: createTableOutcome.GetError().GetErrorType()
Which is: 4-byte object <63-00 00-00>
/Users/jeremy/developer/aws-sdk-cpp/aws-cpp-sdk-dynamodb-integration-tests/TableOperationTest.cpp:269: Failure
Value of: outcome.IsSuccess()
Actual: false
Expected: true

*edit shortened to make this page easier to navigate.

[jeremyong]

How could things "never get consistent?" This sort of confuses me given that the local database is a single shard and there isn't any network link breakage or something that could prevent replica copies from merging.

[prestomation]

Hi Jeremy, you attached the stdout. In the same directory as the executable should be a log file. Please attach this one.

[jeremyong]

http://paste.ubuntu.com/13219062/

[jeremyong]

I disabled the usage of HTTPS to bypass curl error 35 (failure to negotiate https handshake) which resulted in

[TRACE] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000] Making request to http://localhost:8000
[TRACE] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000] Including headers:
[TRACE] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000] content-length: 46
[TRACE] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000] content-type: application/x-amz-json-1.0
[TRACE] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000] host: localhost
[TRACE] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000] user-agent: aws-sdk-cpp/0.9-493-gcbf1dc0 Darwin/15.0.0 x86_64
[TRACE] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000] x-amz-target: DynamoDB_20120810.DescribeTable
[DEBUG] 2015-11-10 14:19:46 CurlHandleContainer [0x7fff75aaa000] Attempting to acquire curl connection.
[DEBUG] 2015-11-10 14:19:46 CurlHandleContainer [0x7fff75aaa000] Returning connection handle 0x7ff981809e00
[DEBUG] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000] Obtained connection handle 0x7ff981809e00
[TRACE] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000] HTTP/1.1 100 Continue

[TRACE] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000]

[TRACE] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000] HTTP/1.1 400 Bad Request

[TRACE] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000] Content-Type: application/x-amz-json-1.0

[TRACE] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000] x-amzn-RequestId: 282a86d2-2055-4685-ab49-ea21f3d81146

[TRACE] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000] Content-Length: 173

[TRACE] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000] Server: Jetty(8.1.12.v20130726)

[TRACE] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000]

[TRACE] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000] 173 bytes written to response.
[DEBUG] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000] Returned http response code 400
[DEBUG] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000] Returned content type application/x-amz-json-1.0
[DEBUG] 2015-11-10 14:19:46 CurlHttpClient [0x7fff75aaa000] Releasing curl handle 0x7ff981809e00
[DEBUG] 2015-11-10 14:19:46 CurlHandleContainer [0x7fff75aaa000] Releasing curl handle 0x7ff981809e00
[DEBUG] 2015-11-10 14:19:46 CurlHandleContainer [0x7fff75aaa000] Notifying waiting threads.
[DEBUG] 2015-11-10 14:19:46 AWSClient [0x7fff75aaa000] Request returned error. Attempting to generate appropriate error codes from response
[TRACE] 2015-11-10 14:19:46 AWSClient [0x7fff75aaa000] Error response is {
"__type" : "com.amazonaws.dynamodb.v20120810#MissingAuthenticationToken",
"message" : "Request must contain either a valid (registered) AWS access key ID or X.509 certificate."
}

[prestomation]

Do you have your credentials file configured?
The C++ SDK follows the same conventions set by the other SDKS. See this blog post: https://blogs.aws.amazon.com/security/post/Tx3D6U6WSFGOK2H/A-New-and-Standardized-Way-to-Manage-Credentials-in-the-AWS-SDKs

[jeremyong]

OK I didn't realize that was necessary for talking to the local db as well. After setting the credentials file and disabling HTTPS the test cases pass.

[prestomation]

Excellent! I didn't realize DynamoDB local required credentials either.
Good to know our tests pass with DynamoDB Local!