[S3Crt]: GetObject should return error specific to self signed cert when Aws::S3Crt::ClientConfiguration caFile is set to self signed cert and verifySSL is true

[csi-amolpawar]

Describe the bug

The Aws S3Crt GetObject request should return error specific to self signed cert. when Aws::S3Crt::ClientConfiguration caFile and verifySSL is configured to self signed cert and true respectively.

Expected Behavior

This should work similar as S3 protocol as we configure the same parameters. See output and code for s3 getObject below

#include <iostream>
#include <string>
#include <openssl/crypto.h>
#include <aws/core/Aws.h>
#include <aws/core/utils/memory/stl/AWSStringStream.h>
#include <aws/core/utils/logging/CRTLogSystem.h>
#include <aws/s3/S3Client.h>
#include <aws/s3/model/GetObjectRequest.h>

static const char ALLOCATION_TAG[] = "s3-getobject-public";

// This is similar retrieving the default dir $ openssl version -d
std::string get_default_openssl_dir()
{
  const std::string OPENSSLDIR_KEY("OPENSSLDIR: ");
  auto ssl_dir = std::string(SSLeay_version(SSLEAY_DIR));
  auto found = ssl_dir.find(OPENSSLDIR_KEY);
  if(found != std::string::npos)
  {
    ssl_dir = ssl_dir.substr(OPENSSLDIR_KEY.size());
    if(auto s = ssl_dir.size(); ssl_dir.at(0) == '"' && ssl_dir.at(s - 1) == '"')
      ssl_dir = ssl_dir.substr(1, s -2);
  }
  return ssl_dir;
}

int main(int argc, char* argv[])
{
  Aws::SDKOptions options;

  options.httpOptions.initAndCleanupCurl = false;
  options.cryptoOptions.initAndCleanupOpenSSL = false;

  options.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Info;

  Aws::InitAPI(options);
  {
    Aws::Client::ClientConfiguration config;
    config.region = Aws::Region::US_EAST_1;

    config.caFile = "/tmp/self-signed.crt";
    config.caPath = get_default_openssl_dir();
    config.verifySSL = true;

    Aws::S3::S3Client s3Client(config, Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Never, true);
    Aws::String bucket("my_bucket");
    Aws::String objectKey("test/my_object");

    Aws::S3::Model::GetObjectRequest request;
    request.SetBucket(bucket);
    request.SetKey(objectKey);
    
    if(auto outcome = s3Client.GetObject(request); outcome.IsSuccess())
      std::cout << outcome.GetResult().GetBody().rdbuf() << std::endl;
    else
      std::cerr << "GetObject error:" << outcome.GetError().GetMessage() << std::endl;
  }
  Aws::ShutdownAPI(options);
  return 0;
}

Output:

[root@e52bcd5686db ca_file_self_signed]# ./s3_get_object_w_self_signed_cert 
GetObject error:curlCode: 77, Problem with the SSL CA cert (path? access rights?)

Current Behavior

It able to read the data from object

[root@e52bcd5686db ca_file_self_signed]# ./s3crt_get_object_w_self_signed_cert 
This is test file for testing aws issues

Reproduction Steps

It is easily re-producible using below code snippet

#include <iostream>
#include <string>
#include <openssl/crypto.h>
#include <aws/core/Aws.h>
#include <aws/core/utils/memory/stl/AWSStringStream.h>
#include <aws/core/utils/logging/CRTLogSystem.h>
#include <aws/s3-crt/S3CrtClient.h>
#include <aws/s3-crt/model/GetObjectRequest.h>

static const char ALLOCATION_TAG[] = "s3-crt-getobject-public";

// This is similar retrieving the default dir $ openssl version -d
std::string get_default_openssl_dir()
{
  const std::string OPENSSLDIR_KEY("OPENSSLDIR: ");
  auto ssl_dir = std::string(SSLeay_version(SSLEAY_DIR));
  auto found = ssl_dir.find(OPENSSLDIR_KEY);
  if(found != std::string::npos)
  {
    ssl_dir = ssl_dir.substr(OPENSSLDIR_KEY.size());
    if(auto s = ssl_dir.size(); ssl_dir.at(0) == '"' && ssl_dir.at(s - 1) == '"')
      ssl_dir = ssl_dir.substr(1, s -2);
  }
  return ssl_dir;
}

int main(int argc, char* argv[])
{
  Aws::SDKOptions options;

  options.httpOptions.initAndCleanupCurl = false;
  options.cryptoOptions.initAndCleanupOpenSSL = false;

  options.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Info;
  options.loggingOptions.crt_logger_create_fn = [=]() {
    return Aws::MakeShared<Aws::Utils::Logging::DefaultCRTLogSystem>(
      ALLOCATION_TAG, options.loggingOptions.logLevel);
  };

  Aws::InitAPI(options);
  {
    Aws::S3Crt::ClientConfiguration config;
    config.region = Aws::Region::US_EAST_1;

    config.caFile = "/tmp/self-signed.crt";
    config.caPath = get_default_openssl_dir();
    config.verifySSL = true;

    Aws::S3Crt::S3CrtClient s3CrtClient(config, Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Never);
    Aws::String bucket("my_object");
    Aws::String objectKey("test/my_object");

    Aws::S3Crt::Model::GetObjectRequest request;
    request.SetBucket(bucket);
    request.SetKey(objectKey);
    
    if(auto outcome = s3CrtClient.GetObject(request); outcome.IsSuccess())
      std::cout << outcome.GetResult().GetBody().rdbuf() << std::endl;
    else
      std::cerr << "GetObject error:" << outcome.GetError().GetMessage() << std::endl;
  }
  Aws::ShutdownAPI(options);
  return 0;
}

Possible Solution

NA

Additional Information/Context

NA

AWS CPP SDK version used

1.11.95

Compiler and Version used

g++ (GCC) 11.2.0

Operating System and version

Linux e52bcd5686db 5.14.0-1057-oem #64-Ubuntu SMP Mon Jan 23 17:02:19 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

[yasminetalby]

Hello @csi-amolpawar ,

Thank you very much for your submission and for bringing this up to our attention.
I was able to reproduce the difference in behavior between S3-Crt and S3.

This issue seems different from the issue you have submitted previously (#2430) as the outcome of the GetObjectRequest (or other request such as PutObjectRequest) is successful using the S3 CRT vs S3.

I will discuss this with the team and post updates here.

Thank you very much once again for your feedback and collaboration.

Best regards,

Yasmine

[yasminetalby]

Hello @csi-amolpawar ,

This issue should be solved by #2530. The fix will be released in v1.11.97.

We really appreciate your feedback and collaboration.

Sincerely,

Yasmine

[csi-amolpawar]

Hi @yasminetalby,

I could see the error message now crtCode: 1029, AWS_IO_TLS_ERROR_NEGOTIATION_FAILURE - TLS (SSL) negotiation failed which is quite different than s3 curlCode: 60, SSL peer certificate or SSH remote key was not OK.

I see this is due to the Aws S3 CRT is not using curl hence returning crt specific things.

Please confirm the same.

Thanks, Amol.

[yasminetalby]

Hello @csi-amolpawar ,

Yes you are right, the error difference between the use of S3CRT and S3 is expected.
As you can see in our documentation : The default HTTP client for Windows is WinHTTP. The default HTTP client for all other platforms is curl..
For AWS S3 CRT, the CRT HTTP client is being used hence the difference in error message.

Let me know if you have any further question or if this issue can be closed as completed.

Best regards,

Yasmine

[csi-amolpawar]

Hi @yasminetalby,

You can close this ticket

Thanks, Amol.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.