Hadoop 18154 branch 2.10.1

[jclarysse]

Description of PR

The PR addresses a requirement to comply with AWS security concept IAM roles for service accounts (IRSA) while operating Delta sharing in Amazon Elastic Kubernetes Service (EKS).
The code change consists in adding a new credentials provider class org.apache.hadoop.fs.s3a.OIDCTokenCredentialsProvider to the module hadoop-aws in Hadoop release 2.10.1. In addition, the dependency aws-java-sdk-bundle-1.11.271 was upgraded to its latest version 1.12.167 as AWS WebIdentityTokenCredentialsProvider class was not yet available in original version.

How was this patch tested?

No new unit-test or integration-test was created on-purpose. The patch was "only" tested as part of our specific use-case, using Delta sharing server 0.4.0 with the following Hadoop configuration (core-site.xml):

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
  <property>
    <name>fs.s3a.aws.credentials.provider</name>
    <value>org.apache.hadoop.fs.s3a.OIDCTokenCredentialsProvider</value>
  </property>
  <property>
    <name>fs.s3a.jwt.path</name>
    <value>/var/run/secrets/eks.amazonaws.com/serviceaccount/token</value>
  </property>
  <property>
    <name>fs.s3a.role.arn</name>
    <value>my_iam_role_arn</value>
  </property>
  <property>
    <name>fs.s3a.session.name</name>
    <value>my_iam_session_name</value>
  </property>
  <property>
      <name>fs.s3a.server-side-encryption-algorithm</name>
      <value>SSE-KMS</value>
  </property>
  <property>
      <name>fs.s3a.server-side-encryption.key</name>
      <value>my_kms_key_id</value>
  </property>      
</configuration>

For code changes:

• The title or this PR starts with the corresponding JIRA issue 'HADOOP-18154'
• Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
• No new dependency was added to the code, however one dependency was upgraded.

[jclarysse]

PR replaced by #4070