Skip to content

HADOOP-17183. ABFS: Enabling checkaccess on ABFS #2331

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
steveloughran merged 4 commits into from
Oct 1, 2020
Merged

HADOOP-17183. ABFS: Enabling checkaccess on ABFS #2331

steveloughran merged 4 commits into from
Oct 1, 2020

Conversation

@bilaharith
Copy link
Contributor

@bilaharith bilaharith commented Sep 23, 2020 •
edited
Loading

ABFS: Enabling checkaccess API

#2316
The commit got reverted due to test failure.
Checkaccess testcase creates a test FS instance created with auth type as OAuth with a client credentials for an app without any RBAC.
The test failure was happening when the tests were ran with SharedKey auth and the fs.azure.account.oauth.provider.type config is missing. Tests did not fail for OAuth and client credentials because in that case this config will be present.

Also these tests needs to be ran mandatorily. So we are letting the test to fail in the absence of the test specific configs.

Driver test results against taccounts in Central India region

With Check access test configs

HNS-OAuth

[INFO] Results:
[INFO]
[INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[WARNING] Tests run: 457, Failures: 0, Errors: 0, Skipped: 64
[INFO] Results:
[INFO]
[WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 24

HNS-SharedKey

[INFO] Results:
[INFO]
[INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[WARNING] Tests run: 457, Failures: 0, Errors: 0, Skipped: 24
[INFO] Results:
[INFO]
[WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 16

NonHNS-SharedKey

[INFO] Results:
[INFO]
[INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[WARNING] Tests run: 457, Failures: 0, Errors: 0, Skipped: 245
[INFO] Results:
[INFO]
[WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 16

Without checkaccess configs

HNS-OAuth

[INFO] Results:
[INFO]
[INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[WARNING] Tests run: 457, Failures: 0, Errors: 0, Skipped: 72
[INFO] Results:
[INFO]
[WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 24

Without check access and OAuth configs

HNS-SharedKey

[INFO] Results:
[INFO]
[INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[WARNING] Tests run: 457, Failures: 0, Errors: 0, Skipped: 32
[INFO] Results:
[INFO]
[WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 16

NonHNS-SharedKey

[INFO] Results:
[INFO]
[INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[WARNING] Tests run: 457, Failures: 0, Errors: 0, Skipped: 246
[INFO] Results:
[INFO]
[WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 16

@bilaharith bilaharith changed the title Hadoop 17183. ABFS: Enabling checkaccess on ABFS HADOOP-17183. ABFS: Enabling checkaccess on ABFS Sep 24, 2020
@bilaharith bilaharith force-pushed the HADOOP-17183-enablecheckaccess branch from 562445f to 0a6011f Compare September 27, 2020 20:27
@bilaharith
Copy link
Contributor Author

bilaharith commented Sep 28, 2020

Test results with accounts in Sentral India

Without checkaccess specific configs

HNS-OAuth

[INFO] Results:
[INFO]
[INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemCheckAccess.testCheckAccessForNonExistentFile » Unexp...
[ERROR] ITestAzureBlobFileSystemCheckAccess.testFsActionALL:293->checkPrerequisites:311->checkIfConfigIsSet:319 » IllegalArgument
[ERROR] ITestAzureBlobFileSystemCheckAccess.testFsActionEXECUTE:218->checkPrerequisites:311->checkIfConfigIsSet:319 » IllegalArgument
[ERROR] ITestAzureBlobFileSystemCheckAccess.testFsActionNONE:204->checkPrerequisites:311->checkIfConfigIsSet:319 » IllegalArgument
[ERROR] ITestAzureBlobFileSystemCheckAccess.testFsActionREAD:233->checkPrerequisites:311->checkIfConfigIsSet:319 » IllegalArgument
[ERROR] ITestAzureBlobFileSystemCheckAccess.testFsActionREADEXECUTE:263->checkPrerequisites:311->checkIfConfigIsSet:319 » IllegalArgument
[ERROR] ITestAzureBlobFileSystemCheckAccess.testFsActionWRITE:248->checkPrerequisites:311->checkIfConfigIsSet:319 » IllegalArgument
[ERROR] ITestAzureBlobFileSystemCheckAccess.testFsActionWRITEEXECUTE:278->checkPrerequisites:311->checkIfConfigIsSet:319 » IllegalArgument
[INFO]
[ERROR] Tests run: 457, Failures: 0, Errors: 8, Skipped: 64
[INFO] Results:
[INFO]
[WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 24

HNS-SharedKey

[INFO] Results:
[INFO]
[INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemCheckAccess.testCheckAccessForNonExistentFile » Unexp...
[ERROR] ITestAzureBlobFileSystemCheckAccess.testFsActionALL:293->checkPrerequisites:311->checkIfConfigIsSet:319 » IllegalArgument
[ERROR] ITestAzureBlobFileSystemCheckAccess.testFsActionEXECUTE:218->checkPrerequisites:311->checkIfConfigIsSet:319 » IllegalArgument
[ERROR] ITestAzureBlobFileSystemCheckAccess.testFsActionNONE:204->checkPrerequisites:311->checkIfConfigIsSet:319 » IllegalArgument
[ERROR] ITestAzureBlobFileSystemCheckAccess.testFsActionREAD:233->checkPrerequisites:311->checkIfConfigIsSet:319 » IllegalArgument
[ERROR] ITestAzureBlobFileSystemCheckAccess.testFsActionREADEXECUTE:263->checkPrerequisites:311->checkIfConfigIsSet:319 » IllegalArgument
[ERROR] ITestAzureBlobFileSystemCheckAccess.testFsActionWRITE:248->checkPrerequisites:311->checkIfConfigIsSet:319 » IllegalArgument
[ERROR] ITestAzureBlobFileSystemCheckAccess.testFsActionWRITEEXECUTE:278->checkPrerequisites:311->checkIfConfigIsSet:319 » IllegalArgument
[INFO]
[ERROR] Tests run: 457, Failures: 0, Errors: 8, Skipped: 24
[INFO] Results:
[INFO]
[WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 16

NonHNS-SharedKey

[INFO] Results:
[INFO]
[INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS:174->checkIfConfigIsSet:319 » IllegalArgument
[INFO]
[ERROR] Tests run: 457, Failures: 0, Errors: 1, Skipped: 245
[INFO] Results:
[INFO]
[WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 16

With checkaccess specific configs

HNS-OAuth

[INFO] Results:
[INFO]
[INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[WARNING] Tests run: 457, Failures: 0, Errors: 0, Skipped: 64
[INFO] Results:
[INFO]
[WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 24

HNS-SharedKey

[INFO] Results:
[INFO]
[INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[WARNING] Tests run: 457, Failures: 0, Errors: 0, Skipped: 24
[INFO] Results:
[INFO]
[WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 16

NonHNS-SharedKey

[INFO] Results:
[INFO]
[INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[WARNING] Tests run: 457, Failures: 0, Errors: 0, Skipped: 245
[INFO] Results:
[INFO]
[WARNING] Tests run: 207, Failures: 0, Errors: 0, Skipped: 16

@steveloughran steveloughran merged commit 51598d8 into apache:trunk Oct 1, 2020
asfgit pushed a commit that referenced this pull request Oct 1, 2020
@bilaharith @steveloughran
HADOOP-17183. ABFS: Enabling checkaccess on ABFS (#2331)
d80dfad 
Contributed by Bilahari TH

Change-Id: If4224697deed733d6db44145994cdd85547c27d1
@steveloughran
Copy link
Contributor

steveloughran commented Oct 1, 2020

+1, merged to 3.3 and trunk

@mehakmeet mehakmeet mentioned this pull request Jan 7, 2021
Merged
jojochuang pushed a commit to jojochuang/hadoop that referenced this pull request May 23, 2023
@bilaharith
CDPD-24203. HADOOP-17183. ABFS: Enabling checkaccess on ABFS (apache#…
21a0d29 
…2331)

Contributed by Bilahari TH

Change-Id: If4224697deed733d6db44145994cdd85547c27d1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@bilaharith @steveloughran