Skip to content

Conversation

@haiyang1987
Copy link
Contributor

Description of PR

HDFS-16879
For the block of the ec file run hdfs fsck -blockId xxx can add shows number of redundant internal block replicas.

For code changes:

for example: the current blockgroup has 10 live replicas, and it will show there are 9 live replicas.
actually, there is a live replica that should be in the redundant state, can add shows "No. of redundant Replica: 1"

hdfs fsck -blockId blk_-xxx

Block Id: blk_-xxx
Block belongs to: /ec/file1
No. of Expected Replica: 9
No. of live Replica: 9
No. of excess Replica: 0
No. of stale Replica: 0
No. of decommissioned Replica: 0
No. of decommissioning Replica: 0
No. of corrupted Replica: 0
Block replica on datanode/rack: ip-xxx1 is HEALTHY
Block replica on datanode/rack: ip-xxx2 is HEALTHY
Block replica on datanode/rack: ip-xxx3 is HEALTHY
Block replica on datanode/rack: ip-xxx4 is HEALTHY
Block replica on datanode/rack: ip-xxx5 is HEALTHY
Block replica on datanode/rack: ip-xxx6 is HEALTHY
Block replica on datanode/rack: ip-xxx7 is HEALTHY
Block replica on datanode/rack: ip-xxx8 is HEALTHY
Block replica on datanode/rack: ip-xxx9 is HEALTHY
Block replica on datanode/rack: ip-xxx10 is HEALTHY

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 47s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 40m 55s trunk passed
+1 💚 compile 1m 28s trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚 compile 1m 19s trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚 checkstyle 1m 7s trunk passed
+1 💚 mvnsite 1m 29s trunk passed
+1 💚 javadoc 1m 6s trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚 javadoc 1m 30s trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚 spotbugs 3m 31s trunk passed
+1 💚 shadedclient 25m 17s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 18s the patch passed
+1 💚 compile 1m 21s the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚 javac 1m 21s the patch passed
+1 💚 compile 1m 14s the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚 javac 1m 14s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 54s the patch passed
+1 💚 mvnsite 1m 21s the patch passed
+1 💚 javadoc 0m 52s the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚 javadoc 1m 25s the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚 spotbugs 3m 28s the patch passed
+1 💚 shadedclient 25m 7s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 383m 1s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
+1 💚 asflicense 0m 41s The patch does not generate ASF License warnings.
496m 32s
Reason Tests
Failed junit tests hadoop.hdfs.TestLeaseRecovery2
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5264/1/artifact/out/Dockerfile
GITHUB PR #5264
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux e4c1bbe0ebc5 4.15.0-197-generic #208-Ubuntu SMP Tue Nov 1 17:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 18a55dd
Default Java Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5264/1/testReport/
Max. process+thread count 2217 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5264/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@haiyang1987
Copy link
Contributor Author

Hi @ZanderXu @tomscut @tasanuma please help me review this pr when you are available, Thanks.

Copy link
Contributor

@dineshchitlangia dineshchitlangia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 @haiyang1987 thanks for the improvement.
Let us wait for at least one more +1

@haiyang1987
Copy link
Contributor Author

Get, Thanks @dineshchitlangia help review it.

Copy link
Contributor

@ZanderXu ZanderXu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @haiyang1987 for your report.

@ZanderXu ZanderXu merged commit 35ce60e into apache:trunk Jan 4, 2023
@ZanderXu
Copy link
Contributor

ZanderXu commented Jan 4, 2023

Merged, thanks @haiyang1987 for your report. Thanks @dineshchitlangia for your review.

@haiyang1987
Copy link
Contributor Author

Thanks @ZanderXu @dineshchitlangia help review and merged.

@ZanderXu
Copy link
Contributor

ZanderXu commented Jan 4, 2023

The failed UT hadoop.hdfs.TestLeaseRecovery2 is traced by HDFS-16853.

LiuGuH pushed a commit to LiuGuH/hadoop that referenced this pull request Mar 26, 2024
symious pushed a commit to symious/hadoop that referenced this pull request Nov 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants