HDFS文件系统检查工具fsck

fsck

hdfsdatanodes

fsck是一个HDFS文件系统检查工具。

使用用法

hadoop fsck [[GENERIC_OPTIONS]] [-move | -delete | -openforwrite] [-files [-blocks [-locations | -racks]]]

• ：检查的起始目录。

• -move：移动受损文件到/lost+found

• -delete：删除受损文件。

• -files：打印出正被检查的文件。

• -blocks： 打印出块信息报告。

• -locations：打印出每个块的位置信息。

• -racks：打印出data-node的网络拓扑结构。

使用案例

[zhangsan@node1 subdir0]$ hdfs fsck /user/zhangsan/hadoop-2.7.3.tar.gz -files -blocks -locations
Connecting to namenode via http://node1:50070/fsck?ugi=zhangsan&files=1&blocks=1&locations=1&path=%2Fuser%2Fzhangsan%2Fhadoop-2.7.3.tar.gz
FSCK started by zhangsan (auth:SIMPLE) from /192.168.179.101 for path /user/zhangsan/hadoop-2.7.3.tar.gz at Mon Apr 04 09:33:56 CST 2022
# 下面可以看到 文件、文件大小204.17MB、block数量、状态
/user/zhangsan/hadoop-2.7.3.tar.gz 214092195 bytes, 2 block(s):  OK
# BP：Block Pool；
0. BP-792177474-192.168.179.101-1649033535606:blk_1073741825_1001 
# len：block长度134217728 Byte, 即128MB； 
len=134217728
# repl：replication = 3, 即副本数为3
repl=3
# 副本存放详细信息
[DatanodeInfoWithStorage[192.168.179.103:50010,DS-dfb7c7fc-8820-454b-a9d9-c84a257b1558,DISK], 
DatanodeInfoWithStorage[192.168.179.102:50010,DS-7da4de31-9108-4c2b-95ae-f139922b7869,DISK],
DatanodeInfoWithStorage[192.168.179.101:50010,DS-e542e372-6db3-4c2f-90aa-0520df3e7066,DISK]] 
# 第二个block
1. BP-792177474-192.168.179.101-1649033535606:blk_1073741826_1002 
# 79874467(第二个数据块中的数据大小) = 214092195(文件总大小) - 134217728(第一个数据块中的数据大小)
len=79874467 
repl=3 
[DatanodeInfoWithStorage[192.168.179.102:50010,DS-7da4de31-9108-4c2b-95ae-f139922b7869,DISK], 
DatanodeInfoWithStorage[192.168.179.103:50010,DS-dfb7c7fc-8820-454b-a9d9-c84a257b1558,DISK], 
DatanodeInfoWithStorage[192.168.179.101:50010,DS-e542e372-6db3-4c2f-90aa-0520df3e7066,DISK]]

Status: HEALTHY
 Total size:	214092195 B
 Total dirs:	0
 Total files:	1
 Total symlinks:		0
 Total blocks (validated):	2 (avg. block size 107046097 B)
 Minimally replicated blocks:	2 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	0 (0.0 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	3.0
 Corrupt blocks:		0
 Missing replicas:		0 (0.0 %)
 Number of data-nodes:		3
 Number of racks:		1
FSCK ended at Mon Apr 04 09:33:56 CST 2022 in 1 milliseconds

The filesystem under path '/user/zhangsan/hadoop-2.7.3.tar.gz' is HEALTHY

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

https://hadoop.apache.org/docs/r1.0.4/cn/commands_manual.html#fsck