mdadm命令详解及实验过程

mdadm命令详解及实验过程

⼀.概念

mdadm是multiple devices admin的简称，它是Linux下的⼀款标准的软件 RAID 管理⼯具，作者是Neil Brown

⼆.特点

mdadm能够诊断、监控和收集详细的阵列信息
mdadm是⼀个单独集成化的程序⽽不是⼀些分散程序的集合，因此对不同RAID管令有共通的语法
mdadm能够执⾏⼏乎所有的功能⽽不需要配置⽂件(也没有默认的配置⽂件）
1
2
3

三.作⽤ （引⽤）

在linux系统中⽬前以MD(Multiple Devices)虚拟块设备的⽅式实现软件RAID,利⽤多个底层的块设备虚拟出⼀个新的虚拟设备,并且利⽤
条带化(stripping)技术将数据块均匀分布到多个磁盘上来提⾼虚拟设备的读写性能,利⽤不同的数据冗祭算法来保护⽤户数据不会因为某个块设备的故障⽽完全丢失,⽽且还能在设备被替换后将丢失的数据恢复到新的设备上. 

⽬前MD⽀持linear,multipath,raid0(stripping),raid1(mirror),raid4,raid5,raid6,raid10等不同的冗余级别和级成⽅式,当然也能⽀持多个RAID
陈列的层叠组成raid1 0,raid5 1等类型的陈列。
1
2
3
4
5

四.实验

试题：建⽴4个⼤⼩为1G的磁盘，并将其中3个创建为raid5的阵列磁盘，1个为热备份磁盘。测试热备份磁盘替换阵列中的磁盘并同步数据。移除损坏的磁盘，添加⼀个新磁盘作为热备份磁盘。最后要求开机⾃动挂载。

4.1创建磁盘

[root@xiao ~]# fdisk /dev/sda
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): n
First cylinder (10486-13054, default 10486):
Using default value 10486
Last cylinder, +cylinders or +size{K,M,G} (10486-13054, default 13054): +1G
Command (m for help): n
First cylinder (10618-13054, default 10618):
Using default value 10618
Last cylinder, +cylinders or +size{K,M,G} (10618-13054, default 13054): +1G
Command (m for help): n
First cylinder (10750-13054, default 10750):
Using default value 10750
Last cylinder, +cylinders or +size{K,M,G} (10750-13054, default 13054): +1G
Command (m for help): n
First cylinder (10882-13054, default 10882):
Using default value 10882
Last cylinder, +cylinders or +size{K,M,G} (10882-13054, default 13054): +1G
Command (m for help): t
Partition number (1-8): 8
Hex code (type L to list codes): fd
Changed system type of partition 8 to fd (Linux raid autodetect)
Command (m for help): t
Partition number (1-8): 7
Hex code (type L to list codes): fd
Changed system type of partition 7 to fd (Linux raid autodetect)
Command (m for help): t
Partition number (1-8): 6
Hex code (type L to list codes): fd
Changed system type of partition 6 to fd (Linux raid autodetect)
Command (m for help): t
Partition number (1-8): 5
Hex code (type L to list codes): fd
Changed system type of partition 5 to fd (Linux raid autodetect)
Command (m for help): p
Disk /dev/sda: 107.4 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0008ed57
Device Boot Start End Blocks Id System
/dev/sda1 * 1 26 204800 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2 26 10225 81920000 83 Linux
/dev/sda3 10225 10486 2097152 82 Linux swap / Solaris
/dev/sda4 10486 13054 20633279 5 Extended
/dev/sda5 10486 10617 1058045 fd Linux raid autodetect
/dev/sda6 10618 10749 1060258+ fd Linux raid autodetect
/dev/sda7 10750 10881 1060258+ fd Linux raid autodetect
/dev/sda8 10882 11013 1060258+ fd Linux raid autodetect
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: 设备或资源忙.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

4.2加载内核

[root@xiao ~]# partx -a /dev/sda5 /dev/sda
[root@xiao ~]# partx -a /dev/sda6 /dev/sda
[root@xiao ~]# partx -a /dev/sda7 /dev/sda
[root@xiao ~]# partx -a /dev/sda8 /dev/sda
1
2
3
4

4.3创建raid5及其热备份盘

[root@xiao ~]# mdadm -C /dev/md0 -l 5 -n 3 -x 1 /dev/sda{5,6,7,8}
mdadm: /dev/sda5 appears to be part of a raid array:
level=raid5 devices=3 ctime=Wed Dec 17 00:58:24 2014
mdadm: /dev/sda6 appears to be part of a raid array:
level=raid5 devices=3 ctime=Wed Dec 17 00:58:24 2014
mdadm: /dev/sda7 appears to be part of a raid array:
level=raid5 devices=3 ctime=Wed Dec 17 00:58:24 2014
mdadm: /dev/sda8 appears to be part of a raid array:
level=raid5 devices=3 ctime=Wed Dec 17 00:58:24 2014
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
1
2
3
4
5
6
7
8
9
10
11
12

4.4初始化时间和磁盘阵列的读写的应⽤相关

使⽤cat /proc/mdstat信息查询RAID阵列当前重构的速度和预期的完成时间。

[root@xiao ~]# cat /proc/mdstat
Personalities : [raid1] [raid0] [raid6] [raid5] [raid4]
md0 : active raid5 sda7[4] sda8[3](S) sda6[1] sda5[0]
2113536 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [UU_]
[=========>...........] recovery = 45.5% (482048/1056768) finish=0.3min speed=30128K/sec
unused devices: <none>
[root@xiao ~]# cat /proc/mdstat
Personalities : [raid1] [raid0] [raid6] [raid5] [raid4]
md0 : active raid5 sda7[4] sda8[3](S) sda6[1] sda5[0]
2113536 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]
unused devices: <none>
[root@xiao ~]# mke2fs -t ext3 /dev/md0 //格式化raid
1
2
3
4
5
6
7
8
9
10
11
12

4.5挂载raid到/mnt⽬录下，并查看是否正常（显⽰lost+found为正常）

[root@xiao ~]# mount /dev/md0 /mnt
[root@xiao ~]# ls /mnt
lost+found
1
2
3

4.6查看raid阵列的详细信息

[root@xiao ~]# mdadm -D /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Wed Dec 17 03:38:08 2014
Raid Level : raid5
Array Size : 2113536 (2.02 GiB 2.16 GB)
Used Dev Size : 1056768 (1032.17 MiB 1082.13 MB)
Raid Devices : 3
Total Devices : 4
Persistence : Superblock is persistent
Update Time : Wed Dec 17 03:55:11 2014
State : clean
Active Devices : 3
Working Devices : 4
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 512K
Name : xiao:0 (local to host xiao)
UUID : bce110f2:34f3fbf1:8de472ed:633a374f
Events : 18
Number Major Minor RaidDevice State
0 8 5 0 active sync /dev/sda5
1 8 6 1 active sync /dev/sda6
4 8 7 2 active sync /dev/sda7
3 8 8 - spare /dev/sda8
4.7模拟损坏其中的⼀个磁盘，这⾥我选择 /dev/sda6磁盘
[root@xiao ~]# mdadm /dev/md0 --fail /dev/sda6
mdadm: set /dev/sda6 faulty in /dev/md0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

4.7查看raid阵列详细信息，发现/dev/sda8⾃动替换了损坏的/dev/sda6磁盘。

[root@xiao ~]# mdadm -D /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Wed Dec 17 03:38:08 2014
Raid Level : raid5
Array Size : 2113536 (2.02 GiB 2.16 GB)
Used Dev Size : 1056768 (1032.17 MiB 1082.13 MB)
Raid Devices : 3
Total Devices : 4
Persistence : Superblock is persistent
Update Time : Wed Dec 17 04:13:59 2014
State : clean, degraded, recovering
Active Devices : 2
Working Devices : 3
Failed Devices : 1
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 512K
Rebuild Status : 43% complete
Name : xiao:0 (local to host xiao)
UUID : bce110f2:34f3fbf1:8de472ed:633a374f
Events : 26
Number Major Minor RaidDevice State
0 8 5 0 active sync /dev/sda5
3 8 8 1 spare rebuilding /dev/sda8
4 8 7 2 active sync /dev/sda7
1 8 6 - faulty /dev/sda6
[root@xiao ~]# cat /proc/mdstat
Personalities : [raid1] [raid0] [raid6] [raid5] [raid4]
md0 : active raid5 sda7[4] sda8[3] sda6[1](F) sda5[0]
2113536 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] #正常的情况会是[UUU],若第⼀个磁盘损坏则显⽰[ _UU ].
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

4.8 移除损坏的硬盘

[root@xiao ~]# mdadm /dev/md0 -r /dev/sda6
mdadm: hot removed /dev/sda6 from /dev/md0
1
2

4.9添加⼀个新硬盘作为热备份盘

[root@xiao ~]# fdisk /dev/sda
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): n
First cylinder (11014-13054, default 11014):
Using default value 11014
Last cylinder, +cylinders or +size{K,M,G} (11014-13054, default 13054): +1G
Command (m for help): t
Partition number (1-9): 9
Hex code (type L to list codes): fd
Changed system type of partition 9 to fd (Linux raid autodetect)
Command (m for help): p
Disk /dev/sda: 107.4 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0008ed57
Device Boot Start End Blocks Id System
/dev/sda1 * 1 26 204800 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2 26 10225 81920000 83 Linux
/dev/sda3 10225 10486 2097152 82 Linux swap / Solaris
/dev/sda4 10486 13054 20633279 5 Extended
/dev/sda5 10486 10617 1058045 fd Linux raid autodetect
/dev/sda6 10618 10749 1060258+ fd Linux raid autodetect
/dev/sda7 10750 10881 1060258+ fd Linux raid autodetect
/dev/sda8 10882 11013 1060258+ fd Linux raid autodetect
/dev/sda9 11014 11145 1060258+ fd Linux raid autodetect
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: 设备或资源忙.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
[root@xiao ~]# partx -a /dev/sda9 /dev/sda
[root@xiao ~]# mdadm /dev/md0 --add /dev/sda9
mdadm: added /dev/sda9
[root@xiao ~]# mdadm -D /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Wed Dec 17 03:38:08 2014
Raid Level : raid5
Array Size : 2113536 (2.02 GiB 2.16 GB)
Used Dev Size : 1056768 (1032.17 MiB 1082.13 MB)
Raid Devices : 3
Total Devices : 4
Persistence : Superblock is persistent
Update Time : Wed Dec 17 04:39:35 2014
State : clean
Active Devices : 3
Working Devices : 4
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 512K
Name : xiao:0 (local to host xiao)
UUID : bce110f2:34f3fbf1:8de472ed:633a374f
Events : 41
Number Major Minor RaidDevice State
0 8 5 0 active sync /dev/sda5
3 8 8 1 active sync /dev/sda8
4 8 7 2 active sync /dev/sda7
5 8 9 - spare /dev/sda9
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

5.开机⾃动挂载

编辑/etc/fsab⽂件
/dev/md0 /mnt ext3 defaults 0 0
:wq
1
2
3

6.mdadm中⽂man(引⽤)

基本语法 ： mdadm [mode] [options]
[mode] 有7种：
Assemble：将以前定义的某个阵列加⼊当前在⽤阵列。
Build：Build a legacy array ，每个device 没有 superblocks
Create：创建⼀个新的阵列，每个device 具有 superblocks
Manage： 管理阵列，⽐如 add 或 remove
Misc：允许单独对阵列中的某个 device 做操作，⽐如抹去superblocks 或 终⽌在⽤的阵列。
Follow or Monitor:监控 raid 1,4,5,6 和 multipath 的状态
Grow：改变raid 容量或 阵列中的 device 数⽬
可⽤的 [options]:
-A, --assemble：加⼊⼀个以前定义的阵列
-B, --build：Build a legacy array without superblocks.
-C, --create：创建⼀个新的阵列
-Q, --query：查看⼀个device，判断它为⼀个 md device 或是 ⼀个 md 阵列的⼀部分
-D, --detail：打印⼀个或多个 md device 的详细信息
-E, --examine：打印 device 上的 md superblock 的内容
-F, --follow, --monitor：选择 Monitor 模式
-G, --grow：改变在⽤阵列的⼤⼩或形态
-h, --help：帮助信息，⽤在以上选项后，则显⽰该选项信息
--help-options
-V, --version
-v, --verbose：显⽰细节
-b, --brief：较少的细节。⽤于 --detail 和 --examine 选项
-f, --force
-c, --config= ：指定配置⽂件，缺省为 /etc/mdadm/mdadm.conf
-s, --scan：扫描配置⽂件或 /proc/mdstat以搜寻丢失的信息。配置⽂件/etc/mdadm/mdadm.conf
create 或 build 使⽤的选项:
-c, --chunk=:Specify chunk size of kibibytes. 缺省为 64.
--rounding=: Specify rounding factor for linear array (==chunk size)
-l, --level=:设定 raid level.
--create可⽤:linear, raid0, 0, stripe, raid1,1, mirror, raid4, 4, raid5, 5, raid6, 6, multipath, mp.
--build可⽤：linear, raid0, 0, stripe.
-p, --parity=：设定 raid5 的奇偶校验规则:eft-asymmetric, left-symmetric, right-asymmetric, right-symmetric, la, ra, ls, rs.缺省为left-symmetric
--layout=:类似于--parity
-n, --raid-devices=:指定阵列中可⽤ device 数⽬，这个数⽬只能由 --grow 修改
-x, --spare-devices=：指定初始阵列的富余device 数⽬
-z, --size=：组建RAID1/4/5/6后从每个device获取的空间总数
--assume-clean:⽬前仅⽤于 --build 选项
-R, --run:阵列中的某⼀部分出现在其他阵列或⽂件系统中时，mdadm会确认该阵列。此选项将不作确认。
-f, --force:通常mdadm不允许只⽤⼀个device 创建阵列，⽽且创建raid5时会使⽤⼀个device作为missing drive。此选项正相反。
-a, --auto{=no,yes,md,mdp,part,p}{NN}:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41