Cheap VPS & Xen Server


Residential Proxy Network - Hourly & Monthly Packages

How To Set Up Software RAID1 On A Running LVM System (Incl. GRUB Configuration) (CentOS 5.3)


This guide explains how to set up software RAID1 on an already running LVM system (CentOS 5.3). The GRUB bootloader will be configured in such a way that the system will still be able to boot if one of the hard drives fails (no matter which one).

I do not issue any guarantee that this will work for you!

 

1 Preliminary Note

In this tutorial I’m using a CentOS 5.3 system with two hard drives, /dev/sda and /dev/sdb which are identical in size. /dev/sdb is currently unused, and /dev/sda has the following partitions (this is the default CentOS partitioning scheme – you should find something similar on your system unless you chose to manually partition during the installation of the system):

  • /dev/sda1: /boot partition, ext3;
  • /dev/sda2: is used for LVM (volume group VolGroup00) and contains / (volume LogVol00) and swap (volume LogVol01).

In the end I want to have the following situation:

  • /dev/md0 (made up of /dev/sda1 and /dev/sdb1): /boot partition, ext3;
  • /dev/md1 (made up of /dev/sda2 and /dev/sdb2): LVM (volume group VolGroup00), contains / (volume LogVol00) and swap (volume LogVol01).

This is the current situation:

df -h

[root@server1 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
8.6G  1.4G  6.8G  17% /
/dev/sda1              99M   13M   82M  14% /boot
tmpfs                 250M     0  250M   0% /dev/shm
[root@server1 ~]#

fdisk -l

[root@server1 ~]# fdisk -l

Disk /dev/sda: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14        1305    10377990   8e  Linux LVM

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdb doesn’t contain a valid partition table
[root@server1 ~]#

pvdisplay

[root@server1 ~]# pvdisplay
/dev/hdc: open failed: No medium found
— Physical volume —
PV Name               /dev/sda2
VG Name               VolGroup00
PV Size               9.90 GB / not usable 22.76 MB
Allocatable           yes (but full)
PE Size (KByte)       32768
Total PE              316
Free PE               0
Allocated PE          316
PV UUID               aikFEP-FB15-nB9C-Nfq0-eGMG-hQid-GOsDuj

[root@server1 ~]#

vgdisplay

[root@server1 ~]# vgdisplay
/dev/hdc: open failed: No medium found
— Volume group —
VG Name               VolGroup00
System ID
Format                lvm2
Metadata Areas        1
Metadata Sequence No  3
VG Access             read/write
VG Status             resizable
MAX LV                0
Cur LV                2
Open LV               2
Max PV                0
Cur PV                1
Act PV                1
VG Size               9.88 GB
PE Size               32.00 MB
Total PE              316
Alloc PE / Size       316 / 9.88 GB
Free  PE / Size       0 / 0
VG UUID               ZPvC10-cN09-fI0S-Vc8l-vOuZ-wM6F-tlz0Mj

[root@server1 ~]#

lvdisplay

[root@server1 ~]# lvdisplay
/dev/hdc: open failed: No medium found
— Logical volume —
LV Name                /dev/VolGroup00/LogVol00
VG Name                VolGroup00
LV UUID                vYlky0-Ymx4-PNeK-FTpk-qxvm-PmoZ-3vcNTd
LV Write Access        read/write
LV Status              available
# open                 1
LV Size                8.88 GB
Current LE             284
Segments               1
Allocation             inherit
Read ahead sectors     auto
– currently set to     256
Block device           253:0

— Logical volume —
LV Name                /dev/VolGroup00/LogVol01
VG Name                VolGroup00
LV UUID                Ml9MMN-DcOA-Lb6V-kWPU-h6IK-P0ww-Gp9vd2
LV Write Access        read/write
LV Status              available
# open                 1
LV Size                1.00 GB
Current LE             32
Segments               1
Allocation             inherit
Read ahead sectors     auto
– currently set to     256
Block device           253:1

[root@server1 ~]#

 

2 Installing mdadm

The most important tool for setting up RAID is mdadm. Let’s install it like this:

yum install mkinitrd mdadm

Afterwards, we load a few kernel modules (to avoid a reboot):

modprobe linear
modprobe multipath
modprobe raid0
modprobe raid1
modprobe raid5
modprobe raid6
modprobe raid10

Now run

cat /proc/mdstat

The output should look as follows:

[root@server1 ~]# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
unused devices: <none>
[root@server1 ~]#

 

3 Preparing /dev/sdb

To create a RAID1 array on our already running system, we must prepare the /dev/sdb hard drive for RAID1, then copy the contents of our /dev/sda hard drive to it, and finally add /dev/sda to the RAID1 array.

First, we copy the partition table from /dev/sda to /dev/sdb so that both disks have exactly the same layout:

sfdisk -d /dev/sda | sfdisk /dev/sdb

The output should be as follows:

[root@server1 ~]# sfdisk -d /dev/sda | sfdisk /dev/sdb
Checking that no-one is using this disk right now …
OK

Disk /dev/sdb: 1305 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
/dev/sdb: unrecognized partition table type
Old situation:
No partitions found
New situation:
Units = sectors of 512 bytes, counting from 0

Device Boot    Start       End   #sectors  Id  System
/dev/sdb1   *        63    208844     208782  83  Linux
/dev/sdb2        208845  20964824   20755980  8e  Linux LVM
/dev/sdb3             0         –          0   0  Empty
/dev/sdb4             0         –          0   0  Empty
Successfully wrote the new partition table

Re-reading the partition table …

If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
[root@server1 ~]#

The command

fdisk -l

should now show that both HDDs have the same layout:

[root@server1 ~]# fdisk -l

Disk /dev/sda: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14        1305    10377990   8e  Linux LVM

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1          13      104391   83  Linux
/dev/sdb2              14        1305    10377990   8e  Linux LVM
[root@server1 ~]#

Next we must change the partition type of our two partitions on /dev/sdb to Linux raid autodetect:

fdisk /dev/sdb

[root@server1 ~]# fdisk /dev/sdb

The number of cylinders for this disk is set to 1305.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): <– m
Command action
a   toggle a bootable flag
b   edit bsd disklabel
c   toggle the dos compatibility flag
d   delete a partition
l   list known partition types
m   print this menu
n   add a new partition
o   create a new empty DOS partition table
p   print the partition table
q   quit without saving changes
s   create a new empty Sun disklabel
t   change a partition’s system id
u   change display/entry units
v   verify the partition table
w   write table to disk and exit
x   extra functionality (experts only)

Command (m for help): <– t
Partition number (1-4): <– 1
Hex code (type L to list codes): <– L

0  Empty           1e  Hidden W95 FAT1 80  Old Minix       bf  Solaris
1  FAT12           24  NEC DOS         81  Minix / old Lin c1  DRDOS/sec (FAT-
2  XENIX root      39  Plan 9          82  Linux swap / So c4  DRDOS/sec (FAT-
3  XENIX usr       3c  PartitionMagic  83  Linux           c6  DRDOS/sec (FAT-
4  FAT16 <32M      40  Venix 80286     84  OS/2 hidden C:  c7  Syrinx
5  Extended        41  PPC PReP Boot   85  Linux extended  da  Non-FS data
6  FAT16           42  SFS             86  NTFS volume set db  CP/M / CTOS / .
7  HPFS/NTFS       4d  QNX4.x          87  NTFS volume set de  Dell Utility
8  AIX             4e  QNX4.x 2nd part 88  Linux plaintext df  BootIt
9  AIX bootable    4f  QNX4.x 3rd part 8e  Linux LVM       e1  DOS access
a  OS/2 Boot Manag 50  OnTrack DM      93  Amoeba          e3  DOS R/O
b  W95 FAT32       51  OnTrack DM6 Aux 94  Amoeba BBT      e4  SpeedStor
c  W95 FAT32 (LBA) 52  CP/M            9f  BSD/OS          eb  BeOS fs
e  W95 FAT16 (LBA) 53  OnTrack DM6 Aux a0  IBM Thinkpad hi ee  EFI GPT
f  W95 Ext’d (LBA) 54  OnTrackDM6      a5  FreeBSD         ef  EFI (FAT-12/16/
10  OPUS            55  EZ-Drive        a6  OpenBSD         f0  Linux/PA-RISC b
11  Hidden FAT12    56  Golden Bow      a7  NeXTSTEP        f1  SpeedStor
12  Compaq diagnost 5c  Priam Edisk     a8  Darwin UFS      f4  SpeedStor
14  Hidden FAT16 <3 61  SpeedStor       a9  NetBSD          f2  DOS secondary
16  Hidden FAT16    63  GNU HURD or Sys ab  Darwin boot     fb  VMware VMFS
17  Hidden HPFS/NTF 64  Novell Netware  b7  BSDI fs         fc  VMware VMKCORE
18  AST SmartSleep  65  Novell Netware  b8  BSDI swap       fd  Linux raid auto
1b  Hidden W95 FAT3 70  DiskSecure Mult bb  Boot Wizard hid fe  LANstep
1c  Hidden W95 FAT3 75  PC/IX           be  Solaris boot    ff  BBT
Hex code (type L to list codes):
 <– fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): <– t
Partition number (1-4): <– 2
Hex code (type L to list codes): <– fd
Changed system type of partition 2 to fd (Linux raid autodetect)

Command (m for help): <– w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@server1 ~]#

The command

fdisk -l

should now show that /dev/sdb1 and /dev/sdb2 are of the type Linux raid autodetect:

[root@server1 ~]# fdisk -l

Disk /dev/sda: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14        1305    10377990   8e  Linux LVM

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1          13      104391   fd  Linux raid autodetect
/dev/sdb2              14        1305    10377990   fd  Linux raid autodetect
[root@server1 ~]#

To make sure that there are no remains from previous RAID installations on /dev/sdb, we run the following commands:

mdadm –zero-superblock /dev/sdb1
mdadm –zero-superblock /dev/sdb2

If there are no remains from previous RAID installations, each of the above commands will throw an error like this one (which is nothing to worry about):

[root@server1 ~]# mdadm –zero-superblock /dev/sdb1
mdadm: Unrecognised md component device – /dev/sdb1
[root@server1 ~]#

Otherwise the commands will not display anything at all.

4 Creating Our RAID Arrays

Now let’s create our RAID arrays /dev/md0 and /dev/md1. /dev/sdb1 will be added to /dev/md0 and/dev/sdb2 to /dev/md1. /dev/sda1 and /dev/sda2 can’t be added right now (because the system is currently running on them), therefore we use the placeholder missing in the following two commands:

mdadm –create /dev/md0 –level=1 –raid-disks=2 missing /dev/sdb1
mdadm –create /dev/md1 –level=1 –raid-disks=2 missing /dev/sdb2

The command

cat /proc/mdstat

should now show that you have two degraded RAID arrays ([_U] or [U_] means that an array is degraded while [UU] means that the array is ok):

[root@server1 ~]# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sdb2[1]
10377920 blocks [2/1] [_U]

md0 : active raid1 sdb1[1]
104320 blocks [2/1] [_U]

unused devices: <none>
[root@server1 ~]#

Next we create a filesystem (ext3) on our non-LVM RAID array /dev/md0:

mkfs.ext3 /dev/md0

Now we come to our LVM RAID array /dev/md1. To prepare it for LVM, we run:

pvcreate /dev/md1

Then we add /dev/md1 to our volume group VolGroup00:

vgextend VolGroup00 /dev/md1

The output of

pvdisplay

should now be similar to this:

[root@server1 ~]# pvdisplay
— Physical volume —
PV Name               /dev/sda2
VG Name               VolGroup00
PV Size               9.90 GB / not usable 22.76 MB
Allocatable           yes (but full)
PE Size (KByte)       32768
Total PE              316
Free PE               0
Allocated PE          316
PV UUID               aikFEP-FB15-nB9C-Nfq0-eGMG-hQid-GOsDuj

— Physical volume —
PV Name               /dev/md1
VG Name               VolGroup00
PV Size               9.90 GB / not usable 22.69 MB
Allocatable           yes
PE Size (KByte)       32768
Total PE              316
Free PE               316
Allocated PE          0
PV UUID               u6IZfM-5Zj8-kFaG-YN8K-kjAd-3Kfv-0oYk7J

[root@server1 ~]#

The output of

vgdisplay

should be as follows:

[root@server1 ~]# vgdisplay
— Volume group —
VG Name               VolGroup00
System ID
Format                lvm2
Metadata Areas        2
Metadata Sequence No  4
VG Access             read/write
VG Status             resizable
MAX LV                0
Cur LV                2
Open LV               2
Max PV                0
Cur PV                2
Act PV                2
VG Size               19.75 GB
PE Size               32.00 MB
Total PE              632
Alloc PE / Size       316 / 9.88 GB
Free  PE / Size       316 / 9.88 GB
VG UUID               ZPvC10-cN09-fI0S-Vc8l-vOuZ-wM6F-tlz0Mj

[root@server1 ~]#

Next we create /etc/mdadm.conf as follows:

mdadm –examine –scan > /etc/mdadm.conf

Display the contents of the file:

cat /etc/mdadm.conf

In the file you should now see details about our two (degraded) RAID arrays:

ARRAY /dev/md0 level=raid1 num-devices=2 UUID=0a96be0f:bf0f4631:a910285b:0f337164
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=f9e691e2:8d25d314:40f42444:7dbe1da1

Next we modify /etc/fstab. Replace LABEL=/boot with /dev/md0 so that the file looks as follows:

vi /etc/fstab

/dev/VolGroup00/LogVol00 /                       ext3    defaults        1 1
/dev/md0             /boot                   ext3    defaults        1 2
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
/dev/VolGroup00/LogVol01 swap                    swap    defaults        0 0

Next replace /dev/sda1 with /dev/md0 in /etc/mtab:

vi /etc/mtab

/dev/mapper/VolGroup00-LogVol00 / ext3 rw 0 0
proc /proc proc rw 0 0
sysfs /sys sysfs rw 0 0
devpts /dev/pts devpts rw,gid=5,mode=620 0 0
/dev/md0 /boot ext3 rw 0 0
tmpfs /dev/shm tmpfs rw 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0
sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw 0 0

Now up to the GRUB boot loader. Open /boot/grub/menu.lst and add fallback=1 right after default=0:

vi /boot/grub/menu.lst

[...]
default=0
fallback=1
[...]

This makes that if the first kernel (counting starts with 0, so the first kernel is 0) fails to boot, kernel #2 will be booted.

In the same file, go to the bottom where you should find some kernel stanzas. Copy the first of them and paste the stanza before the first existing stanza; replace root (hd0,0) with root (hd1,0):

[...]
title CentOS (2.6.18-128.el5)
        root (hd1,0)
        kernel /vmlinuz-2.6.18-128.el5 ro root=/dev/VolGroup00/LogVol00
        initrd /initrd-2.6.18-128.el5.img
title CentOS (2.6.18-128.el5)
        root (hd0,0)
        kernel /vmlinuz-2.6.18-128.el5 ro root=/dev/VolGroup00/LogVol00
        initrd /initrd-2.6.18-128.el5.img

The whole file should look something like this:

# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/VolGroup00/LogVol00
#          initrd /initrd-version.img
#boot=/dev/sda
default=0
fallback=1
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.18-128.el5)
        root (hd1,0)
        kernel /vmlinuz-2.6.18-128.el5 ro root=/dev/VolGroup00/LogVol00
        initrd /initrd-2.6.18-128.el5.img
title CentOS (2.6.18-128.el5)
        root (hd0,0)
        kernel /vmlinuz-2.6.18-128.el5 ro root=/dev/VolGroup00/LogVol00
        initrd /initrd-2.6.18-128.el5.img

root (hd1,0) refers to /dev/sdb which is already part of our RAID arrays. We will reboot the system in a few moments; the system will then try to boot from our (still degraded) RAID arrays; if it fails, it will boot from /dev/sda (-> fallback=1).

Next we adjust our ramdisk to the new situation:

mv /boot/initrd-`uname -r`.img /boot/initrd-`uname -r`.img_orig
mkinitrd /boot/initrd-`uname -r`.img `uname -r`

 

5 Moving Our Data To The RAID Arrays

Now that we’ve modified all configuration files, we can copy the contents of /dev/sda to /dev/sdb (including the configuration changes we’ve made in the previous chapter).

To move the contents of our LVM partition /dev/sda2 to our LVM RAID array /dev/md1, we use the pvmove command:

pvmove /dev/sda2 /dev/md1

This can take some time, so please be patient.

Afterwards, we remove /dev/sda2 from the volume group VolGroup00

vgreduce VolGroup00 /dev/sda2

… and tell the system to not use /dev/sda2 anymore for LVM:

pvremove /dev/sda2

The output of

pvdisplay

should now be as follows:

[root@server1 ~]# pvdisplay
— Physical volume —
PV Name               /dev/md1
VG Name               VolGroup00
PV Size               9.90 GB / not usable 22.69 MB
Allocatable           yes (but full)
PE Size (KByte)       32768
Total PE              316
Free PE               0
Allocated PE          316
PV UUID               u6IZfM-5Zj8-kFaG-YN8K-kjAd-3Kfv-0oYk7J

[root@server1 ~]#

Next we change the partition type of /dev/sda2 to Linux raid autodetect and add /dev/sda2 to the /dev/md1 array:

fdisk /dev/sda

[root@server1 ~]# fdisk /dev/sda

The number of cylinders for this disk is set to 1305.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): <– t
Partition number (1-4): <– 2
Hex code (type L to list codes): <– fd
Changed system type of partition 2 to fd (Linux raid autodetect)

Command (m for help): <– w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
[root@server1 ~]#

mdadm –add /dev/md1 /dev/sda2

Now take a look at

cat /proc/mdstat

… and you should see that the RAID array /dev/md1 is being synchronized:

[root@server1 ~]# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sda2[2] sdb2[1]
10377920 blocks [2/1] [_U]
[====>…………….]  recovery = 23.4% (2436544/10377920) finish=2.0min speed=64332K/sec

md0 : active raid1 sdb1[1]
104320 blocks [2/1] [_U]

unused devices: <none>
[root@server1 ~]#

(You can run

watch cat /proc/mdstat

to get an ongoing output of the process. To leave watch, press CTRL+C.)

Wait until the synchronization has finished (the output should then look like this:

[root@server1 ~]# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sda2[0] sdb2[1]
10377920 blocks [2/2] [UU]

md0 : active raid1 sdb1[1]
104320 blocks [2/1] [_U]

unused devices: <none>
[root@server1 ~]#

).

Now let’s mount /dev/md0:

mkdir /mnt/md0

mount /dev/md0 /mnt/md0

You should now find the array in the output of

mount

[root@server1 ~]# mount
/dev/mapper/VolGroup00-LogVol00 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/md0 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
/dev/md0 on /mnt/md0 type ext3 (rw)
[root@server1 ~]#

Now we copy the contents of /dev/sda1 to /dev/md0 (which is mounted on /mnt/md0):

cd /boot
cp -dpRx . /mnt/md0

6 Preparing GRUB

Afterwards we must install the GRUB bootloader on the second hard drive /dev/sdb:

grub

On the GRUB shell, type in the following commands:

root (hd0,0)

grub> root (hd0,0)
root (hd0,0)
Filesystem type is ext2fs, partition type 0x83
grub>

setup (hd0)

grub> setup (hd0)
setup (hd0)
Checking if “/boot/grub/stage1” exists… no
Checking if “/grub/stage1” exists… yes
Checking if “/grub/stage2” exists… yes
Checking if “/grub/e2fs_stage1_5” exists… yes
Running “embed /grub/e2fs_stage1_5 (hd0)”…  15 sectors are embedded.
succeeded
Running “install /grub/stage1 (hd0) (hd0)1+15 p (hd0,0)/grub/stage2 /grub/grub.conf”… succeeded
Done.
grub>

root (hd1,0)

grub> root (hd1,0)
root (hd1,0)
Filesystem type is ext2fs, partition type 0xfd
grub>

setup (hd1)

grub> setup (hd1)
setup (hd1)
Checking if “/boot/grub/stage1” exists… no
Checking if “/grub/stage1” exists… yes
Checking if “/grub/stage2” exists… yes
Checking if “/grub/e2fs_stage1_5” exists… yes
Running “embed /grub/e2fs_stage1_5 (hd1)”…  15 sectors are embedded.
succeeded
Running “install /grub/stage1 (hd1) (hd1)1+15 p (hd1,0)/grub/stage2 /grub/grub.conf”… succeeded
Done.
grub>

quit

Now, back on the normal shell, we reboot the system and hope that it boots ok from our RAID arrays:

reboot

 

7 Preparing /dev/sda

If all goes well, you should now find /dev/md0 in the output of

df -h

[root@server1 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
8.6G  1.4G  6.8G  18% /
/dev/md0               99M   16M   79M  17% /boot
tmpfs                 250M     0  250M   0% /dev/shm
[root@server1 ~]#

The output of

cat /proc/mdstat

should be as follows:

[root@server1 ~]# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdb1[1]
104320 blocks [2/1] [_U]

md1 : active raid1 sdb2[1] sda2[0]
10377920 blocks [2/2] [UU]

unused devices: <none>
[root@server1 ~]#

The outputs of pvdisplay, vgdisplay, and lvdisplay should be as follows:

pvdisplay

[root@server1 ~]# pvdisplay
— Physical volume —
PV Name               /dev/md1
VG Name               VolGroup00
PV Size               9.90 GB / not usable 22.69 MB
Allocatable           yes (but full)
PE Size (KByte)       32768
Total PE              316
Free PE               0
Allocated PE          316
PV UUID               u6IZfM-5Zj8-kFaG-YN8K-kjAd-3Kfv-0oYk7J

[root@server1 ~]#

vgdisplay

[root@server1 ~]# vgdisplay
— Volume group —
VG Name               VolGroup00
System ID
Format                lvm2
Metadata Areas        1
Metadata Sequence No  9
VG Access             read/write
VG Status             resizable
MAX LV                0
Cur LV                2
Open LV               2
Max PV                0
Cur PV                1
Act PV                1
VG Size               9.88 GB
PE Size               32.00 MB
Total PE              316
Alloc PE / Size       316 / 9.88 GB
Free  PE / Size       0 / 0
VG UUID               ZPvC10-cN09-fI0S-Vc8l-vOuZ-wM6F-tlz0Mj

[root@server1 ~]#

lvdisplay

[root@server1 ~]# lvdisplay
— Logical volume —
LV Name                /dev/VolGroup00/LogVol00
VG Name                VolGroup00
LV UUID                vYlky0-Ymx4-PNeK-FTpk-qxvm-PmoZ-3vcNTd
LV Write Access        read/write
LV Status              available
# open                 1
LV Size                8.88 GB
Current LE             284
Segments               1
Allocation             inherit
Read ahead sectors     auto
– currently set to     256
Block device           253:0

— Logical volume —
LV Name                /dev/VolGroup00/LogVol01
VG Name                VolGroup00
LV UUID                Ml9MMN-DcOA-Lb6V-kWPU-h6IK-P0ww-Gp9vd2
LV Write Access        read/write
LV Status              available
# open                 1
LV Size                1.00 GB
Current LE             32
Segments               1
Allocation             inherit
Read ahead sectors     auto
– currently set to     256
Block device           253:1

[root@server1 ~]#

Now we must change the partition type of /dev/sda1 to Linux raid autodetect as well:

fdisk /dev/sda

[root@server1 ~]# fdisk /dev/sda

The number of cylinders for this disk is set to 1305.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): <– t
Partition number (1-4): <– 1
Hex code (type L to list codes): <– fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): <– w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
[root@server1 ~]#

Now we can add /dev/sda1 to the /dev/md0 RAID array:

mdadm –add /dev/md0 /dev/sda1

Now take a look at

cat /proc/mdstat

[root@server1 ~]# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sda1[0] sdb1[1]
104320 blocks [2/2] [UU]

md1 : active raid1 sdb2[1] sda2[0]
10377920 blocks [2/2] [UU]

unused devices: <none>
[root@server1 ~]#

Then adjust /etc/mdadm.conf to the new situation:

mdadm –examine –scan > /etc/mdadm.conf

/etc/mdadm.conf should now look something like this:

cat /etc/mdadm.conf

ARRAY /dev/md0 level=raid1 num-devices=2 UUID=0a96be0f:bf0f4631:a910285b:0f337164
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=f9e691e2:8d25d314:40f42444:7dbe1da1

Reboot the system:

reboot

It should boot without problems.

That’s it – you’ve successfully set up software RAID1 on your running LVM system!

8 Testing

Now let’s simulate a hard drive failure. It doesn’t matter if you select /dev/sda or /dev/sdb here. In this example I assume that /dev/sdb has failed.

To simulate the hard drive failure, you can either shut down the system and remove /dev/sdb from the system, or you (soft-)remove it like this:

mdadm –manage /dev/md0 –fail /dev/sdb1
mdadm –manage /dev/md1 –fail /dev/sdb2

mdadm –manage /dev/md0 –remove /dev/sdb1
mdadm –manage /dev/md1 –remove /dev/sdb2

Shut down the system:

shutdown -h now

Then put in a new /dev/sdb drive (if you simulate a failure of /dev/sda, you should now put /dev/sdb in /dev/sda‘s place and connect the new HDD as /dev/sdb!) and boot the system. It should still start without problems.

Now run

cat /proc/mdstat

and you should see that we have a degraded array:

[root@server1 ~]# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sda1[0]
104320 blocks [2/1] [U_]

md1 : active raid1 sda2[0]
10377920 blocks [2/1] [U_]

unused devices: <none>
[root@server1 ~]#

The output of

fdisk -l

should look as follows:

[root@server1 ~]# fdisk -l

Disk /dev/sda: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   fd  Linux raid autodetect
/dev/sda2              14        1305    10377990   fd  Linux raid autodetect

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdb doesn’t contain a valid partition table

Disk /dev/md1: 10.6 GB, 10626990080 bytes
2 heads, 4 sectors/track, 2594480 cylinders
Units = cylinders of 8 * 512 = 4096 bytes

Disk /dev/md1 doesn’t contain a valid partition table

Disk /dev/md0: 106 MB, 106823680 bytes
2 heads, 4 sectors/track, 26080 cylinders
Units = cylinders of 8 * 512 = 4096 bytes

Disk /dev/md0 doesn’t contain a valid partition table
[root@server1 ~]#

Now we copy the partition table of /dev/sda to /dev/sdb:

sfdisk -d /dev/sda | sfdisk /dev/sdb

(If you get an error, you can try the –force option:

sfdisk -d /dev/sda | sfdisk –force /dev/sdb

)

[root@server1 ~]# sfdisk -d /dev/sda | sfdisk /dev/sdb
Checking that no-one is using this disk right now …
OK

Disk /dev/sdb: 1305 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
/dev/sdb: unrecognized partition table type
Old situation:
No partitions found
New situation:
Units = sectors of 512 bytes, counting from 0

Device Boot    Start       End   #sectors  Id  System
/dev/sdb1   *        63    208844     208782  fd  Linux raid autodetect
/dev/sdb2        208845  20964824   20755980  fd  Linux raid autodetect
/dev/sdb3             0         –          0   0  Empty
/dev/sdb4             0         –          0   0  Empty
Successfully wrote the new partition table

Re-reading the partition table …

If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
[root@server1 ~]#

Afterwards we remove any remains of a previous RAID array from /dev/sdb

mdadm –zero-superblock /dev/sdb1
mdadm –zero-superblock /dev/sdb2

… and add /dev/sdb to the RAID array:

mdadm -a /dev/md0 /dev/sdb1
mdadm -a /dev/md1 /dev/sdb2

Now take a look at

cat /proc/mdstat

[root@server1 ~]# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdb1[1] sda1[0]
104320 blocks [2/2] [UU]

md1 : active raid1 sdb2[2] sda2[0]
10377920 blocks [2/1] [U_]
[======>…………..]  recovery = 32.3% (3360768/10377920) finish=1.5min speed=74238K/sec

unused devices: <none>
[root@server1 ~]#

Wait until the synchronization has finished:

[root@server1 ~]# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdb1[1] sda1[0]
104320 blocks [2/2] [UU]

md1 : active raid1 sdb2[1] sda2[0]
10377920 blocks [2/2] [UU]

unused devices: <none>
[root@server1 ~]#

Then run

grub

and install the bootloader on both HDDs:

root (hd0,0)
setup (hd0)
root (hd1,0)
setup (hd1)
quit

That’s it. You’ve just replaced a failed hard drive in your RAID1 array.

 

  • The Software-RAID Howto: http://tldp.org/HOWTO/Software-RAID-HOWTO.html
  • CentOS: http://www.centos.org/

Comments

comments