LVM mirror with data and logs mirrored on same 2 devices

Question

I'm trying to set LVM mirroring with only 2 devices. It works perfecly when I add third device for mirrorlog, or use corelog. But with only 2 devices and --alloc anywhere LVM nearly always creates both mirrors on one device.

Situation:

2 x 50GB devices /dev/xvdf and /dev/xvdg
I need 1 x 40GB partition on /dev/xvdf that will be mirrored on /dev/xvdg
I do not want 2 separate drives for mirrored logs, I want mirrored logs on /dev/xvdf and /dev/xvdg (On some extents)

Problem:

LVM nearly always (If I understand lvs command output correctly) creates both mirrors on /dev/xvdf, and mirrorlog on /dev/xvdg (yeah :-(

Commands that I use:

Scenario 1 - simplest:

$ lvcreate -m 1 --mirrorlog mirrored -L40G -n test forfiter --alloc anywhere

$ lvs -a -o +devices
  LV                   VG       Attr   LSize  Origin Snap%  Move Log       Copy%  Convert Devices                                    
  test                 forfiter mwa-a- 40,00g                    test_mlog   7,09         test_mimage_0(0),test_mimage_1(0)          
  [test_mimage_0]      forfiter Iwi-ao 40,00g                                             /dev/xvdf(0)                               
  [test_mimage_1]      forfiter Iwi-ao 40,00g                                             /dev/xvdf(10240)                           
  [test_mimage_1]      forfiter Iwi-ao 40,00g                                             /dev/xvdg(2)                               
  [test_mlog]          forfiter mwa-ao  4,00m                              100,00         test_mlog_mimage_0(0),test_mlog_mimage_1(0)
  [test_mlog_mimage_0] forfiter iwi-ao  4,00m                                             /dev/xvdg(0)                               
  [test_mlog_mimage_1] forfiter iwi-ao  4,00m                                             /dev/xvdg(1)

As you can see test_mimage_1 is partially on /dev/xvdf and partially on /dev/xvdf There is 12799 PE allocated on xvdf and 7683 allocated on xvdg.

Funniest thing is that LVM created mlog on one device too...

Scenario 2 - I tried to specify number of extents to use:

$ lvcreate  -m 1 --mirrorlog mirrored  -L 40G -n test forfiter /dev/xvdf:6-12700 /dev/xvdg:6-12700 /dev/xvdf:0-4 /dev/xvdg:0-4 --alloc anywhere

$ lvs -a -o +devices
  LV                   VG       Attr   LSize  Origin Snap%  Move Log       Copy%  Convert Devices                                    
  test                 forfiter mwa-a- 40,00g                    test_mlog   2,79         test_mimage_0(0),test_mimage_1(0)          
  [test_mimage_0]      forfiter Iwi-ao 40,00g                                             /dev/xvdf(6)                               
  [test_mimage_1]      forfiter Iwi-ao 40,00g                                             /dev/xvdf(10246)                           
  [test_mimage_1]      forfiter Iwi-ao 40,00g                                             /dev/xvdf(0)                               
  [test_mimage_1]      forfiter Iwi-ao 40,00g                                             /dev/xvdg(7)                               
  [test_mlog]          forfiter mwa-ao  4,00m                              100,00         test_mlog_mimage_0(0),test_mlog_mimage_1(0)
  [test_mlog_mimage_0] forfiter iwi-ao  4,00m                                             /dev/xvdg(6)                               
  [test_mlog_mimage_1] forfiter iwi-ao  4,00m                                             /dev/xvdg(0)

No success :-)

I read many tutorials - everywhere authors suggest to use --alloc anywhere, but for me it looks like results are quite strange (Mirroring works, but not as I expected)

I would like to migrate raid1 from MDADM to LVM.

Did you try `--alloc normal` (which is the default) and if you didn't why not? — , Apr 18 '13 at 12:22
Yes I tried with normal --alloc, and always got "Not enough PVs with free space available for parallel allocation.". This is probably (As I understand) because without **anywhere**, lvm needs another 2 volumes for logs. (I tested with 4 EBS drives and it worked) — Tomasz Olszewski, Apr 18 '13 at 12:38
You could use partitions: `/dev/xvdf1` and `/dev/xvdf2` etc. — , Apr 18 '13 at 12:45

score 1 · Answer 1 · answered Jan 08 '15 at 08:38

Without create partition, your problem is you are using in the same command "--mirrorlog mirrored and --alloc anywhere"

I had the same problem on suse 11 using "--mirrorlog mirrored" but i solved using the lvm command in the following way:

lvcreate -m 1 --mirrorlog mirrored -L40G -n test forfiter /dev/xvdb /dev/xvdc

In redhat or centos you can find this comment in lvm.conf

# Set to 1 to guarantee that mirror logs will always be placed on 
# different PVs from the mirror images.  This was the default
# until version 2.02.85.
mirror_logs_require_separate_pvs = 0

There is many way to solve this issue, without create partition.

score 0 · Accepted Answer · answered Apr 18 '13 at 13:08

Use a partition editor such as parted, cfdisk or fdisk to create partitions: /dev/xvdf1 and /dev/xvdf2 etc. and put them into your forfiter VG.

In this example I use /dev/mapper/loop0p1 etc.

pvcreate /dev/mapper/loop1p1
  Physical volume "/dev/mapper/loop1p1" successfully created
pvcreate /dev/mapper/loop1p2
  Physical volume "/dev/mapper/loop1p2" successfully created

vgcreate forfiter /dev/mapper/loop0p1
  Volume group "forfiter" successfully created

vgextend forfiter /dev/mapper/loop1p1
  Volume group "forfiter" successfully extended
vgextend forfiter /dev/mapper/loop0p2
  Volume group "forfiter" successfully extended
vgextend forfiter /dev/mapper/loop1p2
  Volume group "forfiter" successfully extended

vgs forfiter
  VG       #PV #LV #SN Attr   VSize   VFree  
  forfiter   4   0   0 wz--n- 248.00m 248.00m

ls -l /dev/mapper/loop0p1
  lrwxrwxrwx 1 root root 8 Apr 18 08:59 /dev/mapper/loop0p1 -> ../dm-21


vgdisplay -v forfiter | tail -n mumble

  --- Physical volumes ---
  PV Name               /dev/dm-21     
  PV UUID               uFJpEH-dLFA-gJiM-cnao-cFFm-DEZG-RnNvSM
  PV Status             allocatable
  Total PE / Free PE    15 / 15

  PV Name               /dev/dm-23     
  PV UUID               1T7DIL-Xw4s-4tVy-CVQc-lKDp-aUNA-lyk8v2
  PV Status             allocatable
  Total PE / Free PE    15 / 15

  PV Name               /dev/dm-22     
  PV UUID               T0prpa-KKEO-uWUb-zQU3-cosM-uyEI-ext9F7
  PV Status             allocatable
  Total PE / Free PE    16 / 16

  PV Name               /dev/dm-24     
  PV UUID               PC2aCZ-eKdU-p8eS-SBDc-uWzY-54gG-952ndg
  PV Status             allocatable
  Total PE / Free PE    16 / 16

lvcreate -m 1 --mirrorlog mirrored -L64M -n test forfiter
  The link /dev/forfiter/test_mlog should had been created by udev but it was not found. Falling back to direct link creation.
  The link /dev/forfiter/test_mlog should have been removed by udev but it is still present. Falling back to direct link removal.
  Logical volume "test" created

lvs -a -o +devices forfiter
  LV                   VG       Attr   LSize  Origin Snap%  Move Log       Copy%  Convert Devices                                    
  test                 forfiter mwi-a- 64.00m                    test_mlog 100.00         test_mimage_0(0),test_mimage_1(0)          
  [test_mimage_0]      forfiter iwi-ao 64.00m                                             /dev/dm-22(0)                              
  [test_mimage_1]      forfiter iwi-ao 64.00m                                             /dev/dm-24(0)                              
  [test_mlog]          forfiter mwi-ao  4.00m                              100.00         test_mlog_mimage_0(0),test_mlog_mimage_1(0)
  [test_mlog_mimage_0] forfiter iwi-ao  4.00m                                             /dev/dm-21(0)                              
  [test_mlog_mimage_1] forfiter iwi-ao  4.00m                                             /dev/dm-23(0)

You're welcome! I learned something useful while answering your question. :) — , Apr 18 '13 at 13:23
Simplest and wrong. One should not have to fiddle with partitions. Actually it is harmful having two partitions on single drive in one VG and creating RAID on top of that: LVM could create both data legs of the RAID device on that device and when it's gone it's gone. c4f4t0r's answer is the correct one. — Martian, May 21 '15 at 07:19

LVM mirror with data and logs mirrored on same 2 devices

2 Answers2