kvm cannot boot xen guest using lvm -

kvm cannot boot xen guest using lvm

This booting error is because the Xen PV guest image uses the Xen kernel, this is not compatible with anything but a host running a Xen kernel.

I did a kpartx -av virtual.img and then it created some partitions that showed up in fdisk.

I mounted it and did a chroot into it and removed the xen kernel and installed a normal kernel but Xen still shows the same kernel in Grub (only the Xen one).

This is strange but it seems like this Xen PV guest has some sort of hidden or file based virtual /boot because I can't even see the newly installed kernel.

It's obvious that Xen must have a virtual-boot partition in the form of an image for grub somewhere on the main root filesystem but I cannot find it and as usual the Xen documentation does not refer to this or makes it too hard to find.

fdisk -l VPS.img

Disk VPS.img: 0 MB, 0 bytes
255 heads, 63 sectors/track, 0 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0008c4a7

Device Boot Start End Blocks Id System
VPS.img1 * 1 13 104391 83 Linux
VPS.img2 14 1305 10377990 8e Linux LVM
Partition 2 has different physical/logical endings:
phys=(1023, 254, 63) logical=(1304, 254, 63)

I didn't notice the VPS.img1 which was created as loop0p1 with kartpx which is actually a separate boot partition. Copy your non-xen kernel to the mounted Partition #1. Edit the boot line to reflect the non-Xen kernel and initrd

*Important note

Remove the console=/dev/xvc0 from the kernel line or the kernel will not even start to boot (this should be removed permanently). Also make sure you do a kpartx -dv VPS.img before booting and deactivate any LVM partitions from it or groups.

kpartx -dv VPS.img
device-mapper: remove ioctl on loop0p2 failed: Device or resource busy
ioctl: LOOP_CLR_FD: Device or resource busy
can't del loop : /dev/loop0

vgchange -a n /dev/VolGroup00

kpartx -dv VPS.img

del devmap : loop0p2
del devmap : loop0p1
loop deleted : /dev/loop0

For some reasonI'm getting a kernel panic:

switchroot: mount failed: No such file or directory

Kernel panic - not syncing: Attempted to kill init

I believe this is because the Centos initrd for the non-Xen kernel was installed using chroot and doesn't understand it needs /dev/mapper

I used the non-Xen kernel with the Xen initrd and the process gets farther but can't access the volume groups still due to it looking for non-existent modules.

I need to create a proper initrd in Centos

cd /boot/
[root@2 boot]# ls
config-2.6.18-238.el5xen lost+found System.map-2.6.18-308.24.1.el5
config-2.6.18-308.24.1.el5 message vmlinuz-2.6.18-238.el5xen
grub symvers-2.6.18-238.el5xen.gz vmlinuz-2.6.18-308.24.1.el5
initrd-2.6.18-238.el5xen.img symvers-2.6.18-308.24.1.el5.gz xen.gz-2.6.18-238.el5
initrd-2.6.18-308.24.1.el5.img System.map-2.6.18-238.el5xen xen-syms-2.6.18-238.el5
[root@2 boot]# mv initrd-2.6.18-308.24.1.el5.img initrd-2.6.18-308.24.1.el5.img.old
[root@2 boot]# mkinitrd initrd-2.6.18-308.24.1.el5.img 2.6.18-308.24.1.el5
error opening /sys/block: No such file or directory
error opening /sys/block: No such file or directory
No module xenblk found for kernel 2.6.18-308.24.1.el5, aborting.

mkinitrd --with lvm --force-lvm-probe -v -f initrd-2.6.18-308.24.1.el5.img 2.6.18-308.24.1.el5
Creating initramfs
Looking for deps of module ehci-hcd
Looking for deps of module ohci-hcd
Looking for deps of module uhci-hcd
Looking for deps of module ext3: jbd
Looking for deps of module jbd
Looking for driver for device VolGroup00/LogVol00
error opening /sys/block: No such file or directory
Looking for driver for device VolGroup00/LogVol01
error opening /sys/block: No such file or directory
Looking for deps of module ide-disk
Looking for deps of module dm-mem-cache
Looking for deps of module dm-region_hash: dm-mod dm-log
Looking for deps of module dm-mod
Looking for deps of module dm-log: dm-mod
Looking for deps of module dm-message
Looking for deps of module dm-raid45: dm-message dm-mod dm-mem-cache dm-log dm-region_hash
Looking for deps of module lvm
No module lvm found for kernel 2.6.18-308.24.1.el5, aborting.

before chrooting, do a mount -o bind /sys /root/mnt/sys

mkinitrd --with lvm --force-lvm-probe -v -f initrd-2.6.18-308.24.1.el5.img 2.6.18-308.24.1.el5
Creating initramfs
Looking for deps of module ehci-hcd
Looking for deps of module ohci-hcd
Looking for deps of module uhci-hcd
Looking for deps of module ext3: jbd
Looking for deps of module jbd
Looking for driver for device VolGroup00/LogVol00
Looking for driver for device VolGroup00/LogVol01
Looking for deps of module ide-disk
Looking for deps of module dm-mem-cache
Looking for deps of module dm-region_hash: dm-mod dm-log
Looking for deps of module dm-mod
Looking for deps of module dm-log: dm-mod
Looking for deps of module dm-message
Looking for deps of module dm-raid45: dm-message dm-mod dm-mem-cache dm-log dm-region_hash
Looking for deps of module lvm
No module lvm found for kernel 2.6.18-308.24.1.el5, aborting.

mkinitrd -v -f initrd-2.6.18-308.24.1.el5.img 2.6.18-308.24.1.el5Creating initramfs
Looking for deps of module ehci-hcd
Looking for deps of module ohci-hcd
Looking for deps of module uhci-hcd
Looking for deps of module ext3: jbd
Looking for deps of module jbd
Looking for driver for device VolGroup00/LogVol00
Looking for driver for device VolGroup00/LogVol01
Looking for deps of module ide-disk
Looking for deps of module dm-mem-cache
Looking for deps of module dm-region_hash: dm-mod dm-log
Looking for deps of module dm-mod
Looking for deps of module dm-log: dm-mod
Looking for deps of module dm-message
Looking for deps of module dm-raid45: dm-message dm-mod dm-mem-cache dm-log dm-region_hash
Using modules: /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/usb/host/ehci-hcd.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/usb/host/ohci-hcd.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/usb/host/uhci-hcd.ko /lib/modules/2.6.18-308.24.1.el5/kernel/fs/jbd/jbd.ko /lib/modules/2.6.18-308.24.1.el5/kernel/fs/ext3/ext3.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-mem-cache.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-mod.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-log.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-region_hash.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-message.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-raid45.ko
/sbin/nash -> /tmp/initrd.124034/bin/nash
/sbin/insmod.static -> /tmp/initrd.124034/bin/insmod
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/usb/host/ehci-hcd.ko' [elf32-i386] to `/tmp/initrd.124034/lib/ehci-hcd.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/usb/host/ohci-hcd.ko' [elf32-i386] to `/tmp/initrd.124034/lib/ohci-hcd.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/usb/host/uhci-hcd.ko' [elf32-i386] to `/tmp/initrd.124034/lib/uhci-hcd.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/fs/jbd/jbd.ko' [elf32-i386] to `/tmp/initrd.124034/lib/jbd.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/fs/ext3/ext3.ko' [elf32-i386] to `/tmp/initrd.124034/lib/ext3.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-mem-cache.ko' [elf32-i386] to `/tmp/initrd.124034/lib/dm-mem-cache.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-mod.ko' [elf32-i386] to `/tmp/initrd.124034/lib/dm-mod.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-log.ko' [elf32-i386] to `/tmp/initrd.124034/lib/dm-log.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-region_hash.ko' [elf32-i386] to `/tmp/initrd.124034/lib/dm-region_hash.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-message.ko' [elf32-i386] to `/tmp/initrd.124034/lib/dm-message.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-raid45.ko' [elf32-i386] to `/tmp/initrd.124034/lib/dm-raid45.ko' [elf32-i386]
/sbin/dmraid.static -> /tmp/initrd.124034/bin/dmraid
/sbin/kpartx.static -> /tmp/initrd.124034/bin/kpartx
Adding module ehci-hcd
Adding module ohci-hcd
Adding module uhci-hcd
Adding module jbd
Adding module ext3
Adding module dm-mem-cache
Adding module dm-mod
Adding module dm-log
Adding module dm-region_hash
Adding module dm-message
Adding module dm-raid45

this did it, the mnt proc and dev

mount -o bind /proc mnt/proc/
[root@12 ~]# mount -o bind /dev mnt/dev/
[root@12 ~]# chroot mnt
[root@12 /]# cd /boot/
[root@12 boot]# mkinitrd -v -f initrd-2.6.18-308.24.1.el5.img 2.6.18-308.24.1.el5
Creating initramfs
Looking for deps of module ehci-hcd
Looking for deps of module ohci-hcd
Looking for deps of module uhci-hcd
Looking for deps of module ext3: jbd
Looking for deps of module jbd
Looking for driver for device mapper/loop0p2
Found DM device mapper/loop0p2
Looking for deps of module ide-disk
Looking for deps of module dm-mod
Looking for deps of module dm-mirror: dm-mod dm-log
Looking for deps of module dm-log: dm-mod
Looking for deps of module dm-zero: dm-mod
Looking for deps of module dm-snapshot: dm-mod
Looking for deps of module dm-mem-cache
Looking for deps of module dm-region_hash: dm-mod dm-log
Looking for deps of module dm-message
Looking for deps of module dm-raid45: dm-message dm-mod dm-mem-cache dm-log dm-region_hash
Using modules: /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/usb/host/ehci-hcd.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/usb/host/ohci-hcd.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/usb/host/uhci-hcd.ko /lib/modules/2.6.18-308.24.1.el5/kernel/fs/jbd/jbd.ko /lib/modules/2.6.18-308.24.1.el5/kernel/fs/ext3/ext3.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-mod.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-log.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-mirror.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-zero.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-snapshot.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-mem-cache.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-region_hash.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-message.ko /lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-raid45.ko
/sbin/nash -> /tmp/initrd.125242/bin/nash
/sbin/insmod.static -> /tmp/initrd.125242/bin/insmod
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/usb/host/ehci-hcd.ko' [elf32-i386] to `/tmp/initrd.125242/lib/ehci-hcd.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/usb/host/ohci-hcd.ko' [elf32-i386] to `/tmp/initrd.125242/lib/ohci-hcd.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/usb/host/uhci-hcd.ko' [elf32-i386] to `/tmp/initrd.125242/lib/uhci-hcd.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/fs/jbd/jbd.ko' [elf32-i386] to `/tmp/initrd.125242/lib/jbd.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/fs/ext3/ext3.ko' [elf32-i386] to `/tmp/initrd.125242/lib/ext3.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-mod.ko' [elf32-i386] to `/tmp/initrd.125242/lib/dm-mod.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-log.ko' [elf32-i386] to `/tmp/initrd.125242/lib/dm-log.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-mirror.ko' [elf32-i386] to `/tmp/initrd.125242/lib/dm-mirror.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-zero.ko' [elf32-i386] to `/tmp/initrd.125242/lib/dm-zero.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-snapshot.ko' [elf32-i386] to `/tmp/initrd.125242/lib/dm-snapshot.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-mem-cache.ko' [elf32-i386] to `/tmp/initrd.125242/lib/dm-mem-cache.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-region_hash.ko' [elf32-i386] to `/tmp/initrd.125242/lib/dm-region_hash.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-message.ko' [elf32-i386] to `/tmp/initrd.125242/lib/dm-message.ko' [elf32-i386]
copy from `/lib/modules/2.6.18-308.24.1.el5/kernel/drivers/md/dm-raid45.ko' [elf32-i386] to `/tmp/initrd.125242/lib/dm-raid45.ko' [elf32-i386]
/sbin/lvm.static -> /tmp/initrd.125242/bin/lvm
/sbin/dmraid.static -> /tmp/initrd.125242/bin/dmraid
/sbin/kpartx.static -> /tmp/initrd.125242/bin/kpartx
Adding module ehci-hcd
Adding module ohci-hcd
Adding module uhci-hcd
Adding module jbd
Adding module ext3
Adding module dm-mod
Adding module dm-log
Adding module dm-mirror
Adding module dm-zero
Adding module dm-snapshot
Adding module dm-mem-cache
Adding module dm-region_hash
Adding module dm-message
Adding module dm-raid45

The above got it booting but during init or after it says

INIT: Id "co" respawning too fast: disabled for 5 minutes

This is an easy fix, just edit:

vi /etc/inittab

Type: "/^co" or navigate to the line that has:

co:2345:respawn:/sbin/agetty xvc0 9600 vt100-nav

Comment the above line out, then uncomment the following which will be commented out below the co line:

1:2345:respawn:/sbin/mingetty tty1
2:2345:respawn:/sbin/mingetty tty2
3:2345:respawn:/sbin/mingetty tty3
4:2345:respawn:/sbin/mingetty tty4
5:2345:respawn:/sbin/mingetty tty5
6:2345:respawn:/sbin/mingetty tty6

After that, reboot and your conversion from Xen PV is over and your system should be working perfectly now.

As we can see, having a separate boot partition and LVM really complicate the migration process.


  • Linux bash how to list files of a certain date with the statu command for scripting
  • cPanel How to set SSL and Dedicated IP in cPanel
  • OpenVZ vzctl update changes how iptables work - iptables v1.3.5: can't initialize iptables table `nat': Table does not exist (do you need to insmod?) Perhaps iptables or your kernel needs to be upgraded. updated solution
  • Wordpress Plugin manual install how-to
  • Disable linux baxh proxy ENV export variable
  • Linux bash read exif data tags from image jpeg/tiff/jpg
  • PHP Issues with Joomla Strict Standards: Non-static method JLoader::import() should not be called statically
  • esniper and curl errors https://signin.ebay.com/ws/eBayISAPI.dll?SignIn: Couldn't connect to server: couldn't connect to host
  • Disable cphulk in CPanel if you can't login even with the right password
  • bash increment how to
  • Centos automatic ifcfg-eth0 scripts based on list of IPs from bash
  • Vbox/Virtualbox devices grayed out for non-root user solution
  • libguestfs tools howto guide for managing virtual machine images
  • NS_ERROR_FAILURE (0x80004005) The virtual machine 'XP' has terminated unexpectedly during startup with exit code 1.
  • Directadmin compile error solution cc: Internal error: Killed (program cc1)
  • Samsung C410W/CLP410W not printing in Linux solution
  • Samsung Printer Password Login to change settings CLP
  • bash shellshock how to manually patch when there is no update for Centos/Debian/Ubuntu/Fedora
  • How to root and unlock S4 in Canada Bell/Rogers/Telus/Fido I337 4.4.2
  • OpenVZ Howto disable ploop and switch back to SIMFS