Bug 558463

Summary: Unable to access VM disk created by RHEV
Product: [Retired] CloudForms Cloud Engine Reporter: Olivier Renault <orenault>
Component: aeolus-conductorAssignee: Richard W.M. Jones <rjones>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: medium Docs Contact:
Priority: low    
Version: 0.3.1CC: deltacloud-maint, mbooth, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-01-25 18:44:12 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
libguestfs-test-tool output
none
initrd file as requested none

Description Olivier Renault 2010-01-25 12:52:50 UTC 
Created attachment 386610 [details]
libguestfs-test-tool output

Description of problem:
Not able to access the disk of a VM.

Version-Release number of selected component (if applicable):
libguestfs-1.0.76-1.el5.4.x86_64
libguestfs-tools-1.0.76-1.el5.4.x86_64

Hypervisor is running RHEL 5
Linux rh2.rh.hpintelco.org 2.6.18-164.9.1.el5 #1 SMP Wed Dec 9 03:27:37 EST 2009 x86_64 x86_64 x86_64 GNU/Linux


How reproducible:
Always

Steps to Reproduce:
I am trying to access the disk from a VM which has been created by RHEV-M. 

1. Get the name of the disk through some vdsClient magic.
2. guestfish -v
3. <fs> add /dev/496c8d8c-f07f-4805-9d74-fa4d9620ab8d/478a83d2-d79b-423a-84ab-a36f24f0d13b
4. run
  
Actual results:
[root@rh2 ~]#  guestfish -v -a /dev/496c8d8c-f07f-4805-9d74-fa4d9620ab8d/478a83d2-d79b-423a-84ab-a36f24f0d13b

Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for help with commands
      'quit' to quit the shell

><fs> run
looking for supermin appliance in appliance
looking for supermin appliance in /usr/lib64/guestfs
null_vmchannel_sock = 45818
/usr/bin/qemu-system-x86_64 \
    -drive file=/dev/496c8d8c-f07f-4805-9d74-fa4d9620ab8d/478a83d2-d79b-423a-84ab-a36f24f0d13b,cache=off,if=ide \
    -m 500 \
    -no-reboot \
    -nographic \
    -serial stdio \
    -no-hpet \
    -rtc-td-hack \
    -net user,vlan=0,net=10.0.2.0/8 \
    -net nic,model=virtio,vlan=0 \
    -kernel /tmp/libguestfsXN796r/kernel \
    -initrd /tmp/libguestfsXN796r/initrd \
    -append 'panic=1 console=ttyS0 udevtimeout=300 noapic acpi=off cgroup_disable=memory selinux=0 guestfs_vmchannel=tcp:10.0.2.2:45818 guestfs_verbose=1 '
accept_from_daemon: 0xaf49310 g->state = 0
Could not open '/dev/kqemu' - QEMU acceleration layer not activated: No such file or directory
Linux version 2.6.18-164.9.1.el5 (mockbuild.bos.redhat.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)) #1 SMP Wed Dec 9 03:27:37 EST 2009
Command line: panic=1 console=ttyS0 udevtimeout=300 noapic acpi=off cgroup_disable=memory selinux=0 guestfs_vmchannel=tcp:10.0.2.2:45818 guestfs_verbose=1 
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000010000 - 000000000009f000 (usable)
 BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 000000001f3f0000 (usable)
 BIOS-e820: 000000001f3f0000 - 000000001f400000 (ACPI data)
 BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
DMI 2.4 present.
No NUMA configuration found
Faking a node at 0000000000000000-000000001f3f0000
Bootmem setup node 0 0000000000000000-000000001f3f0000
Memory for crash kernel (0x0 to 0x0) notwithin permissible range
disabling kdump
Intel MultiProcessor Specification v1.4
    Virtual Wire compatibility mode.
OEM ID: QEMUCPU  Product ID: 0.1          APIC at: 0xFEE00000
Processor #0 6:0 APIC version 17
I/O APIC #1 Version 17 at 0xFEC00000.
Setting APIC routing to physical flat
Processors: 1
Nosave address range: 000000000009f000 - 00000000000a0000
Nosave address range: 00000000000a0000 - 00000000000e8000
Nosave address range: 00000000000e8000 - 0000000000100000
Allocating PCI resources starting at 20000000 (gap: 1f400000:e0bc0000)
SMP: Allowing 1 CPUs, 0 hotplug CPUs
Built 1 zonelists.  Total pages: 124834
Kernel command line: panic=1 console=ttyS0 udevtimeout=300 noapic acpi=off cgroup_disable=memory selinux=0 guestfs_vmchannel=tcp:10.0.2.2:45818 guestfs_verbose=1 
Initializing CPU#0
PID hash table entries: 2048 (order: 11, 16384 bytes)
Console: colour VGA+ 80x25
Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)
Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)
Checking aperture...
ACPI: DMAR not present
Memory: 408732k/511936k available (2550k kernel code, 102752k reserved, 1291k data, 208k init)
Calibrating delay loop (skipped), value calculated using timer frequency.. 5067.52 BogoMIPS (lpj=2533761)
Security Framework v1.0.0 initialized
SELinux:  Disabled at boot.
Capability LSM initialized
Mount-cache hash table entries: 256
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 512K (64 bytes/line)
CPU 0/0 -> Node 0
SMP alternatives: switching to UP code
Freeing SMP alternatives: 32k freed
Using local APIC timer interrupts.
Detected 62.508 MHz APIC timer.
Brought up 1 CPUs
testing NMI watchdog ... <4>WARNING: CPU#0: NMI appears to be stuck (0->0)!
time.c: Using 1.193182 MHz WALL PIT GTOD PIT/TSC timer.
time.c: Detected 2533.761 MHz processor.
checking if image is initramfs... it is
Freeing initrd memory: 89321k freed
NET: Registered protocol family 16
PCI: Using configuration type 1
ACPI: Interpreter disabled.
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI: disabled
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: Probing PCI hardware
PCI quirk: region b000-b03f claimed by PIIX4 ACPI
PCI quirk: region b100-b10f claimed by PIIX4 SMB
PCI: Using IRQ router PIIX/ICH [8086/7000] at 0000:00:01.0
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
ACPI: DMAR not present
PCI-GART: No AMD northbridge found.
NET: Registered protocol family 2
IP route cache hash table entries: 4096 (order: 3, 32768 bytes)
TCP established hash table entries: 16384 (order: 6, 262144 bytes)
TCP bind hash table entries: 8192 (order: 5, 131072 bytes)
TCP: Hash tables configured (established 16384 bind 8192)
TCP reno registered
audit: initializing netlink socket (disabled)
type=2000 audit(1264423760.339:1): initialized
Total HugeTLB memory allocated, 0
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
Initializing Cryptographic API
alg: No test for crc32c (crc32c-generic)
ksign: Installing public key data
Loading keyring
- Added public key 984BA14248E4E6F4
- User ID: Red Hat, Inc. (Kernel Module GPG key)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
Limiting direct PCI/PCI transfers.
PCI: PIIX3: Enabling Passive Release on 0000:00:01.0
Activating ISA DMA hang workarounds.
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
Real Time Clock Driver v1.12ac
Non-volatile memory driver v1.2
Linux agpgart interface v0.101 (c) Dave Jones
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled
�serial8250: ttyS0 at I/O 0x3f8 (irq = [root@rh2 ~]#  guestfish -v -a /dev/496c8d8c-f07f-4805-9d74-fa4d9620ab8d/478a83d2-d79b-423a-84ab-a36f24f0d13b

Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for help with commands
      'quit' to quit the shell

><fs> run
looking for supermin appliance in appliance
looking for supermin appliance in /usr/lib64/guestfs
null_vmchannel_sock = 45818
/usr/bin/qemu-system-x86_64 \
    -drive file=/dev/496c8d8c-f07f-4805-9d74-fa4d9620ab8d/478a83d2-d79b-423a-84ab-a36f24f0d13b,cache=off,if=ide \
    -m 500 \
    -no-reboot \
    -nographic \
    -serial stdio \
    -no-hpet \
    -rtc-td-hack \
    -net user,vlan=0,net=10.0.2.0/8 \
    -net nic,model=virtio,vlan=0 \
    -kernel /tmp/libguestfsXN796r/kernel \
    -initrd /tmp/libguestfsXN796r/initrd \
    -append 'panic=1 console=ttyS0 udevtimeout=300 noapic acpi=off cgroup_disable=memory selinux=0 guestfs_vmchannel=tcp:10.0.2.2:45818 guestfs_verbose=1 '
accept_from_daemon: 0xaf49310 g->state = 0
Could not open '/dev/kqemu' - QEMU acceleration layer not activated: No such file or directory
Linux version 2.6.18-164.9.1.el5 (mockbuild.bos.redhat.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)) #1 SMP Wed Dec 9 03:27:37 EST 2009
Command line: panic=1 console=ttyS0 udevtimeout=300 noapic acpi=off cgroup_disable=memory selinux=0 guestfs_vmchannel=tcp:10.0.2.2:45818 guestfs_verbose=1 
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000010000 - 000000000009f000 (usable)
 BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 000000001f3f0000 (usable)
 BIOS-e820: 000000001f3f0000 - 000000001f400000 (ACPI data)
 BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
DMI 2.4 present.
No NUMA configuration found
Faking a node at 0000000000000000-000000001f3f0000
Bootmem setup node 0 0000000000000000-000000001f3f0000
Memory for crash kernel (0x0 to 0x0) notwithin permissible range
disabling kdump
Intel MultiProcessor Specification v1.4
    Virtual Wire compatibility mode.
OEM ID: QEMUCPU  Product ID: 0.1          APIC at: 0xFEE00000
Processor #0 6:0 APIC version 17
I/O APIC #1 Version 17 at 0xFEC00000.
Setting APIC routing to physical flat
Processors: 1
Nosave address range: 000000000009f000 - 00000000000a0000
Nosave address range: 00000000000a0000 - 00000000000e8000
Nosave address range: 00000000000e8000 - 0000000000100000
Allocating PCI resources starting at 20000000 (gap: 1f400000:e0bc0000)
SMP: Allowing 1 CPUs, 0 hotplug CPUs
Built 1 zonelists.  Total pages: 124834
Kernel command line: panic=1 console=ttyS0 udevtimeout=300 noapic acpi=off cgroup_disable=memory selinux=0 guestfs_vmchannel=tcp:10.0.2.2:45818 guestfs_verbose=1 
Initializing CPU#0
PID hash table entries: 2048 (order: 11, 16384 bytes)
Console: colour VGA+ 80x25
Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)
Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)
Checking aperture...
ACPI: DMAR not present
Memory: 408732k/511936k available (2550k kernel code, 102752k reserved, 1291k data, 208k init)
Calibrating delay loop (skipped), value calculated using timer frequency.. 5067.52 BogoMIPS (lpj=2533761)
Security Framework v1.0.0 initialized
SELinux:  Disabled at boot.
Capability LSM initialized
Mount-cache hash table entries: 256
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 512K (64 bytes/line)
CPU 0/0 -> Node 0
SMP alternatives: switching to UP code
Freeing SMP alternatives: 32k freed
Using local APIC timer interrupts.
Detected 62.508 MHz APIC timer.
Brought up 1 CPUs
testing NMI watchdog ... <4>WARNING: CPU#0: NMI appears to be stuck (0->0)!
time.c: Using 1.193182 MHz WALL PIT GTOD PIT/TSC timer.
time.c: Detected 2533.761 MHz processor.
checking if image is initramfs... it is
Freeing initrd memory: 89321k freed
NET: Registered protocol family 16
PCI: Using configuration type 1
ACPI: Interpreter disabled.
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI: disabled
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: Probing PCI hardware
PCI quirk: region b000-b03f claimed by PIIX4 ACPI
PCI quirk: region b100-b10f claimed by PIIX4 SMB
PCI: Using IRQ router PIIX/ICH [8086/7000] at 0000:00:01.0
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
ACPI: DMAR not present
PCI-GART: No AMD northbridge found.
NET: Registered protocol family 2
IP route cache hash table entries: 4096 (order: 3, 32768 bytes)
TCP established hash table entries: 16384 (order: 6, 262144 bytes)
TCP bind hash table entries: 8192 (order: 5, 131072 bytes)
TCP: Hash tables configured (established 16384 bind 8192)
TCP reno registered
audit: initializing netlink socket (disabled)
type=2000 audit(1264423760.339:1): initialized
Total HugeTLB memory allocated, 0
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
Initializing Cryptographic API
alg: No test for crc32c (crc32c-generic)
ksign: Installing public key data
Loading keyring
- Added public key 984BA14248E4E6F4
- User ID: Red Hat, Inc. (Kernel Module GPG key)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
Limiting direct PCI/PCI transfers.
PCI: PIIX3: Enabling Passive Release on 0000:00:01.0
Activating ISA DMA hang workarounds.
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
Real Time Clock Driver v1.12ac
Non-volatile memory driver v1.2
Linux agpgart interface v0.101 (c) Dave Jones
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled
�serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
brd: module loaded
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
PIIX3: IDE controller at PCI slot 0000:00:01.1
PIIX3: chipset revision 0
PIIX3: not 100% native mode: will probe irqs later
    ide0: BM-DMA at 0xc000-0xc007, BIOS settings: hda:pio, hdb:pio
    ide1: BM-DMA at 0xc008-0xc00f, BIOS settings: hdc:pio, hdd:pio
hda: QEMU HARDDISK, ATA DISK drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
hdc: QEMU DVD-ROM, ATAPI CD/DVD-ROM drive
ide1 at 0x170-0x177,0x376 on irq 15
hda: max request size: 512KiB
hda: 20971520 sectors (10737 MB) w/256KiB Cache, CHS=16383/255/63, (U)DMA
hda: cache flushes supported
 hda: hda1 hda2
ide-floppy driver 0.99.newide
usbcore: registered new driver hiddev
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.6:USB HID core driver
PNP: No PS/2 controller found. Probing ports directly.
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
TCP bic registered
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
Initalizing network drop monitor service
Freeing unused kernel memory: 208k freed
Write protecting the kernel read-only data: 497k
Failed to execute /init
Kernel panic - not syncing: No init found.  Try passing init= option to kernel.
 <0>Rebooting in 1 seconds..
4) is a 16550A
brd: module loaded
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
PIIX3: IDE controller at PCI slot 0000:00:01.1
PIIX3: chipset revision 0
PIIX3: not 100% native mode: will probe irqs later
    ide0: BM-DMA at 0xc000-0xc007, BIOS settings: hda:pio, hdb:pio
    ide1: BM-DMA at 0xc008-0xc00f, BIOS settings: hdc:pio, hdd:pio
hda: QEMU HARDDISK, ATA DISK drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
hdc: QEMU DVD-ROM, ATAPI CD/DVD-ROM drive
ide1 at 0x170-0x177,0x376 on irq 15
hda: max request size: 512KiB
hda: 20971520 sectors (10737 MB) w/256KiB Cache, CHS=16383/255/63, (U)DMA
hda: cache flushes supported
 hda: hda1 hda2
ide-floppy driver 0.99.newide
usbcore: registered new driver hiddev
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.6:USB HID core driver
PNP: No PS/2 controller found. Probing ports directly.
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
TCP bic registered
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
Initalizing network drop monitor service
Freeing unused kernel memory: 208k freed
Write protecting the kernel read-only data: 497k
Failed to execute /init
Kernel panic - not syncing: No init found.  Try passing init= option to kernel.
 <0>Rebooting in 1 seconds..


Expected results:
Being able to access my disk 

Additional info:
Comment 1 Richard W.M. Jones 2010-01-25 13:03:14 UTC 
Please run the following command:

libguestfs-supermin-helper /usr/lib64/guestfs /tmp/kernel /tmp/initrd

(You don't need to be root)

This will create two files called /tmp/kernel and /tmp/initrd.

/tmp/kernel should be a symlink.

/tmp/initrd should be a large file (~100MB).  Please email /tmp/initrd to me.

-----

For example, when I do this on my RHEL 5.4 machine:

$ libguestfs-supermin-helper /usr/lib64/guestfs /tmp/kernel /tmp/initrd
$ ls -l /tmp/kernel /tmp/initrd 
-rw-r--r-- 1 rjones rjones 99412480 Jan 25 13:01 /tmp/initrd
lrwxrwxrwx 1 rjones rjones       28 Jan 25 13:01 /tmp/kernel -> /boot/vmlinuz-2.6.18-164.el5
Comment 2 Olivier Renault 2010-01-25 15:36:53 UTC 
Created attachment 386652 [details]
initrd file as requested
Comment 3 Richard W.M. Jones 2010-01-25 16:49:56 UTC 
Possible sign of desperation but can you check the install is not
corrupted in any way:

$ rpm -V $(rpm -qa | grep guestf)
Comment 4 Richard W.M. Jones 2010-01-25 16:55:08 UTC 
Also, can you check how much free space you have on /tmp.

Also, when you ran libguestfs-supermin-helper (comment 1) did you
get any other warnings or error messages?
Comment 5 Richard W.M. Jones 2010-01-25 17:32:28 UTC 
A comment on the appliance (initrd) attached in comment 2.  I can understand
why the kernel cannot run /init from that appliance.  The appliance contains
an x86_64 binary of /bin/bash, but only i386 libraries!

The only thing I know that can really screw up a libguestfs installation that much
is multilib[1].

So that is why I really need to know the answer to comment 3.

[1] http://rwmj.wordpress.com/2009/11/16/please-someone-shoot-multilib/#content
Comment 6 Richard W.M. Jones 2010-01-25 18:33:26 UTC 
The reporter neglected to mention:

<monolive> rwmjones, I've cop[ied] initramfs.epel-5.i686.supermin.hostfiles initramfs.epel-5.i686.supermin.img in /usr/lib64/guestfs as it was complaining that he was missing them in

I think this is the multilib issue from comment 5 above.
Comment 7 Richard W.M. Jones 2010-01-25 18:44:12 UTC 

*** This bug has been marked as a duplicate of bug 558593 ***
Comment 8 wes hayutin 2011-12-08 14:08:53 UTC 
perm close