Bug 471079

Summary: Install trees on engarchive appear to be malformed
Product: Red Hat Enterprise Linux 5 Reporter: Mike Belangia <mbelangia>
Component: python-virtinstAssignee: Daniel Berrangé <berrange>
Status: CLOSED DUPLICATE QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 5.2CC: berrange, crobinso, dwmw2, james.brown
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-11-11 18:52:10 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Host system sosreport none

Description Mike Belangia 2008-11-11 17:23:45 UTC
Description of problem:

When trying to create a paravirt guest (host is RHEL5 latest updates) via:

virt-install --paravirt --ram=1024 --vcpus=2 --name=FedoraWorkstation --bridge=xenbr0 --file=/dev/Storage/FedoraWorkstation --location=/mnt/engarchive/released/F-10/Beta/Fedora/x86_64/os   

I get the following stack trace:

[root@dhcp243-97 ~]# virt-install --paravirt --ram=1024 --vcpus=2 --name=FedoraWorkstation --bridge=xenbr0 --file=/dev/Storage/FedoraWorkstation --location=/mnt/engarchive/released/F-10/Beta/Fedora/x86_64/os
Would you like to enable graphics support? (yes or no) no


Starting install...
Invalid file location given: No such file or directory
Domain installation may not have been
 successful.  If it was, you can restart your domain
 by running 'virsh start FedoraWorkstation'; otherwise, please
 restart your installation.
Tue, 11 Nov 2008 12:07:31 ERROR    Invalid file location given: No such file or directory
Traceback (most recent call last):
  File "/usr/sbin/virt-install", line 502, in ?
    main()
  File "/usr/sbin/virt-install", line 462, in main
    dom = guest.start_install(conscb,progresscb)
  File "/usr/lib/python2.4/site-packages/virtinst/Guest.py", line 811, in start_install
    self._prepare_install(meter)
  File "/usr/lib/python2.4/site-packages/virtinst/ParaVirtGuest.py", line 56, in _prepare_install
    self._installer.prepare(guest = self, meter = meter)
  File "/usr/lib/python2.4/site-packages/virtinst/DistroManager.py", line 239, in prepare
    self._prepare_kernel_and_initrd(guest, distro, meter)
  File "/usr/lib/python2.4/site-packages/virtinst/DistroManager.py", line 209, in _prepare_kernel_and_initrd
    distro = distro)
  File "/usr/lib/python2.4/site-packages/virtinst/DistroManager.py", line 123, in acquireKernel
    return store.acquireKernel(fetcher, progresscb)
  File "/usr/lib/python2.4/site-packages/virtinst/OSDistro.py", line 62, in acquireKernel
    kernel = fetcher.acquireFile(kernelpath, progresscb)
  File "/usr/lib/python2.4/site-packages/virtinst/ImageFetcher.py", line 127, in acquireFile
    raise ValueError, _("Invalid file location given: ") + msg
ValueError: Invalid file location given: No such file or directory




This occurs with both i386 and x86_64 trees when using either Beta or Preview.  Using Alpha starts the install but results in a panic:

[root@dhcp243-97 ~]# virt-install --paravirt --ram=1024 --vcpus=2 --name=FedoraWorkstation --bridge=xenbr0 --file=/dev/Storage/FedoraWorkstation --location=/mnt/engarchive/released/F-10/Alpha/Fedora/i386/os
Would you like to enable graphics support? (yes or no) no


Starting install...
Creating domain...                                                 0 B 00:00
------------[ cut here ]------------
WARNING: at kernel/lockdep.c:2068 trace_hardirqs_on_caller+0xad/0x11a() (Not tainted)
Modules linked in:
Pid: 0, comm: swapper Not tainted 2.6.27-0.2.rc0.git6.fc10.i686.xen #1
 [<c043247d>] warn_on_slowpath+0x4c/0x91
 [<c04099b3>] ? mcount_call+0x5/0xa
 [<c04099b3>] ? mcount_call+0x5/0xa
 [<c04099b3>] ? mcount_call+0x5/0xa
 [<c0404d2e>] ? xen_mc_flush+0x13d/0x18c
 [<c0403745>] ? xen_save_fl+0x8/0x21
 [<c044f3a4>] ? trace_hardirqs_off_caller+0x35/0x91
 [<c0450208>] ? trace_hardirqs_on+0xb/0xd
 [<c0450190>] trace_hardirqs_on_caller+0xad/0x11a
 [<c0450208>] trace_hardirqs_on+0xb/0xd
 [<c0405de3>] xen_set_pud_hyper+0x8a/0xa2
 [<c07fe3f4>] xen_start_kernel+0x67c/0x7ea
 =======================
---[ end trace 4eaa2a86a8e2da22 ]---
Reserving virtual address space above 0xf57fe000
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.27-0.2.rc0.git6.fc10.i686.xen (mockbuild@x86-2) (gcc version 4.3.1 20080708 (Red Hat 4.3.1-4) (GCC) ) #1 SMP Fri Jul 18 12:16:23 EDT 2008
ACPI in unprivileged domain disabled
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 00000000000a0000 (usable)
 Xen: 00000000000a0000 - 0000000000100000 (reserved)
 Xen: 0000000000100000 - 000000000194a000 (usable)
 Xen: 000000000194a000 - 0000000001a4d000 (reserved)
 Xen: 0000000001a4d000 - 0000000040000000 (usable)
last_pfn = 0x40000 max_arch_pfn = 0x1000000
kernel direct mapping tables up to 2d7fe000 @ 100000-270000
NX (Execute Disable) protection: active
Using x86 segment limits to approximate NX protection
RAMDISK: 00c5a000 - 0194a000
DMI not present or invalid.
Entering add_active_range(0, 0x0, 0xa0) 0 entries of 256 used
Entering add_active_range(0, 0x100, 0x194a) 1 entries of 256 used
Entering add_active_range(0, 0x1a4d, 0x40000) 2 entries of 256 used
296MB HIGHMEM available.
727MB LOWMEM available.
  mapped low ram: 0 - 2d7fe000
  low ram: 00000000 - 2d7fe000
  bootmap 00007000 - 0000cb00
(8 early reservations) ==> bootmem
  #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
  #1 [0000001000 - 0000002000]    EX TRAMPOLINE ==> [0000001000 - 0000002000]
  #2 [0000006000 - 0000007000]       TRAMPOLINE ==> [0000006000 - 0000007000]
  #3 [0000400000 - 0000c59194]    TEXT DATA BSS ==> [0000400000 - 0000c59194]
  #4 [0000c5a000 - 000194a000]          RAMDISK ==> [0000c5a000 - 000194a000]
  #5 [0001a4d000 - 0001a5d000]    INIT_PG_TABLE ==> [0001a4d000 - 0001a5d000]
  #6 [0000100000 - 000025d000]          PGTABLE ==> [0000100000 - 000025d000]
  #7 [0000007000 - 000000d000]          BOOTMAP ==> [0000007000 - 000000d000]
Scan SMP from c0000000 for 1024 bytes.
Scan SMP from c009fc00 for 1024 bytes.
Scan SMP from c00f0000 for 65536 bytes.
Zone PFN ranges:
  DMA      0x00000000 -> 0x00001000
  Normal   0x00001000 -> 0x0002d7fe
  HighMem  0x0002d7fe -> 0x00040000
Movable zone start PFN for each node
early_node_map[3] active PFN ranges
    0: 0x00000000 -> 0x000000a0
    0: 0x00000100 -> 0x0000194a
    0: 0x00001a4d -> 0x00040000
On node 0 totalpages: 261789
free_area_init_node: node 0, pgdat c07bab00, node_mem_map c1a5e000
  DMA zone: 56 pages used for memmap
  DMA zone: 0 pages reserved
  DMA zone: 3944 pages, LIFO batch:0
  Normal zone: 2492 pages used for memmap
  Normal zone: 179519 pages, LIFO batch:31
  HighMem zone: 1037 pages used for memmap
  HighMem zone: 74741 pages, LIFO batch:15
  Movable zone: 0 pages used for memmap
Using APIC driver default
SMP: Allowing 2 CPUs, 0 hotplug CPUs
Local APIC disabled by BIOS -- you can enable it with "lapic"
mapped APIC to f57f9000 (0286e000)
PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
PM: Registered nosave memory: 000000000194a000 - 0000000001a4d000
Allocating PCI resources starting at 50000000 (gap: 40000000:c0000000)
PERCPU: Allocating 340660 bytes of per cpu data
NR_CPUS: 32, nr_cpu_ids: 2, nr_node_ids 1
trying to map vcpu_info 0 at c2870014, mfn 4110b, offset 20
register_vcpu_info failed: err=-38
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 258204
Kernel command line:
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
CPU 0 irqstacks, hard=c08c0000 soft=c08a0000
PID hash table entries: 4096 (order: 12, 16384 bytes)
Detected 2599.996 MHz processor.
Console: colour dummy device 80x25
console [tty0] enabled
console [hvc0] enabled
Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
... MAX_LOCKDEP_SUBCLASSES:    8
... MAX_LOCK_DEPTH:          48
... MAX_LOCKDEP_KEYS:        2048
... CLASSHASH_SIZE:           1024
... MAX_LOCKDEP_ENTRIES:     8192
... MAX_LOCKDEP_CHAINS:      16384
... CHAINHASH_SIZE:          8192
 memory used by lock dependency info: 1088 kB
 per task-struct memory footprint: 2688 bytes
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Memory: 1007508k/1048576k available (2658k kernel code, 39160k reserved, 1385k data, 628k init, 303112k highmem)
virtual kernel memory layout:
    fixmap  : 0xf5455000 - 0xf57fd000   (3744 kB)
    pkmap   : 0xf5000000 - 0xf5200000   (2048 kB)
    vmalloc : 0xee000000 - 0xf4ffe000   ( 111 MB)
    lowmem  : 0xc0000000 - 0xed7fe000   ( 727 MB)
      .init : 0xc07f8000 - 0xc0895000   ( 628 kB)
      .data : 0xc06988a8 - 0xc07f2e90   (1385 kB)
      .text : 0xc0400000 - 0xc06988a8   (2658 kB)
Checking if this processor honours the WP bit even in supervisor mode...Ok.
CPA: page pool initialized 16 of 16 pages preallocated
1 multicall(s) failed: cpu 0
Pid: 0, comm: swapper Tainted: G        W 2.6.27-0.2.rc0.git6.fc10.i686.xen #1
 [<c0404cb1>] xen_mc_flush+0xc0/0x18c
 [<c0405dc2>] xen_set_pud_hyper+0x69/0xa2
 [<c0405e34>] xen_set_pud+0x39/0x3e
 [<c041fb80>] zap_low_mappings+0x34/0x4c
 [<c080e390>] mem_init+0x2c7/0x2cf
 [<c07f898a>] start_kernel+0x262/0x31b
 [<c07f8096>] i386_start_kernel+0x85/0x8d
 [<c07fe55a>] xen_start_kernel+0x7e2/0x7ea
 =======================
  call  1/1: op=1 arg=[c2870854] result=-22
------------[ cut here ]------------
kernel BUG at arch/x86/xen/multicalls.c:104!
invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
Modules linked in:

Pid: 0, comm: swapper Tainted: G        W (2.6.27-0.2.rc0.git6.fc10.i686.xen #1)
EIP: e019:[<c0404d71>] EFLAGS: 00010002 CPU: 0
EIP is at xen_mc_flush+0x180/0x18c
EAX: c2870054 EBX: 00000000 ECX: c04329d1 EDX: c0404d2e
ESI: 00000001 EDI: 00000001 EBP: c07f3efc ESP: c07f3ee8
 DS: e021 ES: e021 FS: 00d8 GS: 0000 SS: e021
Process swapper (pid: 0, ti=c07f3000 task=c07b13fc task.ti=c07f3000)
Stack: c2870054 00000000 0202d000 5a414001 00000000 c07f3f24 c0405dc2 c07e1000
       4f4c0000 00000000 5a414001 00000000 c07e1000 5a414001 00000000 c07f3f38
       c0405e34 c07e1000 c07e2000 00000000 c07f3f50 c041fb80 5a414001 f4ffe000
Call Trace:
 [<c0405dc2>] ? xen_set_pud_hyper+0x69/0xa2
 [<c0405e34>] ? xen_set_pud+0x39/0x3e
 [<c041fb80>] ? zap_low_mappings+0x34/0x4c
 [<c080e390>] ? mem_init+0x2c7/0x2cf
 [<c07f898a>] ? start_kernel+0x262/0x31b
 [<c07f8096>] ? i386_start_kernel+0x85/0x8d
 [<c07fe55a>] ? xen_start_kernel+0x7e2/0x7ea
 =======================
Code: 8b 55 ec 8b 84 da 04 0a 00 00 ff 94 da 00 0a 00 00 43 8b 45 ec 3b 98 08 0b 00 00 72 e3 85 ff c7 80 08 0b 00 00 00 00 00 00 74 04 <0f> 0b eb fe 8d 65 f4 5b 5e 5f 5d c3 55 89 e5 57 56 53 e8 1c 4c
EIP: [<c0404d71>] xen_mc_flush+0x180/0x18c SS:ESP e021:c07f3ee8
---[ end trace 4eaa2a86a8e2da22 ]---
Kernel panic - not syncing: Attempted to kill the idle task!
------------[ cut here ]------------
WARNING: at kernel/smp.c:288 smp_call_function_mask+0x38/0x131() (Tainted: G      D W)
Modules linked in:
Pid: 0, comm: swapper Tainted: G      D W 2.6.27-0.2.rc0.git6.fc10.i686.xen #1
 [<c043247d>] warn_on_slowpath+0x4c/0x91
 [<c0696b64>] ? __atomic_notifier_call_chain+0x3c/0x5f
 [<c0694463>] ? _spin_unlock+0x22/0x25
 [<c0583ff3>] ? vt_console_print+0x286/0x28e
 [<c069464c>] ? _spin_lock_irqsave+0x28/0x6d
 [<c069442a>] ? _spin_unlock_irqrestore+0x3e/0x55
 [<c044f40b>] ? trace_hardirqs_off+0xb/0xd
 [<c069442a>] ? _spin_unlock_irqrestore+0x3e/0x55
 [<c044f40b>] ? trace_hardirqs_off+0xb/0xd
 [<c044f384>] ? trace_hardirqs_off_caller+0x15/0x91
 [<c04099b3>] ? mcount_call+0x5/0xa
 [<c069442a>] ? _spin_unlock_irqrestore+0x3e/0x55
 [<c04329d1>] ? release_console_sem+0x1ad/0x1c0
 [<c04099b3>] ? mcount_call+0x5/0xa
 [<c04068ed>] ? stop_self+0x0/0x32
 [<c0455b0c>] smp_call_function_mask+0x38/0x131
 [<c04068ed>] ? stop_self+0x0/0x32
 [<c04099b3>] ? mcount_call+0x5/0xa
 [<c069442a>] ? _spin_unlock_irqrestore+0x3e/0x55
 [<c04329d1>] ? release_console_sem+0x1ad/0x1c0
 [<c045f386>] ? crash_kexec+0xd/0xad
 [<c04329d1>] ? release_console_sem+0x1ad/0x1c0
 [<c044f40b>] ? trace_hardirqs_off+0xb/0xd
 [<c04327f6>] ? wake_up_klogd+0x8/0x36
 [<c04099b3>] ? mcount_call+0x5/0xa
 [<c069442a>] ? _spin_unlock_irqrestore+0x3e/0x55
 [<c04329d1>] ? release_console_sem+0x1ad/0x1c0
 [<c04325fb>] ? is_console_locked+0x8/0xf
 [<c0455c1c>] smp_call_function+0x17/0x19
 [<c04069fc>] xen_smp_send_stop+0x16/0x18
 [<c0435457>] ? do_exit+0x60/0x75d
 [<c04099b3>] ? mcount_call+0x5/0xa
 [<c0432301>] ? oops_exit+0x28/0x2d
 [<c040a37a>] ? die+0x161/0x169
 [<c069521b>] ? do_trap+0x8f/0xa8
 [<c040a7da>] ? do_invalid_op+0x0/0x7b
 [<c040a84b>] ? do_invalid_op+0x71/0x7b
 [<c0404d71>] ? xen_mc_flush+0x180/0x18c
 [<c044f40b>] ? trace_hardirqs_off+0xb/0xd
 [<c04327f6>] ? wake_up_klogd+0x8/0x36
 [<c04099b3>] ? mcount_call+0x5/0xa
 [<c069442a>] ? _spin_unlock_irqrestore+0x3e/0x55
 [<c04329d1>] ? release_console_sem+0x1ad/0x1c0
 [<c0403aa4>] ? xen_restore_fl+0x8/0x2b
 [<c0432edd>] ? vprintk+0x331/0x356
 [<c069442a>] ? _spin_unlock_irqrestore+0x3e/0x55
 [<c04329d1>] ? release_console_sem+0x1ad/0x1c0
 [<c04099d8>] ? print_trace_stack+0x8/0xc
 [<c040a9e9>] ? show_trace+0x29/0x2e
 [<c0694aea>] ? error_code+0x72/0x78
 [<c04329d1>] ? release_console_sem+0x1ad/0x1c0
 [<c0404d2e>] ? xen_mc_flush+0x13d/0x18c
 [<c0404d71>] ? xen_mc_flush+0x180/0x18c
 [<c0405dc2>] ? xen_set_pud_hyper+0x69/0xa2
 [<c0405e34>] ? xen_set_pud+0x39/0x3e
 [<c041fb80>] ? zap_low_mappings+0x34/0x4c
 [<c080e390>] ? mem_init+0x2c7/0x2cf
 [<c07f898a>] ? start_kernel+0x262/0x31b
 [<c07f8096>] ? i386_start_kernel+0x85/0x8d
 [<c07fe55a>] ? xen_start_kernel+0x7e2/0x7ea
 =======================
---[ end trace 4eaa2a86a8e2da22 ]---



Source tree is mounted via:
[root@dhcp243-97 ~]# cat /etc/fstab |grep engarchive
engarchive:/engineering/archives2       /mnt/engarchive nfs     defaults       0 0

Comment 1 Mike Belangia 2008-11-11 17:28:18 UTC
Created attachment 323198 [details]
Host system sosreport

Comment 2 David Woodhouse 2008-11-11 17:29:14 UTC
Not a problem with the 0xFFFF package. Please reopen and file against the correct package if the problem persists.

Comment 3 Mike Belangia 2008-11-11 17:58:53 UTC
Of course the problem persists.  Can you tell me what the correct package is to file against the install tree itself?  I have no clue which of the 100s of packages in fedora would be the correct one as it's not a part of any package I'm aware of.

Comment 4 David Woodhouse 2008-11-11 18:08:16 UTC
I've no idea; sorry. Probably the package that provides virt-install, for the python backtrace? And the kernel, for the errors you were seeing from that?

I'm trying _not_ to get lumbered with a slew of triage just because I own the 0xFFFF package though.

Comment 5 Daniel Berrangé 2008-11-11 18:27:08 UTC
Since this is about a virt-install on a RHEL-5 host, it shouldn't be filed against Fedora either. Changing to RHEL product


For more info please re-run virt-install, adding the --debug argument, and attach the complete unedited output to this bug.

Comment 6 Mike Belangia 2008-11-11 18:35:30 UTC
[root@dhcp243-97 ~]# virt-install --paravirt --ram=1024 --vcpus=1 --debug --name=FedoraWorkstation --bridge=xenbr0 --file=/dev/Storage/FedoraWorkstation --location=/mnt/engarchive/released/F-10/Preview/Fedora/i386/os
Tue, 11 Nov 2008 13:36:50 DEBUG    Path is block file: Assuming Block disk type.
Would you like to enable graphics support? (yes or no) no
Tue, 11 Nov 2008 13:36:52 DEBUG    DistroInstaller location is a local file/path: /mnt/engarchive/released/F-10/Preview/Fedora/i386/os


Starting install...
Tue, 11 Nov 2008 13:36:52 DEBUG    Acquiring file from /mnt/engarchive/released/F-10/Preview/Fedora/i386/os/.treeinfo
Tue, 11 Nov 2008 13:36:52 DEBUG    Saved file to /var/lib/xen/virtinst-.treeinfo.GN6hsf
Tue, 11 Nov 2008 13:36:52 DEBUG    Detected a Fedora distro
Tue, 11 Nov 2008 13:36:52 DEBUG    Acquiring file from /mnt/engarchive/released/F-10/Preview/Fedora/i386/os/images/xen/vmlinuz
Invalid file location given: No such file or directory
Domain installation may not have been
 successful.  If it was, you can restart your domain
 by running 'virsh start FedoraWorkstation'; otherwise, please
 restart your installation.
Tue, 11 Nov 2008 13:36:52 ERROR    Invalid file location given: No such file or directory
Traceback (most recent call last):
  File "/usr/sbin/virt-install", line 502, in ?
    main()
  File "/usr/sbin/virt-install", line 462, in main
    dom = guest.start_install(conscb,progresscb)
  File "/usr/lib/python2.4/site-packages/virtinst/Guest.py", line 811, in start_install
    self._prepare_install(meter)
  File "/usr/lib/python2.4/site-packages/virtinst/ParaVirtGuest.py", line 56, in _prepare_install
    self._installer.prepare(guest = self, meter = meter)
  File "/usr/lib/python2.4/site-packages/virtinst/DistroManager.py", line 239, in prepare
    self._prepare_kernel_and_initrd(guest, distro, meter)
  File "/usr/lib/python2.4/site-packages/virtinst/DistroManager.py", line 209, in _prepare_kernel_and_initrd
    distro = distro)
  File "/usr/lib/python2.4/site-packages/virtinst/DistroManager.py", line 123, in acquireKernel
    return store.acquireKernel(fetcher, progresscb)
  File "/usr/lib/python2.4/site-packages/virtinst/OSDistro.py", line 62, in acquireKernel
    kernel = fetcher.acquireFile(kernelpath, progresscb)
  File "/usr/lib/python2.4/site-packages/virtinst/ImageFetcher.py", line 127, in acquireFile
    raise ValueError, _("Invalid file location given: ") + msg
ValueError: Invalid file location given: No such file or directory

Comment 7 Mike Belangia 2008-11-11 18:44:25 UTC
Looks like the problem is a lack of "xen" dir in the Beta/Preview directories:

[root@dhcp243-97 tmp]# ls /mnt/engarchive/released/F-10/Preview/Fedora/i386/os/images/
boot.iso  efidisk.img  install.img  pxeboot  README

[root@dhcp243-97 tmp]# ls /mnt/engarchive/released/F-10/Alpha/Fedora/i386/os/images/
boot.iso  efidisk.img  pxeboot  README  stage2.img  xen

[root@dhcp243-97 tmp]# ls /mnt/engarchive/released/F-10/Beta/Fedora/i386/os/images/
boot.iso  efidisk.img  install.img  pxeboot  README

Comment 8 Daniel Berrangé 2008-11-11 18:52:10 UTC
No, the Beta/Preview trees are fine. We use a single kernel image now. The problem is just that you're using an old virt-install which doesn't understand new tree layout with pv_ops world. See bug 460585

*** This bug has been marked as a duplicate of bug 460585 ***