Bug 224464 - virt-install crashes reproducibly on creating a xen domU on fc6
Summary: virt-install crashes reproducibly on creating a xen domU on fc6
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: xen
Version: 6
Hardware: i386
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Daniel Berrangé
QA Contact: Brian Brock
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2007-01-25 20:13 UTC by Max
Modified: 2007-11-30 22:11 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2007-02-21 21:32:19 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
text file with output from various shell cmds to better describe the local setup (22.97 KB, text/plain)
2007-01-25 20:13 UTC, Max
no flags Details

Description Max 2007-01-25 20:13:46 UTC
Hello,

i'm trying to create a domU with virt-install and virt-install crashes always.

Use packages are:
python-virtinst-0.98.0-1.fc6
kernel-xen-2.6.19-1.2895.fc6
xen-libs-3.0.3-3.fc6
xen-3.0.3-3.fc6

How reproducible:
Always


Steps to Reproduce:
root|asrael:~# getenforce
Permissive
root|asrael:~# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0      939     1 r-----    236.2
root|asrael:~# uname -a
Linux asrael.heise.privat 2.6.19-1.2895.fc6xen #1 SMP Wed Jan 10 19:47:12 EST
2007 i686 athlon i386 GNU/Linux
root|asrael:~# virt-install
What is the name of your virtual machine? theta1
 How much RAM should be allocated (in megabytes)? 256
 What would you like to use as the disk (path)? /storage3/xen-images/theta1-hd0
 How large would you like the disk (/storage3/xen-images/theta1-hd0) to be (in
gigabytes)? 6
 Would you like to enable graphics support? (yes or no) yes
 What is the install location? nfs:192.168.6.12:/mnt/mnt1/


Starting install...
libvir: Xen Daemon error : POST operation failed: (xend.err 'destroyDevice()
takes exactly 3 arguments (2 given)')
Failed to get devices for domain theta1
Traceback (most recent call last):
  File "/usr/sbin/virt-install", line 447, in ?
    main()
  File "/usr/sbin/virt-install", line 411, in main
    dom = guest.start_install(conscb)
  File "/usr/lib/python2.4/site-packages/virtinst/ParaVirtGuest.py", line 202,
in start_install
    return XenGuest.XenGuest.start_install(self, consolecb)
  File "/usr/lib/python2.4/site-packages/virtinst/XenGuest.py", line 366, in
start_install
    self.domain = self.conn.createLinux(cxml, 0)
  File "/usr/lib/python2.4/site-packages/libvirt.py", line 329, in createLinux
    if ret is None:raise libvirtError('virDomainCreateLinux() failed', conn=self)
libvirt.libvirtError: virDomainCreateLinux() failed POST operation failed:
(xend.err 'destroyDevice() takes exactly 3 arguments (2 given)')
root|asrael:~# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0      747     1 r-----    254.6
theta1                                     1      256     1 --p---      0.0

  
Actual results:
The theta1 domU is created but virt-install crashes before displaying a window

Additional info:
I can now connect to the theta1 domU with virt-manager and unpause the domU, but
i can't seem to get any network access to continue installation.

I did loop mount the fc6 dvd to /mnt/mnt1 and enable network access to it via
nfs and vsftpd, but the domU cannot access it. I flushed all firewall rules away
just be sure that there is nothing in the way. Also i can't ping the domU after
i provided it with a ip. The domU is not getting any dhcp relpy from my adsl
router. Checked with wireshark that there are no packets from the domU on the
local network at all.

Thanks for your help

Comment 1 Max 2007-01-25 20:13:46 UTC
Created attachment 146615 [details]
text file with output from various shell cmds to better describe the local setup

Comment 2 Jeremy Katz 2007-01-25 20:26:50 UTC
Can you attach /var/log/xen/xend.log and /var/log/xen/xend-debug.log?

Comment 3 Max 2007-01-25 20:40:38 UTC
Please have a look at comment #1. I included everything there.

Comment 4 graeme 2007-02-02 11:41:09 UTC
Hi
Perhaps it should be added that I am getting the same error only in my case an
Enterprise Linux 4 guest environment is bootstrapped to a Logical Volume.

Looking at previos threads that I logged one of the developers / user on the xen
user list mentioned this:
> There is a suspicious[1] patch in unstable, from the 15 Dec 2006 that
> adds a "Force flag" to calls to destroyDevice in DevController.py. If
> someone at RedHat grabbed this patch by mistake or not realizing the
> consequences in some other way, it may well have caused problems. 

More details can be found here:
http://lists.xensource.com/archives/html/xen-users/2007-01/msg01310.html

Hope this helps
Graeme


Comment 5 Michal Schmidt 2007-02-04 22:18:51 UTC
Graeme is right. The broken patch is xen-xenbus-force-option.patch. It adds a 
third parameter to DevController:destroyDevice without updating its callers. 
Perhaps the parameter could be made optional? Like this: "def 
destroyDevice(self, devid, force=None)"?

Comment 6 graeme 2007-02-05 16:15:49 UTC
Hi Michal

Thanks for that.  By updating DevController.py and adding the "force=None"
default parameter the process at least completes with an error that I can
understand.

/usr/lib/python2.4/site-packages/xen/xend/server/DevController.py

Thanks again
Graeme

Comment 7 Mike McGrath 2007-02-20 22:52:35 UTC
Same problem, its on Xen2 if you want access Jeremy, let me know.

Comment 9 Daniel Berrangé 2007-02-21 13:39:22 UTC
The "(xend.err 'destroyDevice() takes exactly 3 arguments (2 given)')" is *not*
the root cause of the problems. As mentioned in comment #6, there is a bug where
not enough args are given to destroyDevice(), however, this bug only occurrs in
the domain cleanup-path. ie by the time this bug hits, the domain is being
killed off due to another failure. The most likely root cause is that one of the
hotplug scripts failed to complete.

I have pushed an updated xen RPM to updates-testing  which fixes the
destroyDevice() bug, so that root cause underlying bug ought to become visible
to the user. Please re-test with  xen-3.0.3-6.fc6  and report the real crash
problem.


Comment 10 Mike McGrath 2007-02-21 14:53:11 UTC
# rpm -Uhv xen-3.0.3-6.fc6.x86_64.rpm xen-libs-3.0.3-6.fc6.x86_64.rpm
warning: xen-3.0.3-6.fc6.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 30c9ecf8
error: Failed dependencies:
error: Failed dependencies:
        python-virtinst is needed by xen-3.0.3-6.fc6.x86_64
        libvirt <= 0.2.0-1.fc6 conflicts with xen-3.0.3-6.fc6.x86_64


This is on an updated FC6 box.  I should note that on my last update
python-virtinst got replaced by python-xeninst.noarch

Comment 11 Daniel Berrangé 2007-02-21 15:02:43 UTC
The updated Xen has dependancies on other packages in updates-testing. These can
be trivially resolved by using yum instead of hand selecting RPMs, eg

  yum --enablerepo=updates-testing  update xen


Comment 12 Mike McGrath 2007-02-21 15:44:17 UTC
[root@xen2 lib]# rpm -q xen
xen-3.0.3-6.fc6
[root@xen2 lib]# xenguest-install -p -n mirrors -r 512 -f
/dev/mapper/VolGroup00-mirrors --nographics -l
nfs:nfsserver:/vol/ftp/fedora.redhat.com/pub/fedora/linux/core/6/x86_64/os/


Starting install...
libvir: Xen Daemon error : POST operation failed: (xend.err 'destroyDevice()
takes exactly 3 arguments (2 given)')
Failed to get devices for domain mirrors
Traceback (most recent call last):
  File "/usr/sbin/xenguest-install", line 380, in ?
    main()
  File "/usr/sbin/xenguest-install", line 344, in main
    dom = guest.start_install(conscb)
  File "/usr/lib/python2.4/site-packages/xeninst/ParaVirtGuest.py", line 194, in
start_install
    return XenGuest.XenGuest.start_install(self, consolecb)
  File "/usr/lib/python2.4/site-packages/xeninst/XenGuest.py", line 280, in
start_install
    self.domain = self.conn.createLinux(cxml, 0)
  File "/usr/lib64/python2.4/site-packages/libvirt.py", line 329, in createLinux
    if ret is None:raise libvirtError('virDomainCreateLinux() failed', conn=self)
libvirt.libvirtError: virDomainCreateLinux() failed POST operation failed:
(xend.err 'destroyDevice() takes exactly 3 arguments (2 given)')


Comment 13 Daniel Berrangé 2007-02-21 15:53:30 UTC
Can you confirm whether you rebooted, or restarted  XenD  after installing the
updated RPM ?  A restart is neccessary to get XenD to load the updated python
modules, and we can't do it automatically in the %post script because it can
have a detrimental impact on any VMs you've got running at the time.

Also, where did 'xenguest-install' come from ? Is this really an FC6 box ? We
renamed the xenguest-install script to be  virt-install  shortly before FC6 GA
release, and the python-xeninst RPM removed from all FC6 related repositories.
The RPM should be called  python-virtinst and the most recent version in
updates-testing is python-virtinst-0.98.0-1.fc6

Comment 14 Mike McGrath 2007-02-21 20:40:43 UTC
Fixed:  Root cause (for me), the NIC's on that box had recently changed and my
new xen bridge is xenbr1, the installer defaults to xenbr0 thus causing the error.

Comment 15 Daniel Berrangé 2007-02-21 21:24:25 UTC
The latest python-virtinst package I listed in comment #13 will automatically
detect which is your primary bridge device & pick the correct one, so it
shouldn't have mattered that your NIC got renamed - it should have automatically
picked xenbr1. Or were you still using the obsolete xenguest-install script ?


Comment 16 Mike McGrath 2007-02-21 21:29:04 UTC
Fails unless I add a -b xenbr1

============== SNIP ===============
Starting install...
libvir: Xen Daemon error : POST operation failed: (xend.err 'Device 0 (vif)
could not be connected. Could not find bridge device xenbr0')
Failed to get devices for domain mirrors
Traceback (most recent call last):
  File "/usr/sbin/xenguest-install", line 380, in ?
    main()
  File "/usr/sbin/xenguest-install", line 344, in main
    dom = guest.start_install(conscb)
  File "/usr/lib/python2.4/site-packages/xeninst/ParaVirtGuest.py", line 194, in
start_install
    return XenGuest.XenGuest.start_install(self, consolecb)
  File "/usr/lib/python2.4/site-packages/xeninst/XenGuest.py", line 280, in
start_install
    self.domain = self.conn.createLinux(cxml, 0)
  File "/usr/lib64/python2.4/site-packages/libvirt.py", line 329, in createLinux
    if ret is None:raise libvirtError('virDomainCreateLinux() failed', conn=self)
libvirt.libvirtError: virDomainCreateLinux() failed POST operation failed:
(xend.err 'Device 0 (vif) could not be connected. Could not find bridge device
xenbr0')

Comment 17 Daniel Berrangé 2007-02-21 21:32:19 UTC
Ok, so you are still using the obsolete xenguest-install script which is no
longer distributed or supported as part of Fedora. If you use 'virt-install'
instead it will detect the bridge correctly.



Note You need to log in before you can comment on or make changes to this bug.