Hide Forgot
+++ This bug was initially created as a clone of Bug #1042505 +++ Version: libvirt-1.1.1-27.el7 I noticed this issue is still present in the latest libvirt RHEL7 build. Any chance this could be fixed? I know it's low priority, but it's a nice 'polishing' bug. Note that it's not only an issue in the default case, which is a bit unlikely (i.e. virtio-serial controller index 0 with more than 31 ports... don't think a lot of people need that many ports). It is a larger issue for virtio-serial controllers with e.g. maxport=4. I've posted patches for this issue we could potentially work from here: https://www.redhat.com/archives/libvir-list/2014-January/msg00187.html Thanks, Jonathan +++ Original description +++ Good afternoon, While experimenting with virtio-serial ports (using SystemTap's stapvirt utility), we noticed something interesting. When stapvirt adds a virtio-serial port, it simply adds the following XML <channel type='unix'> <source mode='bind' path='/my/path/to/sock.sock'/> <target type='virtio' name='org.systemtap.stapsh.0'/> </channel> and relies on libvirt to get assigned to a virtio-serial controller rather than creating an <address> element itself. So the resulting XML might look like this: <channel type='unix'> <source mode='bind' path='/my/path/to/sock.sock'/> <target type='virtio' name='org.systemtap.stapsh.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> When creating another port, libvirt simply looks at previous virtio-serial ports and adds one to the port number. There are a few issues however: 1. libvirt will always use controller='0' and bus='0' even if there is already a controller on a different index. This is not a big deal in itself, since we can make a reasonable assumption that a controller index 0 will exist before index 1, except that 2. libvirt does not respect the maximum number of ports available on a controller. Even if the controller is declared with ports='4', libvirt will gladly create e.g. 10 ports on that controller. This of course will cause qemu to complain, e.g.: error: Failed to start domain TestVM error: internal error: early end of file from monitor: possible problem: qemu-system-x86_64: -device virtserialport,bus=virtio-serial0.0,nr=4,chardev=charchannel3,id=channel3,name=org.systemtap.stapsh.3: virtio-serial-bus: Out-of-range port id specified, max. allowed: 3 qemu-system-x86_64: -device virtserialport,bus=virtio-serial0.0,nr=4,chardev=charchannel3,id=channel3,name=org.systemtap.stapsh.3: Device 'virtserialport' could not be initialized This means that by default, libvirt will implicitly create a controller with index 0, then continuously add ports to that same controller well passed the limit of 31. Expected: libvirt respects the 'ports' attribute of the virtio-serial controller (assume 31 if missing) and creates a new controller if all the controllers are filled. More generally, upon assigning an address, look at all the virtio-serial controllers available (including those to be inferred) and check if there are ports available on any of them. If not, then create a new controller (e.g. set controller to max_controller_index+1 and let it get implicitly created later during the definition process). Thanks
Upstream patches: https://www.redhat.com/archives/libvir-list/2015-March/msg00071.html
Another version of the patches: https://www.redhat.com/archives/libvir-list/2015-March/msg00821.html
v3: https://www.redhat.com/archives/libvir-list/2015-March/msg01233.html
Fixed upstream by: commit 5903378834bafb031407ab02ce37dcc9ec782d1f Author: Ján Tomko <jtomko> CommitDate: 2015-04-02 15:00:13 +0200 Allocate virtio-serial addresses when starting a domain git describe: v1.2.14-30-g5903378 commit 371ea92f058a0df3ba55c0b398f892653cbfca3 Author: Ján Tomko <jtomko> CommitDate: 2015-04-02 15:00:13 +0200 Auto add virtio-serial controllers git describe: v1.2.14-33-g1371ea9
There are some big changes for virtio char devices, I do some sanity testing for those changes. Version: libvirt-1.2.15-1.el7.x86_64 qemu-kvm-rhev-2.3.0-1.el7.x86_64 1. libvirt can find out a free port for the new char device using "virsh edit" [root@localhost ~]# virsh dumpxml r71 | grep "</channel>" -B5 <target type='serial' port='0'/> </console> <channel type='spicevmc'> <target type='virtio' name='com.redhat.spice.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <channel type='pty'> <target type='virtio' name='org.linux-kvm.port.0'/> <address type='virtio-serial' controller='0' bus='0' port='8'/> </channel> Add below 3 line xml to domain: <channel type='pty'> <target type='virtio' name='org.linux-kvm.port.1'/> </channel> [root@localhost ~]# virsh edit r71 Domain r71 XML configuration edited. [root@localhost ~]# virsh dumpxml r71 | grep "</channel>" -B5 <target type='serial' port='0'/> </console> <channel type='spicevmc'> <target type='virtio' name='com.redhat.spice.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <channel type='pty'> <target type='virtio' name='org.linux-kvm.port.0'/> <address type='virtio-serial' controller='0' bus='0' port='8'/> </channel> <channel type='pty'> <target type='virtio' name='org.linux-kvm.port.1'/> <address type='virtio-serial' controller='0' bus='0' port='2'/> </channel> 2. Automatically generated port number for multi char devices add below lines into domain <channel type='spicevmc'> <target type='virtio' name='com.redhat.spice.0'/> </channel> <channel type='pty'> <target type='virtio' name='org.linux-kvm.port.0'/> </channel> <channel type='pty'> <target type='virtio' name='org.linux-kvm.port.0'/> </channel> [root@localhost ~]# virsh edit r71 Domain r71 XML configuration edited. [root@localhost ~]# virsh dumpxml r71 | grep "</channel>" -B3 <channel type='spicevmc'> <target type='virtio' name='com.redhat.spice.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <channel type='pty'> <target type='virtio' name='org.linux-kvm.port.0'/> <address type='virtio-serial' controller='0' bus='0' port='2'/> </channel> <channel type='pty'> <target type='virtio' name='org.linux-kvm.port.0'/> <address type='virtio-serial' controller='0' bus='0' port='3'/> </channel> [root@localhost ~]# 3. Hot-plug/unplug char device for live domain a. add a char with port 4 [root@localhost ~]# virsh dumpxml r71 | grep "</channel>" -B5 </console> <channel type='spicevmc'> <target type='virtio' name='com.redhat.spice.0' state='disconnected'/> <alias name='channel0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <channel type='pty'> <source path='/dev/pts/8'/> <target type='virtio' name='org.linux-kvm.port.0' state='disconnected'/> <alias name='channel1'/> <address type='virtio-serial' controller='0' bus='0' port='2'/> </channel> <channel type='pty'> <source path='/dev/pts/9'/> <target type='virtio' name='org.linux-kvm.port.1' state='disconnected'/> <alias name='channel2'/> <address type='virtio-serial' controller='0' bus='0' port='8'/> </channel> [root@localhost ~]# cat channel.xml <channel type='pty'> <target type='virtio' name='org.linux-kvm.port.3'/> <address type='virtio-serial' controller='0' bus='0' port='4'/> </channel> [root@localhost ~]# virsh attach-device r71 channel.xml Device attached successfully [root@localhost ~]# virsh dumpxml r71 | grep "</channel>" -B5 </console> <channel type='spicevmc'> <target type='virtio' name='com.redhat.spice.0' state='disconnected'/> <alias name='channel0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <channel type='pty'> <source path='/dev/pts/8'/> <target type='virtio' name='org.linux-kvm.port.0' state='disconnected'/> <alias name='channel1'/> <address type='virtio-serial' controller='0' bus='0' port='2'/> </channel> <channel type='pty'> <source path='/dev/pts/9'/> <target type='virtio' name='org.linux-kvm.port.1' state='disconnected'/> <alias name='channel2'/> <address type='virtio-serial' controller='0' bus='0' port='8'/> </channel> <channel type='pty'> <source path='/dev/pts/10'/> <target type='virtio' name='org.linux-kvm.port.3'/> <alias name='channel3'/> <address type='virtio-serial' controller='0' bus='0' port='4'/> </channel> b. add a char without port, search a free port from in 0~30 [root@localhost ~]# cat ch.xml <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/f16x86_64.agent'/> <target type='virtio' name='org.qemu.guest_agent.1'/> </channel> [root@localhost ~]# virsh attach-device r71 ch.xml Device attached successfully [root@localhost ~]# virsh dumpxml r71 | grep "</channel>" -B5 </console> <channel type='spicevmc'> <target type='virtio' name='com.redhat.spice.0' state='disconnected'/> <alias name='channel0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <channel type='pty'> <source path='/dev/pts/8'/> <target type='virtio' name='org.linux-kvm.port.0' state='disconnected'/> <alias name='channel1'/> <address type='virtio-serial' controller='0' bus='0' port='2'/> </channel> <channel type='pty'> <source path='/dev/pts/9'/> <target type='virtio' name='org.linux-kvm.port.1' state='disconnected'/> <alias name='channel2'/> <address type='virtio-serial' controller='0' bus='0' port='8'/> </channel> <channel type='pty'> <source path='/dev/pts/10'/> <target type='virtio' name='org.linux-kvm.port.3'/> <alias name='channel3'/> <address type='virtio-serial' controller='0' bus='0' port='4'/> </channel> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/f16x86_64.agent'/> <target type='virtio' name='org.qemu.guest_agent.1'/> <alias name='channel4'/> <address type='virtio-serial' controller='0' bus='0' port='3'/> </channel> c. detach a char device, its port is 2 [root@localhost ~]# cat de.xml <channel type='pty'> <source path='/dev/pts/8'/> <target type='virtio' name='org.linux-kvm.port.0' state='disconnected'/> <alias name='channel1'/> <address type='virtio-serial' controller='0' bus='0' port='2'/> </channel> [root@localhost ~]# virsh detach-device r71 de.xml Device detached successfully [root@localhost ~]# virsh dumpxml r71 | grep "</channel>" -B5 </console> <channel type='spicevmc'> <target type='virtio' name='com.redhat.spice.0' state='disconnected'/> <alias name='channel0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <channel type='pty'> <source path='/dev/pts/9'/> <target type='virtio' name='org.linux-kvm.port.1' state='disconnected'/> <alias name='channel2'/> <address type='virtio-serial' controller='0' bus='0' port='8'/> </channel> <channel type='pty'> <source path='/dev/pts/10'/> <target type='virtio' name='org.linux-kvm.port.3'/> <alias name='channel3'/> <address type='virtio-serial' controller='0' bus='0' port='4'/> </channel> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/f16x86_64.agent'/> <target type='virtio' name='org.qemu.guest_agent.1'/> <alias name='channel4'/> <address type='virtio-serial' controller='0' bus='0' port='3'/> </channel> d. attach a char dev without port, the port 2 can be used again [root@localhost ~]# cat ch1.xml <channel type='pty'> <target type='virtio' name='org.linux-kvm.port.44'/> </channel> [root@localhost ~]# [root@localhost ~]# virsh attach-device r71 ch1.xml Device attached successfully [root@localhost ~]# virsh dumpxml r71 | grep "</channel>" -B5 </console> <channel type='spicevmc'> <target type='virtio' name='com.redhat.spice.0' state='disconnected'/> <alias name='channel0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <channel type='pty'> <source path='/dev/pts/9'/> <target type='virtio' name='org.linux-kvm.port.1' state='disconnected'/> <alias name='channel2'/> <address type='virtio-serial' controller='0' bus='0' port='8'/> </channel> <channel type='pty'> <source path='/dev/pts/10'/> <target type='virtio' name='org.linux-kvm.port.3'/> <alias name='channel3'/> <address type='virtio-serial' controller='0' bus='0' port='4'/> </channel> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/f16x86_64.agent'/> <target type='virtio' name='org.qemu.guest_agent.1'/> <alias name='channel4'/> <address type='virtio-serial' controller='0' bus='0' port='3'/> </channel> <channel type='pty'> <source path='/dev/pts/8'/> <target type='virtio' name='org.linux-kvm.port.44'/> <alias name='channel5'/> <address type='virtio-serial' controller='0' bus='0' port='2'/> </channel> 4. Checked some exception output: a. duplicated port [root@localhost ~]# cat ch1.xml <channel type='pty'> <target type='virtio' name='org.linux-kvm.port.3'/> <address type='virtio-serial' controller='0' bus='0' port='4'/> </channel> [root@localhost ~]# virsh attach-device r71 ch1.xml error: Failed to attach device from ch1.xml error: XML error: virtio serial port 4 on controller 0 is already occupied b. with a non-existing controller [root@localhost ~]# cat ch1.xml <channel type='pty'> <target type='virtio' name='org.linux-kvm.port.3'/> <address type='virtio-serial' controller='1' bus='0' port='4'/> </channel> [root@localhost ~]# virsh attach-device r71 ch1.xml error: Failed to attach device from ch1.xml error: XML error: virtio serial controller 1 is missing c. exceed the maximum port [root@localhost ~]# cat ch1.xml <channel type='pty'> <target type='virtio' name='org.linux-kvm.port.3'/> <address type='virtio-serial' controller='0' bus='0' port='31'/> </channel> [root@localhost ~]# virsh attach-device r71 ch1.xml error: Failed to attach device from ch1.xml error: XML error: virtio serial controller 0 does not have port 31 d. when no free port on virtio serial controller [root@localhost ~]# virsh attach-device r71 ch1.xml error: Failed to attach device from ch1.xml error: XML error: Unable to find a free virtio-serial port c. libvirt does not support to attach a controller to live domain, only SCSI controller can do that by now. [root@localhost ~]# cat con.xml <controller type='virtio-serial' index='1'> </controller> [root@localhost ~]# virsh attach-device r71 con.xml error: Failed to attach device from con.xml error: Operation not supported: 'virtio-serial' controller cannot be hot plugged. But I found a small issue, with "--config" option libvirt still can accept duplicated port for virtio channel devcies, but based on your patches, the domain can not boot up normally, need we make a small patch to fix this? Thanks. [root@localhost ~]# virsh attach-device r71 channel.xml --config Device attached successfully [root@localhost ~]# vim channel.xml [root@localhost ~]# [root@localhost ~]# virsh attach-device r71 channel.xml --config Device attached successfully [root@localhost ~]# virsh dumpxml r71 | grep "</channel>" -B5 <target type='serial' port='0'/> </console> <channel type='spicevmc'> <target type='virtio' name='com.redhat.spice.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <channel type='pty'> <target type='virtio' name='org.linux-kvm.port.0'/> <address type='virtio-serial' controller='0' bus='0' port='2'/> </channel> <channel type='pty'> <target type='virtio' name='org.linux-kvm.port.1'/> <address type='virtio-serial' controller='0' bus='0' port='8'/> </channel> <channel type='pty'> <target type='virtio' name='org.linux-kvm.port.3'/> <address type='virtio-serial' controller='0' bus='0' port='4'/> </channel> <channel type='pty'> <target type='virtio' name='org.linux-kvm.port.4'/> <address type='virtio-serial' controller='0' bus='0' port='4'/> </channel> [root@localhost ~]# virsh start r71 error: Failed to start domain r71 error: XML error: virtio serial port 4 on controller 0 is already occupied BTW, any suggestion for my above testing results? Is it enough for the bug?
The testing seems sufficient to me. I have sent a patch for the --config issue. It probably deserves a separate bug. https://www.redhat.com/archives/libvir-list/2015-May/msg00184.html
(In reply to Ján Tomko from comment #11) > The testing seems sufficient to me. > Thanks for your review, I'll verify the bug after do more regression testing. > I have sent a patch for the --config issue. It probably deserves a separate > bug. > https://www.redhat.com/archives/libvir-list/2015-May/msg00184.html A new bug for the coldplug issue: Bug 1220195 - Assign virtio-serial addresses after coldplugging a device Thanks.
According to comment 10 and 11, moved to Verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-2202.html