RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1560976 - Hosted Engine VM (deployed in the past) fails to reboot with 'libvirtError: internal error: failed to format device alias for PTY retrieval' due to an error in console device in libvirt XML generated by the engine
Summary: Hosted Engine VM (deployed in the past) fails to reboot with 'libvirtError: i...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.5
Hardware: Unspecified
OS: Unspecified
urgent
high
Target Milestone: rc
: ---
Assignee: Michal Privoznik
QA Contact: yalzhang@redhat.com
URL:
Whiteboard:
Depends On: 1560666
Blocks: 1504606 1566525
TreeView+ depends on / blocked
 
Reported: 2018-03-27 11:58 UTC by Michal Skrivanek
Modified: 2018-10-30 09:55 UTC (History)
15 users (show)

Fixed In Version: libvirt-4.3.0-1.el7
Doc Type: Bug Fix
Doc Text:
Previously, user aliases for PTY devices that were longer than 32 characters were not supported. Consequently, if a domain included a PTY device with a user alias longer than 32 characters, the domain would not start. With this update, a static buffer was replaced with a dynamic buffer. As a result, the domain starts even if the length of the user alias for a PTY device is longer than 32 characters.
Clone Of: 1560666
: 1566525 (view as bug list)
Environment:
Last Closed: 2018-10-30 09:53:26 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:3113 0 None None None 2018-10-30 09:55:50 UTC

Description Michal Skrivanek 2018-03-27 11:58:52 UTC
parent bug is implementing temporary workaround (to use old flow skipping all aliases), we'd like to re-enable that as soon as possible

+++ This bug was initially created as a clone of Bug #1560666 +++

Description of problem:
Since 4.2.2, ovirt-ha-agent extracts the XML from libvirt generated by the engine and pass it to VDSM.
In system deployed in the past (with vintage otopi flow), the hosted-engine VM was containing a console device that gets wrongly rendered in that XML.

        <console type="pty">
            <target port="0" type="virtio"/>
            <alias name="ua-c60aba6e-b6d8-448b-ab6e-8c7b5c29f351"/>
        </console>

Systems deployed with the ansible flow are not affected since the engine VM got create by the engine via REST APIs.


Version-Release number of selected component (if applicable):
4.2.2-rc5 (beta3)

How reproducible:
?

Steps to Reproduce:
1. deploy hosted-engine with the vintage flow
2. wait for the engine to create OVF_STORE disks
3. try to restart the engine VM

Actual results:
the XML generated from the engine contains
        <console type="pty">
            <target port="0" type="virtio"/>
            <alias name="ua-c60aba6e-b6d8-448b-ab6e-8c7b5c29f351"/>
        </console>

the VM fails to start with:
2018-03-26 11:54:31,272-0400 ERROR (vm/eccb9cb6) [virt.vm] (vmId='eccb9cb6-affd-4806-8200-708370581227') The vm start process failed (vm:940)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 869, in _startUnderlyingVm
    self._run()
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2832, in _run
    dom.createWithFlags(flags)
  File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", line 130, in wrapper
    ret = f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 92, in wrapper
    return func(inst, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1099, in createWithFlags
    if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
libvirtError: internal error: failed to format device alias for PTY retrieval

Expected results:
The engine VM is able to restart from systems deployed in the past also booting from the XML created by the engine.

Additional info:
In only affects systems deployed with the vintage flow

Comment 1 yalzhang@redhat.com 2018-03-28 03:21:02 UTC
I can reproduce it on libvirt-3.9.0-14.el7_5.2.x86_64, it seems that the alias name only accept 24 characters

For 24 characters, guest start successfully
# virsh dumpxml rhel
...
<console type='pty'>
      <source path='/dev/pts/4'/>
      <target type='virtio' port='1'/>
      <alias name='ua-123456789012345678901234'/>
    </console>
...
# virsh start rhel
Domain rhel started

For 25 characters, vm can not start
# virsh dumpxml rhel
...
<console type='pty'>
      <target type='virtio' port='1'/>
      <alias name='ua-1234567890123456789012345'/>
    </console>
...
# virsh start rhel
error: Failed to start domain rhel
error: internal error: failed to format device alias for PTY retrieval

Comment 2 yalzhang@redhat.com 2018-03-28 06:49:44 UTC
1. not only for virtio pty console and serial pty console and serial devices, all virtio channel do have such issue, including virtio channel with type=pty/vc/dev/file/pipe/stdio/udp/tcp/unix/spicevmc

# virsh dumpxml rhel | grep ua -A2 -B2
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <alias name='ua-c60aba6e-b6d8-448b-ab6e-8c7b5c29f353'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>

# virsh start rhel
error: Failed to start domain rhel
error: internal error: failed to format device alias for PTY retrieval

And some libvirtd.log for reference:

2018-03-28 03:34:40.927+0000: 8528: error : qemuProcessLookupPTYs:1946 : internal error: failed to format device alias for PTY retrieval
...
2018-03-28 04:54:59.774+0000: 8527: error : qemuProcessRefreshChannelVirtioState:2037 : internal error: failed to format device alias for PTY retrieval

2. channel with target type is guestfwd will ignore any alias setting.
# virsh edit rhel ===> to add alias like below
 <channel type='pty'>
      <target type='guestfwd' address='10.0.2.1' port='4600'/>
     <alias name='ua-justfortest'/>
    </channel>

# virsh dumpxml rhel
...
<channel type='pty'>
      <target type='guestfwd' address='10.0.2.1' port='4600'/>
    </channel>

==> no alias, the alias setting is ignored for channel with guestfwd target type


3. If we will add the alias name length check? I have tried a little, have not found the max length

1) set  <alias name='ua-'/> for interface device, vm can start with qemu option:

 -netdev tap,fd=27,id=hostua-,vhost=on,vhostfd=29 -device virtio-net-pci,netdev=hostua-,id=ua-,mac=52:54:00:03:20:4c,bus=pci.0,addr=0x3


2) tried with alias name as 'ua-{9027 characters}' with interface device, vm can start successfully

Comment 3 Michal Privoznik 2018-03-29 06:52:46 UTC
Patch proposed upstream:

https://www.redhat.com/archives/libvir-list/2018-March/msg01806.html

Comment 4 Michal Privoznik 2018-04-01 10:41:13 UTC
I've just pushed the patch upstream:

commit c4c32cb300beba41ecbca3ee4884c65630bde861
Author:     Michal Privoznik <mprivozn>
AuthorDate: Thu Mar 29 08:50:01 2018 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Sun Apr 1 12:38:38 2018 +0200

    qemu: Use dynamic buffer for storing PTY aliases
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1560976
    
    For historical reasons we've used 32 bytes long static buffer for
    storing PTY aliases. This breaks users scenario where they try to
    start a machine with user alias consisting of "ua-$uuid".
    
    Signed-off-by: Michal Privoznik <mprivozn>

v4.2.0-2-gc4c32cb300

Comment 7 yalzhang@redhat.com 2018-05-30 03:20:10 UTC
test on libvirt-4.3.0-1.el7.x86_64, the result is as expected.

1. Set alias name for serial device in vm xml:
# virsh dumpxml rhel | grep /console -B9
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='ua-3ea66e7d-40f6-4396-a3c0-3160c9bfdabe'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
      <alias name='ua-3ea66e7d-40f6-4396-a3c0-3160c9bfdabe'/>
    </console>
......
 <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-8-rhel/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='ua-62e41900-52d9-4fe7-b788-c04dbb8e61e9'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>

2. Start the vm
# virsh start rhel
Domain rhel started

3. Hotplug a virtio console, pci serial, virtio channel, then hot-unplug, all succeed
# cat console.xml
<console type='dev'>        
 <source path='/dev/ttyS0'/>      
 <target type='virtio' port='1'/> 
<alias name='ua-99d9ca12-dbd1-4c3f-b9c4-cfcccdad86aa'/>
</console>
# virsh attach-device rhel console.xml
Device attached successfully

# cat serial.xml
<serial type='pty'>
 <source path='/dev/pts/4'/>
 <target type='pci-serial' port='0'/> 
<alias name='ua-89a37c8e-363a-4936-b84d-863191eed07e'/>
</serial>
# virsh attach-device rhel serial.xml
Device attached successfully

# cat channel.xml
<channel type='pty'>       
<source path='/dev/pts/5'/>      
 <target type='virtio' name='arbitrary.virtio.serial.port.name'/>   
    <alias name='ua-9b8ebb59-093d-4f9c-b467-99a433e278a7'/>       
 </channel>
# virsh attach-device rhel channel.xml
Device attached successfully

# virsh dumpxml rhel 
==> check all devices exists with the alias name set

4. test duplicate alias
# virsh attach-device rhel ch.xml
error: Failed to attach device from ch.xml
error: XML error: non unique alias detected: ua-3ea66e7d-40f6-4396-a3c0-3160c9bfdabe

Comment 9 errata-xmlrpc 2018-10-30 09:53:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:3113


Note You need to log in before you can comment on or make changes to this bug.